1
|
Lippl S, Kay K, Jensen G, Ferrera VP, Abbott LF. A mathematical theory of relational generalization in transitive inference. Proc Natl Acad Sci U S A 2024; 121:e2314511121. [PMID: 38968113 PMCID: PMC11252811 DOI: 10.1073/pnas.2314511121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 05/30/2024] [Indexed: 07/07/2024] Open
Abstract
Humans and animals routinely infer relations between different items or events and generalize these relations to novel combinations of items. This allows them to respond appropriately to radically novel circumstances and is fundamental to advanced cognition. However, how learning systems (including the brain) can implement the necessary inductive biases has been unclear. We investigated transitive inference (TI), a classic relational task paradigm in which subjects must learn a relation ([Formula: see text] and [Formula: see text]) and generalize it to new combinations of items ([Formula: see text]). Through mathematical analysis, we found that a broad range of biologically relevant learning models (e.g. gradient flow or ridge regression) perform TI successfully and recapitulate signature behavioral patterns long observed in living subjects. First, we found that models with item-wise additive representations automatically encode transitive relations. Second, for more general representations, a single scalar "conjunctivity factor" determines model behavior on TI and, further, the principle of norm minimization (a standard statistical inductive bias) enables models with fixed, partly conjunctive representations to generalize transitively. Finally, neural networks in the "rich regime," which enables representation learning and improves generalization on many tasks, unexpectedly show poor generalization and anomalous behavior on TI. We find that such networks implement a form of norm minimization (over hidden weights) that yields a local encoding mechanism lacking transitivity. Our findings show how minimal statistical learning principles give rise to a classical relational inductive bias (transitivity), explain empirically observed behaviors, and establish a formal approach to understanding the neural basis of relational abstraction.
Collapse
Affiliation(s)
- Samuel Lippl
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, NY10027
- Center for Theoretical Neuroscience, Department of Neuroscience, Columbia University, New York, NY10027
- Department of Neuroscience, Columbia University Medical Center, New York, NY10032
| | - Kenneth Kay
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, NY10027
- Center for Theoretical Neuroscience, Department of Neuroscience, Columbia University, New York, NY10027
- Grossman Center for the Statistics of Mind, Columbia University, New York, NY10027
| | - Greg Jensen
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, NY10027
- Department of Neuroscience, Columbia University Medical Center, New York, NY10032
- Department of Psychology, Reed College, Portland, OR97202
| | - Vincent P. Ferrera
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, NY10027
- Department of Neuroscience, Columbia University Medical Center, New York, NY10032
- Department of Psychiatry, Columbia University Medical Center, New York, NY10032
| | - L. F. Abbott
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, NY10027
- Center for Theoretical Neuroscience, Department of Neuroscience, Columbia University, New York, NY10027
- Department of Neuroscience, Columbia University Medical Center, New York, NY10032
| |
Collapse
|
2
|
Driscoll LN, Shenoy K, Sussillo D. Flexible multitask computation in recurrent networks utilizes shared dynamical motifs. Nat Neurosci 2024; 27:1349-1363. [PMID: 38982201 PMCID: PMC11239504 DOI: 10.1038/s41593-024-01668-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 04/26/2024] [Indexed: 07/11/2024]
Abstract
Flexible computation is a hallmark of intelligent behavior. However, little is known about how neural networks contextually reconfigure for different computations. In the present work, we identified an algorithmic neural substrate for modular computation through the study of multitasking artificial recurrent neural networks. Dynamical systems analyses revealed learned computational strategies mirroring the modular subtask structure of the training task set. Dynamical motifs, which are recurring patterns of neural activity that implement specific computations through dynamics, such as attractors, decision boundaries and rotations, were reused across tasks. For example, tasks requiring memory of a continuous circular variable repurposed the same ring attractor. We showed that dynamical motifs were implemented by clusters of units when the unit activation function was restricted to be positive. Cluster lesions caused modular performance deficits. Motifs were reconfigured for fast transfer learning after an initial phase of learning. This work establishes dynamical motifs as a fundamental unit of compositional computation, intermediate between neuron and network. As whole-brain studies simultaneously record activity from multiple specialized systems, the dynamical motif framework will guide questions about specialization and generalization.
Collapse
Affiliation(s)
- Laura N Driscoll
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
| | - Krishna Shenoy
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Neurobiology, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Bio-X Institute, Stanford University, Stanford, CA, USA
- Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA
| | - David Sussillo
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| |
Collapse
|
3
|
Biao D, Umoh K, Qiguang C, Xiaole W, Ting F, Yuqian Y, Jinchao Z, Fushui L. The Role of Mindfulness Therapy in the Treatment of Chronic Pain. Curr Pain Headache Rep 2024:10.1007/s11916-024-01284-w. [PMID: 38951466 DOI: 10.1007/s11916-024-01284-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/12/2024] [Indexed: 07/03/2024]
Abstract
PURPOSE OF REVIEW Mindfulness therapy is a widely used treatment for many diseases and has been shown to improve pain-related functions. There is growing support for the use of psychotherapy in the treatment of chronic pain. While studies have shown a positive effect of mindfulness therapy, it is important to consider psychosocial factors as there are still a small number of studies that question its effectiveness. RECENT FINDINGS Based on current studies, mindfulness therapy involves cognitive factors related to chronic pain, both in terms of cognitive production and its impact on cognitive control. Psychological and neurobasic studies were reviewed to provide a deeper understanding of these components, which include thought inhibition, attention deficit, pain catastrophizing, and self-efficacy. Mindfulness therapy has the potential to normalize psychology and nerves, and increase internal and external connectivity to work networks related to stress perception, cognition, and emotion. However, further research is needed to fully understand its effects. By exploring the relationship between mindfulness therapy and chronic pain. This review provides a new avenue for future research in psychotherapy for patients with chronic pain.
Collapse
Affiliation(s)
- Deng Biao
- School of Clinical Medicine, Jiangxi University of Traditional Chinese Medicine, Nanchang, China
| | - KuyikAbasi Umoh
- School of Clinical Medicine, Jiangxi University of Traditional Chinese Medicine, Nanchang, China
| | - Cao Qiguang
- Apartment of Acupotomy and Chiropractic, Affiliated Hospital of Jiangxi University of Traditional Chinese Medicine, Nanchang, China
| | - Wang Xiaole
- Apartment of Acupotomy and Chiropractic, Affiliated Hospital of Jiangxi University of Traditional Chinese Medicine, Nanchang, China.
| | - Fang Ting
- Apartment of Acupotomy and Chiropractic, Affiliated Hospital of Jiangxi University of Traditional Chinese Medicine, Nanchang, China
| | - Yang Yuqian
- School of Clinical Medicine, Jiangxi University of Traditional Chinese Medicine, Nanchang, China
| | - Zhu Jinchao
- School of Clinical Medicine, Jiangxi University of Traditional Chinese Medicine, Nanchang, China
| | - Liu Fushui
- Apartment of Acupotomy and Chiropractic, Affiliated Hospital of Jiangxi University of Traditional Chinese Medicine, Nanchang, China.
| |
Collapse
|
4
|
Hafri A, Bonner MF, Landau B, Firestone C. A Phone in a Basket Looks Like a Knife in a Cup: Role-Filler Independence in Visual Processing. Open Mind (Camb) 2024; 8:766-794. [PMID: 38957507 PMCID: PMC11219067 DOI: 10.1162/opmi_a_00146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 04/17/2024] [Indexed: 07/04/2024] Open
Abstract
When a piece of fruit is in a bowl, and the bowl is on a table, we appreciate not only the individual objects and their features, but also the relations containment and support, which abstract away from the particular objects involved. Independent representation of roles (e.g., containers vs. supporters) and "fillers" of those roles (e.g., bowls vs. cups, tables vs. chairs) is a core principle of language and higher-level reasoning. But does such role-filler independence also arise in automatic visual processing? Here, we show that it does, by exploring a surprising error that such independence can produce. In four experiments, participants saw a stream of images containing different objects arranged in force-dynamic relations-e.g., a phone contained in a basket, a marker resting on a garbage can, or a knife sitting in a cup. Participants had to respond to a single target image (e.g., a phone in a basket) within a stream of distractors presented under time constraints. Surprisingly, even though participants completed this task quickly and accurately, they false-alarmed more often to images matching the target's relational category than to those that did not-even when those images involved completely different objects. In other words, participants searching for a phone in a basket were more likely to mistakenly respond to a knife in a cup than to a marker on a garbage can. Follow-up experiments ruled out strategic responses and also controlled for various confounding image features. We suggest that visual processing represents relations abstractly, in ways that separate roles from fillers.
Collapse
Affiliation(s)
- Alon Hafri
- Department of Linguistics and Cognitive Science, University of Delaware
- Department of Cognitive Science, Johns Hopkins University
- Department of Psychological and Brain Sciences, Johns Hopkins University
| | | | - Barbara Landau
- Department of Cognitive Science, Johns Hopkins University
| | - Chaz Firestone
- Department of Cognitive Science, Johns Hopkins University
- Department of Psychological and Brain Sciences, Johns Hopkins University
| |
Collapse
|
5
|
Shain C, Kean H, Casto C, Lipkin B, Affourtit J, Siegelman M, Mollica F, Fedorenko E. Distributed Sensitivity to Syntax and Semantics throughout the Language Network. J Cogn Neurosci 2024; 36:1427-1471. [PMID: 38683732 DOI: 10.1162/jocn_a_02164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/02/2024]
Abstract
Human language is expressive because it is compositional: The meaning of a sentence (semantics) can be inferred from its structure (syntax). It is commonly believed that language syntax and semantics are processed by distinct brain regions. Here, we revisit this claim using precision fMRI methods to capture separation or overlap of function in the brains of individual participants. Contrary to prior claims, we find distributed sensitivity to both syntax and semantics throughout a broad frontotemporal brain network. Our results join a growing body of evidence for an integrated network for language in the human brain within which internal specialization is primarily a matter of degree rather than kind, in contrast with influential proposals that advocate distinct specialization of different brain areas for different types of linguistic functions.
Collapse
Affiliation(s)
| | - Hope Kean
- Massachusetts Institute of Technology
| | | | | | | | | | | | | |
Collapse
|
6
|
Fedorenko E, Piantadosi ST, Gibson EAF. Language is primarily a tool for communication rather than thought. Nature 2024; 630:575-586. [PMID: 38898296 DOI: 10.1038/s41586-024-07522-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Accepted: 05/03/2024] [Indexed: 06/21/2024]
Abstract
Language is a defining characteristic of our species, but the function, or functions, that it serves has been debated for centuries. Here we bring recent evidence from neuroscience and allied disciplines to argue that in modern humans, language is a tool for communication, contrary to a prominent view that we use language for thinking. We begin by introducing the brain network that supports linguistic ability in humans. We then review evidence for a double dissociation between language and thought, and discuss several properties of language that suggest that it is optimized for communication. We conclude that although the emergence of language has unquestionably transformed human culture, language does not appear to be a prerequisite for complex thought, including symbolic thought. Instead, language is a powerful tool for the transmission of cultural knowledge; it plausibly co-evolved with our thinking and reasoning capacities, and only reflects, rather than gives rise to, the signature sophistication of human cognition.
Collapse
Affiliation(s)
- Evelina Fedorenko
- Massachusetts Institute of Technology, Cambridge, MA, USA.
- Speech and Hearing in Bioscience and Technology Program at Harvard University, Boston, MA, USA.
| | | | | |
Collapse
|
7
|
Huang C, Li A, Pang Y, Yang J, Zhang J, Wu X, Mei L. How the intrinsic functional connectivity patterns of the semantic network support semantic processing. Brain Imaging Behav 2024; 18:539-554. [PMID: 38261218 DOI: 10.1007/s11682-024-00849-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/05/2024] [Indexed: 01/24/2024]
Abstract
Semantic processing, a core of language comprehension, involves the activation of brain regions dispersed extensively across the frontal, temporal, and parietal cortices that compose the semantic network. To comprehend the functional structure of this semantic network and how it prepares for semantic processing, we investigated its intrinsic functional connectivity (FC) and the relation between this pattern and semantic processing ability in a large sample from the Human Connectome Project (HCP) dataset. We first defined a well-studied brain network for semantic processing, and then we characterized the within-network connectivity (WNC) and the between-network connectivity (BNC) within this network using a voxel-based global brain connectivity (GBC) method based on resting-state functional magnetic resonance imaging (fMRI). The results showed that 97.73% of the voxels in the semantic network displayed considerably greater WNC than BNC, demonstrating that the semantic network is a fairly encapsulated network. Moreover, multiple connector hubs in the semantic network were identified after applying the criterion of WNC > 1 SD above the mean WNC of the semantic network. More importantly, three of these connector hubs (i.e., the left anterior temporal lobe, angular gyrus, and orbital part of the inferior frontal gyrus) were reliably associated with semantic processing ability. Our findings suggest that the three identified regions use WNC as the central mechanism for supporting semantic processing and that task-independent spontaneous connectivity in the semantic network is essential for semantic processing.
Collapse
Affiliation(s)
- Chengmei Huang
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Aqian Li
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Yingdan Pang
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Jiayi Yang
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Jingxian Zhang
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Xiaoyan Wu
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China
- School of Psychology, South China Normal University, Guangzhou, 510631, China
- Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631, China
| | - Leilei Mei
- Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education, Guangzhou, 510631, China.
| |
Collapse
|
8
|
Webb TW, Frankland SM, Altabaa A, Segert S, Krishnamurthy K, Campbell D, Russin J, Giallanza T, O'Reilly R, Lafferty J, Cohen JD. The relational bottleneck as an inductive bias for efficient abstraction. Trends Cogn Sci 2024:S1364-6613(24)00080-9. [PMID: 38729852 DOI: 10.1016/j.tics.2024.04.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 03/29/2024] [Accepted: 04/01/2024] [Indexed: 05/12/2024]
Abstract
A central challenge for cognitive science is to explain how abstract concepts are acquired from limited experience. This has often been framed in terms of a dichotomy between connectionist and symbolic cognitive models. Here, we highlight a recently emerging line of work that suggests a novel reconciliation of these approaches, by exploiting an inductive bias that we term the relational bottleneck. In that approach, neural networks are constrained via their architecture to focus on relations between perceptual inputs, rather than the attributes of individual inputs. We review a family of models that employ this approach to induce abstractions in a data-efficient manner, emphasizing their potential as candidate models for the acquisition of abstract concepts in the human mind and brain.
Collapse
|
9
|
Desbordes T, King JR, Dehaene S. Tracking the neural codes for words and phrases during semantic composition, working-memory storage, and retrieval. Cell Rep 2024; 43:113847. [PMID: 38412098 DOI: 10.1016/j.celrep.2024.113847] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 11/02/2023] [Accepted: 02/07/2024] [Indexed: 02/29/2024] Open
Abstract
The ability to compose successive words into a meaningful phrase is a characteristic feature of human cognition, yet its neural mechanisms remain incompletely understood. Here, we analyze the cortical mechanisms of semantic composition using magnetoencephalography (MEG) while participants read one-word, two-word, and five-word noun phrases and compared them with a subsequent image. Decoding of MEG signals revealed three processing stages. During phrase comprehension, the representation of individual words was sustained for a variable duration depending on phrasal context. During the delay period, the word code was replaced by a working-memory code whose activation increased with semantic complexity. Finally, the speed and accuracy of retrieval depended on semantic complexity and was faster for surface than for deep semantic properties. In conclusion, we propose that the brain initially encodes phrases using factorized dimensions for successive words but later compresses them in working memory and requires a period of decompression to access them.
Collapse
Affiliation(s)
- Théo Desbordes
- Meta AI, Paris, France; Cognitive Neuroimaging Unit, NeuroSpin Center, 91191 Gif-sur-Yvette, France.
| | - Jean-Rémi King
- Meta AI, Paris, France; École Normale Supérieure, PSL University, Paris, France
| | - Stanislas Dehaene
- Université Paris Saclay, INSERM, CEA, Cognitive Neuroimaging Unit, NeuroSpin Center, 91191 Gif-sur-Yvette, France; Collège de France, PSL University, Paris, France
| |
Collapse
|
10
|
Tafazoli S, Bouchacourt FM, Ardalan A, Markov NT, Uchimura M, Mattar MG, Daw ND, Buschman TJ. Building compositional tasks with shared neural subspaces. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.31.578263. [PMID: 38352540 PMCID: PMC10862921 DOI: 10.1101/2024.01.31.578263] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/22/2024]
Abstract
Cognition is remarkably flexible; we are able to rapidly learn and perform many different tasks1. Theoretical modeling has shown artificial neural networks trained to perform multiple tasks will re-use representations2 and computational components3 across tasks. By composing tasks from these sub-components, an agent can flexibly switch between tasks and rapidly learn new tasks4. Yet, whether such compositionality is found in the brain is unknown. Here, we show the same subspaces of neural activity represent task-relevant information across multiple tasks, with each task compositionally combining these subspaces in a task-specific manner. We trained monkeys to switch between three compositionally related tasks. Neural recordings found task-relevant information about stimulus features and motor actions were represented in subspaces of neural activity that were shared across tasks. When monkeys performed a task, neural representations in the relevant shared sensory subspace were transformed to the relevant shared motor subspace. Subspaces were flexibly engaged as monkeys discovered the task in effect; their internal belief about the current task predicted the strength of representations in task-relevant subspaces. In sum, our findings suggest that the brain can flexibly perform multiple tasks by compositionally combining task-relevant neural representations across tasks.
Collapse
Affiliation(s)
- Sina Tafazoli
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Adel Ardalan
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Nikola T. Markov
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Motoaki Uchimura
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Nathaniel D. Daw
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
- Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Timothy J. Buschman
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
- Department of Psychology, Princeton University, Princeton, NJ, USA
| |
Collapse
|
11
|
Trujillo JP, Holler J. Conversational facial signals combine into compositional meanings that change the interpretation of speaker intentions. Sci Rep 2024; 14:2286. [PMID: 38280963 PMCID: PMC10821935 DOI: 10.1038/s41598-024-52589-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 01/20/2024] [Indexed: 01/29/2024] Open
Abstract
Human language is extremely versatile, combining a limited set of signals in an unlimited number of ways. However, it is unknown whether conversational visual signals feed into the composite utterances with which speakers communicate their intentions. We assessed whether different combinations of visual signals lead to different intent interpretations of the same spoken utterance. Participants viewed a virtual avatar uttering spoken questions while producing single visual signals (i.e., head turn, head tilt, eyebrow raise) or combinations of these signals. After each video, participants classified the communicative intention behind the question. We found that composite utterances combining several visual signals conveyed different meaning compared to utterances accompanied by the single visual signals. However, responses to combinations of signals were more similar to the responses to related, rather than unrelated, individual signals, indicating a consistent influence of the individual visual signals on the whole. This study therefore provides first evidence for compositional, non-additive (i.e., Gestalt-like) perception of multimodal language.
Collapse
Affiliation(s)
- James P Trujillo
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands.
- Donders Institute for Brain, Cognition, and Behaviour, Nijmegen, The Netherlands.
| | - Judith Holler
- Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
- Donders Institute for Brain, Cognition, and Behaviour, Nijmegen, The Netherlands
| |
Collapse
|
12
|
Kazanina N, Poeppel D. The neural ingredients for a language of thought are available. Trends Cogn Sci 2023; 27:996-1007. [PMID: 37625973 DOI: 10.1016/j.tics.2023.07.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Revised: 07/25/2023] [Accepted: 07/28/2023] [Indexed: 08/27/2023]
Abstract
The classical notion of a 'language of thought' (LoT), advanced prominently by the philosopher Jerry Fodor, is an influential position in cognitive science whereby the mental representations underpinning thought are considered to be compositional and productive, enabling the construction of new complex thoughts from more primitive symbolic concepts. LoT theory has been challenged because a neural implementation has been deemed implausible. We disagree. Examples of critical computational ingredients needed for a neural implementation of a LoT have in fact been demonstrated, in particular in the hippocampal spatial navigation system of rodents. Here, we show that cell types found in spatial navigation (border cells, object cells, head-direction cells, etc.) provide key types of representation and computation required for the LoT, underscoring its neurobiological viability.
Collapse
Affiliation(s)
- Nina Kazanina
- University of Bristol, Bristol, UK; Ernst Strüngmann Institute for Neuroscience, Frankfurt, Germany
| | - David Poeppel
- Ernst Strüngmann Institute for Neuroscience, Frankfurt, Germany; New York University, New York, NY, USA.
| |
Collapse
|
13
|
Piantadosi ST. The algorithmic origins of counting. Child Dev 2023; 94:1472-1490. [PMID: 37984061 DOI: 10.1111/cdev.14031] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2022] [Revised: 09/16/2023] [Accepted: 09/19/2023] [Indexed: 11/22/2023]
Abstract
The study of how children learn numbers has yielded one of the most productive research programs in cognitive development, spanning empirical and computational methods, as well as nativist and empiricist philosophies. This paper provides a tutorial on how to think computationally about learning models in a domain like number, where learners take finite data and go far beyond what they directly observe or perceive. To illustrate, this paper then outlines a model which acquires a counting procedure using observations of sets and words, extending the proposal of Piantadosi et al. (2012). This new version of the model responds to several critiques of the original work and outlines an approach which is likely appropriate for acquiring further aspects of mathematics.
Collapse
|
14
|
Schwartenbeck P, Baram A, Liu Y, Mark S, Muller T, Dolan R, Botvinick M, Kurth-Nelson Z, Behrens T. Generative replay underlies compositional inference in the hippocampal-prefrontal circuit. Cell 2023; 186:4885-4897.e14. [PMID: 37804832 PMCID: PMC10914680 DOI: 10.1016/j.cell.2023.09.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 01/23/2023] [Accepted: 09/06/2023] [Indexed: 10/09/2023]
Abstract
Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.
Collapse
Affiliation(s)
- Philipp Schwartenbeck
- University of Tübingen, Tübingen, Germany; Max Planck Institute for Biological Cybernetics, Tübingen, Baden-Württemberg, Germany; Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK.
| | - Alon Baram
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Shirley Mark
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK
| | - Timothy Muller
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Institute of Neurology, University College London, London WC1N 3BG, UK
| | - Raymond Dolan
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Department of Psychiatry, Universitätsmedizin Berlin (Campus Charité Mitte), Berlin, Germany
| | - Matthew Botvinick
- Google DeepMind, London, UK; Gatsby Computational Neuroscience Unit, University College London, London, UK
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Google DeepMind, London, UK
| | - Timothy Behrens
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Sainsbury Wellcome Centre for Neural Circuits and Behaviour, UCL, London W1T 4JG, UK
| |
Collapse
|
15
|
Rabini G, Ubaldi S, Fairhall SL. Task-based activation and resting-state connectivity predict individual differences in semantic capacity for complex semantic knowledge. Commun Biol 2023; 6:1020. [PMID: 37813935 PMCID: PMC10562439 DOI: 10.1038/s42003-023-05400-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Accepted: 09/29/2023] [Indexed: 10/11/2023] Open
Abstract
Our ability to know and access complex factual information has far reaching effects, influencing our scholastic, professional and social lives. Here we employ functional MRI to assess the relationship between individual differences in semantic aptitude in the task-based activation and resting-state functional connectivity. Using psychometric and behavioural measures, we quantified the semantic and executive aptitude of individuals and had them perform a general-knowledge semantic-retrieval task (N = 41) and recorded resting-state data (N = 43). During the semantic-retrieval task, participants accessed general-knowledge facts drawn from four different knowledge-domains (people, places, objects and 'scholastic'). Individuals with greater executive capacity more strongly recruit anterior sections of prefrontal cortex (PFC) and the precuneus, and individuals with lower semantic capacity more strongly activate a posterior section of the dorsomedial PFC (dmPFC). The role of these regions in semantic processing was validated by analysis of independent resting-state data, where increased connectivity between a left anterior PFC and the precuneus predict higher semantic aptitude, and increased connectivity between left anterior PFC and posterior dmPFC predict lower semantic aptitude. Results suggest that coordination between core semantic regions in the precuneus and anterior prefrontal regions associated with executive processes support greater semantic aptitude.
Collapse
Affiliation(s)
- Giuseppe Rabini
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy.
| | - Silvia Ubaldi
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy
| | - Scott L Fairhall
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Trento, Italy
| |
Collapse
|
16
|
Carcassi F, Szymanik J. The Boolean Language of Thought is recoverable from learning data. Cognition 2023; 239:105541. [PMID: 37473608 DOI: 10.1016/j.cognition.2023.105541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Revised: 04/12/2023] [Accepted: 06/27/2023] [Indexed: 07/22/2023]
Abstract
According to the Language of Thought Hypothesis (LoTH), an influential account in philosophy and cognitive science, human cognition is underlain by symbolic reasoning in a formal language. In this account, concepts are expressions in a Language of Thought, deduction is syntactic manipulation in this language, and learning is an inference of expressions in this language from data. This picture raises the question of what LoT humans have, and how to infer it from behavior. In this paper, we pave the way towards answering this question, by approaching a more fundamental question: to what extent is it possible in principle to recover the human LoT from experimental data? To answer this question, we focus on the fragment of LoT that is concerned with representing Boolean categories and simulate the recovery of the Boolean LoT from category learning experiments. Our findings show that in principle the vast majority of Boolean LoTs can be accurately recovered from experimental data. However, we find that this crucially depends on the employed experimental design. Moreover, we find evidence that LoTs with fewer operators can be recovered from category learning data faster.
Collapse
Affiliation(s)
- Fausto Carcassi
- Department of Linguistics, University of Tübingen, Keplerstraße 2, 72074 Tübingen, Germany.
| | - Jakub Szymanik
- Center for Mind/Brain Sciences and Department of Information Engineering and Computer Science, Corso Bettini 31, 38068 Rovereto (TN), Italy.
| |
Collapse
|
17
|
Doerig A, Sommers RP, Seeliger K, Richards B, Ismael J, Lindsay GW, Kording KP, Konkle T, van Gerven MAJ, Kriegeskorte N, Kietzmann TC. The neuroconnectionist research programme. Nat Rev Neurosci 2023:10.1038/s41583-023-00705-w. [PMID: 37253949 DOI: 10.1038/s41583-023-00705-w] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/21/2023] [Indexed: 06/01/2023]
Abstract
Artificial neural networks (ANNs) inspired by biology are beginning to be widely used to model behavioural and neural data, an approach we call 'neuroconnectionism'. ANNs have been not only lauded as the current best models of information processing in the brain but also criticized for failing to account for basic cognitive functions. In this Perspective article, we propose that arguing about the successes and failures of a restricted set of current ANNs is the wrong approach to assess the promise of neuroconnectionism for brain science. Instead, we take inspiration from the philosophy of science, and in particular from Lakatos, who showed that the core of a scientific research programme is often not directly falsifiable but should be assessed by its capacity to generate novel insights. Following this view, we present neuroconnectionism as a general research programme centred around ANNs as a computational language for expressing falsifiable theories about brain computation. We describe the core of the programme, the underlying computational framework and its tools for testing specific neuroscientific hypotheses and deriving novel understanding. Taking a longitudinal view, we review past and present neuroconnectionist projects and their responses to challenges and argue that the research programme is highly progressive, generating new and otherwise unreachable insights into the workings of the brain.
Collapse
Affiliation(s)
- Adrien Doerig
- Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany.
- Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| | - Rowan P Sommers
- Department of Neurobiology of Language, Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands
| | - Katja Seeliger
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Blake Richards
- Department of Neurology and Neurosurgery, McGill University, Montréal, QC, Canada
- School of Computer Science, McGill University, Montréal, QC, Canada
- Mila, Montréal, QC, Canada
- Montréal Neurological Institute, Montréal, QC, Canada
- Learning in Machines and Brains Program, CIFAR, Toronto, ON, Canada
| | | | | | - Konrad P Kording
- Learning in Machines and Brains Program, CIFAR, Toronto, ON, Canada
- Bioengineering, Neuroscience, University of Pennsylvania, Pennsylvania, PA, USA
| | | | | | | | - Tim C Kietzmann
- Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany
| |
Collapse
|
18
|
Guan C, Aflalo T, Kadlec K, Gámez de Leon J, Rosario ER, Bari A, Pouratian N, Andersen RA. Decoding and geometry of ten finger movements in human posterior parietal cortex and motor cortex. J Neural Eng 2023; 20:036020. [PMID: 37160127 PMCID: PMC10209510 DOI: 10.1088/1741-2552/acd3b1] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 03/24/2023] [Accepted: 05/09/2023] [Indexed: 05/11/2023]
Abstract
Objective. Enable neural control of individual prosthetic fingers for participants with upper-limb paralysis.Approach. Two tetraplegic participants were each implanted with a 96-channel array in the left posterior parietal cortex (PPC). One of the participants was additionally implanted with a 96-channel array near the hand knob of the left motor cortex (MC). Across tens of sessions, we recorded neural activity while the participants attempted to move individual fingers of the right hand. Offline, we classified attempted finger movements from neural firing rates using linear discriminant analysis with cross-validation. The participants then used the neural classifier online to control individual fingers of a brain-machine interface (BMI). Finally, we characterized the neural representational geometry during individual finger movements of both hands.Main Results. The two participants achieved 86% and 92% online accuracy during BMI control of the contralateral fingers (chance = 17%). Offline, a linear decoder achieved ten-finger decoding accuracies of 70% and 66% using respective PPC recordings and 75% using MC recordings (chance = 10%). In MC and in one PPC array, a factorized code linked corresponding finger movements of the contralateral and ipsilateral hands.Significance. This is the first study to decode both contralateral and ipsilateral finger movements from PPC. Online BMI control of contralateral fingers exceeded that of previous finger BMIs. PPC and MC signals can be used to control individual prosthetic fingers, which may contribute to a hand restoration strategy for people with tetraplegia.
Collapse
Affiliation(s)
- Charles Guan
- California Institute of Technology, Pasadena, CA, United States of America
| | - Tyson Aflalo
- California Institute of Technology, Pasadena, CA, United States of America
- T&C Chen Brain-Machine Interface Center at Caltech, Pasadena, CA, United States of America
| | - Kelly Kadlec
- California Institute of Technology, Pasadena, CA, United States of America
| | | | - Emily R Rosario
- Casa Colina Hospital and Centers for Healthcare, Pomona, CA, United States of America
| | - Ausaf Bari
- David Geffen School of Medicine at UCLA, Los Angeles, CA, United States of America
| | - Nader Pouratian
- University of Texas Southwestern Medical Center, Dallas, TX, United States of America
| | - Richard A Andersen
- California Institute of Technology, Pasadena, CA, United States of America
- T&C Chen Brain-Machine Interface Center at Caltech, Pasadena, CA, United States of America
| |
Collapse
|
19
|
Vives ML, de Bruin D, van Baar JM, FeldmanHall O, Bhandari A. Uncertainty aversion predicts the neural expansion of semantic representations. Nat Hum Behav 2023; 7:765-775. [PMID: 36997668 DOI: 10.1038/s41562-023-01561-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 02/17/2023] [Indexed: 04/01/2023]
Abstract
Correctly identifying the meaning of a stimulus requires activating the appropriate semantic representation among many alternatives. One way to reduce this uncertainty is to differentiate semantic representations from each other, thereby expanding the semantic space. Here, in four experiments, we test this semantic-expansion hypothesis, finding that uncertainty-averse individuals exhibit increasingly differentiated and separated semantic representations. This effect is mirrored at the neural level, where uncertainty aversion predicts greater distances between activity patterns in the left inferior frontal gyrus when reading words, and enhanced sensitivity to the semantic ambiguity of these words in the ventromedial prefrontal cortex. Two direct tests of the behavioural consequences of semantic expansion further reveal that uncertainty-averse individuals exhibit reduced semantic interference and poorer generalization. Together, these findings show that the internal structure of our semantic representations acts as an organizing principle to make the world more identifiable.
Collapse
Affiliation(s)
- Marc-Lluís Vives
- Department of Cognitive, Linguistic, Psychological Sciences, Brown University, Providence, RI, USA.
- Department of Psychology, Leiden University, Leiden, The Netherlands.
| | - Daantje de Bruin
- Department of Cognitive, Linguistic, Psychological Sciences, Brown University, Providence, RI, USA
| | - Jeroen M van Baar
- Trimbos Institute, Netherlands Institute for Mental Health and Addiction, Utrecht, The Netherlands
| | - Oriel FeldmanHall
- Department of Cognitive, Linguistic, Psychological Sciences, Brown University, Providence, RI, USA.
- Robert J. and Nancy D. Carney Institute for Brain Science, Brown University, Providence, RI, USA.
| | - Apoorva Bhandari
- Department of Cognitive, Linguistic, Psychological Sciences, Brown University, Providence, RI, USA.
| |
Collapse
|
20
|
Varallyay A, Beller N, Subiaul F. Generative cultural learning in children and adults: the role of compositionality and generativity in cultural evolution. Proc Biol Sci 2023; 290:20222418. [PMID: 37122258 PMCID: PMC10130722 DOI: 10.1098/rspb.2022.2418] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Accepted: 03/22/2023] [Indexed: 05/02/2023] Open
Abstract
Are human cultures distinctively cumulative because they are uniquely compositional? We addressed this question using a summative learning paradigm where participants saw different models build different tower elements, consisting of discrete actions and objects: stacking cubes (tower base) and linking squares (tower apex). These elements could be combined to form a tower that was optimal in terms of height and structural soundness. In addition to measuring copying fidelity, we explored whether children and adults (i) extended the knowledge demonstrated to additional tower elements and (ii) productively combined them. Results showed that children and adults copied observed demonstrations and applied them to novel exemplars. However, only adults in the imitation condition combined the two newly derived base and apex, relative to adults in a control group. Nonetheless, there were remarkable similarities between children's and adults' performance across measures. Composite measures capturing errors and overall generativity in children's and adults' performance produced few population by condition interactions. Results suggest that early in development, humans possess a suite of cognitive skills-compositionality and generativity-that transforms phylogenetically widespread social learning competencies into something that may be unique to our species, cultural learning; allowing human cultures to evolve towards greater complexity.
Collapse
Affiliation(s)
- Adrian Varallyay
- The Institute for Social and Economic Research and Policy, Quantitative Methods in the Social Sciences, Columbia University, New York, NY, USA
| | - Nathalia Beller
- Department of Psychological and Brain Sciences, The George Washington University, Washington, DC, USA
| | - Francys Subiaul
- Department of Speech, Language, and Hearing Sciences, The George Washington University, Washington, DC, USA
- Department of Anthropology, Center for the Advanced Study of Human Paleobiology, The George Washington University, Washington, DC, USA
- Mind-Brain Institute, The George Washington University, Washington, DC, USA
| |
Collapse
|
21
|
Flesch T, Saxe A, Summerfield C. Continual task learning in natural and artificial agents. Trends Neurosci 2023; 46:199-210. [PMID: 36682991 PMCID: PMC10914671 DOI: 10.1016/j.tins.2022.12.006] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Revised: 12/07/2022] [Accepted: 12/15/2022] [Indexed: 01/21/2023]
Abstract
How do humans and other animals learn new tasks? A wave of brain recording studies has investigated how neural representations change during task learning, with a focus on how tasks can be acquired and coded in ways that minimise mutual interference. We review recent work that has explored the geometry and dimensionality of neural task representations in neocortex, and computational models that have exploited these findings to understand how the brain may partition knowledge between tasks. We discuss how ideas from machine learning, including those that combine supervised and unsupervised learning, are helping neuroscientists understand how natural tasks are learned and coded in biological brains.
Collapse
Affiliation(s)
- Timo Flesch
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Andrew Saxe
- Gatsby Computational Neuroscience Unit & Sainsbury Wellcome Centre, UCL, London, UK.
| | | |
Collapse
|
22
|
Kurth-Nelson Z, Behrens T, Wayne G, Miller K, Luettgau L, Dolan R, Liu Y, Schwartenbeck P. Replay and compositional computation. Neuron 2023; 111:454-469. [PMID: 36640765 DOI: 10.1016/j.neuron.2022.12.028] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/11/2022] [Accepted: 12/18/2022] [Indexed: 01/15/2023]
Abstract
Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form of compositional computation where entities are assembled into relationally bound structures to derive qualitatively new knowledge. This idea builds on recent advances in neuroscience, which indicate that the hippocampus flexibly binds objects to generalizable roles and that replay strings these role-bound objects into compound statements. We suggest experiments to test our hypothesis, and we end by noting the implications for AI systems which lack the human ability to radically generalize past experience to solve new problems.
Collapse
Affiliation(s)
- Zeb Kurth-Nelson
- DeepMind, London, UK; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK.
| | - Timothy Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | | | - Kevin Miller
- DeepMind, London, UK; Institute of Ophthalmology, University College London, London, UK
| | - Lennart Luettgau
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Ray Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK; Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Philipp Schwartenbeck
- Max Planck Institute for Biological Cybernetics, Tubingen, Germany; University of Tubingen, Tubingen, Germany
| |
Collapse
|
23
|
Abstract
Research on concepts has focused on categorization. Categorization starts with a stimulus. Equally important are episodes that start with a thought. We engage in thinking to draw out new consequences from stored information, or to work out how to act. Each of the concepts out of which thought is constructed provides access to a large body of stored information. Access is not always just a matter of retrieving a stored belief (semantic memory). Often it depends on running a simulation. Simulation allows conceptual thought to draw on information in special-purpose systems, information stored in special-purpose computational dispositions and special-purpose representational structures. While the utility of simulation, prospection or imagination is widely appreciated, the role of concepts in the process is not well understood. This paper turns to cognitive and computational neuroscience for a model of how simulations enable thinkers to reach novel conclusions. Carried over to conceptual thought, the model suggests that concepts are 'plug & play' devices. The distinctive power of thought-driven simulation derives from the ability of concepts to plug into two kinds of structure at once: the combinatorial structure of a thought at one end and special-purpose structural representations at the other. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.
Collapse
Affiliation(s)
- Nicholas Shea
- Faculty of Philosophy, University of Oxford, Radcliffe Humanities, Woodstock Road, Oxford OX2 6GG, UK,Institute of Philosophy, University of London School of Advanced Study, Senate House, Malet Street, London WC1E 7HU, UK
| |
Collapse
|
24
|
McNamee DC, Stachenfeld KL, Botvinick MM, Gershman SJ. Compositional Sequence Generation in the Entorhinal-Hippocampal System. ENTROPY (BASEL, SWITZERLAND) 2022; 24:1791. [PMID: 36554196 PMCID: PMC9778317 DOI: 10.3390/e24121791] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Revised: 11/01/2022] [Accepted: 11/29/2022] [Indexed: 06/17/2023]
Abstract
Neurons in the medial entorhinal cortex exhibit multiple, periodically organized, firing fields which collectively appear to form an internal representation of space. Neuroimaging data suggest that this grid coding is also present in other cortical areas such as the prefrontal cortex, indicating that it may be a general principle of neural functionality in the brain. In a recent analysis through the lens of dynamical systems theory, we showed how grid coding can lead to the generation of a diversity of empirically observed sequential reactivations of hippocampal place cells corresponding to traversals of cognitive maps. Here, we extend this sequence generation model by describing how the synthesis of multiple dynamical systems can support compositional cognitive computations. To empirically validate the model, we simulate two experiments demonstrating compositionality in space or in time during sequence generation. Finally, we describe several neural network architectures supporting various types of compositionality based on grid coding and highlight connections to recent work in machine learning leveraging analogous techniques.
Collapse
Affiliation(s)
- Daniel C. McNamee
- Neuroscience Programme, Champalimaud Research, 1400-038 Lisbon, Portugal
| | | | - Matthew M. Botvinick
- Google DeepMind, London N1C 4DN, UK
- Gatsby Computational Neuroscience Unit, University College London, London W1T 4JG, UK
| | - Samuel J. Gershman
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
- Center for Brains, Minds and Machines, MIT, Cambridge, MA 02139, USA
| |
Collapse
|
25
|
Quilty-Dunn J, Porot N, Mandelbaum E. The best game in town: The reemergence of the language-of-thought hypothesis across the cognitive sciences. Behav Brain Sci 2022; 46:e261. [PMID: 36471543 DOI: 10.1017/s0140525x22002849] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Mental representations remain the central posits of psychology after many decades of scrutiny. However, there is no consensus about the representational format(s) of biological cognition. This paper provides a survey of evidence from computational cognitive psychology, perceptual psychology, developmental psychology, comparative psychology, and social psychology, and concludes that one type of format that routinely crops up is the language-of-thought (LoT). We outline six core properties of LoTs: (i) discrete constituents; (ii) role-filler independence; (iii) predicate-argument structure; (iv) logical operators; (v) inferential promiscuity; and (vi) abstract content. These properties cluster together throughout cognitive science. Bayesian computational modeling, compositional features of object perception, complex infant and animal reasoning, and automatic, intuitive cognition in adults all implicate LoT-like structures. Instead of regarding LoT as a relic of the previous century, researchers in cognitive science and philosophy-of-mind must take seriously the explanatory breadth of LoT-based architectures. We grant that the mind may harbor many formats and architectures, including iconic and associative structures as well as deep-neural-network-like architectures. However, as computational/representational approaches to the mind continue to advance, classical compositional symbolic structures - that is, LoTs - only prove more flexible and well-supported over time.
Collapse
Affiliation(s)
- Jake Quilty-Dunn
- Department of Philosophy and Philosophy-Neuroscience-Psychology Program, Washington University in St. Louis, St. Louis, MO, USA. , sites.google.com/site/jakequiltydunn/
| | - Nicolas Porot
- Africa Institute for Research in Economics and Social Sciences, Mohammed VI Polytechnic University, Rabat, Morocco. , nicolasporot.com
| | - Eric Mandelbaum
- Departments of Philosophy and Psychology, The Graduate Center & Baruch College, CUNY, New York, NY, USA. , ericmandelbaum.com
| |
Collapse
|
26
|
Sablé-Meyer M, Ellis K, Tenenbaum J, Dehaene S. A language of thought for the mental representation of geometric shapes. Cogn Psychol 2022; 139:101527. [PMID: 36403385 DOI: 10.1016/j.cogpsych.2022.101527] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Revised: 10/26/2022] [Accepted: 10/31/2022] [Indexed: 11/18/2022]
Abstract
In various cultures and at all spatial scales, humans produce a rich complexity of geometric shapes such as lines, circles or spirals. Here, we propose that humans possess a language of thought for geometric shapes that can produce line drawings as recursive combinations of a minimal set of geometric primitives. We present a programming language, similar to Logo, that combines discrete numbers and continuous integration to form higher-level structures based on repetition, concatenation and embedding, and we show that the simplest programs in this language generate the fundamental geometric shapes observed in human cultures. On the perceptual side, we propose that shape perception in humans involves searching for the shortest program that correctly draws the image (program induction). A consequence of this framework is that the mental difficulty of remembering a shape should depend on its minimum description length (MDL) in the proposed language. In two experiments, we show that encoding and processing of geometric shapes is well predicted by MDL. Furthermore, our hypotheses predict additive laws for the psychological complexity of repeated, concatenated or embedded shapes, which we confirm experimentally.
Collapse
Affiliation(s)
- Mathias Sablé-Meyer
- Unicog, CEA, INSERM, Université Paris-Saclay, NeuroSpin Center, 91191 Gif/Yvette, France; Collège de France, Université Paris-Sciences-Lettres (PSL), 75005 Paris, France.
| | - Kevin Ellis
- Cornell University, Ithaca, NY, United States
| | - Josh Tenenbaum
- Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Stanislas Dehaene
- Unicog, CEA, INSERM, Université Paris-Saclay, NeuroSpin Center, 91191 Gif/Yvette, France; Collège de France, Université Paris-Sciences-Lettres (PSL), 75005 Paris, France
| |
Collapse
|
27
|
Mandelbaum E, Dunham Y, Feiman R, Firestone C, Green EJ, Harris D, Kibbe MM, Kurdi B, Mylopoulos M, Shepherd J, Wellwood A, Porot N, Quilty-Dunn J. Problems and Mysteries of the Many Languages of Thought. Cogn Sci 2022; 46:e13225. [PMID: 36537721 DOI: 10.1111/cogs.13225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Revised: 11/15/2022] [Accepted: 11/21/2022] [Indexed: 12/24/2022]
Abstract
"What is the structure of thought?" is as central a question as any in cognitive science. A classic answer to this question has appealed to a Language of Thought (LoT). We point to emerging research from disparate branches of the field that supports the LoT hypothesis, but also uncovers diversity in LoTs across cognitive systems, stages of development, and species. Our letter formulates open research questions for cognitive science concerning the varieties of rules and representations that underwrite various LoT-based systems and how these variations can help researchers taxonomize cognitive systems.
Collapse
Affiliation(s)
- Eric Mandelbaum
- Department of Philosophy, Baruch College.,Departments of Philosophy & Psychology, CUNY Graduate Center
| | | | - Roman Feiman
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University
| | - Chaz Firestone
- Department of Psychological and Brain Sciences, Johns Hopkins University
| | - E J Green
- Department of Linguistics and Philosophy, Massachusetts Institute of Technology
| | - Daniel Harris
- Department of Philosophy, Hunter College & CUNY Graduate Center
| | - Melissa M Kibbe
- Department of Psychological & Brain Sciences, Boston University
| | | | - Myrto Mylopoulos
- Departments of Philosophy and Cognitive Science, Carleton University
| | - Joshua Shepherd
- Department of Philosophy, Carleton College.,Department of Philosophy, University of Barcelona
| | | | - Nicolas Porot
- Africa Institute for Research in Economics and Social Sciences, Mohammed VI Polytechnic University
| | - Jake Quilty-Dunn
- Department of Philosophy & Philosophy-Neuroscience-Psychology, Washington University in St Louis
| |
Collapse
|
28
|
Arseniev-Koehler A, Foster JG. Machine Learning as a Model for Cultural Learning: Teaching an Algorithm What it Means to be Fat. SOCIOLOGICAL METHODS & RESEARCH 2022; 51:1484-1539. [PMID: 37974911 PMCID: PMC10653277 DOI: 10.1177/00491241221122603] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/19/2023]
Abstract
Public culture is a powerful source of cognitive socialization; for example, media language is full of meanings about body weight. Yet it remains unclear how individuals process meanings in public culture. We suggest that schema learning is a core mechanism by which public culture becomes personal culture. We propose that a burgeoning approach in computational text analysis - neural word embeddings - can be interpreted as a formal model for cultural learning. Embeddings allow us to empirically model schema learning and activation from natural language data. We illustrate our approach by extracting four lower-order schemas from news articles: the gender, moral, health, and class meanings of body weight. Using these lower-order schemas we quantify how words about body weight "fill in the blanks" about gender, morality, health, and class. Our findings reinforce ongoing concerns that machine-learning models (e.g., of natural language) can encode and reproduce harmful human biases.
Collapse
Affiliation(s)
- Alina Arseniev-Koehler
- Department of Sociology, Purdue University, West Lafayette, IN, USA
- Division of Biomedical Informatics, University of California, San Diego, La Jolla, CA, USA
| | - Jacob G. Foster
- Department of Sociology, University of California, Los Angeles, Los Angeles, CA, USA
| |
Collapse
|
29
|
Liu RG, Frank MJ. Hierarchical clustering optimizes the tradeoff between compositionality and expressivity of task structures for flexible reinforcement learning. ARTIF INTELL 2022; 312:103770. [PMID: 36711165 PMCID: PMC9880792 DOI: 10.1016/j.artint.2022.103770] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
A hallmark of human intelligence, but challenging for reinforcement learning (RL) agents, is the ability to compositionally generalise, that is, to recompose familiar knowledge components in novel ways to solve new problems. For instance, when navigating in a city, one needs to know the location of the destination and how to operate a vehicle to get there, whether it be pedalling a bike or operating a car. In RL, these correspond to the reward function and transition function, respectively. To compositionally generalize, these two components need to be transferable independently of each other: multiple modes of transport can reach the same goal, and any given mode can be used to reach multiple destinations. Yet there are also instances where it can be helpful to learn and transfer entire structures, jointly representing goals and transitions, particularly whenever these recur in natural tasks (e.g., given a suggestion to get ice cream, one might prefer to bike, even in new towns). Prior theoretical work has explored how, in model-based RL, agents can learn and generalize task components (transition and reward functions). But a satisfactory account for how a single agent can simultaneously satisfy the two competing demands is still lacking. Here, we propose a hierarchical RL agent that learns and transfers individual task components as well as entire structures (particular compositions of components) by inferring both through a non-parametric Bayesian model of the task. It maintains a factorised representation of task components through a hierarchical Dirichlet process, but it also represents different possible covariances between these components through a standard Dirichlet process. We validate our approach on a variety of navigation tasks covering a wide range of statistical correlations between task components and show that it can also improve generalisation and transfer in more complex, hierarchical tasks with goal/subgoal structures. Finally, we end with a discussion of our work including how this clustering algorithm could conceivably be implemented by cortico-striatal gating circuits in the brain.
Collapse
Affiliation(s)
- Rex G. Liu
- Carney Institute for Brain Science, Department of Cognitive, Linguistic, & Psychological Sciences, Brown University, Providence, RI 02912, United States of America
| | - Michael J. Frank
- Carney Institute for Brain Science, Department of Cognitive, Linguistic, & Psychological Sciences, Brown University, Providence, RI 02912, United States of America
| |
Collapse
|
30
|
Klostermann F, Wyrobnik M, Boll M, Ehlen F, Tiedt HO. Tracing embodied word production in persons with Parkinson's disease in distinct motor conditions. Sci Rep 2022; 12:16669. [PMID: 36198900 PMCID: PMC9534912 DOI: 10.1038/s41598-022-21106-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2021] [Accepted: 09/22/2022] [Indexed: 11/09/2022] Open
Abstract
Embodied cognition theories posit direct interactions between sensorimotor and mental processing. Various clinical observations have been interpreted in this controversial framework, amongst others, low verb generation in word production tasks performed by persons with Parkinson's disease (PD). If this were the consequence of reduced motor simulation of prevalent action semantics in this word class, reduced PD pathophysiology should result in increased verb production and a general shift of lexical contents towards particular movement-related meanings. 17 persons with PD and bilateral deep brain stimulation (DBS) of the subhtalamic nucleus (STN) and 17 healthy control persons engaged in a semantically unconstrained, phonemic verbal fluency task, the former in both DBS-off and DBS-on states. The analysis referred to the number of words produced, verb use, and the occurrence of different dimensions of movement-related semantics in the lexical output. Persons with PD produced fewer words than controls. In the DBS-off, but not in the DBS-on condition, the proportion of verbs within this reduced output was lower than in controls. Lowered verb production went in parallel with a semantic shift: in persons with PD in the DBS-off, but not the DBS-on condition, the relatedness of produced words to own body-movement was lower than in controls. In persons with PD, DBS induced-changes of the motor condition appear to go along with formal and semantic shifts in word production. The results are compatible with the idea of some impact of motor system states on lexical processing.
Collapse
Affiliation(s)
- Fabian Klostermann
- Department of Neurology, Motor and Cognition Group, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Hindenburgdamm 30, 12200, Berlin, Germany. .,Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Unter den Linden 6, 10099, Berlin, Germany.
| | - Michelle Wyrobnik
- Department of Neurology, Motor and Cognition Group, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Hindenburgdamm 30, 12200, Berlin, Germany.,Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Unter den Linden 6, 10099, Berlin, Germany.,Institute of Psychology, Humboldt-Universität zu Berlin, Rudower Chaussee 18, 12489, Berlin, Germany
| | - Moritz Boll
- Department of Neurology, Motor and Cognition Group, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Hindenburgdamm 30, 12200, Berlin, Germany
| | - Felicitas Ehlen
- Department of Neurology, Motor and Cognition Group, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Hindenburgdamm 30, 12200, Berlin, Germany.,Department of Psychiatry, Jüdisches Krankenhaus Berlin, Heinz-Galinski-Straße 1, 13347, Berlin, Germany
| | - Hannes Ole Tiedt
- Department of Neurology, Motor and Cognition Group, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Hindenburgdamm 30, 12200, Berlin, Germany
| |
Collapse
|
31
|
Bacelar Valente M. Do All Languages Share the Same Conceptual Structure? COGNITIVE SEMANTICS 2022; 8:159-180. [DOI: 10.1163/23526416-08020001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
Abstract
Abstract
In this work, we consider the views of three exponents of major areas of linguistics – Levelt (psycholinguistics), Jackendoff (theoretical linguistics), and Gil (field linguistics) – regarding the issue of the universality or not of the conceptual structure of languages. In Levelt’s view, during language production, the conceptual structure of the preverbal message is language-specific. In Jackendoff’s theoretical approach to language – his parallel architecture – there is a universal conceptual structure shared by all languages, in contradiction to Levelt’s view. In Gil’s work on Riau Indonesian, he proposes a conceptual structure that is quite different from that of English, adopted by Jackendoff as universal. We find no reason to disagree with Gil’s view. In this way, we take Gil’s work as vindicating Levelt’s view that during language production preverbal messages are encoded with different conceptual structures for different languages.
Collapse
Affiliation(s)
- Mario Bacelar Valente
- Department of Physical, Chemical and Natural Systems, Pablo de Olavide University, Seville, Spain,
| |
Collapse
|
32
|
Acunzo DJ, Low DM, Fairhall SL. Deep neural networks reveal topic-level representations of sentences in medial prefrontal cortex, lateral anterior temporal lobe, precuneus, and angular gyrus. Neuroimage 2022; 251:119005. [PMID: 35176493 PMCID: PMC10184870 DOI: 10.1016/j.neuroimage.2022.119005] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Revised: 02/11/2022] [Accepted: 02/13/2022] [Indexed: 11/17/2022] Open
Abstract
When reading a sentence, individual words can be combined to create more complex meaning. In this study, we sought to uncover brain regions that reflect the representation of the meaning of sentences at the topic level, as opposed to the meaning of their individual constituent words when considered irrespective of their context. Using fMRI, we recorded the neural activity of participants while reading sentences. We constructed a topic-level sentence representations using the final layer of a convolutional neural network (CNN) trained to classify Wikipedia sentences into broad semantic categories. This model was contrasted with word-level sentence representations constructed using the average of the word embeddings constituting the sentence. Using representational similarity analysis, we found that the medial prefrontal cortex, lateral anterior temporal lobe, precuneus, and angular gyrus more strongly represent sentence topic-level, compared to word-level, meaning, uncovering the important role of these semantic system regions in the representation of topic-level meaning. Results were comparable when sentence meaning was modelled with a multilayer perceptron that was not sensitive to word order within a sentence, suggesting that the learning objective, in the terms of the topic being modelled, is the critical factor in capturing these neural representational spaces.
Collapse
Affiliation(s)
- David J Acunzo
- CIMeC/University of Trento, Corso Bettini 31, Rovereto 38068, Italy
| | - Daniel M Low
- Program in Speech and Hearing Bioscience and Technology, Harvard Medical School, United States; Brain and Cognitive Sciences Department, MIT, United States
| | - Scott L Fairhall
- CIMeC/University of Trento, Corso Bettini 31, Rovereto 38068, Italy.
| |
Collapse
|
33
|
O’Reilly RC, Ranganath C, Russin JL. The Structure of Systematicity in the Brain. CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE 2022; 31:124-130. [PMID: 35785023 PMCID: PMC9246245 DOI: 10.1177/09637214211049233] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
A hallmark of human intelligence is the ability to adapt to new situations, by applying learned rules to new content (systematicity) and thereby enabling an open-ended number of inferences and actions (generativity). Here, we propose that the human brain accomplishes these feats through pathways in the parietal cortex that encode the abstract structure of space, events, and tasks, and pathways in the temporal cortex that encode information about specific people, places, and things (content). Recent neural network models show how the separation of structure and content might emerge through a combination of architectural biases and learning, and these networks show dramatic improvements in the ability to capture systematic, generative behavior. We close by considering how the hippocampal formation may form integrative memories that enable rapid learning of new structure and content representations.
Collapse
Affiliation(s)
| | - Charan Ranganath
- Department of Psychology
- Center for Neuroscience, University of California, Davis
| | - Jacob L. Russin
- Department of Psychology
- Center for Neuroscience, University of California, Davis
| |
Collapse
|
34
|
He T, Boudewyn MA, Kiat JE, Sagae K, Luck SJ. Neural correlates of word representation vectors in natural language processing models: Evidence from representational similarity analysis of event-related brain potentials. Psychophysiology 2022; 59:e13976. [PMID: 34817867 PMCID: PMC8810574 DOI: 10.1111/psyp.13976] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2021] [Revised: 07/23/2021] [Accepted: 11/03/2021] [Indexed: 11/30/2022]
Abstract
Natural language processing models based on machine learning (ML-NLP models) have been developed to solve practical problems, such as interpreting an Internet search query. These models are not intended to reflect human language comprehension mechanisms, and the word representations used by ML-NLP models and human brains might therefore be quite different. However, because ML-NLP models are trained with the same kinds of inputs that humans must process, and they must solve many of the same computational problems as the human brain, ML-NLP models and human brains may end up with similar word representations. To distinguish between these hypotheses, we used representational similarity analysis to compare the representational geometry of word representations in two ML-NLP models with the representational geometry of the human brain, as indexed with event-related potentials (ERPs). Participants listened to stories while the electroencephalogram was recorded. We extracted averaged ERPs for each of the 100 words that occurred most frequently in the stories, and we calculated the similarity of the neural response for each pair of words. We compared this 100 × 100 similarity matrix to the 100 × 100 similarity matrix for the word pairs according to two ML-NLP models. We found significant representational similarity between the neural data and each ML-NLP model, beginning within 250 ms of word onset. These results indicate that ML-NLP systems that are designed to solve practical technology problems have a representational geometry that is correlated with that of the human brain, presumably because both are influenced by the structural properties and statistics of language.
Collapse
Affiliation(s)
- Taiqi He
- Center for Neuroscience, University of California, Davis, USA,Department of Linguistics, University of California, Davis, USA
| | | | - John E. Kiat
- Center for Mind & Brain, University of California, Davis, USA
| | - Kenji Sagae
- Department of Linguistics, University of California, Davis, USA
| | - Steven J. Luck
- Center for Mind & Brain, University of California, Davis, USA,Department of Psychology, University of California, Davis, USA,Corresponding Author:
| |
Collapse
|
35
|
From words to phrases: neural basis of social event semantic composition. Brain Struct Funct 2022; 227:1683-1695. [PMID: 35184222 PMCID: PMC9098591 DOI: 10.1007/s00429-022-02465-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 01/25/2022] [Indexed: 11/13/2022]
Abstract
Events are typically composed of at least actions and entities. Both actions and entities have been shown to be represented by neural structures respecting domain organizations in the brain, including those of social/animate (face and body; person-directed action) versus inanimate (man-made object or tool; object-directed action) concepts. It is unclear whether the brain combines actions and entities into events in a (relative) domain-specific fashion or via domain-general mechanisms in regions that have been shown to support semantic and syntactic composition. We tested these hypotheses in a functional magnetic resonance imaging experiment where two domains of verb-noun event phrases (social-person versus manipulation-artifact, e.g., “hug mother” versus “fold napkin”) and their component words were contrasted. We found a series of brain region supporting social-composition effects more strongly than the manipulation phrase composition—the bilateral inferior occipital gyrus (IOG), inferior temporal gyrus (ITG) and anterior temporal lobe (ATL)—which either showed stronger activation strength tested by univariate contrast, stronger content representation tested by representation similarity analysis, or stronger relationship between the neural activation patterns of phrases and synthesis (additive and multiplication) of the neural activity patterns of the word constituents. No regions were observed showing evidence of phrase composition for both domains or stronger effects of manipulation phrases. These findings highlight the roles of the visual cortex and ATL in social event compositions, suggesting a domain-preferring, rather than domain-general, mechanisms of verbal event composition.
Collapse
|
36
|
|
37
|
Lumaca M, Vuust P, Baggio G. Network Analysis of Human Brain Connectivity Reveals Neural Fingerprints of a Compositionality Bias in Signaling Systems. Cereb Cortex 2021; 32:1704-1720. [PMID: 34476458 DOI: 10.1093/cercor/bhab307] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Revised: 08/04/2021] [Accepted: 08/05/2021] [Indexed: 12/16/2022] Open
Abstract
Compositionality is a hallmark of human language and other symbolic systems: a finite set of meaningful elements can be systematically combined to convey an open-ended array of ideas. Compositionality is not uniformly distributed over expressions in a language or over individuals' communicative behavior: at both levels, variation is observed. Here, we investigate the neural bases of interindividual variability by probing the relationship between intrinsic characteristics of brain networks and compositional behavior. We first collected functional resting-state and diffusion magnetic resonance imaging data from a large participant sample (N = 51). Subsequently, participants took part in two signaling games. They were instructed to learn and reproduce an auditory symbolic system of signals (tone sequences) associated with affective meanings (human faces expressing emotions). Signal-meaning mappings were artificial and had to be learned via repeated signaling interactions. We identified a temporoparietal network in which connection length was related to the degree of compositionality introduced in a signaling system by each player. Graph-theoretic analysis of resting-state functional connectivity revealed that, within that network, compositional behavior was associated with integration measures in 2 semantic hubs: the left posterior cingulate cortex and the left angular gyrus. Our findings link individual variability in compositional biases to variation in the anatomy of semantic networks and in the functional topology of their constituent units.
Collapse
Affiliation(s)
- Massimo Lumaca
- Center for Music in the Brain, Department of Clinical Medicine, Aarhus University & The Royal Academy of Music, 8000 Aarhus C, Denmark
| | - Peter Vuust
- Center for Music in the Brain, Department of Clinical Medicine, Aarhus University & The Royal Academy of Music, 8000 Aarhus C, Denmark
| | - Giosuè Baggio
- Language Acquisition and Language Processing Lab, Department of Language and Literature, Norwegian University of Science and Technology, 7941 Trondheim, Norway
| |
Collapse
|
38
|
Musslick S, Cohen JD. Rationalizing constraints on the capacity for cognitive control. Trends Cogn Sci 2021; 25:757-775. [PMID: 34332856 DOI: 10.1016/j.tics.2021.06.001] [Citation(s) in RCA: 48] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2020] [Revised: 06/02/2021] [Accepted: 06/02/2021] [Indexed: 12/16/2022]
Abstract
Humans are remarkably limited in: (i) how many control-dependent tasks they can execute simultaneously, and (ii) how intensely they can focus on a single task. These limitations are universal assumptions of most theories of cognition. Yet, a rationale for why humans are subject to these constraints remains elusive. This feature review draws on recent insights from psychology, neuroscience, and machine learning, to suggest that constraints on cognitive control may result from a rational adaptation to fundamental, computational dilemmas in neural architectures. The reviewed literature implies that limitations in multitasking may result from a trade-off between learning efficacy and processing efficiency and that limitations in the intensity of commitment to a single task may reflect a trade-off between cognitive stability and flexibility.
Collapse
Affiliation(s)
- Sebastian Musslick
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA.
| | - Jonathan D Cohen
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Department of Psychology, Princeton University, Princeton, NJ 08544, USA
| |
Collapse
|
39
|
Röder F, Özdemir O, Nguyen PDH, Wermter S, Eppe M. The Embodied Crossmodal Self Forms Language and Interaction: A Computational Cognitive Review. Front Psychol 2021; 12:716671. [PMID: 34484079 PMCID: PMC8415221 DOI: 10.3389/fpsyg.2021.716671] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2021] [Accepted: 07/16/2021] [Indexed: 11/13/2022] Open
Abstract
Human language is inherently embodied and grounded in sensorimotor representations of the self and the world around it. This suggests that the body schema and ideomotor action-effect associations play an important role in language understanding, language generation, and verbal/physical interaction with others. There are computational models that focus purely on non-verbal interaction between humans and robots, and there are computational models for dialog systems that focus only on verbal interaction. However, there is a lack of research that integrates these approaches. We hypothesize that the development of computational models of the self is very appropriate for considering joint verbal and physical interaction. Therefore, they provide the substantial potential to foster the psychological and cognitive understanding of language grounding, and they have significant potential to improve human-robot interaction methods and applications. This review is a first step toward developing models of the self that integrate verbal and non-verbal communication. To this end, we first analyze the relevant findings and mechanisms for language grounding in the psychological and cognitive literature on ideomotor theory. Second, we identify the existing computational methods that implement physical decision-making and verbal interaction. As a result, we outline how the current computational methods can be used to create advanced computational interaction models that integrate language grounding with body schemas and self-representations.
Collapse
Affiliation(s)
- Frank Röder
- Knowledge Technology, Department of Informatics, University of Hamburg, Hamburg, Germany
| | | | | | | | | |
Collapse
|
40
|
Barnett AJ, Reilly W, Dimsdale-Zucker HR, Mizrak E, Reagh Z, Ranganath C. Intrinsic connectivity reveals functionally distinct cortico-hippocampal networks in the human brain. PLoS Biol 2021; 19:e3001275. [PMID: 34077415 PMCID: PMC8202937 DOI: 10.1371/journal.pbio.3001275] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2020] [Revised: 06/14/2021] [Accepted: 05/07/2021] [Indexed: 12/13/2022] Open
Abstract
Episodic memory depends on interactions between the hippocampus and interconnected neocortical regions. Here, using data-driven analyses of resting-state functional magnetic resonance imaging (fMRI) data, we identified the networks that interact with the hippocampus-the default mode network (DMN) and a "medial temporal network" (MTN) that included regions in the medial temporal lobe (MTL) and precuneus. We observed that the MTN plays a critical role in connecting the visual network to the DMN and hippocampus. The DMN could be further divided into 3 subnetworks: a "posterior medial" (PM) subnetwork comprised of posterior cingulate and lateral parietal cortices; an "anterior temporal" (AT) subnetwork comprised of regions in the temporopolar and dorsomedial prefrontal cortex; and a "medial prefrontal" (MP) subnetwork comprised of regions primarily in the medial prefrontal cortex (mPFC). These networks vary in their functional connectivity (FC) along the hippocampal long axis and represent different kinds of information during memory-guided decision-making. Finally, a Neurosynth meta-analysis of fMRI studies suggests new hypotheses regarding the functions of the MTN and DMN subnetworks, providing a framework to guide future research on the neural architecture of episodic memory.
Collapse
Affiliation(s)
- Alexander J. Barnett
- Center for Neuroscience, University of California at Davis, Davis, California, United States of America
| | - Walter Reilly
- Center for Neuroscience, University of California at Davis, Davis, California, United States of America
| | | | - Eda Mizrak
- Center for Neuroscience, University of California at Davis, Davis, California, United States of America
- Department of Psychology, University of Zurich, Zürich, Switzerland
| | - Zachariah Reagh
- Center for Neuroscience, University of California at Davis, Davis, California, United States of America
- Department of Neurology, University of California at Davis, Sacramento, California, United States of America
- Department of Psychological and Brain Sciences, Washington University in St. Louis, St. Louis, Missouri, United States of America
| | - Charan Ranganath
- Center for Neuroscience, University of California at Davis, Davis, California, United States of America
| |
Collapse
|
41
|
Hafri A, Firestone C. The Perception of Relations. Trends Cogn Sci 2021; 25:475-492. [PMID: 33812770 DOI: 10.1016/j.tics.2021.01.006] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2020] [Revised: 01/05/2021] [Accepted: 01/18/2021] [Indexed: 11/16/2022]
Abstract
The world contains not only objects and features (red apples, glass bowls, wooden tables), but also relations holding between them (apples contained in bowls, bowls supported by tables). Representations of these relations are often developmentally precocious and linguistically privileged; but how does the mind extract them in the first place? Although relations themselves cast no light onto our eyes, a growing body of work suggests that even very sophisticated relations display key signatures of automatic visual processing. Across physical, eventive, and social domains, relations such as support, fit, cause, chase, and even socially interact are extracted rapidly, are impossible to ignore, and influence other perceptual processes. Sophisticated and structured relations are not only judged and understood, but also seen - revealing surprisingly rich content in visual perception itself.
Collapse
Affiliation(s)
- Alon Hafri
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD 21218, USA; Department of Cognitive Science, Johns Hopkins University, Baltimore, MD 21218, USA.
| | - Chaz Firestone
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD 21218, USA; Department of Cognitive Science, Johns Hopkins University, Baltimore, MD 21218, USA; Department of Philosophy, Johns Hopkins University, Baltimore, MD 21218, USA.
| |
Collapse
|
42
|
Szollosi A, Newell BR. People as Intuitive Scientists: Reconsidering Statistical Explanations of Decision Making. Trends Cogn Sci 2020; 24:1008-1018. [PMID: 33077380 DOI: 10.1016/j.tics.2020.09.005] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2020] [Revised: 09/09/2020] [Accepted: 09/10/2020] [Indexed: 10/23/2022]
Abstract
A persistent metaphor in decision-making research casts people as intuitive statisticians. Popular explanations based on this metaphor assume that the way in which people represent the environment is specified and fixed a priori. A major flaw in this account is that it is not clear how people know what aspects of an environment are important, how to interpret those aspects, and how to make decisions based on them. We suggest a theoretical reorientation away from assuming people's representations towards a focus on explaining how people themselves specify what is important to represent. This perspective casts decision makers as intuitive scientists able to flexibly construct, modify, and replace the representations of the decision problems they face.
Collapse
Affiliation(s)
- Aba Szollosi
- School of Psychology, University of New South Wales, Sydney, Australia.
| | - Ben R Newell
- School of Psychology, University of New South Wales, Sydney, Australia
| |
Collapse
|
43
|
A Model for Structured Information Representation in Neural Networks of the Brain. eNeuro 2020; 7:ENEURO.0533-19.2020. [PMID: 32381648 PMCID: PMC7266140 DOI: 10.1523/eneuro.0533-19.2020] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2019] [Revised: 04/01/2020] [Accepted: 04/16/2020] [Indexed: 11/21/2022] Open
Abstract
Humans can reason at an abstract level and structure information into abstract categories, but the underlying neural processes have remained unknown. Recent experimental data provide the hint that this is likely to involve specific subareas of the brain from which structural information can be decoded. Based on this data, we introduce the concept of assembly projections, a general principle for attaching structural information to content in generic networks of spiking neurons. According to the assembly projections principle, structure-encoding assemblies emerge and are dynamically attached to content representations through Hebbian plasticity mechanisms. This model provides the basis for explaining a number of experimental data and provides a basis for modeling abstract computational operations of the brain.
Collapse
|
44
|
Frankland SM, Greene JD. Two Ways to Build a Thought: Distinct Forms of Compositional Semantic Representation across Brain Regions. Cereb Cortex 2020; 30:3838-3855. [PMID: 32279078 DOI: 10.1093/cercor/bhaa001] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2019] [Revised: 11/30/2019] [Accepted: 01/02/2020] [Indexed: 12/23/2022] Open
Abstract
To understand a simple sentence such as "the woman chased the dog", the human mind must dynamically organize the relevant concepts to represent who did what to whom. This structured recombination of concepts (woman, dog, chased) enables the representation of novel events, and is thus a central feature of intelligence. Here, we use functional magnetic resonance (fMRI) and encoding models to delineate the contributions of three brain regions to the representation of relational combinations. We identify a region of anterior-medial prefrontal cortex (amPFC) that shares representations of noun-verb conjunctions across sentences: for example, a combination of "woman" and "chased" to encode woman-as-chaser, distinct from woman-as-chasee. This PFC region differs from the left-mid superior temporal cortex (lmSTC) and hippocampus, two regions previously implicated in representing relations. lmSTC represents broad role combinations that are shared across verbs (e.g., woman-as-agent), rather than narrow roles, limited to specific actions (woman-as-chaser). By contrast, a hippocampal sub-region represents events sharing narrow conjunctions as dissimilar. The success of the hippocampal conjunctive encoding model is anti-correlated with generalization performance in amPFC on a trial-by-trial basis, consistent with a pattern separation mechanism. Thus, these three regions appear to play distinct, but complementary, roles in encoding compositional event structure.
Collapse
Affiliation(s)
- Steven M Frankland
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08540
| | - Joshua D Greene
- Department of Psychology, Center for Brain Science, Harvard University, Cambridge, MA 02138
| |
Collapse
|
45
|
Cross Recruitment of Domain-Selective Cortical Representations Enables Flexible Semantic Knowledge. J Neurosci 2020; 40:3096-3103. [PMID: 32152199 DOI: 10.1523/jneurosci.2224-19.2020] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2019] [Revised: 02/12/2020] [Accepted: 02/14/2020] [Indexed: 11/21/2022] Open
Abstract
Knowledge about objects encompasses not only their prototypical features but also complex, atypical, semantic knowledge (e.g., "Pizza was invented in Naples"). This fMRI study of male and female human participants combines univariate and multivariate analyses to consider the cortical representation of this more complex semantic knowledge. Using the categories of food, people, and places, this study investigates whether access to spatially related geographic semantic knowledge (1) involves the same domain-selective neural representations involved in access to prototypical taste knowledge about food; and (2) elicits activation of neural representations classically linked to places when this geographic knowledge is accessed about food and people. In three experiments using word stimuli, domain-relevant and atypical conceptual access for the categories food, people, and places were assessed. Results uncover two principles of semantic representation: food-selective representations in the left insula continue to be recruited when prototypical taste knowledge is task-irrelevant and under conditions of high cognitive demand; access to geographic knowledge for food and people categories involves the additional recruitment of classically place-selective parahippocampal gyrus, retrosplenial complex, and transverse occipital sulcus. These findings underscore the importance of object category in the representation of a broad range of knowledge, while showing how the cross recruitment of specialized representations may endow the considerable flexibility of our complex semantic knowledge.SIGNIFICANCE STATEMENT We know not only stereotypical things about objects (an apple is round, graspable, edible) but can also flexibly combine typical and atypical features to form complex concepts (the metaphorical role an apple plays in Judeo-Christian belief). In this fMRI study, we observe that, when atypical geographic knowledge is accessed about food dishes, domain-selective sensorimotor-related cortical representations continue to be recruited, but that regions classically associated with place perception are additionally engaged. This interplay between categorically driven representations, linked to the object being accessed, and the flexible recruitment of semantic stores linked to the content being accessed, provides a potential mechanism for the broad representational repertoire of our semantic system.
Collapse
|