1
|
Kay K, Biderman N, Khajeh R, Beiran M, Cueva CJ, Shohamy D, Jensen G, Wei XX, Ferrera VP, Abbott LF. Emergent neural dynamics and geometry for generalization in a transitive inference task. PLoS Comput Biol 2024; 20:e1011954. [PMID: 38662797 PMCID: PMC11125559 DOI: 10.1371/journal.pcbi.1011954] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Revised: 05/24/2024] [Accepted: 02/28/2024] [Indexed: 05/25/2024] Open
Abstract
Relational cognition-the ability to infer relationships that generalize to novel combinations of objects-is fundamental to human and animal intelligence. Despite this importance, it remains unclear how relational cognition is implemented in the brain due in part to a lack of hypotheses and predictions at the levels of collective neural activity and behavior. Here we discovered, analyzed, and experimentally tested neural networks (NNs) that perform transitive inference (TI), a classic relational task (if A > B and B > C, then A > C). We found NNs that (i) generalized perfectly, despite lacking overt transitive structure prior to training, (ii) generalized when the task required working memory (WM), a capacity thought to be essential to inference in the brain, (iii) emergently expressed behaviors long observed in living subjects, in addition to a novel order-dependent behavior, and (iv) expressed different task solutions yielding alternative behavioral and neural predictions. Further, in a large-scale experiment, we found that human subjects performing WM-based TI showed behavior inconsistent with a class of NNs that characteristically expressed an intuitive task solution. These findings provide neural insights into a classical relational ability, with wider implications for how the brain realizes relational cognition.
Collapse
Affiliation(s)
- Kenneth Kay
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, United States of America
- Center for Theoretical Neuroscience, Columbia University, New York, New York, United States of America
- Grossman Center for the Statistics of Mind, Columbia University, New York, New York, United States of America
| | - Natalie Biderman
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, United States of America
- Department of Psychology, Columbia University, New York, New York, United States of America
| | - Ramin Khajeh
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, United States of America
- Center for Theoretical Neuroscience, Columbia University, New York, New York, United States of America
| | - Manuel Beiran
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, United States of America
- Center for Theoretical Neuroscience, Columbia University, New York, New York, United States of America
| | - Christopher J. Cueva
- Department of Brain and Cognitive Sciences, MIT, Cambridge, Massachusetts, United States of America
| | - Daphna Shohamy
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, United States of America
- Department of Psychology, Columbia University, New York, New York, United States of America
- The Kavli Institute for Brain Science, Columbia University, New York, New York, United States of America
| | - Greg Jensen
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, United States of America
- Department of Neuroscience, Columbia University Medical Center, New York, New York, United States of America
- Department of Psychology at Reed College, Portland, Oregon, United States of America
| | - Xue-Xin Wei
- Departments of Neuroscience and Psychology, The University of Texas at Austin, Austin, Texas, United States of America
| | - Vincent P. Ferrera
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, United States of America
- Department of Neuroscience, Columbia University Medical Center, New York, New York, United States of America
- Department of Psychiatry, Columbia University Medical Center, New York, New York, United States of America
| | - LF Abbott
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, United States of America
- Center for Theoretical Neuroscience, Columbia University, New York, New York, United States of America
- The Kavli Institute for Brain Science, Columbia University, New York, New York, United States of America
- Department of Neuroscience, Columbia University Medical Center, New York, New York, United States of America
| |
Collapse
|
2
|
Ramawat S, Marc IB, Ceccarelli F, Ferrucci L, Bardella G, Ferraina S, Pani P, Brunamonti E. The transitive inference task to study the neuronal correlates of memory-driven decision making: A monkey neurophysiology perspective. Neurosci Biobehav Rev 2023; 152:105258. [PMID: 37268179 DOI: 10.1016/j.neubiorev.2023.105258] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Revised: 05/15/2023] [Accepted: 05/30/2023] [Indexed: 06/04/2023]
Abstract
A vast amount of literature agrees that rank-ordered information as A>B>C>D>E>F is mentally represented in spatially organized schemas after learning. This organization significantly influences the process of decision-making, using the acquired premises, i.e. deciding if B is higher than D is equivalent to comparing their position in this space. The implementation of non-verbal versions of the transitive inference task has provided the basis for ascertaining that different animal species explore a mental space when deciding among hierarchically organized memories. In the present work, we reviewed several studies of transitive inference that highlighted this ability in animals and, consequently, the animal models developed to study the underlying cognitive processes and the main neural structures supporting this ability. Further, we present the literature investigating which are the underlying neuronal mechanisms. Then we discuss how non-human primates represent an excellent model for future studies, providing ideal resources for better understanding the neuronal correlates of decision-making through transitive inference tasks.
Collapse
Affiliation(s)
- Surabhi Ramawat
- Department of Physiology and Pharmacology, Sapienza University, Rome, Italy
| | - Isabel Beatrice Marc
- Department of Physiology and Pharmacology, Sapienza University, Rome, Italy; Behavioral Neuroscience PhD Program, Sapienza University, Rome, Italy
| | | | - Lorenzo Ferrucci
- Department of Physiology and Pharmacology, Sapienza University, Rome, Italy
| | - Giampiero Bardella
- Department of Physiology and Pharmacology, Sapienza University, Rome, Italy
| | - Stefano Ferraina
- Department of Physiology and Pharmacology, Sapienza University, Rome, Italy
| | - Pierpaolo Pani
- Department of Physiology and Pharmacology, Sapienza University, Rome, Italy
| | - Emiliano Brunamonti
- Department of Physiology and Pharmacology, Sapienza University, Rome, Italy.
| |
Collapse
|
3
|
Zhang Y, Pan X, Wang Y. Category learning in a recurrent neural network with reinforcement learning. Front Psychiatry 2022; 13:1008011. [PMID: 36387007 PMCID: PMC9640766 DOI: 10.3389/fpsyt.2022.1008011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/31/2022] [Accepted: 10/10/2022] [Indexed: 11/13/2022] Open
Abstract
It is known that humans and animals can learn and utilize category information quickly and efficiently to adapt to changing environments, and several brain areas are involved in learning and encoding category information. However, it is unclear that how the brain system learns and forms categorical representations from the view of neural circuits. In order to investigate this issue from the network level, we combine a recurrent neural network with reinforcement learning to construct a deep reinforcement learning model to demonstrate how the category is learned and represented in the network. The model consists of a policy network and a value network. The policy network is responsible for updating the policy to choose actions, while the value network is responsible for evaluating the action to predict rewards. The agent learns dynamically through the information interaction between the policy network and the value network. This model was trained to learn six stimulus-stimulus associative chains in a sequential paired-association task that was learned by the monkey. The simulated results demonstrated that our model was able to learn the stimulus-stimulus associative chains, and successfully reproduced the similar behavior of the monkey performing the same task. Two types of neurons were found in this model: one type primarily encoded identity information about individual stimuli; the other type mainly encoded category information of associated stimuli in one chain. The two types of activity-patterns were also observed in the primate prefrontal cortex after the monkey learned the same task. Furthermore, the ability of these two types of neurons to encode stimulus or category information was enhanced during this model was learning the task. Our results suggest that the neurons in the recurrent neural network have the ability to form categorical representations through deep reinforcement learning during learning stimulus-stimulus associations. It might provide a new approach for understanding neuronal mechanisms underlying how the prefrontal cortex learns and encodes category information.
Collapse
Affiliation(s)
- Ying Zhang
- Institute for Cognitive Neurodynamics, East China University of Science and Technology, Shanghai, China
| | - Xiaochuan Pan
- Institute for Cognitive Neurodynamics, East China University of Science and Technology, Shanghai, China
| | - Yihong Wang
- Institute for Cognitive Neurodynamics, East China University of Science and Technology, Shanghai, China
| |
Collapse
|