1
|
Ramírez-Ruiz J, Grytskyy D, Mastrogiuseppe C, Habib Y, Moreno-Bote R. Complex behavior from intrinsic motivation to occupy future action-state path space. Nat Commun 2024; 15:6368. [PMID: 39075046 PMCID: PMC11286966 DOI: 10.1038/s41467-024-49711-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Accepted: 06/13/2024] [Indexed: 07/31/2024] Open
Abstract
Most theories of behavior posit that agents tend to maximize some form of reward or utility. However, animals very often move with curiosity and seem to be motivated in a reward-free manner. Here we abandon the idea of reward maximization and propose that the goal of behavior is maximizing occupancy of future paths of actions and states. According to this maximum occupancy principle, rewards are the means to occupy path space, not the goal per se; goal-directedness simply emerges as rational ways of searching for resources so that movement, understood amply, never ends. We find that action-state path entropy is the only measure consistent with additivity and other intuitive properties of expected future action-state path occupancy. We provide analytical expressions that relate the optimal policy and state-value function and prove convergence of our value iteration algorithm. Using discrete and continuous state tasks, including a high-dimensional controller, we show that complex behaviors such as "dancing", hide-and-seek, and a basic form of altruistic behavior naturally result from the intrinsic motivation to occupy path space. All in all, we present a theory of behavior that generates both variability and goal-directedness in the absence of reward maximization.
Collapse
Affiliation(s)
- Jorge Ramírez-Ruiz
- Center for Brain and Cognition, Departament d'Enginyeria i Escola d'Enginyeria, Universitat Pompeu Fabra, Barcelona, Spain.
| | - Dmytro Grytskyy
- Center for Brain and Cognition, Departament d'Enginyeria i Escola d'Enginyeria, Universitat Pompeu Fabra, Barcelona, Spain
| | - Chiara Mastrogiuseppe
- Center for Brain and Cognition, Departament d'Enginyeria i Escola d'Enginyeria, Universitat Pompeu Fabra, Barcelona, Spain
| | - Yamen Habib
- Center for Brain and Cognition, Departament d'Enginyeria i Escola d'Enginyeria, Universitat Pompeu Fabra, Barcelona, Spain
| | - Rubén Moreno-Bote
- Center for Brain and Cognition, Departament d'Enginyeria i Escola d'Enginyeria, Universitat Pompeu Fabra, Barcelona, Spain
- Serra Húnter Fellow Programme, Universitat Pompeu Fabra, Barcelona, Spain
| |
Collapse
|
2
|
Carlson BM, Mitchell BA, Dougherty K, Westerberg JA, Cox MA, Maier A. Does V1 response suppression initiate binocular rivalry? iScience 2023; 26:107359. [PMID: 37520732 PMCID: PMC10382945 DOI: 10.1016/j.isci.2023.107359] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Revised: 06/02/2023] [Accepted: 07/07/2023] [Indexed: 08/01/2023] Open
Abstract
During binocular rivalry (BR) only one eye's view is perceived. Neural underpinnings of BR are debated. Recent studies suggest that primary visual cortex (V1) initiates BR. One trigger might be response suppression across most V1 neurons at the onset of BR. Here, we utilize a variant of BR called binocular rivalry flash suppression (BRFS) to test this hypothesis. BRFS is identical to BR, except stimuli are shown with a ∼1s delay. If V1 response suppression was required to initiate BR, it should occur during BRFS as well. To test this, we compared V1 spiking in two macaques observing BRFS. We found that BRFS resulted in response facilitation rather than response suppression across V1 neurons. However, BRFS still reduces responses in a subset of V1 neurons due to the adaptive effects of asynchronous stimulus presentation. We argue that this selective response suppression could serve as an alternate initiator of BR.
Collapse
Affiliation(s)
- Brock M. Carlson
- Department of Psychology, College of Arts and Science, Vanderbilt Vision Research Center, Center for Integrative and Cognitive Neuroscience, Vanderbilt University, Nashville, TN 37235, USA
| | - Blake A. Mitchell
- Department of Psychology, College of Arts and Science, Vanderbilt Vision Research Center, Center for Integrative and Cognitive Neuroscience, Vanderbilt University, Nashville, TN 37235, USA
| | - Kacie Dougherty
- Department of Psychology, College of Arts and Science, Vanderbilt Vision Research Center, Center for Integrative and Cognitive Neuroscience, Vanderbilt University, Nashville, TN 37235, USA
- Department of Psychology, Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08540, USA
| | - Jacob A. Westerberg
- Department of Psychology, College of Arts and Science, Vanderbilt Vision Research Center, Center for Integrative and Cognitive Neuroscience, Vanderbilt University, Nashville, TN 37235, USA
- Department of Vision and Cognition, Netherlands Institute for Neuroscience, Royal Netherlands Academy of Arts and Sciences, Amsterdam 1105 BA, the Netherlands
| | - Michele A. Cox
- Department of Psychology, College of Arts and Science, Vanderbilt Vision Research Center, Center for Integrative and Cognitive Neuroscience, Vanderbilt University, Nashville, TN 37235, USA
- Center for Visual Science, University of Rochester, Rochester, NY 14627, USA
| | - Alexander Maier
- Department of Psychology, College of Arts and Science, Vanderbilt Vision Research Center, Center for Integrative and Cognitive Neuroscience, Vanderbilt University, Nashville, TN 37235, USA
| |
Collapse
|
3
|
Barkdoll K, Lu Y, Barranca VJ. New insights into binocular rivalry from the reconstruction of evolving percepts using model network dynamics. Front Comput Neurosci 2023; 17:1137015. [PMID: 37034441 PMCID: PMC10079880 DOI: 10.3389/fncom.2023.1137015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Accepted: 03/07/2023] [Indexed: 04/11/2023] Open
Abstract
When the two eyes are presented with highly distinct stimuli, the resulting visual percept generally switches every few seconds between the two monocular images in an irregular fashion, giving rise to a phenomenon known as binocular rivalry. While a host of theoretical studies have explored potential mechanisms for binocular rivalry in the context of evoked model dynamics in response to simple stimuli, here we investigate binocular rivalry directly through complex stimulus reconstructions based on the activity of a two-layer neuronal network model with competing downstream pools driven by disparate monocular stimuli composed of image pixels. To estimate the dynamic percept, we derive a linear input-output mapping rooted in the non-linear network dynamics and iteratively apply compressive sensing techniques for signal recovery. Utilizing a dominance metric, we are able to identify when percept alternations occur and use data collected during each dominance period to generate a sequence of percept reconstructions. We show that despite the approximate nature of the input-output mapping and the significant reduction in neurons downstream relative to stimulus pixels, the dominant monocular image is well-encoded in the network dynamics and improvements are garnered when realistic spatial receptive field structure is incorporated into the feedforward connectivity. Our model demonstrates gamma-distributed dominance durations and well obeys Levelt's four laws for how dominance durations change with stimulus strength, agreeing with key recurring experimental observations often used to benchmark rivalry models. In light of evidence that individuals with autism exhibit relatively slow percept switching in binocular rivalry, we corroborate the ubiquitous hypothesis that autism manifests from reduced inhibition in the brain by systematically probing our model alternation rate across choices of inhibition strength. We exhibit sufficient conditions for producing binocular rivalry in the context of natural scene stimuli, opening a clearer window into the dynamic brain computations that vary with the generated percept and a potential path toward further understanding neurological disorders.
Collapse
|
4
|
Safavi S, Dayan P. Multistability, perceptual value, and internal foraging. Neuron 2022; 110:3076-3090. [PMID: 36041434 DOI: 10.1016/j.neuron.2022.07.024] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Revised: 07/03/2022] [Accepted: 07/25/2022] [Indexed: 11/19/2022]
Abstract
Substantial experimental, theoretical, and computational insights into sensory processing have been derived from the phenomena of perceptual multistability-when two or more percepts alternate or switch in response to a single sensory input. Here, we review a range of findings suggesting that alternations can be seen as internal choices by the brain responding to values. We discuss how elements of external, experimenter-controlled values and internal, uncertainty- and aesthetics-dependent values influence multistability. We then consider the implications for the involvement in switching of regions, such as the anterior cingulate cortex, which are more conventionally tied to value-dependent operations such as cognitive control and foraging.
Collapse
Affiliation(s)
- Shervin Safavi
- University of Tübingen, Tübingen, Germany; Max Planck Institute for Biological Cybernetics, Tübingen, Germany.
| | - Peter Dayan
- University of Tübingen, Tübingen, Germany; Max Planck Institute for Biological Cybernetics, Tübingen, Germany
| |
Collapse
|
5
|
Maier A, Cox MA, Westerberg JA, Dougherty K. Binocular Integration in the Primate Primary Visual Cortex. Annu Rev Vis Sci 2022; 8:345-360. [PMID: 35676095 DOI: 10.1146/annurev-vision-100720-112922] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
binocular vision, binocular fusion, binocular combination, LGN, V1.
Collapse
Affiliation(s)
- A Maier
- Department of Psychological Sciences, Vanderbilt University, Nashville, Tennessee 37240;
| | - M A Cox
- Center for Visual Science, Rochester University, Rochester, New York 14642
| | - J A Westerberg
- Department of Psychological Sciences, Vanderbilt University, Nashville, Tennessee 37240;
| | - K Dougherty
- Neuroscience Institute, Princeton University, Princeton, New Jersey 08544
| |
Collapse
|
6
|
Miller SM. Fluctuations of consciousness, mood, and science: The interhemispheric switch and sticky switch models two decades on. J Comp Neurol 2020; 528:3171-3197. [DOI: 10.1002/cne.24943] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2020] [Revised: 04/21/2020] [Accepted: 04/22/2020] [Indexed: 12/13/2022]
Affiliation(s)
- Steven M. Miller
- Perceptual and Clinical Neuroscience Laboratory, Department of Physiology Monash Biomedicine Discovery Institute, School of Biomedical Sciences, Monash University Melbourne Victoria Australia
- Monash Alfred Psychiatry Research Centre Central Clinical School, Monash University and Alfred Health Melbourne Victoria Australia
| |
Collapse
|