1
|
Ramírez-Ruiz J, Grytskyy D, Mastrogiuseppe C, Habib Y, Moreno-Bote R. Complex behavior from intrinsic motivation to occupy future action-state path space. Nat Commun 2024; 15:6368. [PMID: 39075046 PMCID: PMC11286966 DOI: 10.1038/s41467-024-49711-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Accepted: 06/13/2024] [Indexed: 07/31/2024] Open
Abstract
Most theories of behavior posit that agents tend to maximize some form of reward or utility. However, animals very often move with curiosity and seem to be motivated in a reward-free manner. Here we abandon the idea of reward maximization and propose that the goal of behavior is maximizing occupancy of future paths of actions and states. According to this maximum occupancy principle, rewards are the means to occupy path space, not the goal per se; goal-directedness simply emerges as rational ways of searching for resources so that movement, understood amply, never ends. We find that action-state path entropy is the only measure consistent with additivity and other intuitive properties of expected future action-state path occupancy. We provide analytical expressions that relate the optimal policy and state-value function and prove convergence of our value iteration algorithm. Using discrete and continuous state tasks, including a high-dimensional controller, we show that complex behaviors such as "dancing", hide-and-seek, and a basic form of altruistic behavior naturally result from the intrinsic motivation to occupy path space. All in all, we present a theory of behavior that generates both variability and goal-directedness in the absence of reward maximization.
Collapse
Affiliation(s)
- Jorge Ramírez-Ruiz
- Center for Brain and Cognition, Departament d'Enginyeria i Escola d'Enginyeria, Universitat Pompeu Fabra, Barcelona, Spain.
| | - Dmytro Grytskyy
- Center for Brain and Cognition, Departament d'Enginyeria i Escola d'Enginyeria, Universitat Pompeu Fabra, Barcelona, Spain
| | - Chiara Mastrogiuseppe
- Center for Brain and Cognition, Departament d'Enginyeria i Escola d'Enginyeria, Universitat Pompeu Fabra, Barcelona, Spain
| | - Yamen Habib
- Center for Brain and Cognition, Departament d'Enginyeria i Escola d'Enginyeria, Universitat Pompeu Fabra, Barcelona, Spain
| | - Rubén Moreno-Bote
- Center for Brain and Cognition, Departament d'Enginyeria i Escola d'Enginyeria, Universitat Pompeu Fabra, Barcelona, Spain
- Serra Húnter Fellow Programme, Universitat Pompeu Fabra, Barcelona, Spain
| |
Collapse
|
2
|
Papanikolaou C, Sharma A, Lind PG, Lencastre P. Lévy Flight Model of Gaze Trajectories to Assist in ADHD Diagnoses. ENTROPY (BASEL, SWITZERLAND) 2024; 26:392. [PMID: 38785640 PMCID: PMC11120544 DOI: 10.3390/e26050392] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Revised: 03/29/2024] [Accepted: 04/25/2024] [Indexed: 05/25/2024]
Abstract
The precise mathematical description of gaze patterns remains a topic of ongoing debate, impacting the practical analysis of eye-tracking data. In this context, we present evidence supporting the appropriateness of a Lévy flight description for eye-gaze trajectories, emphasizing its beneficial scale-invariant properties. Our study focuses on utilizing these properties to aid in diagnosing Attention-Deficit and Hyperactivity Disorder (ADHD) in children, in conjunction with standard cognitive tests. Using this method, we found that the distribution of the characteristic exponent of Lévy flights statistically is different in children with ADHD. Furthermore, we observed that these children deviate from a strategy that is considered optimal for searching processes, in contrast to non-ADHD children. We focused on the case where both eye-tracking data and data from a cognitive test are present and show that the study of gaze patterns in children with ADHD can help in identifying this condition. Since eye-tracking data can be gathered during cognitive tests without needing extra time-consuming specific tasks, we argue that it is in a prime position to provide assistance in the arduous task of diagnosing ADHD.
Collapse
Affiliation(s)
- Christos Papanikolaou
- Department of Computer Science, Oslo Metropolitan University, N-0130 Oslo, Norway; (C.P.); (A.S.); (P.G.L.)
| | - Akriti Sharma
- Department of Computer Science, Oslo Metropolitan University, N-0130 Oslo, Norway; (C.P.); (A.S.); (P.G.L.)
| | - Pedro G. Lind
- Department of Computer Science, Oslo Metropolitan University, N-0130 Oslo, Norway; (C.P.); (A.S.); (P.G.L.)
- OsloMet Artificial Intelligence Lab, Pilestredet 52, N-0166 Oslo, Norway
- NordSTAR—Nordic Center for Sustainable and Trustworthy AI Research, Pilestredet 52, N-0166 Oslo, Norway
- Simula Research Laboratory, Numerical Analysis and Scientific Computing, N-0164 Oslo, Norway
| | - Pedro Lencastre
- Department of Computer Science, Oslo Metropolitan University, N-0130 Oslo, Norway; (C.P.); (A.S.); (P.G.L.)
- OsloMet Artificial Intelligence Lab, Pilestredet 52, N-0166 Oslo, Norway
- NordSTAR—Nordic Center for Sustainable and Trustworthy AI Research, Pilestredet 52, N-0166 Oslo, Norway
| |
Collapse
|
3
|
Campeau W, Simons AM, Stevens B. Intermittent Search, Not Strict Lévy Flight, Evolves under Relaxed Foraging Distribution Constraints. Am Nat 2024; 203:513-527. [PMID: 38489781 DOI: 10.1086/729220] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/17/2024]
Abstract
AbstractThe survival of an animal depends on its success as a forager, and understanding the adaptations that result in successful foraging strategies is an enduring endeavour of behavioral ecology. Random walks are one of the primary mathematical descriptions of foraging behavior. Power law distributions are often used to model random walks, as they can characterize a wide range of behaviors, including Lévy walks. Empirical evidence indicates the prevalence and efficiency of Lévy walks as a foraging strategy, and theoretical work suggests an evolutionary origin. However, previous evolutionary models have assumed a priori that move lengths are drawn from a power law or other families of distributions. Here, we remove this restriction with a model that allows for the evolution of any distribution. Instead of Lévy walks, our model unfailingly results in the evolution of intermittent search, a random walk composed of two disjoint modes-frequent localized walks and infrequent extensive moves-that consistently outcompeted Lévy walks. We also demonstrate that foraging using intermittent search may resemble a Lévy walk because of interactions with the resources within an environment. These extrinsically generated Lévy-like walks belie an underlying behavior and may explain the prevalence of Lévy walks reported in the literature.
Collapse
|
4
|
Mirmiran C, Fraser M, Maler L. Finding food in the dark: how trajectories of a gymnotiform fish change with spatial learning. J Exp Biol 2022; 225:285892. [PMID: 36366924 DOI: 10.1242/jeb.244590] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Accepted: 11/08/2022] [Indexed: 11/13/2022]
Abstract
We analyzed the trajectories of freely foraging Gymnotus sp., a pulse-type gymnotiform weakly electric fish, swimming in a dark arena. For each fish, we compared the its initial behavior as it learned the relative location of landmarks and food with its behavior after learning was complete, i.e. after time/distance to locate food had reached a minimal asymptotic level. During initial exploration when the fish did not know the arena layout, trajectories included many sharp angle head turns that occurred at nearly completely random intervals. After spatial learning was complete, head turns became far smoother. Interestingly, the fish still did not take a stereotyped direct route to the food but instead took smooth but variable curved trajectories. We also measured the fish's heading angle error (heading angle - heading angle towards food). After spatial learning, the fish's initial heading angle errors were strongly biased to zero, i.e. the fish mostly turned towards the food. As the fish approached closer to the food, they switched to a random search strategy with a more uniform distribution of heading angle errors.
Collapse
Affiliation(s)
- Camille Mirmiran
- Department of Mathematics and Statistics, University of Ottawa, Ottawa, ON, Canada, K1N 6N5
| | - Maia Fraser
- Department of Mathematics and Statistics, University of Ottawa, Ottawa, ON, Canada, K1N 6N5.,Centre for Neural Dynamics, University of Ottawa, Ottawa, ON, Canada, K1N 6N5
| | - Leonard Maler
- Department of Cellular and Molecular Medicine, University of Ottawa, Ottawa, ON, Canada, K1H 8M5.,Centre for Neural Dynamics, University of Ottawa, Ottawa, ON, Canada, K1N 6N5
| |
Collapse
|
5
|
Nauta J, Simoens P, Khaluf Y, Martinez-Garcia R. Foraging behaviour and patch size distribution jointly determine population dynamics in fragmented landscapes. J R Soc Interface 2022; 19:20220103. [PMID: 35730173 PMCID: PMC9214291 DOI: 10.1098/rsif.2022.0103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Accepted: 05/31/2022] [Indexed: 11/12/2022] Open
Abstract
Increased fragmentation caused by habitat loss represents a major threat to the persistence of animal populations. How fragmentation affects populations depends on the rate at which individuals move between spatially separated patches. Whereas negative effects of habitat loss on biodiversity are well known, the effects of fragmentation per se on population dynamics and ecosystem stability remain less well understood. Here, we use a spatially explicit predator-prey model to investigate how the interplay between fragmentation and optimal foraging behaviour affects predator-prey interactions and, subsequently, ecosystem stability. We study systems wherein prey occupies isolated patches and are consumed by predators that disperse following Lévy random walks. Our results show that the Lévy exponent and the degree of fragmentation jointly determine coexistence probabilities. In highly fragmented landscapes, Brownian and ballistic predators go extinct and only scale-free predators can coexist with prey. Furthermore, our results confirm that predation causes irreversible habitat loss in fragmented landscapes owing to overexploitation of smaller patches of prey. Moreover, we show that predator dispersal can reduce, but not prevent or minimize, the amount of lost habitat. Our results suggest that integrating optimal foraging theory into population and landscape ecology is crucial to assessing the impact of fragmentation on biodiversity and ecosystem stability.
Collapse
Affiliation(s)
- Johannes Nauta
- Department of Information Technology–IDLab, Ghent University-IMEC, Technologiepark Zwijnaarde 126, 9052 Ghent, Belgium
| | - Pieter Simoens
- Department of Information Technology–IDLab, Ghent University-IMEC, Technologiepark Zwijnaarde 126, 9052 Ghent, Belgium
| | - Yara Khaluf
- Wageningen University and Research, Department of Social Sciences–Information Technology Group, Hollandseweg 1, 6706KN Wageningen, The Netherlands
| | - Ricardo Martinez-Garcia
- ICTP South American Institute for Fundamental Research and Instituto de Física Teórica, Universidade Estadual Paulista–UNESP, Rua Dr Bento Teobaldo Ferraz 271, Bloco 2 – Barra Funda, 01140-070 São Paulo, Brazil
| |
Collapse
|