1
|
Zhang W, Valencia A, Chang NB. Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023; 34:2170-2190. [PMID: 34473633 DOI: 10.1109/tnnls.2021.3106777] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Agent-based modeling (ABM) involves developing models in which agents make adaptive decisions in a changing environment. Machine-learning (ML) based inference models can improve sequential decision-making by learning agents' behavioral patterns. With the aid of ML, this emerging area can extend traditional agent-based schemes that hardcode agents' behavioral rules into an adaptive model. Even though there are plenty of studies that apply ML in ABMs, the generalized applicable scenarios, frameworks, and procedures for implementations are not well addressed. In this article, we provide a comprehensive review of applying ML in ABM based on four major scenarios, i.e., microagent-level situational awareness learning, microagent-level behavior intervention, macro-ABM-level emulator, and sequential decision-making. For these four scenarios, the related algorithms, frameworks, procedures of implementations, and multidisciplinary applications are thoroughly investigated. We also discuss how ML can improve prediction in ABMs by trading off the variance and bias and how ML can improve the sequential decision-making of microagent and macrolevel policymakers via a mechanism of reinforced behavioral intervention. At the end of this article, future perspectives of applying ML in ABMs are discussed with respect to data acquisition and quality issues, the possible solution of solving the convergence problem of reinforcement learning, interpretable ML applications, and bounded rationality of ABM.
Collapse
|
2
|
Abstract
Linking cognitive performance with fitness outcomes, measured using both reproductive and survival metrics, of free-living animals is crucial for understanding the evolution of cognition. Although there is increasing evidence showing a link between cognitive traits and reproductive success metrics, studies specifically exploring the link between cognition and survival are scarce. We first explore which cognitive traits related to survival have been investigated in free-living animals. We also discuss the challenges associated with investigating the links between cognition and survival. We then review studies that specifically consider survival of animals of known cognitive abilities that are either free-living or in captivity and later released into the wild. We found nine studies exploring cognitive traits in wild populations. The relationships between these cognitive traits and survival were equivocal. We found a further nine studies in captive-reared populations trained to predator cues and later released into the wild. Training to recognize predator cues was correlated with increased survival in the majority of studies. Finally, different individual intrinsic characteristics (i.e., age, body condition, personality, sex) showed varied effects between studies. We argue that finding ecologically relevant cognitive traits is crucial for gaining a better understanding of how selection impacts certain cognitive traits, and how these might contribute to an individual’s survival. We also suggest possible standardized, easy to implement, cognitive tests that can be used in long-term studies, which would generate large sample sizes, take into account intrinsic characteristics, and provide an opportunity to understand the mechanisms, development and evolution of cognition.
Collapse
|
3
|
Zamora-Camacho FJ. The relationships between toad behaviour, antipredator defences, and spatial and sexual variation in predation pressure. PeerJ 2022; 10:e12985. [PMID: 35194533 PMCID: PMC8858576 DOI: 10.7717/peerj.12985] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Accepted: 02/01/2022] [Indexed: 01/11/2023] Open
Abstract
BACKGROUND Animal behaviour is under strong selection. Selection on behaviour, however, might not act in isolation from other fitness-related traits. Since predators represent outstanding selective forces, animal behaviour could covary with antipredator defences, such that individuals better suited against predators could afford facing the costs of riskier behaviours. Moreover, not all individuals undergo equivalent degrees of predation pressure, which can vary across sexes or habitats. Individuals under lower predation pressure might also exhibit riskier behaviours. METHODS In this work, I tested these hypotheses on natterjack toads (Epidalea calamita). Specifically, I gauged activity time, exploratory behaviour and boldness in standard laboratory conditions, and assessed whether they correlated with body size and antipredator strategies, namely sprint speed, parotoid gland area and parotoid gland colour contrast. Additionally, I compared these traits between sexes and individuals from an agrosystem and pine grove, since there is evidence that males and agrosystem individuals are subjected to greater predation pressure. RESULTS Sprint speed as well as parotoid gland contrast and size appeared unrelated to the behavioural traits studied. In turn, body mass was negatively related to activity time, boldness and exploration. This trend is consistent with the fact that larger toads could be more detectable to their predators, which are mostly gape unconstrained and could easily consume them. As predicted, females exhibited riskier behaviours. Nonetheless, agrosystem toads did not differ from pine grove toads in the behavioural traits measured, despite being under stronger predation pressure.
Collapse
Affiliation(s)
- Francisco Javier Zamora-Camacho
- Departamento de Sistemas Físicos, Químicos y Naturales, Universidad Pablo de Olavide, Seville, Spain,Departamento de Biodiversidad, Ecología y Evolución, Universidad Complutense de Madrid, Madrid, Spain
| |
Collapse
|
4
|
Abstract
Abstract
The expression of an individual animal’s behaviour can be placed along many different personality spectra. Parasite load can alter animal behaviour and, thus, fitness. The personality traits of rusty crayfish, Faxonius rusticus, were analysed in three different behavioural contexts: foraging, exploration, and threatened. Each crayfish was tested in each context 3 times, giving a total of 9 assays per crayfish. After assays were completed, crayfish were dissected, and the hepatopancreas of each crayfish was photo analysed to determine the parasite load of the trematode, Microphallus spp. A composite personality score for each assay and parasite load was loaded into a PCA. The PCA model showed that as parasite load increased, crayfish became bolder in threatening contexts and less exploratory in novel environments, whether or not a food stimulus was present. Thus, parasite load alters the placement of crayfish on different personality spectra, but this change is context specific.
Collapse
Affiliation(s)
- Rebecca Noel MacKay
- Department of Biological Sciences, Bowling Green State University, 217 Life Sciences, N College Drive, Bowling Green, OH 43403, USA
- University of Michigan Biological Station, University of Michigan, 9133 Biological Road, Pellston, MI 49769, USA
| | - Paul A. Moore
- Department of Biological Sciences, Bowling Green State University, 217 Life Sciences, N College Drive, Bowling Green, OH 43403, USA
- University of Michigan Biological Station, University of Michigan, 9133 Biological Road, Pellston, MI 49769, USA
- J.P. Scott Center for Neurosciences, Mind, and Behavior, Bowling Green State University, Bowling Green, OH 43403, USA
| |
Collapse
|
5
|
Anwer H, Mason D, Zajitschek S, Noble DWA, Hesselson D, Morris MJ, Lagisz M, Nakagawa S. An efficient new assay for measuring zebrafish anxiety: Tall tanks that better characterize between-individual differences. J Neurosci Methods 2021; 356:109138. [PMID: 33753125 DOI: 10.1016/j.jneumeth.2021.109138] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Revised: 01/31/2021] [Accepted: 03/11/2021] [Indexed: 12/14/2022]
Abstract
BACKGROUND Zebrafish (Danio rerio) are increasingly being used to model anxiety. A common behavioral assay employed for assessing anxiety-like behaviors in zebrafish is the "novel tank test". We hypothesized that using deeper tanks in this test would result in greater between-individual variation in behavioral responses and a more 'repeatable' assay. NEW METHODS After mapping the literature and identifying common behavioral parameters used in analysis, we performed novel tank anxiety tests in both custom-designed 'tall' tanks with increased depth and 'short' trapezoidal tanks. We compared the repeatability of the behavioral parameters between tall and short tanks and also investigated sex differences. RESULTS Overall, regardless of tank depth, almost all behavioral parameters associated with anxiety in zebrafish were significantly repeatable (R = 0.24 to 0.60). Importantly, our tall tanks better captured between-individual differences, resulting in higher repeatability estimates (average repeatability tall tanks: R = 0.46; average repeatability short tanks: R = 0.36) and clearer sex differences. CONCLUSIONS Our assay using tall tanks has advantages over tests based on short tanks which underestimate repeatability. We argue that use of deeper tanks will improve the reliability of behavioral data across studies using novel tank tests for zebrafish. Our results also call for increased attention in designing the most appropriate assay in biomedical and behavioral sciences as current methods may lack the sensitivity to detect subtle, yet important, information, such as between-individual variation, an important component in assessing the reliability of behavioral data.
Collapse
Affiliation(s)
- Hamza Anwer
- Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, 2052, Australia; Diabetes and Metabolism Division, Garvan Institute of Medical Research, Darlinghurst, Sydney, NSW, 2010, Australia.
| | - Dominic Mason
- Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, 2052, Australia; Diabetes and Metabolism Division, Garvan Institute of Medical Research, Darlinghurst, Sydney, NSW, 2010, Australia
| | - Susanne Zajitschek
- Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, 2052, Australia; Diabetes and Metabolism Division, Garvan Institute of Medical Research, Darlinghurst, Sydney, NSW, 2010, Australia; Liverpool John Moores University, School of Biological and Environmental Sciences, Liverpool, L3 3 AF, United Kingdom
| | - Daniel W A Noble
- Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, 2052, Australia; Division of Ecology and Evolution, Research School of Biology, The Australian National University, Canberra, ACT, 0200, Australia
| | - Daniel Hesselson
- Diabetes and Metabolism Division, Garvan Institute of Medical Research, Darlinghurst, Sydney, NSW, 2010, Australia; Centenary Institute and Faculty of Medicine and Health, University of Sydney, Sydney, NSW, 2006, Australia
| | - Margaret J Morris
- Department of Pharmacology, School of Medical Sciences, University of New South Wales, Sydney, NSW, 2052, Australia
| | - Malgorzata Lagisz
- Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, 2052, Australia; Diabetes and Metabolism Division, Garvan Institute of Medical Research, Darlinghurst, Sydney, NSW, 2010, Australia
| | - Shinichi Nakagawa
- Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, 2052, Australia; Diabetes and Metabolism Division, Garvan Institute of Medical Research, Darlinghurst, Sydney, NSW, 2010, Australia
| |
Collapse
|
6
|
Behavioural repeatability in Sardinian warblers (Sylvia melanocephala): larger individuals are more aggressive. Acta Ethol 2021. [DOI: 10.1007/s10211-020-00358-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
7
|
MacPherson B, Scott R, Gras R. Sex and recombination purge the genome of deleterious alleles: An Individual Based Modeling Approach. ECOLOGICAL COMPLEXITY 2021. [DOI: 10.1016/j.ecocom.2021.100910] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
8
|
Budaev S, Kristiansen TS, Giske J, Eliassen S. Computational animal welfare: towards cognitive architecture models of animal sentience, emotion and wellbeing. ROYAL SOCIETY OPEN SCIENCE 2020; 7:201886. [PMID: 33489298 PMCID: PMC7813262 DOI: 10.1098/rsos.201886] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Accepted: 12/04/2020] [Indexed: 05/08/2023]
Abstract
To understand animal wellbeing, we need to consider subjective phenomena and sentience. This is challenging, since these properties are private and cannot be observed directly. Certain motivations, emotions and related internal states can be inferred in animals through experiments that involve choice, learning, generalization and decision-making. Yet, even though there is significant progress in elucidating the neurobiology of human consciousness, animal consciousness is still a mystery. We propose that computational animal welfare science emerges at the intersection of animal behaviour, welfare and computational cognition. By using ideas from cognitive science, we develop a functional and generic definition of subjective phenomena as any process or state of the organism that exists from the first-person perspective and cannot be isolated from the animal subject. We then outline a general cognitive architecture to model simple forms of subjective processes and sentience. This includes evolutionary adaptation which contains top-down attention modulation, predictive processing and subjective simulation by re-entrant (recursive) computations. Thereafter, we show how this approach uses major characteristics of the subjective experience: elementary self-awareness, global workspace and qualia with unity and continuity. This provides a formal framework for process-based modelling of animal needs, subjective states, sentience and wellbeing.
Collapse
Affiliation(s)
- Sergey Budaev
- Department of Biological Sciences, University of Bergen, PO Box 7803, 5020 Bergen, Norway
| | - Tore S. Kristiansen
- Research Group Animal Welfare, Institute of Marine Research, PO Box 1870, 5817 Bergen, Norway
| | - Jarl Giske
- Department of Biological Sciences, University of Bergen, PO Box 7803, 5020 Bergen, Norway
| | - Sigrunn Eliassen
- Department of Biological Sciences, University of Bergen, PO Box 7803, 5020 Bergen, Norway
| |
Collapse
|
9
|
Lamb SD, Chia JHZ, Johnson SL. Paternal exposure to a common herbicide alters the behavior and serotonergic system of zebrafish offspring. PLoS One 2020; 15:e0228357. [PMID: 32275662 PMCID: PMC7147785 DOI: 10.1371/journal.pone.0228357] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2019] [Accepted: 01/13/2020] [Indexed: 12/18/2022] Open
Abstract
Increasingly, studies are revealing that endocrine disrupting chemicals (EDCs) can alter animal behavior. Early life exposure to EDCs may permanently alter phenotypes through to adulthood. In addition, the effects of EDCs may not be isolated to a single generation − offspring may indirectly be impacted, via non-genetic processes. Here, we analyzed the effects of paternal atrazine exposure on behavioral traits (distance moved, exploration, bottom-dwelling time, latency to enter the top zone, and interaction with a mirror) and whole-brain mRNA of genes involved in the serotonergic system regulation (slc6a4a, slc6a4b, htr1Aa, htr1B, htr2B) of zebrafish (Danio rerio). F0 male zebraFIsh were exposed to atrazine at 0.3, 3 or 30 part per billion (ppb) during early juvenile development, the behavior of F1 progeny was tested at adulthood, and the effect of 0.3 ppb atrazine treatment on mRNA transcription was quantified. Paternal exposure to atrazine significantly reduced interactions with a mirror (a proxy for aggression) and altered the latency to enter the top zone of a tank in unexposed F1 offspring. Bottom-dwelling time (a proxy for anxiety) also appeared to be somewhat affected, and activity (distance moved) was reduced in the context of aggression. slc6a4a and htr1Aa mRNA transcript levels were found to correlate positively with anxiety levels in controls, but we found that this relationship was disrupted in the 0.3 ppb atrazine treatment group. Overall, paternal atrazine exposure resulted in alterations across a variety of behavioral traits and showed signs of serotonergic system dysregulation, demonstrating intergenerational effects. Further research is needed to explore transgenerational effects on behavior and possible mechanisms underpinning behavioral effects.
Collapse
Affiliation(s)
- Simon D. Lamb
- Department of Zoology, University of Otago, Dunedin, Otago, New Zealand
- * E-mail: (SDL); (SLJ)
| | - Jolyn H. Z. Chia
- Department of Zoology, University of Otago, Dunedin, Otago, New Zealand
| | - Sheri L. Johnson
- Department of Zoology, University of Otago, Dunedin, Otago, New Zealand
- * E-mail: (SDL); (SLJ)
| |
Collapse
|
10
|
Bhattacharjee S, MacPherson B, Wang RF, Gras R. Animal communication of fear and safety related to foraging behavior and fitness: An individual-based modeling approach. ECOL INFORM 2019. [DOI: 10.1016/j.ecoinf.2019.101011] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
11
|
Budaev S, Jørgensen C, Mangel M, Eliassen S, Giske J. Decision-Making From the Animal Perspective: Bridging Ecology and Subjective Cognition. Front Ecol Evol 2019. [DOI: 10.3389/fevo.2019.00164] [Citation(s) in RCA: 33] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
|