1
|
Rasanan AHH, Evans NJ, Fontanesi L, Manning C, Huang-Pollock C, Matzke D, Heathcote A, Rieskamp J, Speekenbrink M, Frank MJ, Palminteri S, Lucas CG, Busemeyer JR, Ratcliff R, Rad JA. Beyond discrete-choice options. Trends Cogn Sci 2024:S1364-6613(24)00175-X. [PMID: 39138030 DOI: 10.1016/j.tics.2024.07.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Revised: 07/12/2024] [Accepted: 07/14/2024] [Indexed: 08/15/2024]
Abstract
While decision theories have evolved over the past five decades, their focus has largely been on choices among a limited number of discrete options, even though many real-world situations have a continuous-option space. Recently, theories have attempted to address decisions with continuous-option spaces, and several computational models have been proposed within the sequential sampling framework to explain how we make a decision in continuous-option space. This article aims to review the main attempts to understand decisions on continuous-option spaces, give an overview of applications of these types of decisions, and present puzzles to be addressed by future developments.
Collapse
Affiliation(s)
| | - Nathan J Evans
- School of Psychology, The University of Queensland, St Lucia, QLD 4072, Australia; Department of Psychology, Ludwig Maximilian University of Munich, Munich, Germany
| | - Laura Fontanesi
- Department of Psychology, University of Basel, Missionsstrasse 62A, 4055, Basel, Switzerland
| | | | | | - Dora Matzke
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
| | - Andrew Heathcote
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands; School of Psychological Sciences, University of Newcastle, Newcastle, Australia
| | - Jörg Rieskamp
- Department of Psychology, University of Basel, Missionsstrasse 62A, 4055, Basel, Switzerland
| | | | - Michael J Frank
- Department of Cognitive, Linguistic, and Psychological Sciences and Carney Institute for Brain Science, Brown University, Providence, RI, USA
| | - Stefano Palminteri
- Laboratoire de Neurosciences Cognitives Computationnelles, Institut National de la Santé et Recherche Médicale, Paris, France; Département d'Etudes Cognitives, Ecole Normale Supérieure, PSL Research University, Paris, France
| | | | - Jerome R Busemeyer
- Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN 47405, USA
| | - Roger Ratcliff
- The Ohio State University, 1835 Neil Avenue, Columbus, OH, 43210, USA
| | - Jamal Amani Rad
- Choice Modelling Centre and Institute for Transport Studies, University of Leeds, Leeds LS2 9JT, UK.
| |
Collapse
|
2
|
Fine JM, Yoo SBM, Hayden BY. Control over a mixture of policies determines change of mind topology during continuous choice. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.18.590154. [PMID: 38712284 PMCID: PMC11071291 DOI: 10.1101/2024.04.18.590154] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
Behavior is naturally organized into categorically distinct states with corresponding patterns of neural activity; how does the brain control those states? We propose that states are regulated by specific neural processes that implement meta-control that can blend simpler control processes. To test this hypothesis, we recorded from neurons in the dorsal anterior cingulate cortex (dACC) and dorsal premotor cortex (PMd) while macaques performed a continuous pursuit task with two moving prey that followed evasive strategies. We used a novel control theoretic approach to infer subjects' moment-to-moment latent control variables, which in turn dictated their blend of distinct identifiable control processes. We identified low-dimensional subspaces in neuronal responses that reflected the current strategy, the value of the pursued target, and the relative value of the two targets. The top two principal components of activity tracked changes of mind in abstract and change-type-specific formats, respectively. These results indicate that control of behavioral state reflects the interaction of brain processes found in dorsal prefrontal regions that implement a mixture over low-level control policies.
Collapse
|
3
|
Thomas T, Straub D, Tatai F, Shene M, Tosik T, Kersting K, Rothkopf CA. Modelling dataset bias in machine-learned theories of economic decision-making. Nat Hum Behav 2024; 8:679-691. [PMID: 38216691 PMCID: PMC11045447 DOI: 10.1038/s41562-023-01784-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Accepted: 11/14/2023] [Indexed: 01/14/2024]
Abstract
Normative and descriptive models have long vied to explain and predict human risky choices, such as those between goods or gambles. A recent study reported the discovery of a new, more accurate model of human decision-making by training neural networks on a new online large-scale dataset, choices13k. Here we systematically analyse the relationships between several models and datasets using machine-learning methods and find evidence for dataset bias. Because participants' choices in stochastically dominated gambles were consistently skewed towards equipreference in the choices13k dataset, we hypothesized that this reflected increased decision noise. Indeed, a probabilistic generative model adding structured decision noise to a neural network trained on data from a laboratory study transferred best, that is, outperformed all models apart from those trained on choices13k. We conclude that a careful combination of theory and data analysis is still required to understand the complex interactions of machine-learning models and data of human risky choices.
Collapse
Affiliation(s)
- Tobias Thomas
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany.
- Hessian Center for Artificial Intelligence, Darmstadt, Germany.
| | - Dominik Straub
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany
| | - Fabian Tatai
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany
| | - Megan Shene
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany
| | - Tümer Tosik
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany
| | - Kristian Kersting
- Hessian Center for Artificial Intelligence, Darmstadt, Germany
- Centre for Cognitive Science and Computer Science Department, Technical University of Darmstadt, Darmstadt, Germany
| | - Constantin A Rothkopf
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany
- Hessian Center for Artificial Intelligence, Darmstadt, Germany
| |
Collapse
|
4
|
Webb J, Steffan P, Hayden BY, Lee D, Kemere C, McGinley M. Foraging Under Uncertainty Follows the Marginal Value Theorem with Bayesian Updating of Environment Representations. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.03.30.587253. [PMID: 38585964 PMCID: PMC10996644 DOI: 10.1101/2024.03.30.587253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/09/2024]
Abstract
Foraging theory has been a remarkably successful approach to understanding the behavior of animals in many contexts. In patch-based foraging contexts, the marginal value theorem (MVT) shows that the optimal strategy is to leave a patch when the marginal rate of return declines to the average for the environment. However, the MVT is only valid in deterministic environments whose statistics are known to the forager; naturalistic environments seldom meet these strict requirements. As a result, the strategies used by foragers in naturalistic environments must be empirically investigated. We developed a novel behavioral task and a corresponding computational framework for studying patch-leaving decisions in head-fixed and freely moving mice. We varied between-patch travel time, as well as within-patch reward depletion rate, both deterministically and stochastically. We found that mice adopt patch residence times in a manner consistent with the MVT and not explainable by simple ethologically motivated heuristic strategies. Critically, behavior was best accounted for by a modified form of the MVT wherein environment representations were updated based on local variations in reward timing, captured by a Bayesian estimator and dynamic prior. Thus, we show that mice can strategically attend to, learn from, and exploit task structure on multiple timescales simultaneously, thereby efficiently foraging in volatile environments. The results provide a foundation for applying the systems neuroscience toolkit in freely moving and head-fixed mice to understand the neural basis of foraging under uncertainty.
Collapse
Affiliation(s)
- James Webb
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
- Jan and Dan Duncan Neurological Research Institute, Texas Children’s Hospital, Houston, TX, USA
| | - Paul Steffan
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
| | - Benjamin Y. Hayden
- Department of Neurosurgery, Baylor College of Medicine, Houston, TX, USA
| | - Daeyeol Lee
- The Zanvyl Krieger Mind/Brain Institute, The Solomon H Snyder Department of Neuroscience, Department of Psychological and Brain Sciences, Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD, USA
| | - Caleb Kemere
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, USA
| | - Matthew McGinley
- Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA
- Jan and Dan Duncan Neurological Research Institute, Texas Children’s Hospital, Houston, TX, USA
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, USA
| |
Collapse
|
5
|
Coudiere A, de Rugy A, Danion FR. Right-left hand asymmetry in manual tracking: when poorer control is associated with better adaptation and interlimb transfer. PSYCHOLOGICAL RESEARCH 2024; 88:594-606. [PMID: 37466674 DOI: 10.1007/s00426-023-01858-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 07/10/2023] [Indexed: 07/20/2023]
Abstract
To date, interlimb transfer following visuomotor adaptation has been mainly investigated through discrete reaching movements. Here we explored this issue in the context of continuous manual tracking, a task in which the contribution of online feedback mechanisms is crucial, and in which there is a well-established right (dominant) hand advantage under baseline conditions. We had two objectives (1) to determine whether this preexisting hand asymmetry would persist under visuomotor rotation, (2) to examine interlimb transfer by assessing whether prior experience with the rotation by one hand benefit to the other hand. To address these, 44 right-handed participants were asked to move a joystick and to track a visual target following a rather unpredictable trajectory. Visuomotor adaptation was elicited by introducing a 90° rotation between the joystick motion and the cursor motion. Half of the participants adapted to the rotation first with the right hand, and then with the left, while the other half performed the opposite protocol. As expected during baseline trials, the left hand was less accurate while also exhibiting more variable and exploratory behavior. However, participants exhibited a left hand advantage during first exposure to the rotation. Moreover, interlimb transfer was observed albeit more strongly from the left to the right hand. We suggest that the less effective and more variable/exploratory control strategy of the left hand promoted its adaptation, which incidentally favored transfer from left to right hand. Altogether, this study speaks for further attention to the dominant/non-dominant asymmetry during baseline before examining interlimb transfer of adaptation.
Collapse
Affiliation(s)
- Adrien Coudiere
- CNRS, Université de Poitiers, Université de Tours, CeRCA, UMR 7295, Poitiers, France
| | - Aymar de Rugy
- Université de Bordeaux, CNRS, Institut de Neurosciences Cognitives et Intégratives d'Aquitaine, UMR 5287, Bordeaux, France
| | - Frederic R Danion
- CNRS, Université de Poitiers, Université de Tours, CeRCA, UMR 7295, Poitiers, France.
| |
Collapse
|
6
|
Pearson J, Kumar A. Sophisticated models, minimum descriptions, and the Goldilocks zone of behavior: Comment on "Beyond simple laboratory studies: Developing sophisticated models to study rich behavior" by Maselli et al. Phys Life Rev 2023; 47:137-138. [PMID: 37898103 DOI: 10.1016/j.plrev.2023.10.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Accepted: 10/17/2023] [Indexed: 10/30/2023]
Affiliation(s)
- John Pearson
- Department of Neurobiology, Duke University, Durham NC 27710, USA; Department of Biostatistics & Bioinformatics, Duke University, Durham NC 27710, USA; Department of Psychology & Neuroscience, Duke University, Durham NC 27710, USA; Department of Electrical & Computer Engineering, Duke University, Durham NC 27710, USA.
| | - Achint Kumar
- Department of Physics, Duke University, Durham NC 27710, USA
| |
Collapse
|
7
|
Tao L, Wechsler SP, Bhandawat V. Sensorimotor transformation underlying odor-modulated locomotion in walking Drosophila. Nat Commun 2023; 14:6818. [PMID: 37884581 PMCID: PMC10603174 DOI: 10.1038/s41467-023-42613-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Accepted: 10/17/2023] [Indexed: 10/28/2023] Open
Abstract
Most real-world behaviors - such as odor-guided locomotion - are performed with incomplete information. Activity in olfactory receptor neuron (ORN) classes provides information about odor identity but not the location of its source. In this study, we investigate the sensorimotor transformation that relates ORN activation to locomotion changes in Drosophila by optogenetically activating different combinations of ORN classes and measuring the resulting changes in locomotion. Three features describe this sensorimotor transformation: First, locomotion depends on both the instantaneous firing frequency (f) and its change (df); the two together serve as a short-term memory that allows the fly to adapt its motor program to sensory context automatically. Second, the mapping between (f, df) and locomotor parameters such as speed or curvature is distinct for each pattern of activated ORNs. Finally, the sensorimotor mapping changes with time after odor exposure, allowing information integration over a longer timescale.
Collapse
Affiliation(s)
- Liangyu Tao
- School of Biomedical Engineering and Health Sciences, Drexel University, Philadelphia, PA, USA
| | - Samuel P Wechsler
- School of Biomedical Engineering and Health Sciences, Drexel University, Philadelphia, PA, USA
- Department of Neurobiology and Anatomy, Drexel University, Philadelphia, PA, USA
| | - Vikas Bhandawat
- School of Biomedical Engineering and Health Sciences, Drexel University, Philadelphia, PA, USA.
| |
Collapse
|
8
|
Garcia M, Gupta S, Wikenheiser AM. Sex differences in patch-leaving foraging decisions in rats. OXFORD OPEN NEUROSCIENCE 2023; 2:kvad011. [PMID: 38596244 PMCID: PMC11003400 DOI: 10.1093/oons/kvad011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 10/11/2023] [Accepted: 10/12/2023] [Indexed: 04/11/2024]
Abstract
The ubiquity, importance, and sophistication of foraging behavior makes it an ideal platform for studying naturalistic decision making in animals. We developed a spatial patch-foraging task for rats, in which subjects chose how long to remain in one foraging patch as the rate of food earnings steadily decreased. The cost of seeking out a new location was varied across sessions. The behavioral task was designed to mimic the structure of natural foraging problems, where distinct spatial locations are associated with different reward statistics, and decisions require navigation and movement through space. Male and female Long-Evans rats generally followed the predictions of theoretical models of foraging, albeit with a consistent tendency to persist with patches for too long compared to behavioral strategies that maximize food intake rate. The tendency to choose overly-long patch residence times was stronger in male rats. We also observed sex differences in locomotion as rats performed the task, but these differences in movement only partially accounted for the differences in patch residence durations observed between male and female rats. Together, these results suggest a nuanced relationship between movement, sex, and foraging decisions.
Collapse
Affiliation(s)
- Marissa Garcia
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA 90095, USA
| | - Sukriti Gupta
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA 90095, USA
| | - Andrew M Wikenheiser
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA 90095, USA
- Brain Research Institute, University of California, Los Angeles, Los Angeles, CA 90095, USA
| |
Collapse
|
9
|
Garcia M, Gupta S, Wikenheiser AM. Sex differences in patch-leaving foraging decisions in rats. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.02.19.529135. [PMID: 36824852 PMCID: PMC9949151 DOI: 10.1101/2023.02.19.529135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/21/2023]
Abstract
The ubiquity, importance, and sophistication of foraging behavior makes it an ideal platform for studying naturalistic decision making in animals. We developed a spatial patch-foraging task for rats, in which subjects chose how long to remain in one foraging patch as the rate of food earnings steadily decreased. The cost of seeking out a new location was varied across sessions. The behavioral task was designed to mimic the structure of natural foraging problems, where distinct spatial locations are associated with different reward statistics, and decisions require navigation and movement through space. Male and female Long-Evans rats generally followed the predictions of theoretical models of foraging, albeit with a consistent tendency to persist with patches for too long compared to behavioral strategies that maximize food intake rate. The tendency to choose overly-long patch residence times was stronger in male rats. We also observed sex differences in locomotion as rats performed the task, but these differences in movement only partially accounted for the differences in patch residence durations observed between male and female rats. Together, these results suggest a nuanced relationship between movement, sex, and foraging decisions.
Collapse
Affiliation(s)
- Marissa Garcia
- Department of Psychology, University of California, Los Angeles, Los Angeles, California 90095
- Current address: Neurosciences Graduate Program, University of California, San Diego, San Diego, CA 92093
| | - Sukriti Gupta
- Department of Psychology, University of California, Los Angeles, Los Angeles, California 90095
| | - Andrew M. Wikenheiser
- Department of Psychology, University of California, Los Angeles, Los Angeles, California 90095
- Brain Research Institute, University of California, Los Angeles, Los Angeles, California 90095
| |
Collapse
|
10
|
Drew T, Fortier-Lebel N, Nakajima T. Cortical contribution to visuomotor coordination in locomotion and reaching. Curr Opin Neurobiol 2023; 82:102755. [PMID: 37633106 DOI: 10.1016/j.conb.2023.102755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Revised: 07/10/2023] [Accepted: 07/10/2023] [Indexed: 08/28/2023]
Abstract
One of the hallmarks of mammals is their ability to make precise visually guided limb movements to attain objects. This is best exemplified by the reach and grasp movements of primates, although it is not unique to this mammalian order. Precise, coordinated, visually guided movements are equally as important during locomotion in many mammalian species, especially in predators. In this context, vision is used to guide paw trajectory and placement. In this review we examine the contribution of the fronto-parietal network in the control of such movements. We suggest that this network is responsible for visuomotor coordination across behaviours and species. We further argue for analogies between cytoarchitectonically similar cortical areas in primates and cats.
Collapse
Affiliation(s)
- Trevor Drew
- Département de Neurosciences, Centre Interdisciplinaire de Recherche sur le Cerveau et l'Apprentissage (CIRCA), Groupe de recherche sur la signalisation neurale et la circuiterie (SNC), Université de Montréal, Pavillon Paul-G. Desmarais, C.P. 6128, Succursale Centre-ville, Montréal, Québec, H3C 3J7, Canada.
| | - Nicolas Fortier-Lebel
- Département de Neurosciences, Centre Interdisciplinaire de Recherche sur le Cerveau et l'Apprentissage (CIRCA), Groupe de recherche sur la signalisation neurale et la circuiterie (SNC), Université de Montréal, Pavillon Paul-G. Desmarais, C.P. 6128, Succursale Centre-ville, Montréal, Québec, H3C 3J7, Canada
| | - Toshi Nakajima
- Department of Integrative Neuroscience, Graduate School of Medicine and Pharmaceutical Sciences, University of Toyama, 2630 Sugitani, Toyama, 930-0194, Japan
| |
Collapse
|
11
|
Voloh B, Maisson DJN, Cervera RL, Conover I, Zambre M, Hayden B, Zimmermann J. Hierarchical action encoding in prefrontal cortex of freely moving macaques. Cell Rep 2023; 42:113091. [PMID: 37656619 PMCID: PMC10591875 DOI: 10.1016/j.celrep.2023.113091] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 06/23/2023] [Accepted: 08/18/2023] [Indexed: 09/03/2023] Open
Abstract
Our natural behavioral repertoires include coordinated actions of characteristic types. To better understand how neural activity relates to the expression of actions and action switches, we studied macaques performing a freely moving foraging task in an open environment. We developed a novel analysis pipeline that can identify meaningful units of behavior, corresponding to recognizable actions such as sitting, walking, jumping, and climbing. On the basis of transition probabilities between these actions, we found that behavior is organized in a modular and hierarchical fashion. We found that, after regressing out many potential confounders, actions are associated with specific patterns of firing in each of six prefrontal brain regions and that, overall, encoding of action category is progressively stronger in more dorsal and more caudal prefrontal regions. Together, these results establish a link between selection of units of primate behavior on one hand and neuronal activity in prefrontal regions on the other.
Collapse
Affiliation(s)
- Benjamin Voloh
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | - David J-N Maisson
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | | | - Indirah Conover
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | - Mrunal Zambre
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | - Benjamin Hayden
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA; Department of Neurosurgery, Baylor College of Medicine, Houston, TX 77030, USA
| | - Jan Zimmermann
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA; Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455, USA.
| |
Collapse
|
12
|
Jerjian SJ, Harsch DR, Fetsch CR. Self-motion perception and sequential decision-making: where are we heading? Philos Trans R Soc Lond B Biol Sci 2023; 378:20220333. [PMID: 37545301 PMCID: PMC10404932 DOI: 10.1098/rstb.2022.0333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 06/18/2023] [Indexed: 08/08/2023] Open
Abstract
To navigate and guide adaptive behaviour in a dynamic environment, animals must accurately estimate their own motion relative to the external world. This is a fundamentally multisensory process involving integration of visual, vestibular and kinesthetic inputs. Ideal observer models, paired with careful neurophysiological investigation, helped to reveal how visual and vestibular signals are combined to support perception of linear self-motion direction, or heading. Recent work has extended these findings by emphasizing the dimension of time, both with regard to stimulus dynamics and the trade-off between speed and accuracy. Both time and certainty-i.e. the degree of confidence in a multisensory decision-are essential to the ecological goals of the system: terminating a decision process is necessary for timely action, and predicting one's accuracy is critical for making multiple decisions in a sequence, as in navigation. Here, we summarize a leading model for multisensory decision-making, then show how the model can be extended to study confidence in heading discrimination. Lastly, we preview ongoing efforts to bridge self-motion perception and navigation per se, including closed-loop virtual reality and active self-motion. The design of unconstrained, ethologically inspired tasks, accompanied by large-scale neural recordings, raise promise for a deeper understanding of spatial perception and decision-making in the behaving animal. This article is part of the theme issue 'Decision and control processes in multisensory perception'.
Collapse
Affiliation(s)
- Steven J. Jerjian
- Solomon H. Snyder Department of Neuroscience, Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Devin R. Harsch
- Solomon H. Snyder Department of Neuroscience, Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD 21218, USA
- Center for Neuroscience and Department of Neurobiology, University of Pittsburgh, Pittsburgh, PA 15213, USA
| | - Christopher R. Fetsch
- Solomon H. Snyder Department of Neuroscience, Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD 21218, USA
| |
Collapse
|
13
|
Johnston WJ, Fine JM, Yoo SBM, Ebitz RB, Hayden BY. Semi-orthogonal subspaces for value mediate a tradeoff between binding and generalization. ARXIV 2023:arXiv:2309.07766v1. [PMID: 37744462 PMCID: PMC10516109] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 09/26/2023]
Abstract
When choosing between options, we must associate their values with the action needed to select them. We hypothesize that the brain solves this binding problem through neural population subspaces. To test this hypothesis, we examined neuronal responses in five reward-sensitive regions in macaques performing a risky choice task with sequential offers. Surprisingly, in all areas, the neural population encoded the values of offers presented on the left and right in distinct subspaces. We show that the encoding we observe is sufficient to bind the values of the offers to their respective positions in space while preserving abstract value information, which may be important for rapid learning and generalization to novel contexts. Moreover, after both offers have been presented, all areas encode the value of the first and second offers in orthogonal subspaces. In this case as well, the orthogonalization provides binding. Our binding-by-subspace hypothesis makes two novel predictions borne out by the data. First, behavioral errors should correlate with putative spatial (but not temporal) misbinding in the neural representation. Second, the specific representational geometry that we observe across animals also indicates that behavioral errors should increase when offers have low or high values, compared to when they have medium values, even when controlling for value difference. Together, these results support the idea that the brain makes use of semi-orthogonal subspaces to bind features together.
Collapse
Affiliation(s)
- W. Jeffrey Johnston
- Center for Theoretical Neuroscience and Mortimer B. Zuckerman Mind, Brain, and Behavior Institute, Columbia University, New York, New York, United States of America
| | - Justin M. Fine
- Department of Neurosurgery, Baylor College of Medicine, Houston, Texas, United States of America
| | - Seng Bum Michael Yoo
- Department of Biomedical Engineering, Sunkyunkwan University, and Center for Neuroscience Imaging Research, Institute of Basic Sciences, Suwon, South Korea, Republic of Korea, 16419
| | - R. Becket Ebitz
- Department of Neuroscience, Université de Montréal, Montréal, Quebec, Canada
| | - Benjamin Y. Hayden
- Department of Neurosurgery, Baylor College of Medicine, Houston, Texas, United States of America
| |
Collapse
|
14
|
Maselli A, Gordon J, Eluchans M, Lancia GL, Thiery T, Moretti R, Cisek P, Pezzulo G. Beyond simple laboratory studies: Developing sophisticated models to study rich behavior. Phys Life Rev 2023; 46:220-244. [PMID: 37499620 DOI: 10.1016/j.plrev.2023.07.006] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Accepted: 07/06/2023] [Indexed: 07/29/2023]
Abstract
Psychology and neuroscience are concerned with the study of behavior, of internal cognitive processes, and their neural foundations. However, most laboratory studies use constrained experimental settings that greatly limit the range of behaviors that can be expressed. While focusing on restricted settings ensures methodological control, it risks impoverishing the object of study: by restricting behavior, we might miss key aspects of cognitive and neural functions. In this article, we argue that psychology and neuroscience should increasingly adopt innovative experimental designs, measurement methods, analysis techniques and sophisticated computational models to probe rich, ecologically valid forms of behavior, including social behavior. We discuss the challenges of studying rich forms of behavior as well as the novel opportunities offered by state-of-the-art methodologies and new sensing technologies, and we highlight the importance of developing sophisticated formal models. We exemplify our arguments by reviewing some recent streams of research in psychology, neuroscience and other fields (e.g., sports analytics, ethology and robotics) that have addressed rich forms of behavior in a model-based manner. We hope that these "success cases" will encourage psychologists and neuroscientists to extend their toolbox of techniques with sophisticated behavioral models - and to use them to study rich forms of behavior as well as the cognitive and neural processes that they engage.
Collapse
Affiliation(s)
- Antonella Maselli
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Jeremy Gordon
- University of California, Berkeley, Berkeley, CA, 94704, United States
| | - Mattia Eluchans
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Gian Luca Lancia
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Thomas Thiery
- Department of Psychology, University of Montréal, Montréal, Québec, Canada
| | - Riccardo Moretti
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Paul Cisek
- Department of Neuroscience, University of Montréal, Montréal, Québec, Canada
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| |
Collapse
|
15
|
Maisson DJN, Cervera RL, Voloh B, Conover I, Zambre M, Zimmermann J, Hayden BY. Widespread coding of navigational variables in prefrontal cortex. Curr Biol 2023; 33:3478-3488.e3. [PMID: 37541250 PMCID: PMC10984098 DOI: 10.1016/j.cub.2023.07.024] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Revised: 06/01/2023] [Accepted: 07/13/2023] [Indexed: 08/06/2023]
Abstract
To navigate effectively, we must represent information about our location in the environment. Traditional research highlights the role of the hippocampal complex in this process. Spurred by recent research highlighting the widespread cortical encoding of cognitive and motor variables previously thought to have localized function, we hypothesized that navigational variables would be likewise encoded widely, especially in the prefrontal cortex, which is associated with volitional behavior. We recorded neural activity from six prefrontal regions while macaques performed a foraging task in an open enclosure. In all regions, we found strong encoding of allocentric position, allocentric head direction, boundary distance, and linear and angular velocity. These encodings were not accounted for by distance, time to reward, or motor factors. The strength of coding of all variables increased along a ventral-to-dorsal gradient. Together, these results argue that encoding of navigational variables is not localized to the hippocampus and support the hypothesis that navigation is continuous with other forms of flexible cognition in the service of action.
Collapse
Affiliation(s)
- David J-N Maisson
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Roberto Lopez Cervera
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Benjamin Voloh
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Indirah Conover
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Mrunal Zambre
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Jan Zimmermann
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Benjamin Y Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA.
| |
Collapse
|
16
|
Voloh B, Eisenreich BR, Maisson DJN, Ebitz RB, Park HS, Hayden BY, Zimmermann J. Hierarchical organization of rhesus macaque behavior. OXFORD OPEN NEUROSCIENCE 2023; 2:kvad006. [PMID: 37577290 PMCID: PMC10421634 DOI: 10.1093/oons/kvad006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 06/11/2023] [Accepted: 06/12/2023] [Indexed: 08/15/2023]
Abstract
Primatologists, psychologists and neuroscientists have long hypothesized that primate behavior is highly structured. However, delineating that structure has been impossible due to the difficulties of precision behavioral tracking. Here we analyzed a dataset consisting of continuous measures of the 3D position of two male rhesus macaques (Macaca mulatta) performing three different tasks in a large unrestrained environment over several hours. Using an unsupervised embedding approach on the tracked joints, we identified commonly repeated pose patterns, which we call postures. We found that macaques' behavior is characterized by 49 distinct postures, lasting an average of 0.6 seconds. We found evidence that behavior is hierarchically organized, in that transitions between poses tend to occur within larger modules, which correspond to identifiable actions; these actions are further organized hierarchically. Our behavioral decomposition allows us to identify universal (cross-individual and cross-task) and unique (specific to each individual and task) principles of behavior. These results demonstrate the hierarchical nature of primate behavior, provide a method for the automated ethogramming of primate behavior, and provide important constraints on neural models of pose generation.
Collapse
Affiliation(s)
- Benjamin Voloh
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, 1 Baylor Plaza, Houston, TX 77030
| | - Benjamin R Eisenreich
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, 1 Baylor Plaza, Houston, TX 77030
| | - David J-N Maisson
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, 1 Baylor Plaza, Houston, TX 77030
| | - R Becket Ebitz
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, 1 Baylor Plaza, Houston, TX 77030
| | - Hyun Soo Park
- Department of Computer Science and Engineering, University of Minnesota, 40 Church St, Minneapolis, MN 55455, USA
| | - Benjamin Y Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, 1 Baylor Plaza, Houston, TX 77030
| | - Jan Zimmermann
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, 1 Baylor Plaza, Houston, TX 77030
| |
Collapse
|
17
|
Abstract
The parcellation of the primate cerebral cortex into numbered regions, based on cytoarchitecture, has greatly helped neuroscientists in our quest to understand how the brain implements cognition. Nonetheless, these maps provide an unnecessarily constraining view of how we should do functional neuroanatomy. It is time to think more broadly. Doing so will help advance the goal of incorporating ideas about emergentist organization and interactional complexity into neuroscience.
Collapse
|
18
|
Moeller S, Unakafov AM, Fischer J, Gail A, Treue S, Kagan I. Human and macaque pairs employ different coordination strategies in a transparent decision game. eLife 2023; 12:81641. [PMID: 36633125 PMCID: PMC9937648 DOI: 10.7554/elife.81641] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Accepted: 12/28/2022] [Indexed: 01/13/2023] Open
Abstract
Many real-world decisions in social contexts are made while observing a partner's actions. To study dynamic interactions during such decisions, we developed a setup where two agents seated face-to-face to engage in game-theoretical tasks on a shared transparent touchscreen display ('transparent games'). We compared human and macaque pairs in a transparent version of the coordination game 'Bach-or-Stravinsky', which entails a conflict about which of two individually-preferred opposing options to choose to achieve coordination. Most human pairs developed coordinated behavior and adopted dynamic turn-taking to equalize the payoffs. All macaque pairs converged on simpler, static coordination. Remarkably, two animals learned to coordinate dynamically after training with a human confederate. This pair selected the faster agent's preferred option, exhibiting turn-taking behavior that was captured by modeling the visibility of the partner's action before one's own movement. Such competitive turn-taking was unlike the prosocial turn-taking in humans, who equally often initiated switches to and from their preferred option. Thus, the dynamic coordination is not restricted to humans but can occur on the background of different social attitudes and cognitive capacities in rhesus monkeys. Overall, our results illustrate how action visibility promotes the emergence and maintenance of coordination when agents can observe and time their mutual actions.
Collapse
Affiliation(s)
- Sebastian Moeller
- Cognitive Neuroscience Laboratory, German Primate Center – Leibniz Institute for Primate ResearchGöttingenGermany
- Leibniz ScienceCampus Primate CognitionGöttingenGermany
| | - Anton M Unakafov
- Cognitive Neuroscience Laboratory, German Primate Center – Leibniz Institute for Primate ResearchGöttingenGermany
- Leibniz ScienceCampus Primate CognitionGöttingenGermany
- Georg-Elias-Müller-Institute of Psychology, University of GottingenGöttingenGermany
- Max Planck Institute for Dynamics and Self-OrganizationGöttingenGermany
- Campus Institute for Dynamics of Biological NetworksGottingenGermany
| | - Julia Fischer
- Leibniz ScienceCampus Primate CognitionGöttingenGermany
- Cognitive Ethology Laboratory, German Primate Center – Leibniz Institute for Primate ResearchGöttingenGermany
- Department of Primate Cognition, Johann-Friedrich-Blumenbach Institute for Zoology and Anthropology, University of GottingenGöttingenGermany
| | - Alexander Gail
- Cognitive Neuroscience Laboratory, German Primate Center – Leibniz Institute for Primate ResearchGöttingenGermany
- Leibniz ScienceCampus Primate CognitionGöttingenGermany
- Georg-Elias-Müller-Institute of Psychology, University of GottingenGöttingenGermany
- Bernstein Center for Computational NeuroscienceGöttingenGermany
| | - Stefan Treue
- Cognitive Neuroscience Laboratory, German Primate Center – Leibniz Institute for Primate ResearchGöttingenGermany
- Leibniz ScienceCampus Primate CognitionGöttingenGermany
- Georg-Elias-Müller-Institute of Psychology, University of GottingenGöttingenGermany
- Bernstein Center for Computational NeuroscienceGöttingenGermany
| | - Igor Kagan
- Cognitive Neuroscience Laboratory, German Primate Center – Leibniz Institute for Primate ResearchGöttingenGermany
- Leibniz ScienceCampus Primate CognitionGöttingenGermany
| |
Collapse
|
19
|
Humans account for cognitive costs when finding shortcuts: An information-theoretic analysis of navigation. PLoS Comput Biol 2023; 19:e1010829. [PMID: 36608145 PMCID: PMC9851521 DOI: 10.1371/journal.pcbi.1010829] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 01/19/2023] [Accepted: 12/19/2022] [Indexed: 01/09/2023] Open
Abstract
When faced with navigating back somewhere we have been before we might either retrace our steps or seek a shorter path. Both choices have costs. Here, we ask whether it is possible to characterize formally the choice of navigational plans as a bounded rational process that trades off the quality of the plan (e.g., its length) and the cognitive cost required to find and implement it. We analyze the navigation strategies of two groups of people that are firstly trained to follow a "default policy" taking a route in a virtual maze and then asked to navigate to various known goal destinations, either in the way they want ("Go To Goal") or by taking novel shortcuts ("Take Shortcut"). We address these wayfinding problems using InfoRL: an information-theoretic approach that formalizes the cognitive cost of devising a navigational plan, as the informational cost to deviate from a well-learned route (the "default policy"). In InfoRL, optimality refers to finding the best trade-off between route length and the amount of control information required to find it. We report five main findings. First, the navigational strategies automatically identified by InfoRL correspond closely to different routes (optimal or suboptimal) in the virtual reality map, which were annotated by hand in previous research. Second, people deliberate more in places where the value of investing cognitive resources (i.e., relevant goal information) is greater. Third, compared to the group of people who receive the "Go To Goal" instruction, those who receive the "Take Shortcut" instruction find shorter but less optimal solutions, reflecting the intrinsic difficulty of finding optimal shortcuts. Fourth, those who receive the "Go To Goal" instruction modulate flexibly their cognitive resources, depending on the benefits of finding the shortcut. Finally, we found a surprising amount of variability in the choice of navigational strategies and resource investment across participants. Taken together, these results illustrate the benefits of using InfoRL to address navigational planning problems from a bounded rational perspective.
Collapse
|
20
|
Cazares C, Schreiner DC, Valencia ML, Gremel CM. Orbitofrontal cortex populations are differentially recruited to support actions. Curr Biol 2022; 32:4675-4687.e5. [PMID: 36195096 PMCID: PMC9643660 DOI: 10.1016/j.cub.2022.09.022] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2022] [Revised: 08/03/2022] [Accepted: 09/09/2022] [Indexed: 11/09/2022]
Abstract
The ability to use information from one's prior actions is necessary for decision-making. While orbitofrontal cortex (OFC) has been hypothesized as key for inferences made using cue and value-related information, whether OFC populations contribute to the use of information from volitional actions to guide behavior is not clear. Here, we used a self-paced lever-press hold-down task in which mice infer prior lever-press durations to guide subsequent action performance. We show that the activity of genetically identified lateral OFC (lOFC) subpopulations differentially instantiate current and prior action information during ongoing action execution. Transient state-dependent lOFC circuit disruptions of specified subpopulations reduced the encoding of ongoing press durations but did not disrupt the use of prior action information to guide future action performance. In contrast, a chronic functional loss of lOFC circuit activity resulted in increased reliance on recently executed lever-press durations and impaired contingency reversal, suggesting the recruitment of compensatory mechanisms that resulted in repetitive action control. Our results identify a novel role for lOFC in the integration of action information to guide adaptive behavior.
Collapse
Affiliation(s)
- Christian Cazares
- The Neurosciences Graduate Program, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA
| | - Drew C Schreiner
- Department of Psychology, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA
| | - Mariela Lopez Valencia
- Department of Psychology, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA
| | - Christina M Gremel
- The Neurosciences Graduate Program, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA; Department of Psychology, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA.
| |
Collapse
|
21
|
Noel JP, Balzani E, Avila E, Lakshminarasimhan KJ, Bruni S, Alefantis P, Savin C, Angelaki DE. Coding of latent variables in sensory, parietal, and frontal cortices during closed-loop virtual navigation. eLife 2022; 11:80280. [PMID: 36282071 PMCID: PMC9668339 DOI: 10.7554/elife.80280] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2022] [Accepted: 10/24/2022] [Indexed: 11/13/2022] Open
Abstract
We do not understand how neural nodes operate and coordinate within the recurrent action-perception loops that characterize naturalistic self-environment interactions. Here, we record single-unit spiking activity and local field potentials (LFPs) simultaneously from the dorsomedial superior temporal area (MSTd), parietal area 7a, and dorsolateral prefrontal cortex (dlPFC) as monkeys navigate in virtual reality to 'catch fireflies'. This task requires animals to actively sample from a closed-loop virtual environment while concurrently computing continuous latent variables: (i) the distance and angle travelled (i.e., path integration) and (ii) the distance and angle to a memorized firefly location (i.e., a hidden spatial goal). We observed a patterned mixed selectivity, with the prefrontal cortex most prominently coding for latent variables, parietal cortex coding for sensorimotor variables, and MSTd most often coding for eye movements. However, even the traditionally considered sensory area (i.e., MSTd) tracked latent variables, demonstrating path integration and vector coding of hidden spatial goals. Further, global encoding profiles and unit-to-unit coupling (i.e., noise correlations) suggested a functional subnetwork composed by MSTd and dlPFC, and not between these and 7a, as anatomy would suggest. We show that the greater the unit-to-unit coupling between MSTd and dlPFC, the more the animals' gaze position was indicative of the ongoing location of the hidden spatial goal. We suggest this MSTd-dlPFC subnetwork reflects the monkeys' natural and adaptive task strategy wherein they continuously gaze toward the location of the (invisible) target. Together, these results highlight the distributed nature of neural coding during closed action-perception loops and suggest that fine-grain functional subnetworks may be dynamically established to subserve (embodied) task strategies.
Collapse
Affiliation(s)
- Jean-Paul Noel
- Center for Neural Science, New York University, New York City, United States
| | - Edoardo Balzani
- Center for Neural Science, New York University, New York City, United States
| | - Eric Avila
- Center for Neural Science, New York University, New York City, United States
| | - Kaushik J Lakshminarasimhan
- Center for Neural Science, New York University, New York City, United States.,Center for Theoretical Neuroscience, Columbia University, New York, United States
| | - Stefania Bruni
- Center for Neural Science, New York University, New York City, United States
| | - Panos Alefantis
- Center for Neural Science, New York University, New York City, United States
| | - Cristina Savin
- Center for Neural Science, New York University, New York City, United States
| | - Dora E Angelaki
- Center for Neural Science, New York University, New York City, United States
| |
Collapse
|
22
|
Voigt L, Friedrich J, Grove P, Heinrich N, Ittlinger S, Iskra M, Koop L, Michirev A, Sparascio S, Raab M. Advancing judgment and decision-making research in sport psychology by using the body as an informant in embodied choices. ASIAN JOURNAL OF SPORT AND EXERCISE PSYCHOLOGY 2022. [DOI: 10.1016/j.ajsep.2022.09.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/08/2023]
|
23
|
Hayden BY, Park HS, Zimmermann J. Automated pose estimation in primates. Am J Primatol 2022; 84:e23348. [PMID: 34855257 PMCID: PMC9160209 DOI: 10.1002/ajp.23348] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2021] [Revised: 11/05/2021] [Accepted: 11/09/2021] [Indexed: 11/11/2022]
Abstract
Understanding the behavior of primates is important for primatology, for psychology, and for biology more broadly. It is also important for biomedicine, where primates are an important model organism, and whose behavior is often an important variable of interest. Our ability to rigorously quantify behavior has, however, long been limited. On one hand, we can rigorously quantify low-information measures like preference, looking time, and reaction time; on the other, we can use more gestalt measures like behavioral categories tracked via ethogram, but at high cost and with high variability. Recent technological advances have led to a major revolution in behavioral measurement that offers affordable and scalable rigor. Specifically, digital video cameras and automated pose tracking software can provide measures of full-body position (i.e., pose) of primates over time (i.e., behavior) with high spatial and temporal resolution. Pose-tracking technology in turn can be used to infer behavioral states, such as eating, sleeping, and mating. We call this technological approach behavioral imaging. In this review, we situate the behavioral imaging revolution in the history of the study of behavior, argue for investment in and development of analytical and research techniques that can profit from the advent of the era of big behavior, and propose that primate centers and zoos will take on a more central role in relevant fields of research than they have in the past.
Collapse
Affiliation(s)
- Benjamin Y. Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, Department of Biomedical Engineering
| | - Hyun Soo Park
- Department of Computer Science and Engineering, University of Minnesota, Minneapolis MN 55455
| | - Jan Zimmermann
- Department of Neuroscience, Center for Magnetic Resonance Research, Department of Biomedical Engineering
| |
Collapse
|
24
|
Knaebe B, Weiss CC, Zimmermann J, Hayden BY. The Promise of Behavioral Tracking Systems for Advancing Primate Animal Welfare. Animals (Basel) 2022; 12:ani12131648. [PMID: 35804547 PMCID: PMC9265027 DOI: 10.3390/ani12131648] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2022] [Revised: 06/17/2022] [Accepted: 06/21/2022] [Indexed: 11/16/2022] Open
Abstract
Simple Summary Computerized tracking systems for primates and other animals are one of the great inventions of the 21st century. These systems have already revolutionized the study of primatology, psychology, neuroscience, and biomedicine. Less discussed is that they also promise to greatly enhance animal welfare. Their potential benefits include identifying and reducing pain, suffering, and distress in captive populations, improving laboratory animal welfare, and applying our understanding of animal behavior to increase the “natural” behaviors in captive and wild populations, especially those under threat. We are optimistic that these changes will greatly increase the welfare of primates, including those in laboratories, zoos, primate centers, and in the wild. Abstract Recent years have witnessed major advances in the ability of computerized systems to track the positions of animals as they move through large and unconstrained environments. These systems have so far been a great boon in the fields of primatology, psychology, neuroscience, and biomedicine. Here, we discuss the promise of these technologies for animal welfare. Their potential benefits include identifying and reducing pain, suffering, and distress in captive populations, improving laboratory animal welfare within the context of the three Rs of animal research (reduction, refinement, and replacement), and applying our understanding of animal behavior to increase the “natural” behaviors in captive and wild populations facing human impact challenges. We note that these benefits are often incidental to the designed purpose of these tracking systems, a reflection of the fact that animal welfare is not inimical to research progress, but instead, that the aligned interests between basic research and welfare hold great promise for improvements to animal well-being.
Collapse
|
25
|
De Silva D, Alahakoon D. An artificial intelligence life cycle: From conception to production. PATTERNS (NEW YORK, N.Y.) 2022; 3:100489. [PMID: 35755876 PMCID: PMC9214328 DOI: 10.1016/j.patter.2022.100489] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/15/2021] [Revised: 03/05/2022] [Accepted: 03/16/2022] [Indexed: 05/21/2023]
Abstract
This paper presents the "CDAC AI life cycle," a comprehensive life cycle for the design, development, and deployment of artificial intelligence (AI) systems and solutions. It addresses the void of a practical and inclusive approach that spans beyond the technical constructs to also focus on the challenges of risk analysis of AI adoption, transferability of prebuilt models, increasing importance of ethics and governance, and the composition, skills, and knowledge of an AI team required for successful completion. The life cycle is presented as the progression of an AI solution through its distinct phases-design, develop, and deploy-and 19 constituent stages from conception to production as applicable to any AI initiative. This life cycle addresses several critical gaps in the literature where related work on approaches and methodologies are adapted and not designed specifically for AI. A technical and organizational taxonomy that synthesizes the functional value of AI is a further contribution of this article.
Collapse
Affiliation(s)
- Daswin De Silva
- Centre for Data Analytics and Cognition (CDAC), La Trobe University, Bundoora, VIC, Australia
- Corresponding author
| | - Damminda Alahakoon
- Centre for Data Analytics and Cognition (CDAC), La Trobe University, Bundoora, VIC, Australia
| |
Collapse
|
26
|
Schreiner DC, Cazares C, Renteria R, Gremel CM. Information normally considered task-irrelevant drives decision-making and affects premotor circuit recruitment. Nat Commun 2022; 13:2134. [PMID: 35440120 PMCID: PMC9018678 DOI: 10.1038/s41467-022-29807-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Accepted: 03/24/2022] [Indexed: 02/02/2023] Open
Abstract
Decision-making is a continuous and dynamic process with prior experience reflected in and used by the brain to guide adaptive behavior. However, most neurobiological studies constrain behavior and/or analyses to task-related variables, not accounting for the continuous internal and temporal space in which they occur. We show mice rely on information learned through recent and longer-term experience beyond just prior actions and reward - including checking behavior and the passage of time - to guide self-initiated, self-paced, and self-generated actions. These experiences are represented in secondary motor cortex (M2) activity and its projections into dorsal medial striatum (DMS). M2 integrates this information to bias strategy-level decision-making, and DMS projections reflect specific aspects of this recent experience to guide actions. This suggests diverse aspects of experience drive decision-making and its neural representation, and shows premotor corticostriatal circuits are crucial for using selective aspects of experiential information to guide adaptive behavior.
Collapse
Affiliation(s)
- Drew C Schreiner
- Department of Psychology, University of California San Diego, La Jolla, CA, 92093, USA
| | - Christian Cazares
- The Neurosciences Graduate Program, University of California San Diego, La Jolla, CA, 92093, USA
| | - Rafael Renteria
- Department of Psychology, University of California San Diego, La Jolla, CA, 92093, USA
| | - Christina M Gremel
- Department of Psychology, University of California San Diego, La Jolla, CA, 92093, USA.
- The Neurosciences Graduate Program, University of California San Diego, La Jolla, CA, 92093, USA.
| |
Collapse
|
27
|
Frömer R, Shenhav A. Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making. Neurosci Biobehav Rev 2022; 134:104483. [PMID: 34902441 PMCID: PMC8844247 DOI: 10.1016/j.neubiorev.2021.12.006] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2021] [Revised: 12/01/2021] [Accepted: 12/04/2021] [Indexed: 12/26/2022]
Abstract
While often seeming to investigate rather different problems, research into value-based decision making and cognitive control have historically offered parallel insights into how people select thoughts and actions. While the former studies how people weigh costs and benefits to make a decision, the latter studies how they adjust information processing to achieve their goals. Recent work has highlighted ways in which decision-making research can inform our understanding of cognitive control. Here, we provide the complementary perspective: how cognitive control research has informed understanding of decision-making. We highlight three particular areas of research where this critical interchange has occurred: (1) how different types of goals shape the evaluation of choice options, (2) how people use control to adjust the ways they make their decisions, and (3) how people monitor decisions to inform adjustments to control at multiple levels and timescales. We show how adopting this alternate viewpoint offers new insight into the determinants of both decisions and control; provides alternative interpretations for common neuroeconomic findings; and generates fruitful directions for future research.
Collapse
Affiliation(s)
- R Frömer
- Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Science, Brown University, Providence, RI, United States.
| | - A Shenhav
- Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Science, Brown University, Providence, RI, United States.
| |
Collapse
|
28
|
Abstract
The nervous system is a product of evolution. That is, it was constructed through a long series of modifications, within the strong constraints of heredity, and continuously subjected to intense selection pressures. As a result, the organization and functions of the brain are shaped by its history. We believe that this fact, underappreciated in contemporary systems neuroscience, offers an invaluable aid for helping us resolve the brain's mysteries. Indeed, we think that the consideration of evolutionary history ought to take its place alongside other intellectual tools used to understand the brain, such as behavioural experiments, studies of anatomical structure and functional characterization based on recordings of neural activity. In this introduction, we argue for the importance of evolution by highlighting specific examples of ways that evolutionary theory can enhance neuroscience. The rest of the theme issue elaborates this point, emphasizing the conservative nature of neural evolution, the important consequences of specific transitions that occurred in our history, and the ways in which considerations of evolution can shed light on issues ranging from specific mechanisms to fundamental principles of brain organization. This article is part of the theme issue ‘Systems neuroscience through the lens of evolutionary theory’.
Collapse
Affiliation(s)
- Paul Cisek
- Department of Neuroscience, University of Montréal, 2960 chemin de la tour, local 1107 Montréal, QC H3T 1J4 Canada
| | - Benjamin Y Hayden
- Department of Neuroscience, Department of Biomedical Engineering, and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455, USA
| |
Collapse
|
29
|
Fine JM, Hayden BY. The whole prefrontal cortex is premotor cortex. Philos Trans R Soc Lond B Biol Sci 2022; 377:20200524. [PMID: 34957853 PMCID: PMC8710885 DOI: 10.1098/rstb.2020.0524] [Citation(s) in RCA: 38] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2021] [Accepted: 10/01/2021] [Indexed: 11/12/2022] Open
Abstract
We propose that the entirety of the prefrontal cortex (PFC) can be seen as fundamentally premotor in nature. By this, we mean that the PFC consists of an action abstraction hierarchy whose core function is the potentiation and depotentiation of possible action plans at different levels of granularity. We argue that the apex of the hierarchy should revolve around the process of goal-selection, which we posit is inherently a form of optimization over action abstraction. Anatomical and functional evidence supports the idea that this hierarchy originates on the orbital surface of the brain and extends dorsally to motor cortex. Accordingly, our viewpoint positions the orbitofrontal cortex in a key role in the optimization of goal-selection policies, and suggests that its other proposed roles are aspects of this more general function. Our proposed perspective will reframe outstanding questions, open up new areas of inquiry and align theories of prefrontal function with evolutionary principles. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.
Collapse
Affiliation(s)
- Justin M. Fine
- Department of Neuroscience, Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Benjamin Y. Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| |
Collapse
|
30
|
Bernier PM, Mathew J, Danion FR. Composition and decomposition of visuomotor maps during manual tracking. J Neurophysiol 2021; 126:1685-1697. [PMID: 34614368 DOI: 10.1152/jn.00058.2021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Adapting hand movements to changes in our body or the environment is essential for skilled motor behavior, as is the ability to flexibly combine experience gathered in separate contexts. However, it has been shown that when adapting hand movements to two different visuomotor perturbations in succession, interference effects can occur. Here, we investigate whether these interference effects compromise our ability to adapt to the superposition of the two perturbations. Participants tracked with a joystick, a visual target that followed a smooth but an unpredictable trajectory. Four separate groups of participants (total n = 83) completed one block of 50 trials under each of three mappings: one in which the cursor was rotated by 90° (ROTATION), one in which the cursor mimicked the behavior of a mass-spring system (SPRING), and one in which the SPRING and ROTATION mappings were superimposed (SPROT). The order of the blocks differed across groups. Although interference effects were found when switching between SPRING and ROTATION, participants who performed these blocks first performed better in SPROT than participants who had no prior experience with SPRING and ROTATION (i.e., composition). Moreover, participants who started with SPROT exhibited better performance under SPRING and ROTATION than participants who had no prior experience with each of these mappings (i.e., decomposition). Additional analyses confirmed that these effects resulted from components of learning that were specific to the rotational and spring perturbations. These results show that interference effects do not preclude the ability to compose/decompose various forms of visuomotor adaptation.NEW & NOTEWORTHY The ability to compose/decompose task representations is critical for both cognitive and behavioral flexibility. Here, we show that this ability extends to two forms of visuomotor adaptation in which humans have to perform visually guided hand movements. Despite the presence of interference effects when switching between visuomotor maps, we show that participants are able to flexibly compose or decompose knowledge acquired in previous sessions. These results further demonstrate the flexibility of sensorimotor adaptation in humans.
Collapse
Affiliation(s)
- Pierre-Michel Bernier
- Département de Kinanthropologie, Université de Sherbrooke, Sherbrooke, Quebec, Canada
| | - James Mathew
- Institut Neurosci Timone, Aix Marseille Univ, CNRS, INT, Marseille, France.,Institute of Neuroscience, Institute of Communication & Information Technologies, Electronics & Applied Mathematics, Université Catholique de Louvain, Louvain-la-neuve, Belgium
| | - Frederic R Danion
- Institut Neurosci Timone, Aix Marseille Univ, CNRS, INT, Marseille, France.,Center for Research on Cognition and Learning (CERCA) UMR 7295, University of Poitiers, CNRS, Poitiers, France
| |
Collapse
|
31
|
Ebitz RB, Hayden BY. The population doctrine in cognitive neuroscience. Neuron 2021; 109:3055-3068. [PMID: 34416170 PMCID: PMC8725976 DOI: 10.1016/j.neuron.2021.07.011] [Citation(s) in RCA: 69] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2021] [Revised: 07/02/2021] [Accepted: 07/13/2021] [Indexed: 01/08/2023]
Abstract
A major shift is happening within neurophysiology: a population doctrine is drawing level with the single-neuron doctrine that has long dominated the field. Population-level ideas have so far had their greatest impact in motor neuroscience, but they hold great promise for resolving open questions in cognition as well. Here, we codify the population doctrine and survey recent work that leverages this view to specifically probe cognition. Our discussion is organized around five core concepts that provide a foundation for population-level thinking: (1) state spaces, (2) manifolds, (3) coding dimensions, (4) subspaces, and (5) dynamics. The work we review illustrates the progress and promise that population-level thinking holds for cognitive neuroscience-for delivering new insight into attention, working memory, decision-making, executive function, learning, and reward processing.
Collapse
Affiliation(s)
- R Becket Ebitz
- Department of Neurosciences, Faculté de médecine, Université de Montréal, Montréal, QC, Canada.
| | - Benjamin Y Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, and Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
32
|
Value-free reinforcement learning: policy optimization as a minimal model of operant behavior. Curr Opin Behav Sci 2021; 41:114-121. [DOI: 10.1016/j.cobeha.2021.04.020] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
|
33
|
Schreiner DC, Yalcinbas EA, Gremel CM. A Push For Examining Subjective Experience in Value-Based Decision-Making. Curr Opin Behav Sci 2021; 41:45-49. [PMID: 34056054 PMCID: PMC8153361 DOI: 10.1016/j.cobeha.2021.03.020] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
Subjective experience is a powerful contributor to value-based decision-making. Not every decision is the same, nor made in isolation. Rather, decision-making relies on historical information and internal states for adaptive control. Hence, it is inherently continuous with respect to time - one decision or action evolves into the next. However, forays into the neurobiological underpinnings of decision-making have too frequently ignored the contribution of such continuous subjective experience, instead tying circuit activity and brain area involvement to discrete averaged behaviors and task parameters. While much information has been gained through these investigations, recent works have demonstrated the potential for a greater understanding of neural mechanisms when the continuous, experiential nature of behavior is integrated into the investigation. Such integration has important implications for disease states with disordered decision-making such as addiction, where subjective experience is a large contributor to the disorder.
Collapse
Affiliation(s)
- Drew C. Schreiner
- The Department of Psychology, University of California San Diego, La Jolla, CA 92093, USA
| | - Ege A. Yalcinbas
- The Neurosciences Graduate Program, University of California San Diego, La Jolla, CA 92093, USA
| | - Christina M. Gremel
- The Department of Psychology, University of California San Diego, La Jolla, CA 92093, USA
- The Neurosciences Graduate Program, University of California San Diego, La Jolla, CA 92093, USA
| |
Collapse
|
34
|
The road towards understanding embodied decisions. Neurosci Biobehav Rev 2021; 131:722-736. [PMID: 34563562 PMCID: PMC7614807 DOI: 10.1016/j.neubiorev.2021.09.034] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Revised: 09/16/2021] [Accepted: 09/19/2021] [Indexed: 01/05/2023]
Abstract
Most current decision-making research focuses on classical economic scenarios, where choice offers are prespecified and where action dynamics play no role in the decision. However, our brains evolved to deal with different choice situations: "embodied decisions". As examples of embodied decisions, consider a lion that has to decide which gazelle to chase in the savannah or a person who has to select the next stone to jump on when crossing a river. Embodied decision settings raise novel questions, such as how people select from time-varying choice options and how they track the most relevant choice attributes; but they have long remained challenging to study empirically. Here, we summarize recent progress in the study of embodied decisions in sports analytics and experimental psychology. Furthermore, we introduce a formal methodology to identify the relevant dimensions of embodied choices (present and future affordances) and to map them into the attributes of classical economic decisions (probabilities and utilities), hence aligning them. Studying embodied decisions will greatly expand our understanding of what decision-making is.
Collapse
|
35
|
Maisson DJN, Cash-Padgett TV, Wang MZ, Hayden BY, Heilbronner SR, Zimmermann J. Choice-relevant information transformation along a ventrodorsal axis in the medial prefrontal cortex. Nat Commun 2021; 12:4830. [PMID: 34376663 PMCID: PMC8355277 DOI: 10.1038/s41467-021-25219-w] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2020] [Accepted: 07/27/2021] [Indexed: 11/17/2022] Open
Abstract
Choice-relevant brain regions in prefrontal cortex may progressively transform information about options into choices. Here, we examine responses of neurons in four regions of the medial prefrontal cortex as macaques performed two-option risky choices. All four regions encode economic variables in similar proportions and show similar putative signatures of key choice-related computations. We provide evidence to support a gradient of function that proceeds from areas 14 to 25 to 32 to 24. Specifically, we show that decodability of twelve distinct task variables increases along that path, consistent with the idea that regions that are higher in the anatomical hierarchy make choice-relevant variables more separable. We also show progressively longer intrinsic timescales in the same series. Together these results highlight the importance of the medial wall in choice, endorse a specific gradient-based organization, and argue against a modular functional neuroanatomy of choice.
Collapse
Affiliation(s)
- David J-N Maisson
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA.
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA.
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA.
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA.
| | - Tyler V Cash-Padgett
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Maya Z Wang
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Benjamin Y Hayden
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Sarah R Heilbronner
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Jan Zimmermann
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
36
|
Dyson BJ, Asad A. A micro-genesis account of longer-form reinforcement learning in structured and unstructured environments. NPJ SCIENCE OF LEARNING 2021; 6:19. [PMID: 34162885 PMCID: PMC8222288 DOI: 10.1038/s41539-021-00098-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/11/2021] [Accepted: 05/25/2021] [Indexed: 06/13/2023]
Abstract
We explored the possibility that in order for longer-form expressions of reinforcement learning (win-calmness, loss-restlessness) to manifest across tasks, they must first develop because of micro-transactions within tasks. We found no evidence of win-calmness or loss-restlessness when wins could not be maximised (unexploitable opponents), nor when the threat of win minimisation was presented (exploiting opponents), but evidence of win-calmness (but not loss-restlessness) when wins could be maximised (exploitable opponents).
Collapse
Affiliation(s)
- Benjamin James Dyson
- University of Alberta, Edmonton, AB, Canada.
- University of Sussex, Falmer, UK.
- Ryerson University, Toronto, ON, Canada.
| | - Ahad Asad
- University of Alberta, Edmonton, AB, Canada
| |
Collapse
|
37
|
Yoo SBM, Tu JC, Hayden BY. Multicentric tracking of multiple agents by anterior cingulate cortex during pursuit and evasion. Nat Commun 2021; 12:1985. [PMID: 33790275 PMCID: PMC8012621 DOI: 10.1038/s41467-021-22195-z] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2020] [Accepted: 03/01/2021] [Indexed: 12/16/2022] Open
Abstract
Successful pursuit and evasion require rapid and precise coordination of navigation with adaptive motor control. We hypothesize that the dorsal anterior cingulate cortex (dACC), which communicates bidirectionally with both the hippocampal complex and premotor/motor areas, would serve a mapping role in this process. We recorded responses of dACC ensembles in two macaques performing a joystick-controlled continuous pursuit/evasion task. We find that dACC carries two sets of signals, (1) world-centric variables that together form a representation of the position and velocity of all relevant agents (self, prey, and predator) in the virtual world, and (2) avatar-centric variables, i.e. self-prey distance and angle. Both sets of variables are multiplexed within an overlapping set of neurons. Our results suggest that dACC may contribute to pursuit and evasion by computing and continuously updating a multicentric representation of the unfolding task state, and support the hypothesis that it plays a high-level abstract role in the control of behavior.
Collapse
Affiliation(s)
- Seng Bum Michael Yoo
- Department of Neuroscience, Center for Magnetic Resonance Research, and Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA. .,Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon, Republic of Korea. .,Department of Biomedical Engineering, Sungkyunkwan University, Suwon, Republic of Korea. .,Department of Brain and Cognitive Sciences, Massachusetts Institution of Technology, Cambridge, MA, USA.
| | - Jiaxin Cindy Tu
- Department of Neuroscience, Center for Magnetic Resonance Research, and Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA.,Department of Neuroscience, Washington University at St.Louis, St.Louis, MO, USA
| | - Benjamin Yost Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, and Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
38
|
Bourgeois-Gironde S, Addessi E, Boraud T. Economic behaviours among non-human primates. Philos Trans R Soc Lond B Biol Sci 2021; 376:20190676. [PMID: 33423625 PMCID: PMC7815433 DOI: 10.1098/rstb.2019.0676] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/12/2020] [Indexed: 12/24/2022] Open
Abstract
Do we have any valid reasons to affirm that non-human primates display economic behaviour in a sufficiently rich and precise sense of the phrase? To address this question, we have to develop a set of criteria to assess the vast array of experimental studies and field observations on individual cognitive and behavioural competences as well as the collective organization of non-human primates. We review a sample of these studies and assess how they answer to the following four main challenges. (i) Do we see any economic organization or institutions emerge among groups of non-human primates? (ii) Are the cognitive abilities, and often biases, that have been evidenced as underlying typical economic decision-making among humans, also present among non-human primates? (iii) Can we draw positive lessons from performance comparisons among primate species, humans and non-humans but also across non-human primate species, as elicited by canonical game-theoretical experimental paradigms, especially as far as economic cooperation and coordination are concerned? And (iv) in which way should we improve models and paradigms to obtain more ecological data and conclusions? Articles discussed in this paper most often bring about positive answers and promising perspectives to support the existence and prevalence of economic behaviours among non-human primates. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.
Collapse
Affiliation(s)
- Sacha Bourgeois-Gironde
- Institut Jean Nicod, Département d’études cognitives, ENS, EHESS, CNRS, PSLUniversity, France
| | - Elsa Addessi
- Unità di Primatologia Cognitiva e Centro Primati, Istituto di Scienze e Tecnologie della Cognizione, Consiglio Nazionale delle Ricerche, 00197 Rome, Italy
| | - Thomas Boraud
- CNRS, UMR 5293, IMN, 33000 Bordeaux, France
- University of Bordeaux, UMR 5293, IMN, 33000, Bordeaux, France
- CHU de Bordeaux, IMN Clinique, 33000 Bordeaux, France
| |
Collapse
|