1
|
Van de Maele T, Verbelen T, Çatal O, De Boom C, Dhoedt B. Active Vision for Robot Manipulators Using the Free Energy Principle. Front Neurorobot 2021; 15:642780. [PMID: 33746730 PMCID: PMC7973267 DOI: 10.3389/fnbot.2021.642780] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2020] [Accepted: 02/03/2021] [Indexed: 11/14/2022] Open
Abstract
Occlusions, restricted field of view and limited resolution all constrain a robot's ability to sense its environment from a single observation. In these cases, the robot first needs to actively query multiple observations and accumulate information before it can complete a task. In this paper, we cast this problem of active vision as active inference, which states that an intelligent agent maintains a generative model of its environment and acts in order to minimize its surprise, or expected free energy according to this model. We apply this to an object-reaching task for a 7-DOF robotic manipulator with an in-hand camera to scan the workspace. A novel generative model using deep neural networks is proposed that is able to fuse multiple views into an abstract representation and is trained from data by minimizing variational free energy. We validate our approach experimentally for a reaching task in simulation in which a robotic agent starts without any knowledge about its workspace. Each step, the next view pose is chosen by evaluating the expected free energy. We find that by minimizing the expected free energy, exploratory behavior emerges when the target object to reach is not in view, and the end effector is moved to the correct reach position once the target is located. Similar to an owl scavenging for prey, the robot naturally prefers higher ground for exploring, approaching its target once located.
Collapse
Affiliation(s)
- Toon Van de Maele
- IDLab, Department of Information Technology, Ghent University—imec, Ghent, Belgium
| | | | | | | | | |
Collapse
|
2
|
John YJ, Zikopoulos B, Bullock D, Barbas H. Visual Attention Deficits in Schizophrenia Can Arise From Inhibitory Dysfunction in Thalamus or Cortex. COMPUTATIONAL PSYCHIATRY (CAMBRIDGE, MASS.) 2018; 2:223-257. [PMID: 30627672 PMCID: PMC6317791 DOI: 10.1162/cpsy_a_00023] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/27/2018] [Accepted: 10/17/2018] [Indexed: 01/13/2023]
Abstract
Schizophrenia is associated with diverse cognitive deficits, including disorders of attention-related oculomotor behavior. At the structural level, schizophrenia is associated with abnormal inhibitory control in the circuit linking cortex and thalamus. We developed a spiking neural network model that demonstrates how dysfunctional inhibition can degrade attentive gaze control. Our model revealed that perturbations of two functionally distinct classes of cortical inhibitory neurons, or of the inhibitory thalamic reticular nucleus, disrupted processing vital for sustained attention to a stimulus, leading to distractibility. Because perturbation at each circuit node led to comparable but qualitatively distinct disruptions in attentive tracking or fixation, our findings support the search for new eye movement metrics that may index distinct underlying neural defects. Moreover, because the cortico-thalamic circuit is a common motif across sensory, association, and motor systems, the model and extensions can be broadly applied to study normal function and the neural bases of other cognitive deficits in schizophrenia.
Collapse
Affiliation(s)
- Yohan J. John
- Neural Systems Laboratory, Department of Health Sciences, Boston University, Boston, Massachusetts, USA
| | - Basilis Zikopoulos
- Human Systems Neuroscience Laboratory, Department of Health Sciences, Boston University, Boston, Massachusetts, USA
- Graduate Program for Neuroscience, Boston University, and School of Medicine, Boston, Massachusetts, USA
| | - Daniel Bullock
- Graduate Program for Neuroscience, Boston University, and School of Medicine, Boston, Massachusetts, USA
- Department of Psychological and Brain Sciences, Boston University, Boston, Massachusetts, USA
| | - Helen Barbas
- Neural Systems Laboratory, Department of Health Sciences, Boston University, Boston, Massachusetts, USA
- Graduate Program for Neuroscience, Boston University, and School of Medicine, Boston, Massachusetts, USA
| |
Collapse
|
3
|
Calancie OG, Khalid-Khan S, Booij L, Munoz DP. Eye movement desensitization and reprocessing as a treatment for PTSD: current neurobiological theories and a new hypothesis. Ann N Y Acad Sci 2018; 1426:127-145. [PMID: 29931688 DOI: 10.1111/nyas.13882] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2017] [Revised: 04/16/2018] [Accepted: 05/24/2018] [Indexed: 12/25/2022]
Abstract
Eye movement desensitization and reprocessing (EMDR), a form of psychotherapy for individuals with post-traumatic stress disorder (PTSD), has long been a controversial topic, hampered in part by a lack of understanding of the neural mechanisms that contribute to its remedial effect. Here, we review current theories describing EMDR's potential neurobiological mechanisms of action involving working memory, interhemispheric communication, de-arousal, and memory reconsolidation. We then discuss recent studies describing the temporal and spatial aspects of smooth pursuit and predictive saccades, which resemble those made during EMDR, and their neural correlates within the default mode network (DMN) and cerebellum. We hypothesize that if the production of bilateral predictive eye movements is supportive of DMN and cerebellum activation, then therapies that shift the brain towards this state correspondingly would benefit the processes regulated by these structures (i.e., memory retrieval, relaxation, and associative learning), all of which are essential components for PTSD recovery. We propose that the timing of sensory stimulation may be relevant to treatment effect and could be adapted across different patients depending on their baseline saccade metrics. Empirical data in support of this model are reviewed and experimental predictions are discussed.
Collapse
Affiliation(s)
- Olivia G Calancie
- Centre for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada
- Division of Child and Youth Mental Health, Kingston Health Sciences Centre, Kingston, Ontario, Canada
| | - Sarosh Khalid-Khan
- Centre for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada
- Division of Child and Youth Mental Health, Kingston Health Sciences Centre, Kingston, Ontario, Canada
| | - Linda Booij
- Department of Psychology, Concordia University, Montréal, Quebec, Canada
- Department of Psychology, Queen's University, Kingston, Ontario, Canada
| | - Douglas P Munoz
- Centre for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada
- Department of Biomedical and Molecular Sciences, Queen's University, Kingston, Ontario, Canada
| |
Collapse
|
4
|
Grossberg S. Desirability, availability, credit assignment, category learning, and attention: Cognitive-emotional and working memory dynamics of orbitofrontal, ventrolateral, and dorsolateral prefrontal cortices. Brain Neurosci Adv 2018; 2:2398212818772179. [PMID: 32166139 PMCID: PMC7058233 DOI: 10.1177/2398212818772179] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2017] [Accepted: 03/16/2018] [Indexed: 11/17/2022] Open
Abstract
BACKGROUND The prefrontal cortices play an essential role in cognitive-emotional and working memory processes through interactions with multiple brain regions. METHODS This article further develops a unified neural architecture that explains many recent and classical data about prefrontal function and makes testable predictions. RESULTS Prefrontal properties of desirability, availability, credit assignment, category learning, and feature-based attention are explained. These properties arise through interactions of orbitofrontal, ventrolateral prefrontal, and dorsolateral prefrontal cortices with the inferotemporal cortex, perirhinal cortex, parahippocampal cortices; ventral bank of the principal sulcus, ventral prearcuate gyrus, frontal eye fields, hippocampus, amygdala, basal ganglia, hypothalamus, and visual cortical areas V1, V2, V3A, V4, middle temporal cortex, medial superior temporal area, lateral intraparietal cortex, and posterior parietal cortex. Model explanations also include how the value of visual objects and events is computed, which objects and events cause desired consequences and which may be ignored as predictively irrelevant, and how to plan and act to realise these consequences, including how to selectively filter expected versus unexpected events, leading to movements towards, and conscious perception of, expected events. Modelled processes include reinforcement learning and incentive motivational learning; object and spatial working memory dynamics; and category learning, including the learning of object categories, value categories, object-value categories, and sequence categories, or list chunks. CONCLUSION This article hereby proposes a unified neural theory of prefrontal cortex and its functions.
Collapse
Affiliation(s)
- Stephen Grossberg
- Center for Adaptive Systems, Graduate Program in Cognitive and Neural Systems, Departments of Mathematics & Statistics, Psychological & Brain Sciences, Biomedical Engineering, Boston University, Boston, MA, USA
| |
Collapse
|
5
|
Friston K, FitzGerald T, Rigoli F, Schwartenbeck P, Pezzulo G. Active Inference: A Process Theory. Neural Comput 2016; 29:1-49. [PMID: 27870614 DOI: 10.1162/neco_a_00912] [Citation(s) in RCA: 432] [Impact Index Per Article: 54.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
This article describes a process theory based on active inference and belief propagation. Starting from the premise that all neuronal processing (and action selection) can be explained by maximizing Bayesian model evidence-or minimizing variational free energy-we ask whether neuronal responses can be described as a gradient descent on variational free energy. Using a standard (Markov decision process) generative model, we derive the neuronal dynamics implicit in this description and reproduce a remarkable range of well-characterized neuronal phenomena. These include repetition suppression, mismatch negativity, violation responses, place-cell activity, phase precession, theta sequences, theta-gamma coupling, evidence accumulation, race-to-bound dynamics, and transfer of dopamine responses. Furthermore, the (approximately Bayes' optimal) behavior prescribed by these dynamics has a degree of face validity, providing a formal explanation for reward seeking, context learning, and epistemic foraging. Technically, the fact that a gradient descent appears to be a valid description of neuronal activity means that variational free energy is a Lyapunov function for neuronal dynamics, which therefore conform to Hamilton's principle of least action.
Collapse
Affiliation(s)
- Karl Friston
- Wellcome Trust Centre for Neuroimaging, UCL, London WC1N 3BG, U.K.
| | - Thomas FitzGerald
- Wellcome Trust Centre for Neuroimaging, UCL, London WC1N 3BG, U.K., and Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5BE, U.K.
| | - Francesco Rigoli
- Wellcome Trust Centre for Neuroimaging, UCL, London WC1N 3BG, U.K.
| | - Philipp Schwartenbeck
- Wellcome Trust Centre for Neuroimaging, UCL, London WC1N 3BG, U.K.; Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London, WC1B 5BE, U.K.; Centre for Neurocognitive Research, University of Salzburg, 5020 Salzburg, Austria; and Neuroscience Institute, Christian-Doppler-Klinik, Paracelsus Medical University Salzburg, A-5020 Salzburg, Austria
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, 00185 Rome, Italy
| |
Collapse
|
6
|
Mirza MB, Adams RA, Mathys CD, Friston KJ. Scene Construction, Visual Foraging, and Active Inference. Front Comput Neurosci 2016; 10:56. [PMID: 27378899 PMCID: PMC4906014 DOI: 10.3389/fncom.2016.00056] [Citation(s) in RCA: 103] [Impact Index Per Article: 12.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2016] [Accepted: 05/25/2016] [Indexed: 11/13/2022] Open
Abstract
This paper describes an active inference scheme for visual searches and the perceptual synthesis entailed by scene construction. Active inference assumes that perception and action minimize variational free energy, where actions are selected to minimize the free energy expected in the future. This assumption generalizes risk-sensitive control and expected utility theory to include epistemic value; namely, the value (or salience) of information inherent in resolving uncertainty about the causes of ambiguous cues or outcomes. Here, we apply active inference to saccadic searches of a visual scene. We consider the (difficult) problem of categorizing a scene, based on the spatial relationship among visual objects where, crucially, visual cues are sampled myopically through a sequence of saccadic eye movements. This means that evidence for competing hypotheses about the scene has to be accumulated sequentially, calling upon both prediction (planning) and postdiction (memory). Our aim is to highlight some simple but fundamental aspects of the requisite functional anatomy; namely, the link between approximate Bayesian inference under mean field assumptions and functional segregation in the visual cortex. This link rests upon the (neurobiologically plausible) process theory that accompanies the normative formulation of active inference for Markov decision processes. In future work, we hope to use this scheme to model empirical saccadic searches and identify the prior beliefs that underwrite intersubject variability in the way people forage for information in visual scenes (e.g., in schizophrenia).
Collapse
Affiliation(s)
- M Berk Mirza
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London London, UK
| | - Rick A Adams
- Division of Psychiatry, University College LondonLondon, UK; Institute of Cognitive Neuroscience, University College LondonLondon, UK
| | - Christoph D Mathys
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College LondonLondon, UK; Translational Neuromodeling Unit, Institute for Biomedical Engineering, University of Zurich and Swiss Federal Institute of Technology in Zurich (ETHZ)Zurich, Switzerland; Max Planck UCL Centre for Computational Psychiatry and Ageing ResearchLondon, UK
| | - Karl J Friston
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London London, UK
| |
Collapse
|
7
|
Neural Dynamics of the Basal Ganglia During Perceptual, Cognitive, and Motor Learning and Gating. INNOVATIONS IN COGNITIVE NEUROSCIENCE 2016. [DOI: 10.1007/978-3-319-42743-0_19] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
|
8
|
Grossberg S, Srinivasan K, Yazdanbakhsh A. Binocular fusion and invariant category learning due to predictive remapping during scanning of a depthful scene with eye movements. Front Psychol 2015; 5:1457. [PMID: 25642198 PMCID: PMC4294135 DOI: 10.3389/fpsyg.2014.01457] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2014] [Accepted: 11/28/2014] [Indexed: 12/02/2022] Open
Abstract
How does the brain maintain stable fusion of 3D scenes when the eyes move? Every eye movement causes each retinal position to process a different set of scenic features, and thus the brain needs to binocularly fuse new combinations of features at each position after an eye movement. Despite these breaks in retinotopic fusion due to each movement, previously fused representations of a scene in depth often appear stable. The 3D ARTSCAN neural model proposes how the brain does this by unifying concepts about how multiple cortical areas in the What and Where cortical streams interact to coordinate processes of 3D boundary and surface perception, spatial attention, invariant object category learning, predictive remapping, eye movement control, and learned coordinate transformations. The model explains data from single neuron and psychophysical studies of covert visual attention shifts prior to eye movements. The model further clarifies how perceptual, attentional, and cognitive interactions among multiple brain regions (LGN, V1, V2, V3A, V4, MT, MST, PPC, LIP, ITp, ITa, SC) may accomplish predictive remapping as part of the process whereby view-invariant object categories are learned. These results build upon earlier neural models of 3D vision and figure-ground separation and the learning of invariant object categories as the eyes freely scan a scene. A key process concerns how an object's surface representation generates a form-fitting distribution of spatial attention, or attentional shroud, in parietal cortex that helps maintain the stability of multiple perceptual and cognitive processes. Predictive eye movement signals maintain the stability of the shroud, as well as of binocularly fused perceptual boundaries and surface representations.
Collapse
Affiliation(s)
- Stephen Grossberg
- Center for Adaptive Systems, Graduate Program in Cognitive and Neural Systems, Center of Excellence for Learning in Education, Science and Technology, Center for Computational Neuroscience and Neural Technology, and Department of Mathematics Boston University, Boston, MA, USA
| | - Karthik Srinivasan
- Center for Adaptive Systems, Graduate Program in Cognitive and Neural Systems, Center of Excellence for Learning in Education, Science and Technology, Center for Computational Neuroscience and Neural Technology, and Department of Mathematics Boston University, Boston, MA, USA
| | - Arash Yazdanbakhsh
- Center for Adaptive Systems, Graduate Program in Cognitive and Neural Systems, Center of Excellence for Learning in Education, Science and Technology, Center for Computational Neuroscience and Neural Technology, and Department of Mathematics Boston University, Boston, MA, USA
| |
Collapse
|
9
|
Friston K, Adams R, Montague R. What is value-accumulated reward or evidence? Front Neurorobot 2012; 6:11. [PMID: 23133414 PMCID: PMC3487150 DOI: 10.3389/fnbot.2012.00011] [Citation(s) in RCA: 33] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2012] [Accepted: 10/16/2012] [Indexed: 12/04/2022] Open
Abstract
Why are you reading this abstract? In some sense, your answer will cast the exercise as valuable—but what is value? In what follows, we suggest that value is evidence or, more exactly, log Bayesian evidence. This implies that a sufficient explanation for valuable behavior is the accumulation of evidence for internal models of our world. This contrasts with normative models of optimal control and reinforcement learning, which assume the existence of a value function that explains behavior, where (somewhat tautologically) behavior maximizes value. In this paper, we consider an alternative formulation—active inference—that replaces policies in normative models with prior beliefs about the (future) states agents should occupy. This enables optimal behavior to be cast purely in terms of inference: where agents sample their sensorium to maximize the evidence for their generative model of hidden states in the world, and minimize their uncertainty about those states. Crucially, this formulation resolves the tautology inherent in normative models and allows one to consider how prior beliefs are themselves optimized in a hierarchical setting. We illustrate these points by showing that any optimal policy can be specified with prior beliefs in the context of Bayesian inference. We then show how these prior beliefs are themselves prescribed by an imperative to minimize uncertainty. This formulation explains the saccadic eye movements required to read this text and defines the value of the visual sensations you are soliciting.
Collapse
Affiliation(s)
- Karl Friston
- Wellcome Trust Centre for Neuroimaging, University College London London, UK
| | | | | |
Collapse
|
10
|
Friston K, Adams RA, Perrinet L, Breakspear M. Perceptions as hypotheses: saccades as experiments. Front Psychol 2012; 3:151. [PMID: 22654776 PMCID: PMC3361132 DOI: 10.3389/fpsyg.2012.00151] [Citation(s) in RCA: 179] [Impact Index Per Article: 14.9] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2012] [Accepted: 04/26/2012] [Indexed: 11/24/2022] Open
Abstract
If perception corresponds to hypothesis testing (Gregory, 1980); then visual searches might be construed as experiments that generate sensory data. In this work, we explore the idea that saccadic eye movements are optimal experiments, in which data are gathered to test hypotheses or beliefs about how those data are caused. This provides a plausible model of visual search that can be motivated from the basic principles of self-organized behavior: namely, the imperative to minimize the entropy of hidden states of the world and their sensory consequences. This imperative is met if agents sample hidden states of the world efficiently. This efficient sampling of salient information can be derived in a fairly straightforward way, using approximate Bayesian inference and variational free-energy minimization. Simulations of the resulting active inference scheme reproduce sequential eye movements that are reminiscent of empirically observed saccades and provide some counterintuitive insights into the way that sensory evidence is accumulated or assimilated into beliefs about the world.
Collapse
Affiliation(s)
- Karl Friston
- The Wellcome Trust Centre for Neuroimaging, University College LondonLondon, UK
| | - Rick A. Adams
- The Wellcome Trust Centre for Neuroimaging, University College LondonLondon, UK
| | - Laurent Perrinet
- The Wellcome Trust Centre for Neuroimaging, University College LondonLondon, UK
- Institut de Neurosciences de la Timone, CNRS - Aix-Marseille UniversityMarseille, France
| | - Michael Breakspear
- Queensland Institute of Medical Research, Royal Brisbane HospitalBrisbane, QLD, Australia
| |
Collapse
|
11
|
Foley NC, Grossberg S, Mingolla E. Neural dynamics of object-based multifocal visual spatial attention and priming: object cueing, useful-field-of-view, and crowding. Cogn Psychol 2012; 65:77-117. [PMID: 22425615 DOI: 10.1016/j.cogpsych.2012.02.001] [Citation(s) in RCA: 36] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2011] [Revised: 01/07/2012] [Accepted: 02/02/2012] [Indexed: 11/18/2022]
Abstract
How are spatial and object attention coordinated to achieve rapid object learning and recognition during eye movement search? How do prefrontal priming and parietal spatial mechanisms interact to determine the reaction time costs of intra-object attention shifts, inter-object attention shifts, and shifts between visible objects and covertly cued locations? What factors underlie individual differences in the timing and frequency of such attentional shifts? How do transient and sustained spatial attentional mechanisms work and interact? How can volition, mediated via the basal ganglia, influence the span of spatial attention? A neural model is developed of how spatial attention in the where cortical stream coordinates view-invariant object category learning in the what cortical stream under free viewing conditions. The model simulates psychological data about the dynamics of covert attention priming and switching requiring multifocal attention without eye movements. The model predicts how "attentional shrouds" are formed when surface representations in cortical area V4 resonate with spatial attention in posterior parietal cortex (PPC) and prefrontal cortex (PFC), while shrouds compete among themselves for dominance. Winning shrouds support invariant object category learning, and active surface-shroud resonances support conscious surface perception and recognition. Attentive competition between multiple objects and cues simulates reaction-time data from the two-object cueing paradigm. The relative strength of sustained surface-driven and fast-transient motion-driven spatial attention controls individual differences in reaction time for invalid cues. Competition between surface-driven attentional shrouds controls individual differences in detection rate of peripheral targets in useful-field-of-view tasks. The model proposes how the strength of competition can be mediated, though learning or momentary changes in volition, by the basal ganglia. A new explanation of crowding shows how the cortical magnification factor, among other variables, can cause multiple object surfaces to share a single surface-shroud resonance, thereby preventing recognition of the individual objects.
Collapse
Affiliation(s)
- Nicholas C Foley
- Center for Adaptive Systems, Department of Cognitive and Neural Systems, Boston University, 677 Beacon Street, Boston, MA 02215, USA
| | | | | |
Collapse
|
12
|
Grossberg S, Srihasam K, Bullock D. Neural dynamics of saccadic and smooth pursuit eye movement coordination during visual tracking of unpredictably moving targets. Neural Netw 2011; 27:1-20. [PMID: 22078464 DOI: 10.1016/j.neunet.2011.10.011] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2010] [Revised: 10/14/2011] [Accepted: 10/20/2011] [Indexed: 10/15/2022]
Abstract
How does the brain coordinate saccadic and smooth pursuit eye movements to track objects that move in unpredictable directions and speeds? Saccadic eye movements rapidly foveate peripheral visual or auditory targets, and smooth pursuit eye movements keep the fovea pointed toward an attended moving target. Analyses of tracking data in monkeys and humans reveal systematic deviations from predictions of the simplest model of saccade-pursuit interactions, which would use no interactions other than common target selection and recruitment of shared motoneurons. Instead, saccadic and smooth pursuit movements cooperate to cancel errors of gaze position and velocity, and thus to maximize target visibility through time. How are these two systems coordinated to promote visual localization and identification of moving targets? How are saccades calibrated to correctly foveate a target despite its continued motion during the saccade? The neural model proposed here answers these questions. Modeled interactions encompass motion processing areas MT, MST, FPA, DLPN and NRTP; saccade planning and execution areas FEF, LIP, and SC; the saccadic generator in the brain stem; and the cerebellum. Simulations illustrate the model's ability to functionally explain and quantitatively simulate anatomical, neurophysiological and behavioral data about coordinated saccade-pursuit tracking.
Collapse
Affiliation(s)
- Stephen Grossberg
- Center for Adaptive Systems, Department of Cognitive and Neural Systems, Boston University, 677 Beacon Street, Boston, MA 02215, USA.
| | | | | |
Collapse
|
13
|
Mahaffy S, Krauzlis RJ. Inactivation and stimulation of the frontal pursuit area change pursuit metrics without affecting pursuit target selection. J Neurophysiol 2011; 106:347-60. [PMID: 21525365 DOI: 10.1152/jn.00669.2010] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
The frontal pursuit area (FPA) lies posterior to the frontal eye fields in the frontal cortex and contains neurons that are directionally selective for pursuit eye movements. Lesions of the FPA (alternately called "FEFsem") cause deficits in pursuit acceleration and velocity, which are largest for movements directed toward the lesioned side. Conversely, stimulation of the FPA evokes pursuit from fixation and increases the gain of the pursuit response. On the basis of these properties, it has been hypothesized that the FPA could underlie the selection of pursuit direction. To test this possibility, we manipulated FPA activity and measured the effect on target selection behavior in rhesus monkeys. First, we unilaterally inactivated the FPA with the GABA agonist muscimol. We then measured the monkeys' performance on a pursuit-choice task. Second, we applied microstimulation unilaterally to the FPA during pursuit initiation while monkeys performed the same pursuit-choice task. Both of these manipulations produced significant effects on pursuit metrics; the inactivation decreased pursuit velocity and acceleration, and microstimulation evoked pursuit directly. Despite these changes, both manipulations failed to significantly alter choice behavior. These results show that FPA activity is not necessary for pursuit target selection.
Collapse
Affiliation(s)
- Shaun Mahaffy
- Systems Neurobiology Laboratory, Salk Institute for Biological Studies, La Jolla, CA 92037, USA
| | | |
Collapse
|
14
|
Mahaffy S, Krauzlis RJ. Neural activity in the frontal pursuit area does not underlie pursuit target selection. Vision Res 2011; 51:853-66. [PMID: 20970442 PMCID: PMC3046298 DOI: 10.1016/j.visres.2010.10.010] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2010] [Revised: 10/06/2010] [Accepted: 10/07/2010] [Indexed: 11/17/2022]
Abstract
The frontal pursuit area (FPA) contains neurons that are directionally selective for pursuit eye-movements. We found that FPA neurons discriminate target from distracter too late to account for pursuit directional selection. Rather, the timing of neuronal discrimination is linked to pursuit onset, suggesting a role in motor execution. We also found buildup of activity of FPA neurons prior to pursuit onset that correlated with eye acceleration. These results show that the FPA is unlikely to be involved in selection of initial pursuit direction, but could be involved in motor preparation by increasing pursuit gain prior to pursuit onset.
Collapse
Affiliation(s)
- Shaun Mahaffy
- Neurosciences Graduate Program, University of California, San Diego, 9500 Gilman Drive La Jolla, CA 92093-0662, United States
| | | |
Collapse
|
15
|
Grossberg S, Vladusich T. How do children learn to follow gaze, share joint attention, imitate their teachers, and use tools during social interactions? Neural Netw 2010; 23:940-65. [DOI: 10.1016/j.neunet.2010.07.011] [Citation(s) in RCA: 24] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2010] [Accepted: 07/29/2010] [Indexed: 12/01/2022]
|
16
|
Bullock D, Tan CO, John YJ. Computational perspectives on forebrain microcircuits implicated in reinforcement learning, action selection, and cognitive control. Neural Netw 2009; 22:757-65. [PMID: 19592218 PMCID: PMC2746108 DOI: 10.1016/j.neunet.2009.06.008] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2009] [Revised: 05/28/2009] [Accepted: 06/25/2009] [Indexed: 11/19/2022]
Abstract
Abundant new information about signaling pathways in forebrain microcircuits presents many challenges, and opportunities for discovery, to computational neuroscientists who strive to bridge from microcircuits to flexible cognition and action. Accurate treatment of microcircuit pathways is especially critical for creating models that correctly predict the outcomes of candidate neurological therapies. Recent models are trying to specify how cortical circuits that enable planning and voluntary actions interact with adaptive subcortical microcircuits in the basal ganglia. The basal ganglia are strongly implicated in reinforcement learning, and in all behavior and cognition over which the frontal lobes exert flexible control. The persisting role of the basal ganglia shows that ancient vertebrate designs for motivated action selection proved adaptable enough to support many "modern" behavioral innovations, including fluent generation of language and speech. This paper summarizes how recent models have incorporated realistic representations of microcircuit features, and have begun to trace their computational implications. Also summarized are recent empirical discoveries that provide guidance regarding how to formulate the rules for synaptic modification that govern learning in cortico-striatal pathways. Such efforts are contributing to an emerging synthesis based on an interlocking set of computational hypotheses regarding cortical interactions with basal ganglia and thalamic nuclei. These hypotheses specify how specialized microcircuits solve learning and control problems inherent to the brain's parallel design.
Collapse
Affiliation(s)
- Daniel Bullock
- Boston University, Department of Cognitive and Neural Systems, 677 Beacon Street, Boston, MA 02215, United States.
| | | | | |
Collapse
|
17
|
Andrew Browning N, Grossberg S, Mingolla E. Cortical dynamics of navigation and steering in natural scenes: Motion-based object segmentation, heading, and obstacle avoidance. Neural Netw 2009; 22:1383-98. [PMID: 19502003 DOI: 10.1016/j.neunet.2009.05.007] [Citation(s) in RCA: 26] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2008] [Revised: 05/07/2009] [Accepted: 05/18/2009] [Indexed: 10/20/2022]
Abstract
Visually guided navigation through a cluttered natural scene is a challenging problem that animals and humans accomplish with ease. The ViSTARS neural model proposes how primates use motion information to segment objects and determine heading for purposes of goal approach and obstacle avoidance in response to video inputs from real and virtual environments. The model produces trajectories similar to those of human navigators. It does so by predicting how computationally complementary processes in cortical areas MT(-)/MSTv and MT(+)/MSTd compute object motion for tracking and self-motion for navigation, respectively. The model's retina responds to transients in the input stream. Model V1 generates a local speed and direction estimate. This local motion estimate is ambiguous due to the neural aperture problem. Model MT(+) interacts with MSTd via an attentive feedback loop to compute accurate heading estimates in MSTd that quantitatively simulate properties of human heading estimation data. Model MT(-) interacts with MSTv via an attentive feedback loop to compute accurate estimates of speed, direction and position of moving objects. This object information is combined with heading information to produce steering decisions wherein goals behave like attractors and obstacles behave like repellers. These steering decisions lead to navigational trajectories that closely match human performance.
Collapse
Affiliation(s)
- N Andrew Browning
- Department of Cognitive and Neural Systems, Boston University, Boston, MA 02215, USA
| | | | | |
Collapse
|