1
|
Wu EG, Brackbill N, Rhoades C, Kling A, Gogliettino AR, Shah NP, Sher A, Litke AM, Simoncelli EP, Chichilnisky EJ. Fixational eye movements enhance the precision of visual information transmitted by the primate retina. Nat Commun 2024; 15:7964. [PMID: 39261491 PMCID: PMC11390888 DOI: 10.1038/s41467-024-52304-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Accepted: 08/29/2024] [Indexed: 09/13/2024] Open
Abstract
Fixational eye movements alter the number and timing of spikes transmitted from the retina to the brain, but whether these changes enhance or degrade the retinal signal is unclear. To quantify this, we developed a Bayesian method for reconstructing natural images from the recorded spikes of hundreds of retinal ganglion cells (RGCs) in the macaque retina (male), combining a likelihood model for RGC light responses with the natural image prior implicitly embedded in an artificial neural network optimized for denoising. The method matched or surpassed the performance of previous reconstruction algorithms, and provides an interpretable framework for characterizing the retinal signal. Reconstructions were improved with artificial stimulus jitter that emulated fixational eye movements, even when the eye movement trajectory was assumed to be unknown and had to be inferred from retinal spikes. Reconstructions were degraded by small artificial perturbations of spike times, revealing more precise temporal encoding than suggested by previous studies. Finally, reconstructions were substantially degraded when derived from a model that ignored cell-to-cell interactions, indicating the importance of stimulus-evoked correlations. Thus, fixational eye movements enhance the precision of the retinal representation.
Collapse
Affiliation(s)
- Eric G Wu
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
| | - Nora Brackbill
- Department of Physics, Stanford University, Stanford, CA, USA
| | - Colleen Rhoades
- Department of Bioengineering, Stanford University, Stanford, CA, USA
| | - Alexandra Kling
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
- Department of Ophthalmology, Stanford University, Stanford, CA, USA
- Hansen Experimental Physics Laboratory, Stanford University, 452 Lomita Mall, Stanford, 94305, CA, USA
| | - Alex R Gogliettino
- Hansen Experimental Physics Laboratory, Stanford University, 452 Lomita Mall, Stanford, 94305, CA, USA
- Neurosciences PhD Program, Stanford University, Stanford, CA, USA
| | - Nishal P Shah
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | - Alexander Sher
- Santa Cruz Institute for Particle Physics, University of California, Santa Cruz, Santa Cruz, CA, USA
| | - Alan M Litke
- Santa Cruz Institute for Particle Physics, University of California, Santa Cruz, Santa Cruz, CA, USA
| | - Eero P Simoncelli
- Flatiron Institute, Simons Foundation, New York, NY, USA
- Center for Neural Science, New York University, New York, NY, USA
- Courant Institute of Mathematical Sciences, New York University, New York, NY, USA
| | - E J Chichilnisky
- Department of Neurosurgery, Stanford University, Stanford, CA, USA.
- Department of Ophthalmology, Stanford University, Stanford, CA, USA.
- Hansen Experimental Physics Laboratory, Stanford University, 452 Lomita Mall, Stanford, 94305, CA, USA.
| |
Collapse
|
2
|
D'Angelo JC, Tiruveedhula P, Weber RJ, Arathorn DW, Roorda A. A paradoxical misperception of relative motion. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.04.596708. [PMID: 38895454 PMCID: PMC11185587 DOI: 10.1101/2024.06.04.596708] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/21/2024]
Abstract
Motion perception is considered a hyperacuity. The presence of a visual frame of reference to compute relative motion is necessary to achieve this sensitivity [Legge, Gordon E., and F. W. Campbell. "Displacement detection in human vision." Vision Research 21.2 (1981): 205-213.]. However, there is a special condition where humans are unable to accurately detect relative motion: images moving in a direction consistent with retinal slip where the motion is unnaturally amplified can, under some conditions, appear stable [Arathorn, David W., et al. "How the unstable eye sees a stable and moving world." Journal of Vision 13.10.22 (2013)]. In this study, we asked: Is world-fixed retinal image background content necessary for the visual system to compute the direction of eye motion to render in the percept images moving with amplified slip as stable? Or, are non-visual cues sufficient? Subjects adjusted the parameters of a stimulus moving in a random trajectory to match the perceived motion of images moving contingent to the retina. Experiments were done with and without retinal image background content. The perceived motion of stimuli moving with amplified retinal slip was suppressed in the presence of visual content; however, higher magnitudes of motion were perceived under conditions with no visual cues. Our results demonstrate that the presence of retinal image background content is essential for the visual system to compute its direction of motion. The visual content that might be thought to provide a strong frame of reference to detect amplified retinal slips, instead paradoxically drives the misperception of relative motion.
Collapse
Affiliation(s)
- Josephine C D'Angelo
- Herbert Wertheim School of Optometry and Vision Science, University of California, Berkeley, CA 94720
| | - Pavan Tiruveedhula
- Herbert Wertheim School of Optometry and Vision Science, University of California, Berkeley, CA 94720
| | - Raymond J Weber
- Electrical and Computer Engineering Department, Montana Sate University, Bozeman, MT 59717-3780
| | - David W Arathorn
- Electrical and Computer Engineering Department, Montana Sate University, Bozeman, MT 59717-3780
| | - Austin Roorda
- Herbert Wertheim School of Optometry and Vision Science, University of California, Berkeley, CA 94720
| |
Collapse
|
3
|
Seckler H, Szwabiński J, Metzler R. Machine-Learning Solutions for the Analysis of Single-Particle Diffusion Trajectories. J Phys Chem Lett 2023; 14:7910-7923. [PMID: 37646323 DOI: 10.1021/acs.jpclett.3c01351] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/01/2023]
Abstract
Single-particle traces of the diffusive motion of molecules, cells, or animals are by now routinely measured, similar to stochastic records of stock prices or weather data. Deciphering the stochastic mechanism behind the recorded dynamics is vital in understanding the observed systems. Typically, the task is to decipher the exact type of diffusion and/or to determine the system parameters. The tools used in this endeavor are currently being revolutionized by modern machine-learning techniques. In this Perspective we provide an overview of recently introduced methods in machine-learning for diffusive time series, most notably, those successfully competing in the anomalous diffusion challenge. As such methods are often criticized for their lack of interpretability, we focus on means to include uncertainty estimates and feature-based approaches, both improving interpretability and providing concrete insight into the learning process of the machine. We expand the discussion by examining predictions on different out-of-distribution data. We also comment on expected future developments.
Collapse
Affiliation(s)
- Henrik Seckler
- Institute of Physics & Astronomy, University of Potsdam, 14476 Potsdam-Golm, Germany
| | - Janusz Szwabiński
- Hugo Steinhaus Center, Faculty of Pure and Applied Mathematics, Wrocław University of Science and Technology, Wybrzeże Wyspiańskiego 27, 50-370 Wrocław, Poland
| | - Ralf Metzler
- Institute of Physics & Astronomy, University of Potsdam, 14476 Potsdam-Golm, Germany
- Asia Pacific Center for Theoretical Physics, Pohang 37673, Republic of Korea
| |
Collapse
|
4
|
Testa S, Sabatini SP, Canessa A. Active fixation as an efficient coding strategy for neuromorphic vision. Sci Rep 2023; 13:7445. [PMID: 37156822 PMCID: PMC10167324 DOI: 10.1038/s41598-023-34508-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Accepted: 05/03/2023] [Indexed: 05/10/2023] Open
Abstract
Contrary to a photographer, who puts a great effort in keeping the lens still, eyes insistently move even during fixation. This benefits signal decorrelation, which underlies an efficient encoding of visual information. Yet, camera motion is not sufficient alone; it must be coupled with a sensor specifically selective to temporal changes. Indeed, motion induced on standard imagers only results in burring effects. Neuromorphic sensors represent a valuable solution. Here we characterize the response of an event-based camera equipped with fixational eye movements (FEMs) on both synthetic and natural images. Our analyses prove that the system starts an early stage of redundancy suppression, as a precursor of subsequent whitening processes on the amplitude spectrum. This does not come at the price of corrupting structural information contained in local spatial phase across oriented axes. Isotropy of FEMs ensures proper representations of image features without introducing biases towards specific contrast orientations.
Collapse
Affiliation(s)
- Simone Testa
- Department of Informatics, Bioengineering, Robotics and Systems Engineering (DIBRIS), University of Genoa, 16145, Genoa, Italy
| | - Silvio P Sabatini
- Department of Informatics, Bioengineering, Robotics and Systems Engineering (DIBRIS), University of Genoa, 16145, Genoa, Italy
| | - Andrea Canessa
- Department of Informatics, Bioengineering, Robotics and Systems Engineering (DIBRIS), University of Genoa, 16145, Genoa, Italy.
| |
Collapse
|
5
|
Fixational drift is driven by diffusive dynamics in central neural circuitry. Nat Commun 2022; 13:1697. [PMID: 35361753 PMCID: PMC8971408 DOI: 10.1038/s41467-022-29201-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2021] [Accepted: 03/04/2022] [Indexed: 11/08/2022] Open
Abstract
During fixation and between saccades, our eyes undergo diffusive random motion called fixational drift. The role of fixational drift in visual coding and inference has been debated in the past few decades, but the mechanisms that underlie this motion remained unknown. In particular, it has been unclear whether fixational drift arises from peripheral sources, or from central sources within the brain. Here we show that fixational drift is correlated with neural activity, and identify its origin in central neural circuitry within the oculomotor system, upstream to the ocular motoneurons (OMNs). We analyzed a large data set of OMN recordings in the rhesus monkey, alongside precise measurements of eye position, and found that most of the variance of fixational eye drifts must arise upstream of the OMNs. The diffusive statistics of the motion points to the oculomotor integrator, a memory circuit responsible for holding the eyes still between saccades, as a likely source of the motion. Theoretical modeling, constrained by the parameters of the primate oculomotor system, supports this hypothesis by accounting for the amplitude as well as the statistics of the motion. Thus, we propose that fixational ocular drift provides a direct observation of diffusive dynamics in a neural circuit responsible for storage of continuous parameter memory in persistent neural activity. The identification of a mechanistic origin for fixational drift is likely to advance the understanding of its role in visual processing and inference.
Collapse
|
6
|
Microsaccades, Drifts, Hopf Bundle and Neurogeometry. J Imaging 2022; 8:jimaging8030076. [PMID: 35324631 PMCID: PMC8953095 DOI: 10.3390/jimaging8030076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Revised: 03/03/2022] [Accepted: 03/04/2022] [Indexed: 02/01/2023] Open
Abstract
The first part of the paper contains a short review of the image processing in early vision is static, when the eyes and the stimulus are stable, and in dynamics, when the eyes participate in fixation eye movements. In the second part, we give an interpretation of Donders’ and Listing’s law in terms of the Hopf fibration of the 3-sphere over the 2-sphere. In particular, it is shown that the configuration space of the eye ball (when the head is fixed) is the 2-dimensional hemisphere SL+, called Listing hemisphere, and saccades are described as geodesic segments of SL+ with respect to the standard round metric. We study fixation eye movements (drift and microsaccades) in terms of this model and discuss the role of fixation eye movements in vision. A model of fixation eye movements is proposed that gives an explanation of presaccadic shift of receptive fields.
Collapse
|
7
|
Wang W, Cherstvy AG, Kantz H, Metzler R, Sokolov IM. Time averaging and emerging nonergodicity upon resetting of fractional Brownian motion and heterogeneous diffusion processes. Phys Rev E 2021; 104:024105. [PMID: 34525678 DOI: 10.1103/physreve.104.024105] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Accepted: 07/14/2021] [Indexed: 12/12/2022]
Abstract
How different are the results of constant-rate resetting of anomalous-diffusion processes in terms of their ensemble-averaged versus time-averaged mean-squared displacements (MSDs versus TAMSDs) and how does stochastic resetting impact nonergodicity? We examine, both analytically and by simulations, the implications of resetting on the MSD- and TAMSD-based spreading dynamics of particles executing fractional Brownian motion (FBM) with a long-time memory, heterogeneous diffusion processes (HDPs) with a power-law space-dependent diffusivity D(x)=D_{0}|x|^{γ} and their "combined" process of HDP-FBM. We find, inter alia, that the resetting dynamics of originally ergodic FBM for superdiffusive Hurst exponents develops disparities in scaling and magnitudes of the MSDs and mean TAMSDs indicating weak ergodicity breaking. For subdiffusive HDPs we also quantify the nonequivalence of the MSD and TAMSD and observe a new trimodal form of the probability density function. For reset FBM, HDPs and HDP-FBM we compute analytically and verify by simulations the short-time MSD and TAMSD asymptotes and long-time plateaus reminiscent of those for processes under confinement. We show that certain characteristics of these reset processes are functionally similar despite a different stochastic nature of their nonreset variants. Importantly, we discover nonmonotonicity of the ergodicity-breaking parameter EB as a function of the resetting rate r. For all reset processes studied we unveil a pronounced resetting-induced nonergodicity with a maximum of EB at intermediate r and EB∼(1/r)-decay at large r. Alongside the emerging MSD-versus-TAMSD disparity, this r-dependence of EB can be an experimentally testable prediction. We conclude by discussing some implications to experimental systems featuring resetting dynamics.
Collapse
Affiliation(s)
- Wei Wang
- Max Planck Institute for the Physics of Complex Systems, Nöthnitzer Straße 38, 01187 Dresden, Germany
| | - Andrey G Cherstvy
- Institute for Physics & Astronomy University of Potsdam, Karl-Liebknecht-Straße 24/25, 14476 Potsdam-Golm, Germany.,Institut für Physik, Humboldt-Universität zu Berlin, Newtonstraße 15, 12489 Berlin, Germany
| | - Holger Kantz
- Max Planck Institute for the Physics of Complex Systems, Nöthnitzer Straße 38, 01187 Dresden, Germany
| | - Ralf Metzler
- Institute for Physics & Astronomy University of Potsdam, Karl-Liebknecht-Straße 24/25, 14476 Potsdam-Golm, Germany
| | - Igor M Sokolov
- Institut für Physik, Humboldt-Universität zu Berlin, Newtonstraße 15, 12489 Berlin, Germany.,IRIS Adlershof, Zum Großen Windkanal 6, 12489 Berlin, Germany
| |
Collapse
|
8
|
Parisot K, Zozor S, Guérin-Dugué A, Phlypo R, Chauvin A. Micro-pursuit: A class of fixational eye movements correlating with smooth, predictable, small-scale target trajectories. J Vis 2021; 21:9. [PMID: 33444434 PMCID: PMC7838552 DOI: 10.1167/jov.21.1.9] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Humans generate ocular pursuit movements when a moving target is tracked throughout the visual field. In this article, we show that pursuit can be generated and measured at small amplitudes, at the scale of fixational eye movements, and tag these eye movements as micro-pursuits. During micro-pursuits, gaze direction correlates with a target's smooth, predictable target trajectory. We measure similarity between gaze and target trajectories using a so-called maximally projected correlation and provide results in three experimental data sets. A first observation of micro-pursuit is provided in an implicit pursuit task, where observers were tasked to maintain their gaze fixed on a static cross at the center of screen, while reporting changes in perception of an ambiguous, moving (Necker) cube. We then provide two experimental paradigms and their corresponding data sets: a first replicating micro-pursuits in an explicit pursuit task, where observers had to follow a moving fixation cross (Cross), and a second with an unambiguous square (Square). Individual and group analyses provide evidence that micro-pursuits exist in both the Necker and Cross experiments but not in the Square experiment. The interexperiment analysis results suggest that the manipulation of stimulus target motion, task, and/or the nature of the stimulus may play a role in the generation of micro-pursuits.
Collapse
Affiliation(s)
- Kevin Parisot
- CNRS, Institute of Engineering, GIPSA-lab & LPNC, University of Grenoble Alpes, Grenoble, France., https://scholar.google.fr/citations?user=WjGkMmIAAAAJ&hl=fr&oi=ao
| | - Steeve Zozor
- CNRS, Institute of Engineering, GIPSA-lab, University of Grenoble Alpes, Grenoble, France., http://www.gipsa-lab.grenoble-inp.fr/page_pro.php?vid=86
| | - Anne Guérin-Dugué
- CNRS, Institute of Engineering, GIPSA-lab, University of Grenoble Alpes, Grenoble, France., http://www.gipsa-lab.grenoble-inp.fr/page_pro.php?vid=71
| | - Ronald Phlypo
- CNRS, Institute of Engineering, GIPSA-lab, University of Grenoble Alpes, Grenoble, France., http://www.gipsa-lab.grenoble-inp.fr/page_pro.php?vid=2173
| | - Alan Chauvin
- CNRS, LPNC, University of Grenoble Alpes, Grenoble, France., https://lpnc.univ-grenoble-alpes.fr/Alan-Chauvin
| |
Collapse
|
9
|
Mangalam M, Kelty-Stephen DG. Point estimates, Simpson's paradox, and nonergodicity in biological sciences. Neurosci Biobehav Rev 2021; 125:98-107. [PMID: 33621638 DOI: 10.1016/j.neubiorev.2021.02.017] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Revised: 02/02/2021] [Accepted: 02/08/2021] [Indexed: 11/18/2022]
Abstract
Modern biomedical, behavioral and psychological inference about cause-effect relationships respects an ergodic assumption, that is, that mean response of representative samples allow predictions about individual members of those samples. Recent empirical evidence in all of the same fields indicates systematic violations of the ergodic assumption. Indeed, violation of ergodicity in biomedical, behavioral and psychological causes is precisely the inspiration behind our research inquiry. Here, we review the long term costs to scientific progress in these domains and a practical way forward. Specifically, we advocate using statistical measures that can themselves encode the degree and type of nonergodicity in measurements. Taking such steps will lead to a paradigm shift, allowing researchers to investigate the nonstationary, far-from-equilibrium processes that characterize the creativity and emergence of biological and psychological behavior.
Collapse
Affiliation(s)
- Madhur Mangalam
- Department of Physical Therapy, Movement and Rehabilitation Sciences, Northeastern University, Boston, MA, USA.
| | | |
Collapse
|
10
|
Gruber LZ, Ahissar E. Closed loop motor-sensory dynamics in human vision. PLoS One 2020; 15:e0240660. [PMID: 33057398 PMCID: PMC7561174 DOI: 10.1371/journal.pone.0240660] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2020] [Accepted: 09/30/2020] [Indexed: 12/02/2022] Open
Abstract
Vision is obtained with a continuous motion of the eyes. The kinematic analysis of eye motion, during any visual or ocular task, typically reveals two (kinematic) components: saccades, which quickly replace the visual content in the retinal fovea, and drifts, which slowly scan the image after each saccade. While the saccadic exchange of regions of interest (ROIs) is commonly considered to be included in motor-sensory closed-loops, it is commonly assumed that drifts function in an open-loop manner, that is, independent of the concurrent visual input. Accordingly, visual perception is assumed to be based on a sequence of open-loop processes, each initiated by a saccade-triggered retinal snapshot. Here we directly challenged this assumption by testing the dependency of drift kinematics on concurrent visual inputs using real-time gaze-contingent-display. Our results demonstrate a dependency of the trajectory on the concurrent visual input, convergence of speed to condition-specific values and maintenance of selected drift-related motor-sensory controlled variables, all strongly indicative of drifts being included in a closed-loop brain-world process, and thus suggesting that vision is inherently a closed-loop process.
Collapse
Affiliation(s)
| | - Ehud Ahissar
- Department of Neurobiology, Weizmann Institute of Science, Rehovot, Israel
| |
Collapse
|
11
|
Malevich T, Buonocore A, Hafed ZM. Rapid stimulus-driven modulation of slow ocular position drifts. eLife 2020; 9:e57595. [PMID: 32758358 PMCID: PMC7442486 DOI: 10.7554/elife.57595] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2020] [Accepted: 08/05/2020] [Indexed: 11/17/2022] Open
Abstract
The eyes are never still during maintained gaze fixation. When microsaccades are not occurring, ocular position exhibits continuous slow changes, often referred to as drifts. Unlike microsaccades, drifts remain to be viewed as largely random eye movements. Here we found that ocular position drifts can, instead, be very systematically stimulus-driven, and with very short latencies. We used highly precise eye tracking in three well trained macaque monkeys and found that even fleeting (~8 ms duration) stimulus presentations can robustly trigger transient and stimulus-specific modulations of ocular position drifts, and with only approximately 60 ms latency. Such drift responses are binocular, and they are most effectively elicited with large stimuli of low spatial frequency. Intriguingly, the drift responses exhibit some image pattern selectivity, and they are not explained by convergence responses, pupil constrictions, head movements, or starting eye positions. Ocular position drifts have very rapid access to exogenous visual information.
Collapse
Affiliation(s)
- Tatiana Malevich
- Werner Reichardt Centre for Integrative Neuroscience, Tuebingen UniversityTuebingenGermany
- Hertie Institute for Clinical Brain Research, Tuebingen UniversityTuebingenGermany
- Graduate School of Neural and Behavioural Sciences, International Max-Planck Research School, Tuebingen UniversityTuebingenGermany
| | - Antimo Buonocore
- Werner Reichardt Centre for Integrative Neuroscience, Tuebingen UniversityTuebingenGermany
- Hertie Institute for Clinical Brain Research, Tuebingen UniversityTuebingenGermany
| | - Ziad M Hafed
- Werner Reichardt Centre for Integrative Neuroscience, Tuebingen UniversityTuebingenGermany
- Hertie Institute for Clinical Brain Research, Tuebingen UniversityTuebingenGermany
| |
Collapse
|
12
|
Rucci M, Ahissar E, Burr D. Temporal Coding of Visual Space. Trends Cogn Sci 2019; 22:883-895. [PMID: 30266148 DOI: 10.1016/j.tics.2018.07.009] [Citation(s) in RCA: 56] [Impact Index Per Article: 11.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2018] [Revised: 07/16/2018] [Accepted: 07/16/2018] [Indexed: 11/20/2022]
Abstract
Establishing a representation of space is a major goal of sensory systems. Spatial information, however, is not always explicit in the incoming sensory signals. In most modalities it needs to be actively extracted from cues embedded in the temporal flow of receptor activation. Vision, on the other hand, starts with a sophisticated optical imaging system that explicitly preserves spatial information on the retina. This may lead to the assumption that vision is predominantly a spatial process: all that is needed is to transmit the retinal image to the cortex, like uploading a digital photograph, to establish a spatial map of the world. However, this deceptively simple analogy is inconsistent with theoretical models and experiments that study visual processing in the context of normal motor behavior. We argue here that, as with other senses, vision relies heavily on temporal strategies and temporal neural codes to extract and represent spatial information.
Collapse
Affiliation(s)
- Michele Rucci
- Center for Visual Science, University of Rochester, Rochester, NY 14627, USA; Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY 14627, USA.
| | - Ehud Ahissar
- Department of Neurobiology, Weizmann Institute, Rehovot, Israel.
| | - David Burr
- Department of Neuroscience, University of Florence, Florence 50125, Italy; School of Psychology, University of Sydney, Camperdown, NSW 2006, Australia.
| |
Collapse
|
13
|
Leszczynski M, Schroeder CE. The Role of Neuronal Oscillations in Visual Active Sensing. Front Integr Neurosci 2019; 13:32. [PMID: 31396059 PMCID: PMC6664014 DOI: 10.3389/fnint.2019.00032] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2019] [Accepted: 07/03/2019] [Indexed: 01/22/2023] Open
Abstract
Visual perception is most often studied as a "passive" process in which an observer fixates steadily at point in space so that stimuli can be delivered to the system with spatial precision. Analysis of neuronal signals related to vision is generally keyed to stimulus onset, stimulus movement, etc.; i.e., events external to the observer. In natural "active" vision, however, information is systematically acquired by using eye movements including rapid (saccadic) eye movements, as well as smooth ocular pursuit of moving objects and slower drifts. Here we consider the use of alternating saccades and fixations to gather information from a visual scene. The underlying motor sampling plan contains highly reliable information regarding "where" and "when" the eyes will land, this information can be used predictively to modify firing properties of neurons precisely at the time when this "contextual" information is most useful - when a volley of retinal input enters the system at the onset of each fixation. Analyses focusing on neural events leading to and resulting from shifts in fixation, as well as visual events external to the observer, can provide a more complete and mechanistic understanding of visual information processing. Studies thus far suggest that active vision may be a fundamentally different from that process we usually study with more traditional passive viewing paradigms. In this Perspective we note that active saccadic sampling behavior imposes robust temporal patterning on the activity of neuron ensembles and large-scale neural dynamics throughout the brain's visual pathways whose mechanistic effects on information processing are not yet fully understood. The spatio-temporal sequence of eye movements elicits a succession of temporally predictable quasi-rhythmic sensory inputs, whose encoding is enhanced by entrainment of low frequency oscillations to the rate of eye movements. Review of the pertinent findings underscores the fact that temporal coordination between motor and visual cortices is critical for understanding neural dynamics of active vision and posits that phase entrainment of neuronal oscillations plays a mechanistic role in this process.
Collapse
Affiliation(s)
- Marcin Leszczynski
- Department of Neurological Surgery, College of Physicians and Surgeons, Columbia University, New York, NY, United States
- Translational Neuroscience Laboratories, The Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY, United States
| | - Charles E. Schroeder
- Department of Neurological Surgery, College of Physicians and Surgeons, Columbia University, New York, NY, United States
- Translational Neuroscience Laboratories, The Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY, United States
| |
Collapse
|
14
|
Park AS, Bedggood PA, Metha AB, Anderson AJ. The influence of perceptual stabilisation on perceptual grouping of temporally asynchronous stimuli. Vision Res 2019; 160:1-9. [DOI: 10.1016/j.visres.2019.04.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2019] [Revised: 04/11/2019] [Accepted: 04/19/2019] [Indexed: 10/26/2022]
|
15
|
Park ASY, Metha AB, Bedggood PA, Anderson AJ. The influence of retinal image motion on the perceptual grouping of temporally asynchronous stimuli. J Vis 2019; 19:2. [PMID: 30943528 PMCID: PMC6450642 DOI: 10.1167/19.4.2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022] Open
Abstract
Briefly presented stimuli can reveal the lower limit of retinal-based perceptual stabilization mechanisms. This is demonstrated in perceptual grouping of temporally asynchronous stimuli, in which alternate row or column elements of a regular grid are presented over two successive display frames with an imperceptible temporal offset. The grouping phenomenon results from a subtle shift between alternate grid elements due to incomplete compensation of small, fixational eye movements occurring between the two presentation frames. This suggests that larger retinal shifts should amplify the introduced shifts between alternate grid elements and improve grouping performance. However, large shifts are necessarily absent in small eye movements. Furthermore, shifts follow a random walk, making the relationship between shift magnitude and performance difficult to explore systematically. Here, we established a systematic relationship between retinal image motion and perceptual grouping by presenting alternate grid elements (untracked) during smooth pursuit of known velocities. Our results show grouping performance to improve in direct proportion to pursuit velocity. Any potential compensation by extraretinal signals (e.g., efference copy) does not seem to occur.
Collapse
Affiliation(s)
- Adela S Y Park
- Department of Optometry & Vision Sciences, The University of Melbourne, Melbourne, Australia
| | - Andrew B Metha
- Department of Optometry & Vision Sciences, The University of Melbourne, Melbourne, Australia
| | - Phillip A Bedggood
- Department of Optometry & Vision Sciences, The University of Melbourne, Melbourne, Australia
| | - Andrew J Anderson
- Department of Optometry & Vision Sciences, The University of Melbourne, Melbourne, Australia
| |
Collapse
|
16
|
Tian X, Yoshida M, Hafed ZM. Dynamics of fixational eye position and microsaccades during spatial cueing: the case of express microsaccades. J Neurophysiol 2018; 119:1962-1980. [PMID: 29465321 DOI: 10.1152/jn.00752.2017] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Microsaccades are systematically modulated by peripheral spatial cues, and these eye movements have been implicated in perceptual and motor performance changes in cueing tasks. However, an additional oculomotor factor that may also influence performance in these tasks, fixational eye position itself, has been largely neglected so far. Using precise eye tracking and real-time retinal-image stabilization, we carefully analyzed fixational eye position dynamics and related them to microsaccade generation during spatial cueing. As expected, during baseline fixation, microsaccades corrected for a foveal motor error away from the preferred retinal locus of fixation (the so-called ocular position "set point" of the oculomotor system). However, we found that this relationship was violated during a short period immediately after cue onset; a subset of cue-directed "express microsaccades" that were highly precise in time and direction, and that were larger than regular microsaccades, occurred. These movements, having <100-ms latencies from cue onset, were triggered when fixational eye position was already at the oculomotor set point when the cue appeared; they were thus error-increasing rather than error-decreasing. Critically, even when no microsaccades occurred, fixational eye position itself was systematically deviated toward the cue, again with ~100-ms latency, suggesting that the oculomotor system establishes a new set point at different postcue times. This new set point later switched to being away from the cue after ~200-300 ms. Because eye position alters the location of retinal images, our results suggest that both eye position and microsaccades can be associated with performance changes in spatial cueing tasks. NEW & NOTEWORTHY Covert spatial cueing tasks are a workhorse for studying cognitive processing in humans and monkeys, but gaze is not perfectly stable during these tasks. We found that minute fixational eye position changes, independent of the more studied microsaccades, are not random in cueing tasks and are thus not "averaged out" in analyses. These changes can additionally dictate microsaccade times. Thus, in addition to microsaccadic influences, retinal image changes associated with fixational eye position are relevant for performance in cueing tasks.
Collapse
Affiliation(s)
- Xiaoguang Tian
- Werner Reichardt Centre for Integrative Neuroscience, Tuebingen University, Tuebingen, Germany.,Graduate School of Neural and Behavioural Sciences, International Max Planck Research School, Tuebingen University, Tuebingen, Germany.,Hertie Institute for Clinical Brain Research, Tuebingen University, Tuebingen, Germany
| | - Masatoshi Yoshida
- Department of System Neuroscience, National Institute for Physiological Sciences , Okazaki , Japan.,School of Life Science, The Graduate University for Advanced Studies , Hayama , Japan
| | - Ziad M Hafed
- Werner Reichardt Centre for Integrative Neuroscience, Tuebingen University, Tuebingen, Germany.,Hertie Institute for Clinical Brain Research, Tuebingen University, Tuebingen, Germany
| |
Collapse
|