On the possible roles of microsaccades and drifts in visual perception

Behind mouse eyes: The function and control of eye movements in mice

The mouse visual system has become the most popular model to study the cellular and circuit mechanisms of sensory processing. However, the importance of eye movements only started to be appreciated recently. Eye movements provide a basis for predictive sensing and deliver insights into various brain functions and dysfunctions. A plethora of knowledge on the central control of eye movements and their role in perception and behaviour arose from work on primates. However, an overview of various eye movements in mice and a comparison to primates is missing. Here, we review the eye movement types described to date in mice and compare them to those observed in primates. We discuss the central neuronal mechanisms for their generation and control. Furthermore, we review the mounting literature on eye movements in mice during head-fixed and freely moving behaviours. Finally, we highlight gaps in our understanding and suggest future directions for research.

Thalamocortical loops as temporal demodulators across senses

Sensory information is coded in space and in time. The organization of neuronal activity in space maintains straightforward relationships with the spatial organization of the perceived environment. In contrast, the temporal organization of neuronal activity is not trivially related to external features due to sensor motion. Still, the temporal organization shares similar principles across sensory modalities. Likewise, thalamocortical circuits exhibit common features across senses. Focusing on touch, vision, and audition, we review their shared coding principles and suggest that thalamocortical systems include circuits that allow analogous recoding mechanisms in all three senses. These thalamocortical circuits constitute oscillations-based phase-locked loops, that translate temporally-coded sensory information to rate-coded cortical signals, signals that can integrate information across sensory and motor modalities. The loop also allows predictive locking to the onset of future modulations of the sensory signal. The paper thus suggests a theoretical framework in which a common thalamocortical mechanism implements temporal demodulation across senses.

Fixation-related saccadic inhibition in free viewing in response to stimulus saliency

Microsaccades that occur during fixation were studied extensively in response to transient stimuli, showing a typical inhibition (Oculomotor Inhibition, OMI), and a later release with a latency that depends on stimulus saliency, attention, and expectations. Here, we investigated the hypothesis that in free viewing every saccade provides a new transient stimulation that should result in a stimulus-dependent OMI like a flashed presentation during fixation. Participants (N = 16) freely inspected static displays of randomly oriented Gabor texture images, with varied contrast and spatial frequency (SF) for periods of 10 s each. Eye tracking recordings were divided into epochs triggered by saccade landing (> 1 dva), and microsaccade latency relative to fixation onset was computed (msRT). We found that the msRT in free viewing was shorter for more salient stimuli (higher contrast or lower SF), as previously found for flashed stimuli. It increased with saccade size and decreased across successive saccades, but only for higher contrast, suggesting contrast-dependent repetition enhancement in free viewing. Our results indicate that visual stimulus-dependent inhibition of microsaccades also applies to free viewing. These findings are in agreement with the similarity found between event-related and fixation-related potentials and open the way for studies combining both approaches to study natural vision.

Oculo-retinal dynamics can explain the perception of minimal recognizable configurations

Natural vision is a dynamic and continuous process. Under natural conditions, visual object recognition typically involves continuous interactions between ocular motion and visual contrasts, resulting in dynamic retinal activations. In order to identify the dynamic variables that participate in this process and are relevant for image recognition, we used a set of images that are just above and below the human recognition threshold and whose recognition typically requires >2 s of viewing. We recorded eye movements of participants while attempting to recognize these images within trials lasting 3 s. We then assessed the activation dynamics of retinal ganglion cells resulting from ocular dynamics using a computational model. We found that while the saccadic rate was similar between recognized and unrecognized trials, the fixational ocular speed was significantly larger for unrecognized trials. Interestingly, however, retinal activation level was significantly lower during these unrecognized trials. We used retinal activation patterns and oculomotor parameters of each fixation to train a binary classifier, classifying recognized from unrecognized trials. Only retinal activation patterns could predict recognition, reaching 80% correct classifications on the fourth fixation (on average, ∼2.5 s from trial onset). We thus conclude that the information that is relevant for visual perception is embedded in the dynamic interactions between the oculomotor sequence and the image. Hence, our results suggest that ocular dynamics play an important role in recognition and that understanding the dynamics of retinal activation is crucial for understanding natural vision.

Closed loop motor-sensory dynamics in human vision

Vision is obtained with a continuous motion of the eyes. The kinematic analysis of eye motion, during any visual or ocular task, typically reveals two (kinematic) components: saccades, which quickly replace the visual content in the retinal fovea, and drifts, which slowly scan the image after each saccade. While the saccadic exchange of regions of interest (ROIs) is commonly considered to be included in motor-sensory closed-loops, it is commonly assumed that drifts function in an open-loop manner, that is, independent of the concurrent visual input. Accordingly, visual perception is assumed to be based on a sequence of open-loop processes, each initiated by a saccade-triggered retinal snapshot. Here we directly challenged this assumption by testing the dependency of drift kinematics on concurrent visual inputs using real-time gaze-contingent-display. Our results demonstrate a dependency of the trajectory on the concurrent visual input, convergence of speed to condition-specific values and maintenance of selected drift-related motor-sensory controlled variables, all strongly indicative of drifts being included in a closed-loop brain-world process, and thus suggesting that vision is inherently a closed-loop process.

Microsaccadic rate signatures correlate under monocular and binocular stimulation conditions

Microsaccades are involuntary eye movements occurring naturally during fixation. In this study, microsaccades were investigated under monocularly and binocularly stimulated conditions with respect to their directional distribution and rate signature, that refers to a curve reporting the frequency modulation of microsaccades over time. For monocular stimulation the left eye was covered by an infrared filter. In both stimulation conditions, participants fixated a Gabor patch presented randomly in orientation of 45° or 135° over a wide range of spatial frequencies appearing in the center of a monitor. Considering the microsaccadic directions, this study showed microsaccades to be preferably horizontally oriented in their mean direction, regardless of the spatial characteristics of the grating. Furthermore, this outcome was found to be consistent between both stimulation conditions. Moreover, this study found that the microsaccadic rate signature curve correlates between both stimulation conditions, while the curve given for binocular stimulation was already proposed as a tool for estimation of visual performance in the past.

Therefore, this study extends the applicability of microsaccades to clinical use, since parameters as contrast sensitivity, has been measured monocularly in the clinical attitude.

Visual feature tuning of superior colliculus neural reafferent responses after fixational microsaccades

The primate superior colliculus (SC) is causally involved in microsaccade generation. Moreover, visually responsive SC neurons across this structure's topographic map, even at peripheral eccentricities much larger than the tiny microsaccade amplitudes, exhibit significant modulations of evoked response sensitivity when stimuli appear perimicrosaccadically. However, during natural viewing, visual stimuli are normally stably present in the environment and are only shifted on the retina by eye movements. Here we investigated this scenario for the case of microsaccades, asking whether and how SC neurons respond to microsaccade-induced image jitter. We recorded neural activity from two male rhesus macaque monkeys. Within the response field (RF) of a neuron, there was a stable stimulus consisting of a grating of one of three possible spatial frequencies. The grating was stable on the display, but microsaccades periodically jittered the retinotopic RF location over it. We observed clear short-latency visual reafferent responses after microsaccades. These responses were weaker, but earlier (relative to new fixation onset after microsaccade end), than responses to sudden stimulus onsets without microsaccades. The reafferent responses clearly depended on microsaccade amplitude as well as microsaccade direction relative to grating orientation. Our results indicate that one way for microsaccades to influence vision is through modulating how the spatio-temporal landscape of SC visual neural activity represents stable stimuli in the environment. Such representation depends on the specific pattern of temporal luminance modulations expected from the relative relationship between eye movement vector (size and direction) on one hand and spatial visual pattern layout on the other.NEW & NOTEWORTHY Despite being diminutive, microsaccades still jitter retinal images. We investigated how such jitter affects superior colliculus (SC) activity. We found that SC neurons exhibit short-latency visual reafferent bursts after microsaccades. These bursts reflect not only the spatial luminance profiles of visual patterns but also how such profiles are shifted by eye movement size and direction. These results indicate that the SC continuously represents visual patterns, even as they are jittered by the smallest possible saccades.

Temporal Coding of Visual Space

Establishing a representation of space is a major goal of sensory systems. Spatial information, however, is not always explicit in the incoming sensory signals. In most modalities it needs to be actively extracted from cues embedded in the temporal flow of receptor activation. Vision, on the other hand, starts with a sophisticated optical imaging system that explicitly preserves spatial information on the retina. This may lead to the assumption that vision is predominantly a spatial process: all that is needed is to transmit the retinal image to the cortex, like uploading a digital photograph, to establish a spatial map of the world. However, this deceptively simple analogy is inconsistent with theoretical models and experiments that study visual processing in the context of normal motor behavior. We argue here that, as with other senses, vision relies heavily on temporal strategies and temporal neural codes to extract and represent spatial information.

Human gaze tracks attentional focusing in memorized visual space

Brain areas that control gaze are also recruited for covert shifts of spatial attention1-9. In the external space of perception, there is a natural ecological link between the control of gaze and spatial attention, as information sampled at covertly attended locations can inform where to look next2,10,11. Attention can also be directed internally to representations held within the spatial layout of visual working memory12-16. In such cases, the incentive for using attention to direct gaze disappears, as there are no external targets to scan. Here we investigate whether the oculomotor system of the brain also participates in attention focusing within the internal space of memory. Paradoxically, we reveal this participation through gaze behaviour itself. We demonstrate that selecting an item from visual working memory biases gaze in the direction of the memorized location of that item, despite there being nothing to look at and location memory never explicitly being probed. This retrospective 'gaze bias' occurs only when an item is not already in the internal focus of attention, and it predicts the performance benefit associated with the focusing of internal attention. We conclude that the oculomotor system also participates in focusing attention within memorized space, leaving traces all the way to the eyes.

The Importance of Sensory Processing in Mental Health: A Proposed Addition to the Research Domain Criteria (RDoC) and Suggestions for RDoC 2.0

The time is ripe to integrate burgeoning evidence of the important role of sensory and motor functioning in mental health within the National Institute of Mental Health's [NIMH] Research Domain Criteria [RDoC] framework (National Institute of Mental Health, n.d.a), a multi-dimensional method of characterizing mental functioning in health and disease across all neurobiological levels of analysis ranging from genetic to behavioral. As the importance of motor processing in psychopathology has been recognized (Bernard and Mittal, 2015; Garvey and Cuthbert, 2017; National Institute of Mental Health, 2019), here we focus on sensory processing. First, we review the current design of the RDoC matrix, noting sensory features missing despite their prevalence in multiple mental illnesses. We identify two missing classes of sensory symptoms that we widely define as (1) sensory processing, including sensory sensitivity and active sensing, and (2) domains of perceptual signaling, including interoception and proprioception, which are currently absent or underdeveloped in the perception construct of the cognitive systems domain. Then, we describe the neurobiological basis of these psychological constructs and examine why these sensory features are important for understanding psychopathology. Where appropriate, we examine links between sensory processing and the domains currently included in the RDoC matrix. Throughout, we emphasize how the addition of these sensory features to the RDoC matrix is important for understanding a range of mental health disorders. We conclude with the suggestion that a separate sensation and perception domain can enhance the current RDoC framework, while discussing what we see as important principles and promising directions for the future development and use of the RDoC.

Transient and sustained effects of stimulus properties on the generation of microsaccades

Saccades shift the gaze rapidly every few hundred milliseconds from one fixated location to the next, producing a flow of visual input into the visual system even in the absence of changes in the environment. During fixation, small saccades called microsaccades are produced 1-3 times per second, generating a flow of visual input. The characteristics of this visual flow are determined by the timings of the saccades and by the characteristics of the visual stimuli on which they are performed. Previous models of microsaccade generation have accounted for the effects of external stimulation on the production of microsaccades, but they have not considered the effects of the prolonged background stimulus on which microsaccades are performed. The effects of this stimulus on the process of microsaccade generation could be sustained, following its prolonged presentation, or transient, through the visual transients produced by the microsaccades themselves. In four experiments, we varied the properties of the constant displays and examined the resulting modulation of microsaccade properties: their sizes, their timings, and the correlations between properties of consecutive microsaccades. Findings show that displays of higher spatial frequency and contrast produce smaller microsaccades and longer minimal intervals between consecutive microsaccades; and smaller microsaccades are followed by smaller and delayed microsaccades. We explain these findings in light of previous models and suggest a conceptual model by which both sustained and transient effects of the stimulus have central roles in determining the generation of microsaccades.

A theory of how active behavior stabilises neural activity: Neural gain modulation by closed-loop environmental feedback

During active behaviours like running, swimming, whisking or sniffing, motor actions shape sensory input and sensory percepts guide future motor commands. Ongoing cycles of sensory and motor processing constitute a closed-loop feedback system which is central to motor control and, it has been argued, for perceptual processes. This closed-loop feedback is mediated by brainwide neural circuits but how the presence of feedback signals impacts on the dynamics and function of neurons is not well understood. Here we present a simple theory suggesting that closed-loop feedback between the brain/body/environment can modulate neural gain and, consequently, change endogenous neural fluctuations and responses to sensory input. We support this theory with modeling and data analysis in two vertebrate systems. First, in a model of rodent whisking we show that negative feedback mediated by whisking vibrissa can suppress coherent neural fluctuations and neural responses to sensory input in the barrel cortex. We argue this suppression provides an appealing account of a brain state transition (a marked change in global brain activity) coincident with the onset of whisking in rodents. Moreover, this mechanism suggests a novel signal detection mechanism that selectively accentuates active, rather than passive, whisker touch signals. This mechanism is consistent with a predictive coding strategy that is sensitive to the consequences of motor actions rather than the difference between the predicted and actual sensory input. We further support the theory by re-analysing previously published two-photon data recorded in zebrafish larvae performing closed-loop optomotor behaviour in a virtual swim simulator. We show, as predicted by this theory, that the degree to which each cell contributes in linking sensory and motor signals well explains how much its neural fluctuations are suppressed by closed-loop optomotor behaviour. More generally we argue that our results demonstrate the dependence of neural fluctuations, across the brain, on closed-loop brain/body/environment interactions strongly supporting the idea that brain function cannot be fully understood through open-loop approaches alone.

Animals actively exploring or interacting with their surroundings must process a cyclical flow of information from the environment through sensory receptors, the central nervous system, the musculoskeletal system and back to the environment. This closed-loop sensorimotor system is essential for an animal's ability to adapt and survive in complex environments. Importantly, closed loop feedback signals also regulate brainwide neural circuits for behavior. Specifically, the activity of coherent populations of neurons inform motor behaviours and in turn are influenced by sensory feedback signals mediated by the environment. We develop a theory that suggests that this feedback can explain the marked changes in large-scale neural dynamics and sensory processing (together referred to as brain state) that coincide with the onset of active behaviours. This feedback may contribute to flexible context dependent neural computations in brain systems.

A self-avoiding walk with neural delays as a model of fixational eye movements

Fixational eye movements show scaling behaviour of the positional mean-squared displacement with a characteristic transition from persistence to antipersistence for increasing time-lag. These statistical patterns were found to be mainly shaped by microsaccades (fast, small-amplitude movements). However, our re-analysis of fixational eye-movement data provides evidence that the slow component (physiological drift) of the eyes exhibits scaling behaviour of the mean-squared displacement that varies across human participants. These results suggest that drift is a correlated movement that interacts with microsaccades. Moreover, on the long time scale, the mean-squared displacement of the drift shows oscillations, which is also present in the displacement auto-correlation function. This finding lends support to the presence of time-delayed feedback in the control of drift movements. Based on an earlier non-linear delayed feedback model of fixational eye movements, we propose and discuss different versions of a new model that combines a self-avoiding walk with time delay. As a result, we identify a model that reproduces oscillatory correlation functions, the transition from persistence to antipersistence, and microsaccades.

Microsaccades are sensitive to word structure: A novel approach to study language processing

Microsaccades are miniature eye movements that occur involuntarily during fixation. They are typically inhibited following stimulus onset and are released from inhibition about 300 ms post-stimulus. Microsaccade-inhibition is modulated by low level features of visual stimuli, but it is currently unknown whether they are sensitive to higher level, abstract linguistic properties. To address this question, we measured the timing of microsaccades while subjects were presented with written Hebrew words and pronounceable nonwords (pseudowords). We manipulated the underlying structure of pseudowords such that half of them contained real roots while the other half contained invented roots. Importantly, orthographic similarity to real words was equated between the two conditions. Microsaccade onset was significantly slower following real-root compared to invented-root stimuli. Similar results were obtained when considering post-stimulus delay of eye blinks. Moreover, microsaccade-delay was positively and significantly correlated with measures of real-word similarity. These findings demonstrate, for the first time, sensitivity of microsaccades to linguistic structure. Because microsaccades are involuntary and can be measured in the absence of overt response, our results provide initial evidence that they can be used as a novel physiological measure in the study of language processes in healthy and clinical populations.

Dye-enhanced visualization of rat whiskers for behavioral studies

Visualization and tracking of the facial whiskers is required in an increasing number of rodent studies. Although many approaches have been employed, only high-speed videography has proven adequate for measuring whisker motion and deformation during interaction with an object. However, whisker visualization and tracking is challenging for multiple reasons, primary among them the low contrast of the whisker against its background. Here, we demonstrate a fluorescent dye method suitable for visualization of one or more rat whiskers. The process makes the dyed whisker(s) easily visible against a dark background. The coloring does not influence the behavioral performance of rats trained on a vibrissal vibrotactile discrimination task, nor does it affect the whiskers' mechanical properties.

Eye movements in the wild: Oculomotor control, gaze behavior & frames of reference

Understanding the brain's capacity to encode complex visual information from a scene and to transform it into a coherent perception of 3D space and into well-coordinated motor commands are among the outstanding questions in the study of integrative brain function. Eye movement methodologies have allowed us to begin addressing these questions in increasingly naturalistic tasks, where eye and body movements are ubiquitous and, therefore, the applicability of most traditional neuroscience methods restricted. This review explores foundational issues in (1) how oculomotor and motor control in lab experiments extrapolates into more complex settings and (2) how real-world gaze behavior in turn decomposes into more elementary eye movement patterns. We review the received typology of oculomotor patterns in laboratory tasks, and how they map onto naturalistic gaze behavior (or not). We discuss the multiple coordinate systems needed to represent visual gaze strategies, how the choice of reference frame affects the description of eye movements, and the related but conceptually distinct issue of coordinate transformations between internal representations within the brain.

Perception as a closed-loop convergence process

Perception of external objects involves sensory acquisition via the relevant sensory organs. A widely-accepted assumption is that the sensory organ is the first station in a serial chain of processing circuits leading to an internal circuit in which a percept emerges. This open-loop scheme, in which the interaction between the sensory organ and the environment is not affected by its concurrent downstream neuronal processing, is strongly challenged by behavioral and anatomical data. We present here a hypothesis in which the perception of external objects is a closed-loop dynamical process encompassing loops that integrate the organism and its environment and converging towards organism-environment steady-states. We discuss the consistency of closed-loop perception (CLP) with empirical data and show that it can be synthesized in a robotic setup. Testable predictions are proposed for empirical distinction between open and closed loop schemes of perception.