Microsaccades: small steps on a long way

Reading Specific Small Saccades Predict Individual Phonemic Awareness and Reading Speed

Small fixational eye-movements are a fundamental aspect of vision and thought to reflect fine shifts in covert attention during active viewing. While the perceptual benefits of these small eye movements have been demonstrated during a wide range of experimental tasks including during free viewing, their function during reading remains surprisingly unclear. Previous research demonstrated that readers with increased microsaccade rates displayed longer reading speeds. To what extent increased fixational eye movements are, however, specific to reading and might be indicative of reading skill deficits remains, however, unknown. To address this topic, we compared the eye movement scan paths of 13 neurotypical individuals and 13 subjects diagnosed with developmental dyslexia during short story reading and free viewing of natural scenes. We found that during reading only, dyslexics tended to display small eye movements more frequently compared to neurotypicals, though this effect was not significant at the population level, as it could also occur in slow readers not diagnosed as dyslexics. In line with previous research, neurotypical readers had twice as many regressive compared to progressive microsaccades, which did not occur during free viewing. In contrast, dyslexics showed similar amounts of regressive and progressive small fixational eye movements during both reading and free viewing. We also found that participants with smaller fixational saccades from both neurotypical and dyslexic samples displayed reduced reading speeds and lower scores during independent tests of reading skill. Slower readers also displayed greater variability in the landing points and temporal occurrence of their fixational saccades. Both the rate and spatio-temporal variability of fixational saccades were associated with lower phonemic awareness scores. As none of the observed differences between dyslexics and neurotypical readers occurred during control experiments with free viewing, the reported effects appear to be directly related to reading. In summary, our results highlight the predictive value of small saccades for reading skill, but not necessarily for developmental dyslexia.

Fixational eye movements following concussion

The purpose of this study was to evaluate fixational eye movements (FEMs) with high spatial and temporal resolution following concussion, where oculomotor symptoms and impairments are common. Concussion diagnosis was determined using current consensus guidelines. A retinal eye-tracking device, the tracking scanning laser ophthalmoscope (TSLO), was used to measure FEMs in adolescents and young adults following a concussion and in an unaffected control population. FEMs were quantified in two fixational paradigms: (1) when fixating on the center, or (2) when fixating on the corner of the TSLO imaging raster. Fixational saccade amplitude in recent concussion patients (≤ 21 days) was significantly greater, on average, in the concussion group (mean = 1.03°; SD = 0.36°) compared with the controls (mean = 0.82°; SD = 0.31°), when fixating on the center of the imaging raster (t = 2.87, df = 82, p = 0.005). These fixational saccades followed the main sequence and therefore also had greater peak velocity (t = 2.86, df = 82, p = 0.006) and peak acceleration (t = 2.80, df = 82, p = 0.006). These metrics significantly differentiated concussed from controls (AUC = 0.67-0.68, minimum p = 0.005). No group differences were seen for the drift metrics in either task or for any of the FEMs metrics in the corner-of-raster fixation task. Fixational saccade amplitudes were significantly different in the concussion group, but only when fixating on the center of the raster. This task specificity suggests that task optimization may improve differentiation and warrants further study. FEMs measured in the acute-to-subacute period of concussion recovery may provide a quick (<3 minutes), objective, sensitive, and accurate ocular dysfunction assessment. Future work should assess the impact of age, mechanism of injury, and post-concussion recovery on FEM alterations following concussion.

When Eyes Wander Around: Mind-Wandering as Revealed by Eye Movement Analysis with Hidden Markov Models

Mind-wandering has been shown to largely influence our learning efficiency, especially in the digital and distracting era nowadays. Detecting mind-wandering thus becomes imperative in educational scenarios. Here, we used a wearable eye-tracker to record eye movements during the sustained attention to response task. Eye movement analysis with hidden Markov models (EMHMM), which takes both spatial and temporal eye-movement information into account, was used to examine if participants' eye movement patterns can differentiate between the states of focused attention and mind-wandering. Two representative eye movement patterns were discovered through clustering using EMHMM: centralized and distributed patterns. Results showed that participants with the centralized pattern had better performance on detecting targets and rated themselves as more focused than those with the distributed pattern. This study indicates that distinct eye movement patterns are associated with different attentional states (focused attention vs. mind-wandering) and demonstrates a novel approach in using EMHMM to study attention. Moreover, this study provides a potential approach to capture the mind-wandering state in the classroom without interrupting the ongoing learning behavior.

The role of fixation disengagement and oculomotor preparation in gap saccade task is gap-duration dependent

The influence of internal brain state on behavioral performance is well illustrated by the gap-saccade task, in which saccades might be initiated with short latency (express saccade) or with long latency (regular saccade) even though the external visual condition is identical. Accumulated evidence has demonstrated that the internal brain state is different before the initiation of an express saccade than of a regular saccade. However, the reported origin of the fluctuation of internal brain state is disputed among previous studies, e.g., the fixation disengagement theory versus the oculomotor preparation theory. In the present study, we examined these two theories by analyzing the rate and direction of fixational saccades, i.e., small amplitude saccades during fixation period, because they could be modulated by internal brain state. Since fixation disengagement is not spatially tuned, it might affect the rate but not direction of fixational saccade. In contrast, oculomotor preparation can contain the spatial information for upcoming saccade, thus, it might have a distinct effect on fixational saccade direction. We found that the different spatiotemporal characteristics of fixational saccades among tasks with different gap durations reveals different driven force to change the internal brain state. Under short gap duration (100 ms), fixation disengagement plays a primary role in switching internal brain state. Conversely, under medium (200 ms) and long (400 ms) gap durations, oculomotor preparation plays a primary role. These results suggest that both fixation disengagement and oculomotor preparation can change the internal brain state, but their relative contributions are gap-duration dependent.

Role of the frontal eye field in human microsaccade responses: A TMS study

Microsaccade is a type of fixational eye movements that is modulated by various sensory and cognitive processes, and impact our visual perception. Although studies in monkeys have demonstrated a functional role for the superior colliculus and frontal eye field (FEF) in controlling microsaccades, our understanding of the neural mechanisms underlying the generation of microsaccades is still limited. By applying continuous theta-burst stimulation (cTBS) over the right FEF and the vertex, we investigated the role of the FEF in generating human microsaccade responses evoked by salient stimuli or by changes in background luminance. We observed higher microsaccade rates prior to target appearance, and larger rebound in microsaccade occurrence following salient stimuli, when disruptive cTBS was applied over FEF compared to vertex stimulation. Moreover, the microsaccade direction modulation after changes in background luminance was disrupted with FEF stimulation. Together, our results constitute the first evidence of FEF modulation in human microsaccade responses.

Time averaging and emerging nonergodicity upon resetting of fractional Brownian motion and heterogeneous diffusion processes

How different are the results of constant-rate resetting of anomalous-diffusion processes in terms of their ensemble-averaged versus time-averaged mean-squared displacements (MSDs versus TAMSDs) and how does stochastic resetting impact nonergodicity? We examine, both analytically and by simulations, the implications of resetting on the MSD- and TAMSD-based spreading dynamics of particles executing fractional Brownian motion (FBM) with a long-time memory, heterogeneous diffusion processes (HDPs) with a power-law space-dependent diffusivity D(x)=D_{0}|x|^{γ} and their "combined" process of HDP-FBM. We find, inter alia, that the resetting dynamics of originally ergodic FBM for superdiffusive Hurst exponents develops disparities in scaling and magnitudes of the MSDs and mean TAMSDs indicating weak ergodicity breaking. For subdiffusive HDPs we also quantify the nonequivalence of the MSD and TAMSD and observe a new trimodal form of the probability density function. For reset FBM, HDPs and HDP-FBM we compute analytically and verify by simulations the short-time MSD and TAMSD asymptotes and long-time plateaus reminiscent of those for processes under confinement. We show that certain characteristics of these reset processes are functionally similar despite a different stochastic nature of their nonreset variants. Importantly, we discover nonmonotonicity of the ergodicity-breaking parameter EB as a function of the resetting rate r. For all reset processes studied we unveil a pronounced resetting-induced nonergodicity with a maximum of EB at intermediate r and EB∼(1/r)-decay at large r. Alongside the emerging MSD-versus-TAMSD disparity, this r-dependence of EB can be an experimentally testable prediction. We conclude by discussing some implications to experimental systems featuring resetting dynamics.

Vision as oculomotor reward: cognitive contributions to the dynamic control of saccadic eye movements

Humans and other primates are equipped with a foveated visual system. As a consequence, we reorient our fovea to objects and targets in the visual field that are conspicuous or that we consider relevant or worth looking at. These reorientations are achieved by means of saccadic eye movements. Where we saccade to depends on various low-level factors such as a targets' luminance but also crucially on high-level factors like the expected reward or a targets' relevance for perception and subsequent behavior. Here, we review recent findings how the control of saccadic eye movements is influenced by higher-level cognitive processes. We first describe the pathways by which cognitive contributions can influence the neural oculomotor circuit. Second, we summarize what saccade parameters reveal about cognitive mechanisms, particularly saccade latencies, saccade kinematics and changes in saccade gain. Finally, we review findings on what renders a saccade target valuable, as reflected in oculomotor behavior. We emphasize that foveal vision of the target after the saccade can constitute an internal reward for the visual system and that this is reflected in oculomotor dynamics that serve to quickly and accurately provide detailed foveal vision of relevant targets in the visual field.

Looking ahead in working memory to guide sequential behaviour

Working memory can maintain multiple sensory representations to serve unfolding sequential behaviour, such as while making tea or planning a route. How the human mind juggles internal representations as they become relevant to guide sequential behaviour remains poorly understood. Specifically, while there is good evidence that we can flexibly switch priorities among representations in working memory^1-4, it is unclear how and when dormant memory representations are brought into focus during sequential behaviour. Capitalising on a recently established and temporally precise gaze marker of internal selection^5,6, we reveal that the focus in the mind moves to the next-relevant memory representation while behaviour associated with the presently relevant memory representation is still ongoing. Thus, like visual sampling of external objects in the world^7-9, internal visual sampling also 'looks ahead' to the next object in memory during sequential behaviour.

Micro-pursuit: A class of fixational eye movements correlating with smooth, predictable, small-scale target trajectories

Humans generate ocular pursuit movements when a moving target is tracked throughout the visual field. In this article, we show that pursuit can be generated and measured at small amplitudes, at the scale of fixational eye movements, and tag these eye movements as micro-pursuits. During micro-pursuits, gaze direction correlates with a target's smooth, predictable target trajectory. We measure similarity between gaze and target trajectories using a so-called maximally projected correlation and provide results in three experimental data sets. A first observation of micro-pursuit is provided in an implicit pursuit task, where observers were tasked to maintain their gaze fixed on a static cross at the center of screen, while reporting changes in perception of an ambiguous, moving (Necker) cube. We then provide two experimental paradigms and their corresponding data sets: a first replicating micro-pursuits in an explicit pursuit task, where observers had to follow a moving fixation cross (Cross), and a second with an unambiguous square (Square). Individual and group analyses provide evidence that micro-pursuits exist in both the Necker and Cross experiments but not in the Square experiment. The interexperiment analysis results suggest that the manipulation of stimulus target motion, task, and/or the nature of the stimulus may play a role in the generation of micro-pursuits.

Age effects on saccadic suppression of luminance and color

Saccadic eye movements modulate visual perception: they initiate and terminate high acuity vision at a certain location in space, but before and during their execution visual contrast sensitivity is strongly attenuated for 100 to 200 ms. Transient perisaccadic perceptual distortions are assumed to be an important mechanism to maintain visual stability. Little is known about age effects on saccadic suppression, even though for healthy adults other major age-related changes are well documented, like a decrease of visual contrast sensitivity for intermediate and high spatial frequencies or an increase of saccade latencies. Here, we tested saccadic suppression of luminance and isoluminant chromatic flashes in 100 participants from eight to 78 years. To estimate the effect of saccadic suppression on contrast sensitivity, we used a two-alternative forced choice (2AFC) design and an adaptive staircase procedure to modulate the luminance or chromatic contrast of a flashed detection target during fixation and 15 ms after saccade onset. The target was a single horizontal luminance or chromatic line flashed 2° above or below the fixation or saccade target. Compared to fixation, average perisaccadic contrast sensitivity decreased significantly by 66% for luminance and by 36% for color. A significant correlation was found for the strength of saccadic suppression of luminance and color. However, a small age effect was found only for the strength of saccadic suppression of luminance, which increased from 64% to 70% from young to old age. We conclude that saccadic suppression for luminance and color is present in most participants independent of their age and that mechanisms of suppression stay relatively stable during healthy aging.

Microsaccades and attention in a high-acuity visual alignment task

While aiming and shooting, we make tiny eye movements called microsaccades that shift gaze between task-relevant objects within a small region of the visual field. However, in the brief period before pressing the trigger, microsaccades are suppressed. This might be due to the lack of a requirement to shift gaze as the retinal images of the two objects begin to overlap on the fovea. Alternatively, we might actively suppress microsaccades to prevent any disturbances in visual perception caused by microsaccades around the time of their occurrence and their subsequent effect on shooting performance. In this study we looked at microsaccade rates while participants performed a simulated shooting task under two conditions: a normal condition in which they moved their eyes freely, and an eccentric condition in which they maintained gaze on a fixed target while performing the shooting task at 5° eccentricity. As expected, microsaccade rate dropped near the end of the task in the normal viewing condition. However, we also found the same decrease for the eccentric condition in which microsaccades did not shift gaze between the task objects. Microsaccades are also produced in response to shifts in covert attention. To test whether disengagement of covert attention from the eccentric shooting location caused the drop in microsaccade rate, we monitored the location of participants' spatial attention by using a Rapid Serial Visual Presentation (RSVP) task simultaneously at a location opposite to the shooting task. Target letter detection at the RSVP location did not improve during the drop in microsaccade rate, suggesting that covert attention was maintained at the shooting task location. We conclude that in addition to their usual gaze-shifting function, microsaccades during fine-acuity tasks might be modulated by cognitive processes other than spatial attention.

Gaze During Locomotion in Virtual Reality and the Real World

How vision guides gaze in realistic settings has been researched for decades. Human gaze behavior is typically measured in laboratory settings that are well controlled but feature-reduced and movement-constrained, in sharp contrast to real-life gaze control that combines eye, head, and body movements. Previous real-world research has shown environmental factors such as terrain difficulty to affect gaze; however, real-world settings are difficult to control or replicate. Virtual reality (VR) offers the experimental control of a laboratory, yet approximates freedom and visual complexity of the real world (RW). We measured gaze data in 8 healthy young adults during walking in the RW and simulated locomotion in VR. Participants walked along a pre-defined path inside an office building, which included different terrains such as long corridors and flights of stairs. In VR, participants followed the same path in a detailed virtual reconstruction of the building. We devised a novel hybrid control strategy for movement in VR: participants did not actually translate: forward movements were controlled by a hand-held device, rotational movements were executed physically and transferred to the VR. We found significant effects of terrain type (flat corridor, staircase up, and staircase down) on gaze direction, on the spatial spread of gaze direction, and on the angular distribution of gaze-direction changes. The factor world (RW and VR) affected the angular distribution of gaze-direction changes, saccade frequency, and head-centered vertical gaze direction. The latter effect vanished when referencing gaze to a world-fixed coordinate system, and was likely due to specifics of headset placement, which cannot confound any other analyzed measure. Importantly, we did not observe a significant interaction between the factors world and terrain for any of the tested measures. This indicates that differences between terrain types are not modulated by the world. The overall dwell time on navigational markers did not differ between worlds. The similar dependence of gaze behavior on terrain in the RW and in VR indicates that our VR captures real-world constraints remarkably well. High-fidelity VR combined with naturalistic movement control therefore has the potential to narrow the gap between the experimental control of a lab and ecologically valid settings.

Prestimulus inhibition of eye movements reflects temporal expectation rather than time estimation

Eye movements are inhibited prior to the occurrence of temporally predictable events. This 'oculomotor inhibition effect' has been demonstrated with various tasks and modalities. Specifically, it was shown that when intervals between cue and target are fixed, saccade rate prior to the target is lower than when they are varied. However, it is still an open question whether this effect is linked to temporal expectation to the predictable target, or to the duration estimation of the interval preceding it. Here, we examined this question in 20 participants while they performed an implicit temporal expectation and an explicit time estimation task. In each trial, following cue onset, two consecutive grating patches were presented, each preceded by an interval. Temporal expectation was manipulated by setting the first interval duration to be either fixed or varied within each block. Participants were requested to compare either the durations of the two intervals (time estimation), or the tilts of the two grating patches (temporal expectation). Saccade rate, measured prior to the first grating, was lower in the fixed relative to the varied condition of both tasks. This suggests that the inhibition effect is elicited by target predictability and indicates that it is linked to temporal expectation, rather than to time estimation processes. Additionally, this finding suggests that the oculomotor inhibition is independent of motor readiness, as it was elicited even when no response was required. We conclude that the prestimulus oculomotor inhibition effect can be used as a marker of temporal expectation, and discuss its potential underlying mechanisms.

The Tobii Pro Spectrum: A useful tool for studying microsaccades?

Due to its reported high sampling frequency and precision, the Tobii Pro Spectrum is of potential interest to researchers who want to study small eye movements during fixation. We test how suitable the Tobii Pro Spectrum is for research on microsaccades by computing data-quality measures and common properties of microsaccades and comparing these to the currently most used system in this field: the EyeLink 1000 Plus. Results show that the EyeLink data provide higher RMS precision and microsaccade rates compared with data acquired with the Tobii Pro Spectrum. However, both systems provide microsaccades with similar directions and shapes, as well as rates consistent with previous literature. Data acquired at 1200 Hz with the Tobii Pro Spectrum provide results that are more similar to the EyeLink, compared to data acquired at 600 Hz. We conclude that the Tobii Pro Spectrum is a useful tool for researchers investigating microsaccades.

Is apparent fixational drift in eye-tracking data due to filters or eyeball rotation?

Eye trackers are sometimes used to study the miniature eye movements such as drift that occur while observers fixate a static location on a screen. Specifically, analysis of such eye-tracking data can be performed by examining the temporal spectrum composition of the recorded gaze position signal, allowing to assess its color. However, not only rotations of the eyeball but also filters in the eye tracker may affect the signal’s spectral color. Here, we therefore ask whether colored, as opposed to white, signal dynamics in eye-tracking recordings reflect fixational eye movements, or whether they are instead largely due to filters. We recorded gaze position data with five eye trackers from four pairs of human eyes performing fixation sequences, and also from artificial eyes. We examined the spectral color of the gaze position signals produced by the eye trackers, both with their filters switched on, and for unfiltered data. We found that while filtered data recorded from both human and artificial eyes were colored for all eye trackers, for most eye trackers the signal was white when examining both unfiltered human and unfiltered artificial eye data. These results suggest that color in the eye-movement recordings was due to filters for all eye trackers except the most precise eye tracker where it may partly reflect fixational eye movements. As such, researchers studying fixational eye movements should be careful to examine the properties of the filters in their eye tracker to ensure they are studying eyeball rotation and not filter properties.

The role of eye movements in manual interception: A mini-review

When we catch a moving object in mid-flight, our eyes and hands are directed toward the object. Yet, the functional role of eye movements in guiding interceptive hand movements is not yet well understood. This review synthesizes emergent views on the importance of eye movements during manual interception with an emphasis on laboratory studies published since 2015. We discuss the role of eye movements in forming visual predictions about a moving object, and for enhancing the accuracy of interceptive hand movements through feedforward (extraretinal) and feedback (retinal) signals. We conclude by proposing a framework that defines the role of human eye movements for manual interception accuracy as a function of visual certainty and object motion predictability.

Dissociable Cortical and Subcortical Mechanisms for Mediating the Influences of Visual Cues on Microsaccadic Eye Movements

Visual selection in primates is intricately linked to eye movements, which are generated by a network of cortical and subcortical neural circuits. When visual selection is performed covertly, without foveating eye movements toward the selected targets, a class of fixational eye movements, called microsaccades, is still involved. Microsaccades are small saccades that occur when maintaining precise gaze fixation on a stationary point, and they exhibit robust modulations in peripheral cueing paradigms used to investigate covert visual selection mechanisms. These modulations consist of changes in both microsaccade directions and frequencies after cue onsets. Over the past two decades, the properties and functional implications of these modulations have been heavily studied, revealing a potentially important role for microsaccades in mediating covert visual selection effects. However, the neural mechanisms underlying cueing effects on microsaccades are only beginning to be investigated. Here we review the available causal manipulation evidence for these effects' cortical and subcortical substrates. In the superior colliculus (SC), activity representing peripheral visual cues strongly influences microsaccade direction, but not frequency, modulations. In the cortical frontal eye fields (FEF), activity only compensates for early reflexive effects of cues on microsaccades. Using evidence from behavior, theoretical modeling, and preliminary lesion data from the primary visual cortex and microstimulation data from the lower brainstem, we argue that the early reflexive microsaccade effects arise subcortically, downstream of the SC. Overall, studying cueing effects on microsaccades in primates represents an important opportunity to link perception, cognition, and action through unaddressed cortical-subcortical neural interactions. These interactions are also likely relevant in other sensory and motor modalities during other active behaviors.

Investigating Arousal, Saccade Preparation, and Global Luminance Effects on Microsaccade Behavior

Microsaccades, small saccadic eye movements occurring during fixation, have been suggested to be modulated by various sensory, cognitive, and affective processes relating to arousal. Although the modulation of fatigue-related arousal on microsaccade behavior has previously been characterized, the influence of other aspects of arousal, such as emotional arousal, is less understood. Moreover, microsaccades are modulated by cognitive processes (e.g., voluntary saccade preparation) that could also be linked to arousal. To investigate the influence of emotional arousal, saccade preparation, and global luminance levels on microsaccade behavior, emotional auditory stimuli were presented prior to the onset of a fixation cue whose color indicated to look either at the peripheral stimulus (pro-saccade) or in the opposite direction of the stimulus (anti-saccade). Microsaccade behavior was found to be significantly modulated by saccade preparation and global luminance level, but not emotional arousal. In the pro- and anti-saccade task, microsaccade rate was lower during anti-saccade preparation as compared to pro-saccade preparation, though microsaccade dynamics were comparable during both trial types. Our results reveal a differential role of arousal linked to emotion, fatigue, saccade preparation, and global luminance level on microsaccade behavior.

Active vision at the foveal scale in the primate superior colliculus

The primate superior colliculus (SC) has recently been shown to possess both a large foveal representation as well as a varied visual processing repertoire. This structure is also known to contribute to eye movement generation. Here, we describe our current understanding of how SC visual and movement-related signals interact within the realm of small eye movements associated with the foveal scale of visuomotor behavior. Within the SC's foveal representation, there is a full spectrum of visual, visual-motor, and motor-related discharge for fixational eye movements. Moreover, a substantial number of neurons only emit movement-related discharge when microsaccades are visually guided, but not when similar movements are generated toward a blank. This represents a particularly striking example of integrating vision and action at the foveal scale. Beyond that, SC visual responses themselves are strongly modulated, and in multiple ways, by the occurrence of small eye movements. Intriguingly, this impact can extend to eccentricities well beyond the fovea, causing both sensitivity enhancement and suppression in the periphery. Because of large foveal magnification of neural tissue, such long-range eccentricity effects are neurally warped into smaller differences in anatomical space, providing a structural means for linking peripheral and foveal visual modulations around fixational eye movements. Finally, even the retinal-image visual flows associated with tiny fixational eye movements are signaled fairly faithfully by peripheral SC neurons with relatively large receptive fields. These results demonstrate how studying active vision at the foveal scale represents an opportunity for understanding primate vision during natural behaviors involving ever-present foveating eye movements.NEW & NOTEWORTHY The primate superior colliculus (SC) is ideally suited for active vision at the foveal scale: it enables detailed foveal visual analysis by accurately driving small eye movements, and it also possesses a visual processing machinery that is sensitive to active eye movement behavior. Studying active vision at the foveal scale in the primate SC is informative for broader aspects of active perception, including the overt and covert processing of peripheral extra-foveal visual scene locations.

Oculomotor inhibition during smooth pursuit and its dependence on contrast sensitivity

Our eyes are never still, but tend to "freeze" in response to stimulus onset. This effect is termed "oculomotor inhibition" (OMI); its magnitude and time course depend on the stimulus parameters, attention, and expectation. We previously showed that the time course and duration of microsaccade and spontaneous eye-blink inhibition provide an involuntary measure of low-level visual properties such as contrast sensitivity during fixation. We investigated whether this stimulus-dependent inhibition also occurs during smooth pursuit, for both the catch-up saccades and the pursuit itself. Observers followed a target with continuous back-and-forth horizontal motion while a Gabor patch was briefly flashed centrally with varied spatial frequency and contrast. Catch-up saccades of the size of microsaccades had a similar pattern of inhibition as microsaccades during fixation, with stronger inhibition onset and faster inhibition release for more salient stimuli. Moreover, a similar stimulus dependency of inhibition was shown for pursuit latencies and peak velocity. Additionally, microsaccade latencies at inhibition release, peak pursuit velocities, and latencies at minimum pursuit velocity were correlated with contrast sensitivity. We demonstrated the generality of OMI to smooth pursuit for both microsaccades and the pursuit itself and its close relation to the low-level processes that define saliency, such as contrast sensitivity.

Characterizing gaze position signals and synthesizing noise during fixations in eye-tracking data

The magnitude of variation in the gaze position signals recorded by an eye tracker, also known as its precision, is an important aspect of an eye tracker’s data quality. However, data quality of eye-tracking signals is still poorly understood. In this paper, we therefore investigate the following: (1) How do the various available measures characterizing eye-tracking data during fixation relate to each other? (2) How are they influenced by signal type? (3) What type of noise should be used to augment eye-tracking data when evaluating eye-movement analysis methods? To support our analysis, this paper presents new measures to characterize signal type and signal magnitude based on RMS-S2S and STD, two established measures of precision. Simulations are performed to investigate how each of these measures depends on the number of gaze position samples over which they are calculated, and to reveal how RMS-S2S and STD relate to each other and to measures characterizing the temporal spectrum composition of the recorded gaze position signal. Further empirical investigations were performed using gaze position data recorded with five eye trackers from human and artificial eyes. We found that although the examined eye trackers produce gaze position signals with different characteristics, the relations between precision measures derived from simulations are borne out by the data. We furthermore conclude that data with a range of signal type values should be used to assess the robustness of eye-movement analysis methods. We present a method for generating artificial eye-tracker noise of any signal type and magnitude.

Regression-based analysis of combined EEG and eye-tracking data: Theory and applications

Fixation-related potentials (FRPs), neural responses aligned to the end of saccades, are a promising tool for studying the dynamics of attention and cognition under natural viewing conditions. In the past, four methodological problems have complicated the analysis of such combined eye-tracking/electroencephalogram experiments: (1) the synchronization of data streams, (2) the removal of ocular artifacts, (3) the condition-specific temporal overlap between the brain responses evoked by consecutive fixations, and (4) the fact that numerous low-level stimulus and saccade properties also influence the postsaccadic neural responses. Although effective solutions exist for the first two problems, the latter two are only beginning to be addressed. In the current paper, we present and review a unified regression-based framework for FRP analysis that allows us to deconvolve overlapping potentials while also controlling for both linear and nonlinear confounds on the FRP waveform. An open software implementation is provided for all procedures. We then demonstrate the advantages of this proposed (non)linear deconvolution modeling approach for data from three commonly studied paradigms: face perception, scene viewing, and reading. First, for a traditional event-related potential (ERP) face recognition experiment, we show how this technique can separate stimulus ERPs from overlapping muscle and brain potentials produced by small (micro)saccades on the face. Second, in natural scene viewing, we model and isolate multiple nonlinear effects of saccade parameters on the FRP. Finally, for a natural sentence reading experiment using the boundary paradigm, we show how it is possible to study the neural correlates of parafoveal preview after removing spurious overlap effects caused by the associated difference in average fixation time. Our results suggest a principal way of measuring reliable eye movement-related brain activity during natural vision.

Performance Analysis of a Head and Eye Motion-Based Control Interface for Assistive Robots

Assistive robots support people with limited mobility in their everyday life activities and work. However, most of the assistive systems and technologies for supporting eating and drinking require a residual mobility in arms or hands. For people without residual mobility, different hands-free controls have been developed. For hands-free control, the combination of different modalities can lead to great advantages and improved control. The novelty of this work is a new concept to control a robot using a combination of head and eye motions. The control unit is a mobile, compact and low-cost multimodal sensor system. A Magnetic Angular Rate Gravity (MARG)-sensor is used to detect head motion and an eye tracker enables the system to capture the user’s gaze. To analyze the performance of the two modalities, an experimental evaluation with ten able-bodied subjects and one subject with tetraplegia was performed. To assess discrete control (event-based control), a button activation task was performed. To assess two-dimensional continuous cursor control, a Fitts’s Law task was performed. The usability study was related to a use-case scenario with a collaborative robot assisting a drinking action. The results of the able-bodied subjects show no significant difference between eye motions and head motions for the activation time of the buttons and the throughput, while, using the eye tracker in the Fitts’s Law task, the error rate was significantly higher. The subject with tetraplegia showed slightly better performance for button activation when using the eye tracker. In the use-case, all subjects were able to use the control unit successfully to support the drinking action. Due to the limited head motion of the subject with tetraplegia, button activation with the eye tracker was slightly faster than with the MARG-sensor. A further study with more subjects with tetraplegia is planned, in order to verify these results.

Dependence of the stimulus-driven microsaccade rate signature in rhesus macaque monkeys on visual stimulus size and polarity

Microsaccades have a steady rate of occurrence during maintained gaze fixation, which gets transiently modulated by abrupt sensory stimuli. Such modulation, characterized by a rapid reduction in microsaccade frequency followed by a stronger rebound phase of high microsaccade rate, is often described as the microsaccadic rate signature, owing to its stereotyped nature. Here, we investigated the impacts of stimulus polarity (luminance increments or luminance decrements relative to background luminance) and size on the microsaccadic rate signature. We presented brief, behaviorally irrelevant visual flashes consisting of large or small, white or black stimuli over an otherwise gray image background. Both large and small stimuli caused robust early microsaccadic inhibition, but postinhibition microsaccade rate rebound was significantly delayed and weakened for large stimuli when compared with small ones. Critically, small black stimuli were associated with stronger modulations in the microsaccade rate signature than small white stimuli, particularly in the postinhibition rebound phase, and black stimuli also amplified the incidence of early stimulus-directed microsaccades. Our results demonstrate that the microsaccadic rate signature is sensitive to stimulus size and polarity, and they point to dissociable neural mechanisms underlying early microsaccadic inhibition after stimulus onset and later microsaccadic rate rebound at longer times thereafter. These results also demonstrate early access of oculomotor control circuitry to diverse sensory representations, particularly for momentarily inhibiting saccade generation with short latencies.NEW & NOTEWORTHY Microsaccade rate is transiently reduced after sudden stimulus onsets, and then strongly rebounds before returning to baseline. We explored the influence of stimulus polarity (black vs. white) and size on this "rate signature." Large stimuli caused more muted microsaccadic rebound than small ones, and microsaccadic rebound was also differentially affected by black versus white stimuli, particularly with small stimuli. These results suggest dissociated neural mechanisms for microsaccadic inhibition and rebound in the microsaccadic rate signature.

Eye motion correction algorithm for OCT-based corneal topography

With its sequential image acquisition, OCT-based corneal topography is often susceptible to measurement errors due to eye motion. We have developed a novel algorithm to detect eye motion and minimize its impact on OCT topography maps. We applied the eye motion correction algorithm to corneal topographic scans acquired using a 70 kHz spectral-domain OCT device. OCT corneal topographic measurements were compared to those from a rotating Scheimpflug camera topographer. The motion correction algorithm provided a 2-4 fold improvement in the repeatability of OCT topography and its agreement with the standard Scheimpflug topographer. The repeatability of OCT Zernike-based corneal mean power, cardinal astigmatism, and oblique astigmatism after motion detection was 0.14 D, 0.28 D, and 0.24 D, respectively. The average differences between the two devices were 0.19 D for simulated keratometry-based corneal mean power, 0.23 D for cardinal astigmatism, and 0.25 D for oblique astigmatism. Our eye motion detection method can be applied to any OCT device, and it therefore represents a powerful tool for improving OCT topography.

Disentangling top-down and bottom-up influences on blinks in the visual and auditory domain

Sensory input as well as cognitive factors can drive the modulation of blinking. Our aim was to dissociate sensory driven bottom-up from cognitive top-down influences on blinking behavior and compare these influences between the auditory and the visual domain. Using an oddball paradigm, we found a significant pre-stimulus decrease in blink probability for visual input compared to auditory input. Sensory input further led to an early post-stimulus blink increase in both modalities if a task demanded attention to the input. Only visual input caused a pronounced early increase without a task. In case of a target or the omission of a stimulus (as compared to standard input), an additional late increase in blink rate was found in the auditory and visual domain. This suggests that blink modulation must be based on the interpretation of the input, but does not need any sensory input at all to occur. Our results show a complex modulation of blinking based on top-down factors such as prediction and attention in addition to sensory-based influences. The magnitude of the modulation is mainly influenced by general attentional demands, while the latency of this modulation allows dissociating general from specific top-down influences that are independent of the sensory domain.

Custom extraction of macular ganglion cell-inner plexiform layer thickness more precisely co-localizes structural measurements with visual fields test grids

We aimed to evaluate methods of extracting optical coherence tomography (OCT)-derived macular ganglion cell-inner plexiform layer (GCIPL) thickness measurements over retinal locations corresponding to standard visual field (VF) test grids. A custom algorithm was developed to automatically extract GCIPL thickness measurements from locations corresponding to Humphrey Field Analyser 10-2 and 30-2 test grids over Goldmann II, III and V stimulus sizes from a healthy cohort of 478 participants. Differences between GCIPL thickness measurements based on VF test grids (VF-based paradigms) and the 8 × 8 grid, as per instrument review software, were analyzed, as were impacts of fovea to optic disc tilt and areas over which GCIPL thickness measurements were extracted. Significant differences between the VF-based paradigms and the 8 × 8 grid were observed at up to 55% of locations across the macula, with the greatest deviations at the fovea (median 25.5 μm, 95% CI 25.24–25.72 μm, P < .0001). While significant correlations with fovea to optic disc tilt were noted at up to 33% of locations distributed 6°–8° from the foveal center, there were no marked differences in GCIPL thickness measurements between VF-based paradigms using different stimulus sizes. As such, standard high-density OCT measurement paradigms do not adequately reflect GCIPL measurements at retinal locations tested with standard VF patterns, with the central macular region contributing most to the observed differences and with further correction required for fovea to optic disc tilt. Spatial direction of GCIPL thickness measurements will improve future comparisons of structure and function, thereby improving methods designed to detect pathology affecting the inner retina.

Torsional component of microsaccades during fixation and quick phases during optokinetic stimulation

While many studies have characterized the eye movements during visual fixation, includ-ing microsaccades, in most cases only horizontal and vertical components have been rec-orded and analyzed. Thus, little is known about the torsional component of microsaccades. We took advantage of a newly developed software and hardware to record eye movements around the three axes of rotation during fixation and torsional optokinetic stimulus. We found that the average amplitude of the torsional component of microsaccades during fixation was 0.34 ± 0.07 degrees with velocities following a main sequence with a slope comparable to the horizontal and vertical components. We also found the size of the tor-sional displacement during microsaccades was correlated with the horizontal but not the vertical component. In the presence of an optokinetic stimulus a nystagmus was induced producing a more frequent and larger torsional quick phases compared to microsaccades produced during fixation with a stationary stimulus. The torsional component and the vertical vergence component of quick phases grew larger with higher velocities. Addition-ally, our results validate and show the feasibility of recording torsional eye movements using video eye tracking in a desktop mounted setup.

Closed loop motor-sensory dynamics in human vision

Vision is obtained with a continuous motion of the eyes. The kinematic analysis of eye motion, during any visual or ocular task, typically reveals two (kinematic) components: saccades, which quickly replace the visual content in the retinal fovea, and drifts, which slowly scan the image after each saccade. While the saccadic exchange of regions of interest (ROIs) is commonly considered to be included in motor-sensory closed-loops, it is commonly assumed that drifts function in an open-loop manner, that is, independent of the concurrent visual input. Accordingly, visual perception is assumed to be based on a sequence of open-loop processes, each initiated by a saccade-triggered retinal snapshot. Here we directly challenged this assumption by testing the dependency of drift kinematics on concurrent visual inputs using real-time gaze-contingent-display. Our results demonstrate a dependency of the trajectory on the concurrent visual input, convergence of speed to condition-specific values and maintenance of selected drift-related motor-sensory controlled variables, all strongly indicative of drifts being included in a closed-loop brain-world process, and thus suggesting that vision is inherently a closed-loop process.

Reduced fixation stability induced by peripheral viewing does not contribute to crowding

Attending to peripheral visual targets while maintaining central fixation, a process that involves covert attention, reduces fixation stability. Here, we tested the hypothesis that changes in fixation stability induced by peripheral viewing contribute to crowding in peripheral vision by increasing positional uncertainty. We first assessed whether fixation was less stable during peripheral versus central (foveal) viewing for both crowded and uncrowded stimuli. We then tested whether fixation stability during peripheral viewing was associated with the extent of crowding. Fourteen participants performed a tumbling E orientation discrimination task at three different eccentricities (0°, 5°, 10°). The target was presented with or without flankers. Fixational eye movements were measured using an infrared video-based eyetracker. A central fixation cross was provided for the two peripheral viewing conditions, and optotype size was scaled for each eccentricity. Discrimination of appropriately scaled uncrowded stimuli was unaffected by eccentricity, whereas discrimination of crowded stimuli deteriorated dramatically with eccentricity, despite scaling. Both crowded and uncrowded peripheral stimuli were associated with reduced fixation stability, increased microsaccadic amplitude, and a greater proportion of horizontal microsaccades relative to centrally presented stimuli. However, these effects were not associated with the magnitude of crowding. This suggests that reduced fixation stability due to peripheral viewing does not contribute to crowding in peripheral vision.

Microsaccadic Eye Movements but not Pupillary Dilation Response Characterizes the Crossmodal Freezing Effect

In typical spatial orienting tasks, the perception of crossmodal (e.g., audiovisual) stimuli evokes greater pupil dilation and microsaccade inhibition than unisensory stimuli (e.g., visual). The characteristic pupil dilation and microsaccade inhibition has been observed in response to "salient" events/stimuli. Although the "saliency" account is appealing in the spatial domain, whether this occurs in the temporal context remains largely unknown. Here, in a brief temporal scale (within 1 s) and with the working mechanism of involuntary temporal attention, we investigated how eye metric characteristics reflect the temporal dynamics of perceptual organization, with and without multisensory integration. We adopted the crossmodal freezing paradigm using the classical Ternus apparent motion. Results showed that synchronous beeps biased the perceptual report for group motion and triggered the prolonged sound-induced oculomotor inhibition (OMI), whereas the sound-induced OMI was not obvious in a crossmodal task-free scenario (visual localization without audiovisual integration). A general pupil dilation response was observed in the presence of sounds in both visual Ternus motion categorization and visual localization tasks. This study provides the first empirical account of crossmodal integration by capturing microsaccades within a brief temporal scale; OMI but not pupillary dilation response characterizes task-specific audiovisual integration (shown by the crossmodal freezing effect).

Concealed information revealed by involuntary eye movements on the fringe of awareness in a mock terror experiment

Involuntary eye movements during fixation are typically inhibited following stimulus onset (Oculomotor Inhibition, OMI), depending on the stimulus saliency and attention, with an earlier and longer OMI for barely visible familiar faces. However, it is still unclear whether OMI regarding familiarities and perceptual saliencies differ enough to allow a reliable OMI-based concealed information test (CIT). In a “mock terror” experiment with 25 volunteers, 13 made a concealed choice of a “terror-target” (one of eight), associated with 3 probes (face, name, and residence), which they learned watching text and videos, whereas 12 “innocents” pre-learned nothing. All participants then watched ~ 25 min of repeated brief presentations of barely visible (masked) stimuli that included the 8 potential probes, as well as a universally familiar face as a reference, while their eye movements were monitored. We found prolonged and deviant OMI regarding the probes. Incorporated with the individual pattern of responses to the reference, our analysis correctly identified 100% of the terror targets, and was 95% correct in discriminating “terrorists” from “innocents”. Our results provide a “proof of concept” for a novel approach to CIT, based on involuntary oculomotor responses to barely visible stimuli, individually tailored, and with high accuracy and theoretical resistance to countermeasures.

Microsaccadic rate signatures correlate under monocular and binocular stimulation conditions

Microsaccades are involuntary eye movements occurring naturally during fixation. In this study, microsaccades were investigated under monocularly and binocularly stimulated conditions with respect to their directional distribution and rate signature, that refers to a curve reporting the frequency modulation of microsaccades over time. For monocular stimulation the left eye was covered by an infrared filter. In both stimulation conditions, participants fixated a Gabor patch presented randomly in orientation of 45° or 135° over a wide range of spatial frequencies appearing in the center of a monitor. Considering the microsaccadic directions, this study showed microsaccades to be preferably horizontally oriented in their mean direction, regardless of the spatial characteristics of the grating. Furthermore, this outcome was found to be consistent between both stimulation conditions. Moreover, this study found that the microsaccadic rate signature curve correlates between both stimulation conditions, while the curve given for binocular stimulation was already proposed as a tool for estimation of visual performance in the past.

Therefore, this study extends the applicability of microsaccades to clinical use, since parameters as contrast sensitivity, has been measured monocularly in the clinical attitude.

Rapid stimulus-driven modulation of slow ocular position drifts

The eyes are never still during maintained gaze fixation. When microsaccades are not occurring, ocular position exhibits continuous slow changes, often referred to as drifts. Unlike microsaccades, drifts remain to be viewed as largely random eye movements. Here we found that ocular position drifts can, instead, be very systematically stimulus-driven, and with very short latencies. We used highly precise eye tracking in three well trained macaque monkeys and found that even fleeting (~8 ms duration) stimulus presentations can robustly trigger transient and stimulus-specific modulations of ocular position drifts, and with only approximately 60 ms latency. Such drift responses are binocular, and they are most effectively elicited with large stimuli of low spatial frequency. Intriguingly, the drift responses exhibit some image pattern selectivity, and they are not explained by convergence responses, pupil constrictions, head movements, or starting eye positions. Ocular position drifts have very rapid access to exogenous visual information.

Oculomotor freezing reflects tactile temporal expectation and aids tactile perception

The oculomotor system keeps the eyes steady in expectation of visual events. Here, recording microsaccades while people performed a tactile, frequency discrimination task enabled us to test whether the oculomotor system shows an analogous preparatory response for unrelated tactile events. We manipulated the temporal predictability of tactile targets using tactile cues, which preceded the target by either constant (high predictability) or variable (low predictability) time intervals. We find that microsaccades are inhibited prior to tactile targets and more so for constant than variable intervals, revealing a tight crossmodal link between tactile temporal expectation and oculomotor action. These findings portray oculomotor freezing as a marker of crossmodal temporal expectation. Moreover, microsaccades occurring around the tactile target presentation are associated with reduced task performance, suggesting that oculomotor freezing mitigates potential detrimental, concomitant effects of microsaccades and revealing a crossmodal coupling between tactile perception and oculomotor action.

Microsaccades during high speed continuous visual search

Here, we provide an analysis of the microsaccades that occurred during continuous visual search and targeting of small faces that we pasted either into cluttered background photos or into a simple gray background. Subjects continuously used their eyes to target singular 3-degree upright or inverted faces in changing scenes. As soon as the participant's gaze reached the target face, a new face was displayed in a different and random location. Regardless of the experimental context (e.g. background scene, no background scene), or target eccentricity (from 4 to 20 degrees of visual angle), we found that the microsaccade rate dropped to near zero levels within only 12 milliseconds after stimulus onset. There were almost never any microsaccades after stimulus onset and before the first saccade to the face. One subject completed 118 consecutive trials without a single microsaccade. However, in about 20% of the trials, there was a single microsaccade that occurred almost immediately after the preceding saccade's offset. These microsaccades were task oriented because their facial landmark targeting distributions matched those of saccades within both the upright and inverted face conditions. Our findings show that a single feedforward pass through the visual hierarchy for each stimulus is likely all that is needed to effectuate prolonged continuous visual search. In addition, we provide evidence that microsaccades can serve perceptual functions like correcting saccades or effectuating task-oriented goals during continuous visual search.

High-resolution, ultrafast, wide-field retinal eye-tracking for enhanced quantification of fixational and saccadic motion

We introduce a novel, noninvasive retinal eye-tracking system capable of detecting eye displacements with an angular resolution of 0.039 arcmin and a maximum velocity of 300°/s across an 8° span. Our system is designed based on a confocal retinal imaging module similar to a scanning laser ophthalmoscope. It utilizes a 2D MEMS scanner ensuring high image frame acquisition frequencies up to 1.24 kHz. In contrast with leading eye-tracking technology, we measure the eye displacements via the collection of the observed spatial excursions for all the times corresponding a full acquisition cycle, thus obviating the need for both a baseline reference frame and absolute spatial calibration. Using this approach, we demonstrate the precise measurement of eye movements with magnitudes exceeding the spatial extent of a single frame, which is not possible using existing image-based retinal trackers. We describe our retinal tracker, tracking algorithms and assess the performance of our system by using programmed artificial eye movements. We also demonstrate the clinical capabilities of our system with in vivo subjects by detecting microsaccades with angular extents as small as 0.028°. The rich kinematic ocular data provided by our system with its exquisite degree of accuracy and extended dynamic range opens new and exciting avenues in retinal imaging and clinical neuroscience. Several subtle features of ocular motion such as saccadic dysfunction, fixation instability and abnormal smooth pursuit can be readily extracted and inferred from the measured retinal trajectories thus offering a promising tool for identifying biomarkers of neurodegenerative diseases associated with these ocular symptoms.

Neural mechanism of visual information degradation from retina to V1 area

The information processing mechanism of the visual nervous system is an unresolved scientific problem that has long puzzled neuroscientists. The amount of visual information is significantly degraded when it reaches the V1 after entering the retina; nevertheless, this does not affect our visual perception of the outside world. Currently, the mechanisms of visual information degradation from retina to V1 are still unclear. For this purpose, the current study used the experimental data summarized by Marcus E. Raichle to investigate the neural mechanisms underlying the degradation of the large amount of data from topological mapping from retina to V1, drawing on the photoreceptor model first. The obtained results showed that the image edge features of visual information were extracted by the convolution algorithm with respect to the function of synaptic plasticity when visual signals were hierarchically processed from low-level to high-level. The visual processing was characterized by the visual information degradation, and this compensatory mechanism embodied the principles of energy minimization and transmission efficiency maximization of brain activity, which matched the experimental data summarized by Marcus E. Raichle. Our results further the understanding of the information processing mechanism of the visual nervous system.

Microsaccade generation requires a foveal anchor

Visual scene characteristics can affect various aspects of saccade and microsaccade dynamics. For example, blank visual scenes are known to elicit diminished saccade and microsaccade production, compared to natural scenes. Similarly, microsaccades are less frequent in the dark. Yet, the extent to which foveal versus peripheral visual information contribute to microsaccade production remains unclear: because microsaccade directions are biased towards covert attention locations, it follows that peripheral visual stimulation could suffice to produce regular microsaccade dynamics, even without foveal stimulation being present. Here we determined the characteristics of microsaccades as a function of foveal and/or peripheral visual stimulation, while human subjects conducted four types of oculomotor tasks (fixation, free viewing, guided viewing and passive viewing). Foveal information was either available, or made unavailable, by the presentation of simulated scotomas. We found foveal stimulation to be critical for microsaccade production, and peripheral stimulation, by itself, to be insufficient to yield normal microsaccades. In each oculomotor task, microsaccade production decreased when scotomas blocked foveal stimulation. Across comparable foveal stimulation conditions, the type of peripheral stimulation (static versus dynamic) moreover affected microsaccade production, with dynamic backgrounds resulting in lower microsaccadic rates than static backgrounds. These results indicate that a foveal visual anchor is necessary for normal microsaccade generation. Whereas peripheral visual stimulation, on its own, does not suffice for normal microsaccade production, it can nevertheless modulate microsaccadic characteristics. These findings extend our current understanding of the links between visual input and ocular motor control, and may therefore help improve the diagnosis and treatment of ophthalmic conditions that degrade central vision, such as age-related macular degeneration.

Microsaccades in Applied Environments: Real-World Applications of Fixational Eye Movement Measurements

Across a wide variety of research environments, the recording of microsaccades and other fixational eye movements has provided insight and solutions into practical problems. Here we review the literature on fixational eye movements-especially microsaccades-in applied and ecologically-valid scenarios. Recent technical advances allow noninvasive fixational eye movement recordings in real-world contexts, while observers perform a variety of tasks. Thus, fixational eye movement measures have been obtained in a host of real-world scenarios, such as in connection with driver fatigue, vestibular sensory deprivation in astronauts, and elite athletic training, among others. Here we present the state of the art in the practical applications of fixational eye movement research, examine its potential future uses, and discuss the benefits of including microsaccade measures in existing eye movement detection technologies. Current evidence supports the inclusion of fixational eye movement measures in real-world contexts, as part of the development of new or improved oculomotor assessment tools. The real-world applications of fixational eye movement measurements will only grow larger and wider as affordable high-speed and high-spatial resolution eye trackers become increasingly prevalent.

Visual working memory and action: Functional links and bi-directional influences

Working memory bridges perception to action over extended delays, enabling flexible goal-directed behaviour. To date, studies of visual working memory – concerned with detailed visual representations such as shape and colour – have considered visual memory predominantly in the context of visual task demands, such as visual identification and search. Another key purpose of visual working memory is to directly inform and guide upcoming actions. Taking this as a starting point, I review emerging evidence for the pervasive bi-directional links between visual working memory and (planned) action, and discuss these links from the perspective of their common goal of enabling flexible and precise behaviour.

Saliency and priority modulation in a pop-out paradigm: Pupil size and microsaccades

A salient stimulus can trigger a coordinated orienting response consisting of a saccade, pupil, and microsaccadic responses. Saliency models predict that the degree of visual conspicuity of all visual stimuli guides visual orienting. By presenting a multiple-item array that included an oddball colored item (pop-out), randomly mixed colored items (mixed-color), or single-color items (single-color), we examined the effects of saliency and priority (saliency + relevancy) on pupil size and microsaccade responses. Larger pupil responses were produced in the pop-out compared to the mixed-color or single-color conditions after stimulus presentation. However, the saliency modulation on microsaccades was not significant. Furthermore, although goal-relevancy information did not modulate pupil responses and microsaccade rate, microsaccade direction was biased toward the pop-out item when it was the subsequent saccadic target. Together, our results demonstrate saliency modulation on pupil size and priority effects on microsaccade direction during visual pop-out.

Visual feature tuning of superior colliculus neural reafferent responses after fixational microsaccades

The primate superior colliculus (SC) is causally involved in microsaccade generation. Moreover, visually responsive SC neurons across this structure's topographic map, even at peripheral eccentricities much larger than the tiny microsaccade amplitudes, exhibit significant modulations of evoked response sensitivity when stimuli appear perimicrosaccadically. However, during natural viewing, visual stimuli are normally stably present in the environment and are only shifted on the retina by eye movements. Here we investigated this scenario for the case of microsaccades, asking whether and how SC neurons respond to microsaccade-induced image jitter. We recorded neural activity from two male rhesus macaque monkeys. Within the response field (RF) of a neuron, there was a stable stimulus consisting of a grating of one of three possible spatial frequencies. The grating was stable on the display, but microsaccades periodically jittered the retinotopic RF location over it. We observed clear short-latency visual reafferent responses after microsaccades. These responses were weaker, but earlier (relative to new fixation onset after microsaccade end), than responses to sudden stimulus onsets without microsaccades. The reafferent responses clearly depended on microsaccade amplitude as well as microsaccade direction relative to grating orientation. Our results indicate that one way for microsaccades to influence vision is through modulating how the spatio-temporal landscape of SC visual neural activity represents stable stimuli in the environment. Such representation depends on the specific pattern of temporal luminance modulations expected from the relative relationship between eye movement vector (size and direction) on one hand and spatial visual pattern layout on the other.NEW & NOTEWORTHY Despite being diminutive, microsaccades still jitter retinal images. We investigated how such jitter affects superior colliculus (SC) activity. We found that SC neurons exhibit short-latency visual reafferent bursts after microsaccades. These bursts reflect not only the spatial luminance profiles of visual patterns but also how such profiles are shifted by eye movement size and direction. These results indicate that the SC continuously represents visual patterns, even as they are jittered by the smallest possible saccades.

Advantage of detecting visual events in the right hemifield is affected by reading skill

Visual perception is often not homogenous across the visual field and can vary depending on situational demands. The reasons behind this inhomogeneity are not clear. Here we show that directing attention that is consistent with a western reading habit from left to right, results in a ~32% higher sensitivity to detect transient visual events in the right hemifield. This right visual field advantage was largely reduced in individuals with reading difficulties from developmental dyslexia. Similarly, visual detection became more symmetric in skilled readers, when attention was guided opposite to the reading pattern. Taken together, these findings highlight a higher sensitivity in the right visual field for detecting the onset of sudden visual events that is well accounted for by left hemisphere dominated reading habit.

What makes a microsaccade? A review of 70 years of research prompts a new detection method

A new method for detecting microsaccades in eye-movement data is presented, following a review of reported microsaccade properties between the 1940s and today. The review focuses on the parameter ranges within which certain physical markers of microsaccades are thought to occur, as well as any features of microsaccades that have been stably reported over time. One feature of microsaccades, their binocularity, drives the new microsaccade detection method. The binocular correlation method for microsaccade detection is validated on two datasets of binocular eye-movements recorded using video-based systems: one collected as part of this study, and one from Nyström et al, 2017. Comparisons between detection methods are made using precision-recall statistics. This confirms that the binocular correlation method performs well when compared to manual coders and performs favourably compared to the commonly used Engbert & Kliegl (2003) method with subsequent modifications (Engbert & Mergenthaler, 2006). The binocular correlation microsaccade detection method is easy to implement and MATLAB code is made available to download.

Shared Physiological Correlates of Multisensory and Expectation-Based Facilitation

Perceptual performance in a visual task can be enhanced by simultaneous multisensory information, but can also be enhanced by a symbolic or amodal cue inducing a specific expectation. That similar benefits can arise from multisensory information and within-modality expectation raises the question of whether the underlying neurophysiological processes are the same or distinct. We investigated this by comparing the influence of the following three types of auxiliary probabilistic cues on visual motion discrimination in humans: (1) acoustic motion, (2) a premotion visual symbolic cue, and (3) a postmotion symbolic cue. Using multivariate analysis of the EEG data, we show that both the multisensory and preceding visual symbolic cue enhance the encoding of visual motion direction as reflected by cerebral activity arising from occipital regions ∼200–400 ms post-stimulus onset. This suggests a common or overlapping physiological correlate of cross-modal and intramodal auxiliary information, pointing to a neural mechanism susceptive to both multisensory and more abstract probabilistic cues. We also asked how prestimulus activity shapes the cue–stimulus combination and found a differential influence on the cross-modal and intramodal combination: while alpha power modulated the relative weight of visual motion and the acoustic cue, it did not modulate the behavioral influence of a visual symbolic cue, pointing to differences in how prestimulus activity shapes the combination of multisensory and abstract cues with task-relevant information.

The effects of fixational tremor on the retinal image

The study of fixational eye motion has implications for the neural and computational underpinnings of vision. One component of fixational eye motion is tremor, a high-frequency oscillatory jitter reported to be anywhere from ∼11-60 arcseconds in amplitude. In order to isolate the effects of tremor on the retinal image directly and in the absence of optical blur, high-frequency, high-resolution eye traces were collected in six subjects from videos recorded with an adaptive optics scanning laser ophthalmoscope. Videos were acquired while subjects engaged in an active fixation task where they fixated on a tumbling E stimulus and reported changes in its orientation. Spectral analysis was conducted on periods of ocular drift, with all drifts being concatenated together after removal of saccades from the trace. The resultant amplitude spectra showed a slight deviation from the traditional 1/f nature of optical drift in the frequency range of 50-100 Hz, which is indicative of tremor. However, this deviation rarely exceeded 1 arcsecond and the consequent standard deviation of retinal image motion over the tremor band (50-100 Hz) was just over 5 arcseconds. Given such a small amplitude, it is unlikely tremor will contribute in any meaningful way to the visual percept.

Finely tuned eye movements enhance visual acuity

High visual acuity is essential for many tasks, from recognizing distant friends to driving a car. While much is known about how the eye’s optics and anatomy contribute to spatial resolution, possible influences from eye movements are rarely considered. Yet humans incessantly move their eyes, and it has long been suggested that oculomotor activity enhances fine pattern vision. Here we examine the role of eye movements in the most common assessment of visual acuity, the Snellen eye chart. By precisely localizing gaze and actively controlling retinal stimulation, we show that fixational behavior improves acuity by more than 0.15 logMAR, at least 2 lines of the Snellen chart. This improvement is achieved by adapting both microsaccades and ocular drifts to precisely position the image on the retina and adjust its motion. These findings show that humans finely tune their fixational eye movements so that they greatly contribute to normal visual acuity.

Humans are normally not aware that their eyes are always in motion, even when attempting to maintain steady gaze on a point. Here the authors show that these small eye movements are finely controlled and contribute more than two lines in a standard eye-chart test of visual acuity.

Relationship of postsaccadic oscillation with the state of the pupil inside the iris and with cognitive processing

Recent studies using video-based eye tracking have presented accumulating evidence that postsaccadic oscillation defined in reference to the pupil center (PSOp) is larger than that to the iris center (PSOi). This indicates that the relative motion of the pupil reflects the viscoelasticity of the tissue of the iris. It is known that the pupil size controlled by the sphincter/dilator pupillae muscles reflects many aspects of cognition. A hypothesis derived from this fact is that cognitive tasks affect the properties of PSOp due to the change in the state of these muscles. To test this hypothesis, we conducted pro- and antisaccade tasks for human participants and adopted the recent physical model of PSO to evaluate the dynamic properties of PSOp/PSOi. The results showed the dependence of the elasticity coefficient of the PSOp on the antisaccade task, but this effect was not significant for the PSOi. This suggests that cognitive tasks such as antisaccade tasks affect elasticity of the muscle of the iris. We found that the trial-by-trial fluctuation in the presaccade absolute pupil size correlated with the elasticity coefficient of PSOp. We also found the task dependence of the viscosity coefficient and overshoot amount of PSOi, which probably reflects the dynamics of the entire eyeball movement. The difference in task dependence between PSOp and PSOi indicates that the separate measures of these two can be means to distinguish factors related to the oculomotor neural system from those related to the physiological states of the iris tissue.

NEW & NOTEWORTHY The state of the eyeball varies dynamically moment by moment depending on underlying neural/cognitive processing. Combining simultaneous measurements of pupil-centric and iris-centric movements and a recent physical model of postsaccadic oscillation (PSO), we show that the pupil-centric PSO is sensitive to the type of saccade task, suggesting that the physical state of the iris muscles reflects the underlying cognitive processes.

Predicting the Partition of Behavioral Variability in Speed Perception with Naturalistic Stimuli

A core goal of visual neuroscience is to predict human perceptual performance from natural signals. Performance in any natural task can be limited by at least three sources of uncertainty: stimulus variability, internal noise, and suboptimal computations. Determining the relative importance of these factors has been a focus of interest for decades but requires methods for predicting the fundamental limits imposed by stimulus variability on sensory-perceptual precision. Most successes have been limited to simple stimuli and simple tasks. But perception science ultimately aims to understand how vision works with natural stimuli. Successes in this domain have proven elusive. Here, we develop a model of humans based on an image-computable (images in, estimates out) Bayesian ideal observer. Given biological constraints, the ideal optimally uses the statistics relating local intensity patterns in moving images to speed, specifying the fundamental limits imposed by natural stimuli. Next, we propose a theoretical link between two key decision-theoretic quantities that suggests how to experimentally disentangle the impacts of internal noise and deterministic suboptimal computations. In several interlocking discrimination experiments with three male observers, we confirm this link and determine the quantitative impact of each candidate performance-limiting factor. Human performance is near-exclusively limited by natural stimulus variability and internal noise, and humans use near-optimal computations to estimate speed from naturalistic image movies. The findings indicate that the partition of behavioral variability can be predicted from a principled analysis of natural images and scenes. The approach should be extendable to studies of neural variability with natural signals.SIGNIFICANCE STATEMENT Accurate estimation of speed is critical for determining motion in the environment, but humans cannot perform this task without error. Different objects moving at the same speed cast different images on the eyes. This stimulus variability imposes fundamental external limits on the human ability to estimate speed. Predicting these limits has proven difficult. Here, by analyzing natural signals, we predict the quantitative impact of natural stimulus variability on human performance given biological constraints. With integrated experiments, we compare its impact to well-studied performance-limiting factors internal to the visual system. The results suggest that the deterministic computations humans perform are near optimal, and that behavioral responses to natural stimuli can be studied with the rigor and interpretability defining work with simpler stimuli.