Using high frame rate CMOS sensors for three-dimensional eye tracking

Swift-Eye: Towards Anti-blink Pupil Tracking for Precise and Robust High-Frequency Near-Eye Movement Analysis with Event Cameras

Eye tracking has shown great promise in many scientific fields and daily applications, ranging from the early detection of mental health disorders to foveated rendering in virtual reality (VR). These applications all call for a robust system for high-frequency near-eye movement sensing and analysis in high precision, which cannot be guaranteed by the existing eye tracking solutions with CCD/CMOS cameras. To bridge the gap, in this paper, we propose Swift-Eye, an offline precise and robust pupil estimation and tracking framework to support high-frequency near-eye movement analysis, especially when the pupil region is partially occluded. Swift-Eye is built upon the emerging event cameras to capture the high-speed movement of eyes in high temporal resolution. Then, a series of bespoke components are designed to generate high-quality near-eye movement video at a high frame rate over kilohertz and deal with the occlusion over the pupil caused by involuntary eye blinks. According to our extensive evaluations on EV-Eye, a large-scale public dataset for eye tracking using event cameras, Swift-Eye shows high robustness against significant occlusion. It can improve the IoU and F1-score of the pupil estimation by 20% and 12.5% respectively, compared with the second-best competing approach, when over 80% of the pupil region is occluded by the eyelid. Lastly, it provides continuous and smooth traces of pupils in extremely high temporal resolution and can support high-frequency eye movement analysis and a number of potential applications, such as mental health diagnosis, behaviour-brain association, etc. The implementation details and source codes can be found at https://github.com/ztysdu/Swift-Eye.

Lens Thickness Microfluctuations in Young and Prepresbyopic Adults During Steady-State Accommodation

Purpose

To determine whether lens mechanical dynamics change with age and with accommodative demands.

Methods

Lens thickness microfluctuations were measured using a high-speed custom-built spectral domain optical coherence tomography system in five young adults (20 to 25 years old) at 0 diopters (D), 2 D, 4 D, and maximum accommodative demand and in five prepresbyopes (38 to 45 years old) under relaxed and maximal accommodation. For each state, the measurements were repeated four times during the same session. Images of the central 2-mm zone of the lens comprising 170 A-lines/frame were acquired for 10 seconds, and axial lens thickness change was measured. Lens thickness microfluctuations (µm²/Hz) were assessed by integrating the power spectrum of lens thickness microfluctuations between 0 and 4 Hz.

Results

The amplitude of lens microfluctuations was higher in the accommodated states than in the relaxed state in both age groups. Lens microfluctuations were higher in young adult participants than in prepresbyopes, with a significant difference in relaxed and maximally accommodated states (P = 0.04 and P = 0.04). In the young participants, the amplitude of microfluctuations reached a plateau at maximum accommodation.

Conclusions

Lens mechanical dynamics are both age and accommodation dependent. The decrease in lens thickness microfluctuations with age is consistent with an age-related increase in lens stiffness or decrease of the ciliary muscle displacement. The lens does not contribute to the high-frequency component of ocular dioptric microfluctuations.

Visual Attention Relates to Operator Performance in Spacecraft Docking Training

BACKGROUND: Manually controlled docking of a spacecraft to a space station is an operational task that poses high demands on cognitive and perceptual functioning. Effective processing of visual information is crucial for success. Eye tracking can reveal the operator’s attentional focus unobtrusively and objectively. Therefore, our aim was to test the feasibility of eye tracking during a simulation of manual docking and to identify links between visual information processing and performance.METHODS: We hypothesized that duration and number of gazes to specific regions of interest of the simulation (total dwell time and number of dwells) would be associated with docking accuracy. Eye movements were recorded in 10 subjects (30% women, M = 33.4 yr old) during the 6° head-down tilt bed rest study AGBRESA during 20 training sessions with the 6df learning program for spacecraft docking.RESULTS: Subjects’ gaze was directed most frequently and longest to the vizor (185 dwells and 22,355 ms per task) followed by the two instrument displays (together 75 dwells and 4048 ms per task). We observed a significant positive relationship between number and duration of visual checks of speed and distance to the docking point and the accuracy of the docking maneuver.DISCUSSION: In conclusion, eye tracking provides valuable information related to docking accuracy that might prospectively offer the opportunity to improve docking training effectiveness.Piechowski S, Johannes B, Pustowalow W, Arz M, Mulder E, Jordan J, Wolf OT, Rittweger J. Visual attention relates to operator performance in spacecraft docking training. Aerosp Med Hum Perform. 2022; 93(6):480–486.

The Role of Different Afferent Systems in the Modulation of the Otolith-Ocular Reflex After Long-Term Space Flights

Background

The vestibular (otolith) function is highly suppressed during space flight (SF) and the study of these changes is very important for the safety of the space crew during SF missions. The vestibular function (particularly, otolith-ocular reflex–OOcR) in clinical and space medicine is studied using different methodologies. However, different methods and methodologies can influence the outcome results.

Objective

The current study addresses the question of whether the OOcR results obtained by different methods are different, and what the role is of the different afferent systems in the modulation of the OOcR.

Methods

A total of 25 Russian cosmonauts voluntarily took part in our study. They are crewmembers of long duration space missions on the International Space Station (ISS). Cosmonauts were examined in pre- and post-flight “Sensory Adaptation” and “Gaze Spin” experiments, twice before (preflight) and three times after SF (post-flight). We used two different video oculography (VOG) systems for the recording of the OOcR obtained in each experiment.

Results

Comparison of the two VOG systems didn’t result into significant and systematic differences in the OOcR measurements. Analysis of the static torsion otolith–ocular reflex (OOR), static torsion otolith–cervical–ocular reflex (OCOR) and static torsion otolith–ocular reflex during eccentric centrifugation (OOREC) shows that the OOREC results in a lower OOcR response compared to the OOR and OCOR (before flight and late post-flight). However, all OOcRs were significantly decreased in all cosmonauts early post-flight.

Conclusion

Analysis of the results of ocular counter rolling (OCR) obtained by different methods (OOR, OCOR, and OOREC) showed that different afferent systems (tactile-proprioception, neck-cervical, visual and vestibular afferent input) have an impact on the OOcR.

Hybrid FPGA-CPU pupil tracker

An off-axis monocular pupil tracker designed for eventual integration in ophthalmoscopes for eye movement stabilization is described and demonstrated. The instrument consists of light-emitting diodes, a camera, a field-programmable gate array (FPGA) and a central processing unit (CPU). The raw camera image undergoes background subtraction, field-flattening, 1-dimensional low-pass filtering, thresholding and robust pupil edge detection on an FPGA pixel stream, followed by least-squares fitting of the pupil edge pixel coordinates to an ellipse in the CPU. Experimental data suggest that the proposed algorithms require raw images with a minimum of ∼32 gray levels to achieve sub-pixel pupil center accuracy. Tests with two different cameras operating at 575, 1250 and 5400 frames per second trained on a model pupil achieved 0.5-1.5 μm pupil center estimation precision with 0.6-2.1 ms combined image download, FPGA and CPU processing latency. Pupil tracking data from a fixating human subject show that the tracker operation only requires the adjustment of a single parameter, namely an image intensity threshold. The latency of the proposed pupil tracker is limited by camera download time (latency) and sensitivity (precision).

Small eye movements cannot be reliably measured by video-based P-CR eye-trackers

For evaluating whether an eye-tracker is suitable for measuring microsaccades, Poletti & Rucci (2016) propose that a measure called 'resolution' could be better than the more established root-mean-square of the sample-to-sample distances (RMS-S2S). Many open questions exist around the resolution measure, however. Resolution needs to be calculated using data from an artificial eye that can be turned in very small steps. Furthermore, resolution has an unclear and uninvestigated relationship to the RMS-S2S and STD (standard deviation) measures of precision (Holmqvist & Andersson, 2017, p. 159-190), and there is another metric by the same name (Clarke, Ditterich, Drüen, Schönfeld, and Steineke 2002), which instead quantifies the errors of amplitude measurements. In this paper, we present a mechanism, the Stepperbox, for rotating artificial eyes in arbitrary angles from 1' (arcmin) and upward. We then use the Stepperbox to find the minimum reliably detectable rotations in 11 video-based eye-trackers (VOGs) and the Dual Purkinje Imaging (DPI) tracker. We find that resolution correlates significantly with RMS-S2S and, to a lesser extent, with STD. In addition, we find that although most eye-trackers can detect some small rotations of an artificial eye, the rotations of amplitudes up to 2∘ are frequently erroneously measured by video-based eye-trackers. We show evidence that the corneal reflection (CR) feature of these eye-trackers is a major cause of erroneous measurements of small rotations of artificial eyes. Our data strengthen the existing body of evidence that video-based eye-trackers produce errors that may require that we reconsider some results from research on reading, microsaccades, and vergence, where the amplitude of small eye movements have been measured with past or current video-based eye-trackers. In contrast, the DPI reports correct rotation amplitudes down to 1'.

A Robust Method of Eye Torsion Measurement for Medical Applications

The detection of eye torsion is an important element for diagnosis of balance disorders, although it is rarely available in existing eye tracking systems. A novel method is proposed in this paper to provide robust measurement of torsional eye movements. A numerical approach is presented to estimate the iris boundary only according to the gaze direction, so the segmentation of the iris is more robust against occlusions and ambiguities. The perspective distortion of the iris pattern at eccentric eye positions is also corrected, benefiting from the transformation relation that is established for the iris estimation. The angle of the eye torsion is next measured on the unrolled iris patterns via a TM (Template Matching) technique. The principle of the proposed method is validated and its robustness in practice is assessed. A very low mean FPR (False Positive Rate) is reported (i.e., 3.3%) in a gaze test when testing on five participants with very different eye morphologies. The present method always gave correct measurement on the iris patterns with simulated eye torsions and rarely provided mistaken detections in the absence of eye torsion in practical conditions. Therefore, it shows a good potential to be further applied in medical applications.

The Effect of Real-time Headbox Adjustments on Data Quality

Following a patent owned by Tobii, the framerate of a CMOS camera can be increased by reducing the size of the recording window so that it fits the eyes with minimum room to spare. The position of the recording window can be dynamically adjusted within the camera sensor area to follow the eyes as the participant moves the head. Since only a portion of the camera sensor data is communicated to the computer and processed, much higher framerates can be achieved with the same CPU and camera. Eye trackers can be expected to present data at a high speed, with good accuracy and precision, small latency and with minimal loss of data while allowing participants to behave as normally as possible. In this study, the effect of headbox adjustments in real-time is investigated with respect to the above-mentioned parameters. It was found that, for the specific camera model and tracking algorithm, one or two headbox adjustments per second, as would normally be the case during recording of human participants, could be tolerated in favour of a higher framerate. The effect of adjustment of the recording window can be reduced by using a larger recording window at the cost of the framerate.

A no-moving-parts sensor for the detection of eye fixation using polarised light and retinal birefringence information

Polarised near-infra-red light is reflected from the foveal area in a detectable bow-tie pattern of polarisation states, offering the opportunity for eye tracking. A coaxial optical transducer was developed, consisting of a laser diode, a polariser, a filter, and a photodetector. Several such transducers may be used to interrogate different spots on the retina, thus eliminating the requirement for scanning systems with moving parts. To test the signal quality obtainable, using just one transducer, a test subject was asked to fixate successively on twelve targets located on a circle around the transducer, to simulate the retina's being interrogated by twelve sensors placed on a 3⁰ diameter circle surrounding the projection of the fovea. The resulting signal is close to the "ideal" sine wave that would have been recorded from a propeller-type birefringence pattern from a human fovea. The transducer can be used in the detection of fixation for medical and other purposes. It does not require calibration, strict restrictions on head position, or head-mounted appliances.

Knowing what the brain is seeing in three dimensions: A novel, noninvasive, sensitive, accurate, and low-noise technique for measuring ocular torsion

Torsional eye movements are rotations of the eye around the line of sight. Measuring torsion is essential to understanding how the brain controls eye position and how it creates a veridical perception of object orientation in three dimensions. Torsion is also important for diagnosis of many vestibular, neurological, and ophthalmological disorders. Currently, there are multiple devices and methods that produce reliable measurements of horizontal and vertical eye movements. Measuring torsion, however, noninvasively and reliably has been a longstanding challenge, with previous methods lacking real-time capabilities or suffering from intrusive artifacts. We propose a novel method for measuring eye movements in three dimensions using modern computer vision software (OpenCV) and concepts of iris recognition. To measure torsion, we use template matching of the entire iris and automatically account for occlusion of the iris and pupil by the eyelids. The current setup operates binocularly at 100 Hz with noise <0.1° and is accurate within 20° of gaze to the left, to the right, and up and 10° of gaze down. This new method can be widely applicable and fill a gap in many scientific and clinical disciplines.

Controlling Posture and Vergence Eye Movements in Quiet Stance: Effects of Thin Plantar Inserts

The purpose of this study was to assess properties of vergence and saccade eye movements as well as posture in quiet stance, and the effects of thin plantar inserts upon postural and oculomotor control. The performances of 36 young healthy subjects were recorded by a force platform and an eye tracker in three testing conditions: without plantar stimulation, with a 3 millimetre-thick plantar insert, either a Medial or a Lateral Arch Support (MAS/LAS). The results showed a decrease of the Surface and Variance of Speed and a more posterior position of the CoP with either stimulation compared with the control condition. The fractal analysis showed a decrease with MAS. Wavelet analysis in the time-frequency domain revealed an increase in the Cancelling Time of the low frequency band with MAS. These results suggest a better stability for a lower energy cost. Concerning eye movements, the inserts influenced only vergence (not saccades): MAS caused an increase of the phasic amplitude of divergence, and conversely a decrease of the tonic amplitude. In contrast, LAS caused an increase of the tonic amplitude of convergence. Thus, MAS renders divergence less visually driven, while LAS renders convergence more visually driven. We conclude that the CNS uses the podal signal for both postural and vergence control via specific mechanisms. Plantar inserts have an influence upon posture and vergence movements in a different way according to the part of the foot sole being stimulated. These results can be useful to clinicians interested in foot or eye.

Modification of unilateral otolith responses following spaceflight

The aim of the study was to resolve the issue of spaceflight-induced, adaptive modification of the otolith system by measuring unilateral otolith responses in a pre- versus post-flight design. The study represents the first comprehensive approach to examining unilateral otolith function following space flight. Ten astronauts participated in unilateral otolith function tests three times preflight and up to four times after Shuttle flights from landing day through the subsequent 10 days. During unilateral centrifugation, utricular function was examined by the perceptual changes reflected by the subjective visual vertical (SVV) and the otolith-mediated ocular counter-roll, designated as utriculo-ocular response (UOR). Unilateral saccular reflexes were recorded by measurement of collic vestibular evoked myogenic potentials (cVEMP). The findings demonstrate a general increase in interlabyrinth asymmetry of otolith responses on landing day relative to preflight baseline, with subsequent reversal in asymmetry within 2-3 days. Recovery to baseline levels was achieved within 10 days. This fluctuation in asymmetry was consistent for the utricle tests (SVV and UOR) while apparently stronger for SVV. A similar asymmetry was observed during cVEMP testing. In addition, the results provide initial evidence of a dominant labyrinth. The findings require reconsideration of the otolith asymmetry hypothesis; in general, on landing day, the response from one labyrinth was equivalent to preflight values, while the other showed considerable discrepancy. The finding that one otolith response can return to one-g level within hours after re-entry while the other takes considerably longer demonstrates the importance of considering the otolith response as a result of both peripheral and associated central neural processing.

A device for continuous monitoring of true central fixation based on foveal birefringence

A device for continuous monitoring of central fixation utilizes birefringence, the property of the Henle fibers surrounding the human fovea, to change the polarization state of light. A circular scan of retinal birefringence, where the scanning circle encompasses the fovea, allows identification of true central fixation-an assessment much needed in various applications in ophthalmology, psychology, and psychiatry. The device allows continuous monitoring for central fixation over an extended period of time in the presence of fixation targets and distracting stimuli, which may be helpful in detecting attention deficit hyperactivity disorder, autism spectrum disorders, and other disorders characterized by changes in the subject's ability to maintain fixation. A proof-of-concept has been obtained in a small study of ADHD patients and normal control subjects.

Mechanisms compensating for visual field restriction in adolescents with damage to the retro-geniculate visual system

BACKGROUND

To describe visual field (VF) outcome in three adolescents with damage to the optic radiation and to focus on mechanisms that may compensate the practical functional limitations of VF defects.

DESIGN

Descriptive, prospective multi-case study in a hospital setting.

PARTICIPANTS

Three teenagers with cerebral visual dysfunction because of damage to the retro-geniculate visual pathways.

METHODS

Best-corrected visual acuity and eye alignment were assessed. Visual field function was tested with Goldmann perimetry, and with Rarebit, Humphrey Visual Field Analyzer and Esterman computerized techniques. Fixation was registered with video oculography during Rarebit examination. Magnetic resonance imaging of the brain illustrated brain damage and its relation to the posterior visual system.

RESULTS

One of the three subjects had bilateral asymmetric white matter damage of immaturity, early-onset exotropia, and a relative homonymous VF defect, but normal binocular VF. The second subject also had bilateral asymmetric white matter damage of immaturity and showed an inferior right quadrantanopia, confirmed by the binocular field. Registration of fixation revealed automatic scanning during perimetry. The third subject had an almost total left homonymous hemianopia after resection of a brain tumour in the right temporal lobe. The hemianopia could be compensated for by fast voluntary scanning.

CONCLUSION

Congenital and later-acquired homonymous VF defects may, at least in young subjects, be compensated for by scanning. Exotropia may compensate VF defects and, therefore, the VF should be tested before strabismus surgery.

Active collisions in altered gravity reveal eye-hand coordination strategies

Most object manipulation tasks involve a series of actions demarcated by mechanical contact events, and gaze is usually directed to the locations of these events as the task unfolds. Typically, gaze foveates the target 200 ms in advance of the contact. This strategy improves manual accuracy through visual feedback and the use of gaze-related signals to guide the hand/object. Many studies have investigated eye-hand coordination in experimental and natural tasks; most of them highlighted a strong link between eye movements and hand or object kinematics. In this experiment, we analyzed gaze strategies in a collision task but in a very challenging dynamical context. Participants performed collisions while they were exposed to alternating episodes of microgravity, hypergravity and normal gravity. First, by isolating the effects of inertia in microgravity, we found that peak hand acceleration marked the transition between two modes of grip force control. Participants exerted grip forces that paralleled load force profiles, and then increased grip up to a maximum shifted after the collision. Second, we found that the oculomotor strategy adapted visual feedback of the controlled object around the collision, as demonstrated by longer durations of fixation after collision in new gravitational environments. Finally, despite large variability of arm dynamics in altered gravity, we found that saccades were remarkably time-locked to the peak hand acceleration in all conditions. In conclusion, altered gravity allowed light to be shed on predictive mechanisms used by the central nervous system to coordinate gaze, hand and grip motor actions during a mixed task that involved transport of an object and high impact loads.

Review of the major findings about Duane retraction syndrome (DRS) leading to an updated form of classification

In view of all the reported evidence by electromyography in the 1970s, by histology in the 1980s, and by cerebral imagery since the 2000s, Duane retraction syndrome (DRS) has been described as the consequence of a congenital anomaly of the 6th cranial nerve nuclei with aberrant innervations by supply from the 3rd cranial nerve. Both genetic and environmental factors are likely to play a role when the cranial nerves and ocular muscles are developing between the 4th and the 8th week of gestation. New data from eye movement recordings contributed to better understanding the binocular control of saccades. Modeling of saccades in DRS seems promising for the quantification of the innervational deficit and the mechanical properties of the eye plant. The usual clinical classification of DRS needs to be updated in order to match more accurately the underlying dysinnervation of the extra ocular muscles and to illustrate the continuum that exists between the various forms. This review aims to summarize the major findings about DRS and to guide the clinician in the surgical management of this particular form of strabismus.

Neurovestibular considerations for sub-orbital space flight: A framework for future investigation

Commercial sub-orbital operators will soon offer the excitement of traveling to space to thousands of people. Based on previous experience in space flight and parabolic flight, sensorimotor disruptions in eye movements, postural stability, and motor coordination are likely in these travelers. Here we propose a framework for developing strategies to overcome these sensorimotor disruptions. We delineate how approaches should differ from those applied to orbital flight and between sub-orbital passengers and pilots based on differing frequency of flights and mission objectives. Sensorimotor adaptation is one strategy for overcoming disruptions; an important question is whether it occurs quickly enough to be of use during periods of reduced and enhanced gravity lasting less than five minutes. Data are presented showing that sensorimotor adaptation of the pitch vestibulo-ocular reflex during parabolic flight takes a few consecutive days of flying to overcome an initial disruption. We conclude with recommendations for operators and researchers to improve safety and comfort during sub-orbital operations. We recommend using parabolic flight as a tool for pre-adapting sub-orbital passengers, along with further research into the required quantity and timing of these pre-adaptation flights and the tasks conducted during these flights. Likewise, for sub-orbital pilots, we recommend emphasizing recency of experience.

Compensating for camera translation in video eye-movement recordings by tracking a representative landmark selected automatically by a genetic algorithm

It is common in oculomotor and vestibular research to use video or still cameras to acquire data on eye movements. Unfortunately, such data are often contaminated by unwanted motion of the face relative to the camera, especially during experiments in dynamic motion environments. We develop a method for estimating the motion of a camera relative to a highly deformable surface, specifically the movement of a camera relative to the face and eyes. A small rectangular region of interest (ROI) on the face is automatically selected and tracked throughout a set of video frames as a measure of vertical camera translation. The specific goal is to present a process based on a genetic algorithm that selects a suitable ROI for tracking: one whose translation within the camera image accurately matches the actual relative motion of the camera. We find that co-correlation, a statistic describing the time series of a large group of ROIs, predicts the accuracy of the ROIs, and can be used to select the best ROI from a group. After the genetic algorithm finds the best ROIs from a group, it uses recombination to form a new generation of ROIs that inherit properties of the ROIs from the previous generation. We show that the algorithm can select an ROI that will estimate camera translation and determine the direction that the eye is looking with an average accuracy of 0.75 degrees , even with camera translations of 2.5mm at a viewing distance of 120 mm, which would cause an error of 11 degrees without correction.

A novel haploscopic viewing apparatus with a three-axis eye tracker

PURPOSE

To validate the accuracy and precision of a novel haploscope for use in investigating the mechanisms underlying normal eye control as well as the mechanisms of cyclovertical strabismus in pediatric and adult subjects. The accuracy and precision represent the device's ability to reproducibly measure true eye positions.

METHODS

A novel haploscope was developed that allows the measurement of eye positions and movements under specified conditions of vergence, head tilting, and cover testing. Equipped with video oculography, the haploscope can aid in the objective assessment of binocular and adaptive mechanisms that maintain oculomotor alignment. The device's accuracy and precision were assessed using a model eye with 3 axes of rotation. The device was then used to measure ocular torsion during the well-documented phenomenon of ocular counter-roll with head tilt. The eye movements of 6 normal subjects were measured as each subject fixated binocularly on the center of radially symmetric targets during head tilting.

RESULTS

The device yielded Pearson correlations with the model eye of R = 1.0, with residual error (re), a measure of accuracy, about all 3 ocular axes peaking at re = 19 +/- 5 arcmin. For human subjects, average positional error was re = 21 +/- 9 arcmin. Ocular counter-roll averaged 5.7 +/- 0.9 degrees for left and right eyes.

CONCLUSIONS

These results validate the accuracy and precision of this novel haploscope. They support its use in future investigations of the mechanisms of oculomotor control and alignment.

Predictive disjunctive pursuit of virtual images perceived to move in depth

Human and nonhuman primates predictively use smooth pursuit and saccades to track visual targets that move in a fronto-parallel plane. This behaviour is believed to be facilitated by short-term memory of the target motion and/or an efference copy of the subject's motor effort. Subjects in our experiments tracked dichoptically viewed targets that appeared to move vertically, right or left and towards the subject. The virtual image of the target was tracked using disjunctive smooth pursuit and saccades. To reveal predictive tracking, targets were blanked 100 ms after the onset of motion for intervals of 800 ms. During the blanked interval, subjects initiated pursuit and predictively tracked the unseen virtual image using memory guided eye movements. Our data are consistent with recent electrophysiological studies that describe cells that encode target or eye movements in depth when a target is briefly blanked but pursuit is maintained. However, predictive pursuit of a virtual target with disjunctive eye movements poses a challenge for understanding how a short-term memory store might encode the desired eye movement, its coordinate frame, and how it is transformed into motor commands.

Perceived self-orientation in allocentric and egocentric space: effects of visual and physical tilt on saccadic and tactile measures

Do physical tilt and tilt of the visual environment affect perception of allocentric and egocentric space? We addressed this question using two perceptual-motor tasks: alignment of a tactile rod (ROD) and saccadic eye movements (EM). Nine participants indicated the vertical axis of their heads (egocentric task), as well as the direction of gravity (allocentric task). Head orientation (+/-60 degrees and 0 degrees) and visual environment orientation (+/-120 degrees, +/-60 degrees and 0 degrees) were independently manipulated in the fronto-planar roll plane. ROD and EM estimates of both allocentric and egocentric reference directions varied with head and room orientation. Physical tilt dominated allocentric estimates in the dark where overestimates of physical tilt were noted up to 11 degrees using both measures. Allocentric ROD and EM estimates were significantly correlated across all head orientations (r=.70, p<.01) but only when upright for egocentric estimates (r=.38, p<.01). The relative contributions of the visual environment, gravity's direction and long-body axis to the estimation of allocentric and egocentric directions were determined by vector modeling. This modeling found that vision determined about 14% of the allocentric ROD and EM estimates, that the long-axis body reference played no discernible role, and that the largest factor was gravity, the effective direction of which was non-veridical. For egocentric estimates, vision contributed about 3% with the largest factor being the body reference. We conclude that perception of allocentric and egocentric space is likely influenced by multiple senses that define common egocentric and allocentric frames of reference accessible for saccadic and tactile estimates of perceived self-orientation.

Role of ocular convergence in the Romberg quotient

The Romberg test generally shows that postural stability is better with eyes open than eyes closed; the Romberg quotient (RQ) is generally 2.5. This study examines the possible role of vergence angle on the RQ. Eighteen young (25.3+/-2.7 years) and 17 old (61.6+/-4.4 years) subjects were required to fixate a target at 40 cm or at 200 cm inducing different vergence angle (i.e. 8.6 degrees and 1.7 degrees, respectively) either with eyes open or closed. Postural stability of subjects was measured with force platform (TechnoConcept). The RQ was about 2 at 40 cm but dropped to 1 at 200 cm. In a second experiment, 15 subjects (26.7+/-5.5 years) run the Romberg test with eye movement measures (Chronos). Subjects were required to fixate a target placed at 20 cm, 40 cm, 90 cm, 200 cm or 350 cm either in light or in dark. The RQ at 20 cm and 40 cm was close to 2 and dropped to 1 at 90 cm and beyond. In parallel, the vergence angle at 20 cm and 40 cm changed significantly between light and dark, while at 90 cm and beyond it was stable (about 2 degrees both in light and dark). The distance had a significant effect on the co-variance between the RQ based on the anterior-posterior sway, and the change of vergence angle. We suggest different ways of control of posture according to the viewing distance: at near distance and in the light, the CNS uses vision coupled with oculo-motor convergence signals (efferent and afferent) leading to high RQ; at intermediate and far distances, it would use mostly internal signals (vestibular, proprioceptive, somatosensory), and similarly in the light and in the dark.

Control of bimanual rhythmic movements: trading efficiency for robustness depending on the context

This paper investigates how the efficiency and robustness of a skilled rhythmic task compete against each other in the control of a bimanual movement. Human subjects juggled a puck in 2D through impacts with two metallic arms, requiring rhythmic bimanual actuation. The arms kinematics were only constrained by the position, velocity and time of impacts while the rest of the trajectory did not influence the movement of the puck. In order to expose the task robustness, we manipulated the task context in two distinct manners: the task tempo was assigned at four different values (hence manipulating the time available to plan and execute each impact movement individually); and vision was withdrawn during half of the trials (hence reducing the sensory inflows). We show that when the tempo was fast, the actuation was rhythmic (no pause in the trajectory) while at slow tempo, the actuation was discrete (with pause intervals between individual movements). Moreover, the withdrawal of visual information encouraged the rhythmic behavior at the four tested tempi. The discrete versus rhythmic behavior give different answers to the efficiency/robustness trade-off: discrete movements result in energy efficient movements, while rhythmic movements impact the puck with negative acceleration, a property preserving robustness. Moreover, we report that in all conditions the impact velocity of the arms was negatively correlated with the energy of the puck. This correlation tended to stabilize the task and was influenced by vision, revealing again different control strategies. In conclusion, this task involves different modes of control that balance efficiency and robustness, depending on the context.

Compensating for camera translation in video eye movement recordings by tracking a landmark selected automatically by a genetic algorithm

We develop a method for accurately estimating the motion of a camera relative to a highly deformable surface, specifically the movement of a camera relative to the eye. A small rectangular landmark is selected and tracked throughout a set of video frames as a measure of vertical camera translation. The specific goal is to present a process based on a genetic algorithm that selects a suitable landmark. We find that co-correlation, a statistic relating the time series of a large population of landmarks, is a robust predictor of the accuracy of the landmarks. This statistic is used to iteratively select the best landmark from the population. At each iteration new landmarks are created that inherit properties of the previous population of landmarks. We show that the algorithm can select a landmark that will estimate camera translation with an accuracy of 1.8 pixels, which means that the direction the eye is looking can be determined with an accuracy of better than 0.6 degrees.

The application of eye-tracking technology in the study of autism

For many decades, eye-tracking has been used to investigate gaze behaviour in the normal population. Recent studies have extended its use to individuals with disorders on the autism spectrum. Such studies typically focus on the processing of socially salient stimuli. In this review, we discuss the potential for this technique to reveal the strategies adopted by individuals with high-functioning autism when processing social information. Studies suggest that eye-tracking techniques have the potential to offer insight into the downstream difficulties in everyday social interaction which such individuals experience.

Directional eye fixation sensor using birefringence-based foveal detection

We recently developed and reported an eye fixation monitor that detects the fovea by its radial orientation of birefringent nerve fibers. The instrument used a four-quadrant photodetector and a normalized difference function to check for a best match between the detector quadrants and the arms of the bow-tie pattern of polarization states surrounding the fovea. This function had a maximum during central fixation but could not tell where the subject was looking relative to the center. We propose a linear transformation to obtain horizontal and vertical eye position coordinates from the four photodetector signals, followed by correction based on a priori calibration information. The method was verified on both a computer model and on human eyes. The major advantage of this new eye-tracking method is that it uses true information coming from the fovea, rather than reflections from other structures, to identify the direction of foveal gaze.

Automatic detection of camera translation in eye video recordings using multiple methods

In video eye tracking, shifting of the camera relative to the head can introduce artifacts. This work proposes a new combination of image processing techniques to automatically detect and measure the relative translation of cameras separately imaging the left and right eyes. It uses a priori physiological knowledge to improve the accuracy of algorithms. The first method compares an eye image with a reference frame using cross-correlation methods. The second isolates the upper eyelid and compares it with a reference frame to improve approximation of camera translation. The third creates an eyelid template from multiple frames and cross-correlates it with each image frame. The later has the highest accuracy, with a mean error of 1.3 pixels. It is more robust since it eliminates features of the image that may introduce errors. This excellent accuracy makes the method a viable solution for the problem of camera movement relative to the head.

Computation of gaze orientation under unrestrained head movements

Given the high relevance of visual input to human behavior, it is often important to precisely monitor the spatial orientation of the visual axis. One popular and accurate technique for measuring gaze orientation is based on the dual search coil. This technique does not allow for very large displacements of the subject, however, and is not robust with respect to translations of the head. More recently, less invasive procedures have been developed that record eye movements with camera-based systems attached to a helmet worn by the subject. Computational algorithms have also been developed that can calibrate eye orientation when the head's position is fixed. Given that camera-based systems measure the eye's position in its orbit, however, the reconstruction of gaze orientation is not as straightforward when the head is allowed to move. In this paper, we propose a new algorithm and calibration method to compute gaze orientation under unrestrained head conditions. Our method requires only the accurate measurement of orbital eye position (for instance, with a camera-based system), and the position of three points on the head. The calculations are expressed in terms of linear algebra, so can easily be interpreted and related to the geometry of the human body. Our calibration method has been tested experimentally and validated against independent data, proving that is it robust even under large translations, rotations, and torsions of the head.

Aging does not affect the accuracy of vertical saccades nor the quality of their binocular coordination: a study of a special elderly group

Fine binocular coordination of vertical saccades is a complex process requiring appropriate distribution of innervations to all six extraocular muscles. Loss of such coordination causes vertical binocular disparities that are particularly bothersome. We studied the quality of binocular control of vertical saccades in healthy subjects, 11 young adults (20-28 years) and 11 elderly adults (63-75 years). We used LED targets at 7.5 degrees or 15 degrees from the center (fixation), up or down in four conditions: gap and overlap tasks, each done at two distances--near (40 cm) and far (150 cm). Vertical eye movements were recorded with video-oculography (CHRONOS). The results showed: aged subjects performed vertical saccades as accurately as young subjects. Importantly, the binocular coordination of vertical saccades was well preserved in the elderly; the mean difference of vertical saccades between the two eyes was 0.10 degrees and 0.09 degrees in young and elderly subjects, respectively. Upward saccades were associated with divergence, downward ones with convergence. This secondary phenomenon was also the same and of the similar amplitude for young (1.30 degrees ) and elderly (1.25 degrees) subjects. Thus, despite its complexity, the quality of binocular coordination of vertical saccades remains intact with age. The other observations are mostly dependent of several aspects on the direction (up/down), viewing distances and eccentricities; the horizontal vergence during or after vertical saccades was found to be larger for downward saccades than for upward saccades, for saccades at far distance than at close, and for the more eccentric targets (15 degrees versus 7.5 degrees). All these phenomena are the same for both young and elderly subjects. We conclude that the accuracy and the binocular coordination of vertical saccades, at least for target steps less than 15 degrees , are preserved in elderly subjects <75 years who maintain good physical and intellectual form. The data are consistent with the idea of the existence of non-aging system function in the human CNS.

Smooth ocular pursuit during the transient disappearance of an accelerating visual target: the role of reflexive and voluntary control

This study examined the extent to which human subjects predict future target motion for the control of smooth ocular pursuit. Subjects were required to pursue an accelerating target (0, 4 or 8 degrees/s2) that underwent a transient occlusion, and consequently reappeared with the same or increased velocity. Presentations were received in a random or blocked order. Subjects exhibited anticipatory smooth pursuit prior to target motion onset, which in blocked presentations was scaled to the velocity generated by the target acceleration. In random presentations subjects also exhibited anticipatory smooth pursuit, but this was reflected in a more generalized response. During the transient occlusion all subjects exhibited a reduction in eye velocity, which was followed in the majority by a recovery prior to target reappearance. In random presentations, eye velocity decayed and recovered to a level that followed on from the response to the initial ramp. In blocked presentations, there was evidence of improved scaling throughout, which culminated in a significant increase in eye velocity between the start and end of the transient occlusion (8 degrees/s2 only). These findings are difficult to reconcile with reflexive accounts of oculomotor control that perpetuate current eye motion, and hence generate a simple form of prediction using a direct efference copy ("eye-velocity memory"). Rather, they are more consistent with the scaling of smooth pursuit eye movements by means of a more-persistent velocity-based representation, which plays a significant role in both random and blocked stimulus presentations.

Prolongation of latency of horizontal saccades in elderly is distance and task specific

The present study examined horizontal saccades in healthy subjects: 9 adults (20-32 years) and 10 aged subjects (63-83 years), under gap (fixation target extinguishes prior to target onset) and overlap (fixation stays on after target onset). The gap paradigm is known to promote fast initiation of saccades while the overlap paradigm promotes voluntary saccades with longer latency. In real life we perform saccades at various distances. In this study each paradigm was run at three viewing distances-20, 40 and 150 cm, corresponding to a convergence angle of 17.1 degrees, 8.6 degrees and 2.3 degrees, respectively. Eye movements were recorded with the Chronos video eye tracker or with the photoelectric IRIS. The main findings are: (i) increase in latency of saccades with age, with distance and with the overlap condition; (ii) evidence for interaction between these factors, indicating the following anomaly: in the gap condition and at near, aged subjects show short latencies similar to those of young adults; (iii) express type of latencies (between 80 and 120 ms) occur most frequently at near in the gap condition and at similar rates in young (25%) and aged subjects (20%). The specificity of close distance combined with the gap for triggering short latency saccades could be related to both attention and oculomotor fixation disengagement. The strength of coupling between fixation-eye movement control and visual attention control varies for different locations in space, and its decline with aging can be also different.

Dynamic pitch rotation affects eye torsion

CONCLUSION

Most of the subjects studied had eye torsion responses to pitch, although the direction of torsion varied between subjects. Opposite responses may be the result of individual variation in anatomical or physiological vector orientations of hair cells in the anterior or posterior utricle or in the saccule.

OBJECTIVE

This study aimed to determine whether systematic changes in eye torsion occur when subjects are rotated in forward and backward pitch.

MATERIALS AND METHODS

Twenty-one normal subjects were seated in a dual axis human rotator, positioned so that the interaural axis was aligned with the axis of pitch rotation. Fixation LED suppressed vertical or horizontal eye movement. Recordings were carried out in darkness apart from the fixation LED, using a three-dimensional eye tracker based on CMOS image sensors. Subjects were twice tilted from upright to 90 degrees occiput down, then forward to 45 degrees face down.

RESULTS

Most subjects had eye torsion changes in response to pitch, with mean amplitudes of approximately 2 degrees to 90 degrees backward tilt and 1 degree to 45 degrees forward tilt. Ten subjects had clockwise torsion to backward pitch and counterclockwise to forward pitch; six subjects had the opposite responses. Statistical testing of the distributions of the regression slopes between these two groups were significant (p<0.001). Five subjects had unclear responses.

Evidence for Synergy Between Saccades and Smooth Pursuit During Transient Target Disappearance

Visual tracking of moving objects requires prediction to compensate for visual delays and minimize mismatches between eye and target position and velocity. In everyday life, objects often disappear behind an occluder, and prediction is required to align eye and target at reappearance. Earlier studies investigating eye motion during target blanking showed that eye velocity first decayed after disappearance but was sustained or often recovered in a predictive way. Furthermore, saccades were directed toward the unseen target trajectory and therefore appeared to correct for position errors resulting from eye velocity decay. To investigate the synergy between smooth and saccadic eye movements, this study used a target blanking paradigm where both position and velocity of the target at reappearance could vary independently but were presented repeatedly to facilitate prediction. We found that eye velocity at target reappearance was only influenced by expected target velocity, whereas saccades responded to the expected change of target position at reappearance. Moreover, subjects exhibited on-line adaptation, on a trial-by-trial basis, between smooth and saccadic components; i.e., saccades compensated for variability of smooth eye displacement during the blanking period such that gaze at target reappearance was independent of the level of smooth eye displacement. We suggest these results indicate that information arising from efference copies of saccadic and smooth pursuit systems are combined with the goal of adjusting eye position at target reappearance. Based on prior experimental evidence, we hypothesize that this spatial remapping is carried out through interactions between a number of identified neurophysiological structures.

The control of vertical saccades in aged subjects

In real life we produce vertical saccades at different distances and eccentricities, and while our fixation is more or less actively engaged. The goal of this study is to examine vertical saccades in aged and young subjects, taking into consideration all these parameters. Eleven adults (20-28 years) and 11 aged subjects (63-83 years) were recruited. We used LED targets at 7.5 degrees or 15 degrees, up or down in four conditions: gap and overlap tasks, each done at two distances-at near (40 cm) and at far (150 cm). In the gap task fixation target extinguishes prior to target onset, while in the overlap condition it stays on after target onset; consequently, visual attention and fixation are employed differently in the two tasks. Eye movements were recorded with the Chronos video eye tracker. Results showed that vertical saccades were longer for aged subjects than for young adults under almost all conditions. For both aged and young subjects, latencies were shorter under the gap than under the overlap task. Latencies for eccentric targets at 15 degrees were significantly longer than those at 7.5 degrees but for aged subjects only; this effect was more pronounced for upward saccades under the overlap condition. Express type of latencies (80-120 ms) occurred frequently in the gap task and at similar rates for young adults (16%) and aged subjects (12%); in the overlap task express latencies were scarce in young adults (0.4%) and aged subjects (1.8%). Age deteriorates the ability to trigger regular volitional saccades but not the ability to produce express type of saccades. Latency increase with aging is attributed to the degeneration of central areas, e.g. oculomotor cortical areas involved in the initiation of vertical saccades.

Functional Assessment of Head???Eye Coordination During Vehicle Operation

PURPOSE

Visual impairment, resulting from ocular abnormalities or brain lesions, can significantly affect driving performance. The impact of vestibulopathy on head-eye coordination is also a concern in vehicle operation safety, yet to date there has been little functional research in this area. An understanding of decrements in driving ability resulting from visual and vestibular pathology, plus the differences in visual strategies used by novice and experienced drivers, would benefit from an objective analysis of head-eye coordination during vehicle operation.

METHODS

We have developed a laptop-based system for measuring eye, head, and vehicle movement in real time. Digital video cameras mounted on lightweight swimming goggles are used to provide images of the eye and scene, allowing assessment of gaze. In addition, the use of inertial measurement units to simultaneously transduce head and vehicle movement allows us to evaluate the vestibular contribution to stable vision.

RESULTS

Data was obtained from a flight simulator and while driving a car. During banking turns in the flight simulator, there was a sustained roll tilt of the head and eyes toward the scene-derived visual vertical with a combined gain of approximately 25%. One of the most complex visual tasks when driving was exiting a multistory car park, which involved the scanning of hundreds of parked vehicles with an average fixation time of approximately 100 ms. The vertical vestibulo-ocular reflex was also found to make a significant contribution to the maintenance of dynamic visual acuity even while driving on paved surfaces.

CONCLUSION

These results demonstrate the viability of functional assessment of head-eye coordination during vehicle operation, and potential applications of this technology to driver assessment are discussed. Analysis of both active and reflex contributions to gaze may provide a clearer understanding of the impact of visual and vestibular impairment on driving ability.

Automatic detection of camera translation in eye video recordings using multiple methods

A concern with video eye movement tracking is that movement of the camera headset relative to the head creates an artifact of eye movement in pupil-detection software. We describe the development of, and compare the results of, three automatic image processing algorithms to measure camera movement. The best of the algorithms has an average accuracy of 1.3 pixels, equivalent to 0.49 deg with our eye tracking system.

Using video-oculography for galvanic evoked vestibulo-ocular monitoring in comatose patients

Binocular eye movement responses to galvanic vestibular stimulation were measured in comatose patients. Healthy persons have been demonstrated to show a consistent and reproducible relationship between eye movement and galvanic vestibular stimulation. Any pathology of the vestibular integrating structures (brainstem and to some degree the cerebellum and cortex) is assumed to influence this response pathognomonically. A monitoring facility was designed to record eye movements during galvanic labyrinth polarization (GaLa) in order to examine the vestibular response in comatose patients. GaLa was applied by means of two pairs of electrodes attached between the mastoid and interscapular region. A custom-built, battery-driven current-source stimulator served to generate sinusoidal stimuli that could be applied either unilaterally or simultaneously and independently to the left and right labyrinths. The resultant binocular eye movements were recorded using a Chronos Eye Tracker and the digital image sequences stored for subsequent analysis. Repeated testing was performed in five comatose patients with a Glasgow Coma Score of 3, who were intubated and ventilated (subarachnoid hemorrhage (n = 3), traumatic injury (n = 2)). The observed ocular movements varied. In four cases ocular movements in response to GaLa were observed, while one patient showed neither spontaneous nor galvanic-induced eye movements. The latter was diagnosed as brain-dead 2 days after testing. In contrast, the patients who showed ocular movements synchronous with galvanic stimulation left the intensive care unit (ICU) for rehabilitation. These data indicate that it is possible to assess brain function in comatose patients by evaluating the vestibulo-ocular response to galvanic stimulation. The designed monitoring facility can be used on the ICU ward without disconnection of other monitoring equipment.

Characteristics of saccadic intrusions

Primary fixation is never perfectly stable, but is frequently interrupted by slow drifts, microsaccades and saccadic intrusions (SI). SI are involuntary, conjugate movements which take the form of an initial fast movement away from the desired eye position and followed after a short duration, by either a return secondary saccade or a drift. The purpose of this study was to examine the prevalence and metrics of SI in a population of 50 healthy subjects. Using both one and two dimensional recordings we find that all 50 members of the subject group exhibited SI. The SI were bilateral, conjugate and horizontal. No purely vertical SI were detected when examined in three subjects. SI amplitude mean and range was 0.6 degrees +/-0.5 degrees, 0.1 degrees -4.1 degrees; SI frequency mean and range was 18.0+/-14.3 per min, 1.0-54.8 per min; SI duration mean and range was 225+/-150, 20-870 ms. The mean SI amplitude and frequency when SI<0.5 degrees were removed was 0.97 degrees +/-0.56 degrees and 7.0+/-11.4 per min respectively. Age was positively correlated with SI amplitude (p<0.01), but there was no correlation between age and SI frequency. Three of four types of SI monophasic square wave intrusions (MSWI), biphasic square wave intrusions (BSWI) and double saccadic pulses (DSP) were found to be exclusively saccadic, whilst the fourth type, the single saccadic pulses (SSP), were confirmed to exhibit a slow secondary component. MSWI were the most frequently observed SI occurring in 47 out of 50 (94%) of the subjects with a mean amplitude, frequency and duration of 0.7 degrees +/-0.5 degrees, 11.5+/-11.6 per min, and 255+/-147 ms respectively. Mean amplitudes and frequencies for BSWI (n=20), SSP (n=11) and DSP (n=34) were found to be 0.50 degrees +/-0.2 degrees, 1.2+/-2.5 per min; 0.40 degrees +/-0.20 degrees, 0.4+/-1.0 per min and 0.3 degrees +/-0.4 degrees, 5.0+/-8.7 per min respectively. No differences in MSWI characteristics were found between binocular and monocular viewing. Possible explanations for SI occurrence include experimental viewing conditions, subject fatigue and covert shifts in attention.

Predictive Smooth Ocular Pursuit During the Transient Disappearance of a Visual Target

When a moving target disappears and there is a complete absence of visual feedback signals, eye velocity decays rapidly but often recovers to previous levels if there is an expectation the target will reappear further along its trajectory Given that eye velocity cannot be maintained under such circumstances, the anticipatory recovery may function to minimize the developing velocity error. When there is a change in target velocity during a transient, any recovery should ideally be scaled and hence predictive of the expected target velocity at reappearance. This study confirmed that subjects did not maintain eye velocity close to target velocity for the duration of the inter-stimulus interval (ISI). The majority of subjects exhibited an initial reduction in eye velocity followed by a scaled recovery prior to target reappearance. Eye velocity during the ISI was, therefore, predictive of the expected change in target velocity. These behavioral data were simulated using a model in which gain applied to the visuomotor drive is reduced after the loss of visual feedback and then modulated depending on subject’s expectation regarding the target’s future trajectory.

A laser-based eye-tracking system

This paper reports on the development of a new eye-tracking system for noninvasive recording of eye movements. The eye tracker uses a flying-spot laser to selectively image landmarks on the eye and, subsequently, measure horizontal, vertical, and torsional eye movements. Considerable work was required to overcome the adverse effects of specular reflection of the flying-spot from the surface of the eye onto the sensing elements of the eye tracker. These effects have been largely overcome, and the eye-tracker has been used to document eye movement abnormalities, such as abnormal torsional pulsion of saccades, in the clinical setting.