Comparison of visual SLAM and IMU in tracking head movement outdoors

Gaze Scanning on Mid-Block Sidewalks by Pedestrians With Homonymous Hemianopia With or Without Spatial Neglect

Purpose

The purpose of this study was to investigate gaze-scanning by pedestrians with homonymous hemianopia (HH) when walking on mid-block sidewalks.

Methods

Pedestrians with right homonymous hemianopia (RHH), and left homonymous hemianopia (LHH) without and with left spatial neglect (LHSN) walked on city streets wearing a gaze-tracking system. Gaze points were obtained by combining head movement and eye-in-head movement. Mixed-effects regression models were used to compare horizontal gaze scan magnitudes and rates between the side of the hemi-field loss (BlindSide) and the seeing side (SeeingSide), among the three subject groups, and between mid-block walking and street crossing segments.

Results

A total of 7021 gaze scans were obtained from 341 minutes of mid-block walking videos by 19 participants (6 with LHH, 7 with RHH, and 6 with LHSN). The average gaze magnitude and scanning rate in mid-block segments were significantly higher towards the BlindSide than the SeeingSide in LHH (magnitude larger by 1.9° (degrees), P = 0.006; scan rate higher by 4.2 scans/minute, P < 0.001) and RHH subjects (magnitude larger by 3.3°, P < 0.001; scan rate higher by 3.2 scans/minute, P = 0.002), but they were not significantly different in LHSN subjects. The scanning rate, in terms of scans/minute (mean, 95% confidence interval [CI]) was significantly lower in LHSN subjects (mean = 6.9, 95% CI = 5.6-8.7) than LHH (mean = 10.2, 95% CI = 8.0-13.1; P = 0.03) and RHH (mean = 11.1, 95% CI = 9.0-13.7; P = 0.007) subjects. Compared to street-crossings, the scan rate during the mid-block segments was lower by 3.5 scans/minute (P < 0.001) and the gaze magnitude was smaller by 3.8° (P < 0.001) over the 3 groups.

Conclusions

Evidence of compensatory scanning suggests a proactive, top-down mechanism driving gaze in HH. The presence of spatial neglect (SN) appeared to negatively impact the top-down process.

ACE-DNV: Automatic classification of gaze events in dynamic natural viewing

Eye movements offer valuable insights for clinical interventions, diagnostics, and understanding visual perception. The process usually involves recording a participant's eye movements and analyzing them in terms of various gaze events. Manual identification of these events is extremely time-consuming. Although the field has seen the development of automatic event detection and classification methods, these methods have primarily focused on distinguishing events when participants remain stationary. With increasing interest in studying gaze behavior in freely moving participants, such as during daily activities like walking, new methods are required to automatically classify events in data collected under unrestricted conditions. Existing methods often rely on additional information from depth cameras or inertial measurement units (IMUs), which are not typically integrated into mobile eye trackers. To address this challenge, we present a framework for classifying gaze events based solely on eye-movement signals and scene video footage. Our approach, the Automatic Classification of gaze Events in Dynamic and Natural Viewing (ACE-DNV), analyzes eye movements in terms of velocity and direction and leverages visual odometry to capture head and body motion. Additionally, ACE-DNV assesses changes in image content surrounding the point of gaze. We evaluate the performance of ACE-DNV using a publicly available dataset and showcased its ability to discriminate between gaze fixation, gaze pursuit, gaze following, and gaze shifting (saccade) events. ACE-DNV exhibited comparable performance to previous methods, while eliminating the necessity for additional devices such as IMUs and depth cameras. In summary, ACE-DNV simplifies the automatic classification of gaze events in natural and dynamic environments. The source code is accessible at https://github.com/arnejad/ACE-DNV .

An Enhancement of Outdoor Location-Based Augmented Reality Anchor Precision through VSLAM and Google Street View

Outdoor Location-Based Augmented Reality (LAR) applications require precise positioning for seamless integrations of virtual content into immersive experiences. However, common solutions in outdoor LAR applications rely on traditional smartphone sensor fusion methods, such as the Global Positioning System (GPS) and compasses, which often lack the accuracy needed for precise AR content alignments. In this paper, we introduce an innovative approach to enhance LAR anchor precision in outdoor environments. We leveraged Visual Simultaneous Localization and Mapping (VSLAM) technology, in combination with innovative cloud-based methodologies, and harnessed the extensive visual reference database of Google Street View (GSV), to address the accuracy limitation problems. For the evaluation, 10 Point of Interest (POI) locations were used as anchor point coordinates in the experiments. We compared the accuracies between our approach and the common sensor fusion LAR solution comprehensively involving accuracy benchmarking and running load performance testing. The results demonstrate substantial enhancements in overall positioning accuracies compared to conventional GPS-based approaches for aligning AR anchor content in the real world.

Gaze Scanning at Street Crossings by Pedestrians With Homonymous Hemianopia With and Without Hemispatial Neglect

Purpose

To investigate compensatory gaze-scanning behaviors during street crossings by pedestrians with homonymous hemianopia (HH) and hemispatial neglect (HSN).

Methods

Pedestrians with right homonymous hemianopia (RHH) and left homonymous hemianopia without (LHH) and with left spatial-neglect (LHSN) walked on city streets wearing a gaze-tracking system that also captured scene videos. Street-crossing instances were manually annotated, and horizontal gaze scan of magnitude ≥20° and scanning rates were compared within-subject, between the side of the hemifield loss (BlindSide) and the other side (SeeingSide). Proportion of instances with scans to both the left and the right side at nonsignalized crossings (indicative of safe scanning behavior) were compared among the three subject groups.

Results

Data from 19 participants (6 LHH, 7 RHH, and 6 with mild [4] or moderate [2] LHSN), consisting of 521 street-crossing instances of a total duration of 201 minutes and 5375 gaze scans, were analyzed. The overall gaze magnitude (mean [95% confidence interval (CI)]) was significantly larger toward the BlindSide (40.4° [39.1°-41.9°]) than the SeeingSide (36° [34.8°-37.3°]; P < 0.001). The scanning rate (mean [95% CI] scans/min) toward the BlindSide (14 [12.5-15.6]) was significantly higher than the SeeingSide (11.5 [10.3°-12.9°]; P < 0.001). The scanning rate in the LHSN group (10.7 [8.9-12.8]) was significantly lower than the LHH group (14 [11.6-17.0]; P = 0.045). The proportion of nonsignalized crossings with scans to both sides was significantly lower in LHSN (58%; P = 0.039) and RHH (51%; P = 0.003) than LHH (75%) participants.

Conclusions

All groups demonstrated compensatory scanning, making more gaze scans with larger magnitudes to the blind side. Mild to moderate LHSN adversely impacted the scanning rate.

Natural statistics of human head orientation constrain models of vestibular processing

Head orientation relative to gravity determines how gravity-dependent environmental structure is sampled by the visual system, as well as how gravity itself is sampled by the vestibular system. Therefore, both visual and vestibular sensory processing should be shaped by the statistics of head orientation relative to gravity. Here we report the statistics of human head orientation during unconstrained natural activities in humans for the first time, and we explore implications for models of vestibular processing. We find that the distribution of head pitch is more variable than head roll and that the head pitch distribution is asymmetrical with an over-representation of downward head pitch, consistent with ground-looking behavior. We further suggest that pitch and roll distributions can be used as empirical priors in a Bayesian framework to explain previously measured biases in perception of both roll and pitch. Gravitational and inertial acceleration stimulate the otoliths in an equivalent manner, so we also analyze the dynamics of human head orientation to better understand how knowledge of these dynamics can constrain solutions to the problem of gravitoinertial ambiguity. Gravitational acceleration dominates at low frequencies and inertial acceleration dominates at higher frequencies. The change in relative power of gravitational and inertial components as a function of frequency places empirical constraints on dynamic models of vestibular processing, including both frequency segregation and probabilistic internal model accounts. We conclude with a discussion of methodological considerations and scientific and applied domains that will benefit from continued measurement and analysis of natural head movements moving forward.