Accuracy and re-test reliability of mobile eye-tracking in Parkinson's disease and older adults

Assessment of Visual Attention in Teams with or without Dedicated Team Leaders: A Neonatal Simulation-Based Pilot Randomised Cross-Over Trial Utilising Low-Cost Eye-Tracking Technology

BACKGROUND

Eye-tracking technology could be used to study human factors during teamwork.

OBJECTIVES

This work aimed to compare the visual attention (VA) of a team member acting as both a team leader and managing the airway, compared to a team member performing the focused task of managing the airway in the presence of a dedicated team leader. This work also aimed to report differences in team performance, behavioural skills, and workload between the two groups using validated tools.

METHODS

We conducted a simulation-based, pilot randomised controlled study. The participants included were volunteer paediatric trainees, nurse practitioners, and neonatal nurses. Three teams consisting of four team members were formed. Each team participated in two identical neonatal resuscitation simulation scenarios in a random order, once with and once without a team leader. Using a commercially available eye-tracking device, we analysed VA regarding attention to (1) a manikin, (2) a colleague, and (3) a monitor. Only the trainee who was the airway operator would wear eye-tracking glasses in both simulations.

RESULTS

In total, 6 simulation scenarios and 24 individual role allocations were analysed. Participants in a no-team-leader capacity had a greater number of total fixations on manikin and monitors, though this was not significant. There were no significant differences in team performance, behavioural skills, and individual workload. Physical demand was reported as significantly higher by participants in the group without a team leader. During debriefing, all the teams expressed their preference for having a dedicated team leader.

CONCLUSION

In our pilot study using low-cost technology, we could not demonstrate the difference in VA with the presence of a team leader.

Webcam eye tracking close to laboratory standards: Comparing a new webcam-based system and the EyeLink 1000

This paper aims to compare a new webcam-based eye-tracking system, integrated into the Labvanced platform for online experiments, to a "gold standard" lab-based eye tracker (EyeLink 1000 - SR Research). Specifically, we simultaneously recorded data with both eye trackers in five different tasks, analyzing their real-time performance. These tasks were a subset of a standardized test battery for eye trackers, including a Large Grid task, Smooth Pursuit eye movements, viewing natural images, and two Head Movements tasks (roll, yaw). The results show that the webcam-based system achieved an overall accuracy of 1.4°, and a precision of 1.1° (standard deviation (SD) across subjects), an error of about 0.5° larger than the EyeLink system. Interestingly, both accuracy (1.3°) and precision (0.9°) were slightly better for centrally presented targets, the region of interest in many psychophysical experiments. Remarkably, the correlation of raw gaze samples between the EyeLink and webcam-based was at about 90% for the Large Grid task and about 80% for Free View and Smooth Pursuit. Overall, these results put the performance of the webcam-based system roughly on par with mobile eye-tracking devices (Ehinger et al. PeerJ, 7, e7086, 2019; Tonsen et al., 2020) and demonstrate substantial improvement compared to existing webcam eye-tracking solutions (Papoutsaki et al., 2017).

Evaluating the Tobii Pro Glasses 2 and 3 in static and dynamic conditions

Over the past few decades, there have been significant developments in eye-tracking technology, particularly in the domain of mobile, head-mounted devices. Nevertheless, questions remain regarding the accuracy of these eye-trackers during static and dynamic tasks. In light of this, we evaluated the performance of two widely used devices: Tobii Pro Glasses 2 and Tobii Pro Glasses 3. A total of 36 participants engaged in tasks under three dynamicity conditions. In the "seated with a chinrest" trial, only the eyes could be moved; in the "seated without a chinrest" trial, both the head and the eyes were free to move; and during the walking trial, participants walked along a straight path. During the seated trials, participants' gaze was directed towards dots on a wall by means of audio instructions, whereas in the walking trial, participants maintained their gaze on a bullseye while walking towards it. Eye-tracker accuracy was determined using computer vision techniques to identify the target within the scene camera image. The findings showed that Tobii 3 outperformed Tobii 2 in terms of accuracy during the walking trials. Moreover, the results suggest that employing a chinrest in the case of head-mounted eye-trackers is counterproductive, as it necessitates larger eye eccentricities for target fixation, thereby compromising accuracy compared to not using a chinrest, which allows for head movement. Lastly, it was found that participants who reported higher workload demonstrated poorer eye-tracking accuracy. The current findings may be useful in the design of experiments that involve head-mounted eye-trackers.

iVOMS: Instrumented Vestibular / Ocular motor screen in healthy controls and mild traumatic brain injury

Objective Vestibular/ocular deficits occur with mild traumatic brain injury (mTBI). The vestibular/ocular motor screening (VOMS) tool is used to assess individuals post-mTBI, which primarily relies upon subjective self-reported symptoms. Instrumenting the VOMS (iVOMS) with technology may allow for more objective assessment post-mTBI, which reflects actual task performance. This study aimed to validate the iVOMS analytically and clinically in mTBI and controls. Methods Seventy-nine people with sub-acute mTBI (<12 weeks post-injury) and forty-four healthy control participants performed the VOMS whilst wearing a mobile eye-tracking on a one-off visit. People with mTBI were included if they were within 12 weeks of a physician diagnosis. Participants were excluded if they had any musculoskeletal, neurological or sensory deficits which could explain dysfunction. A series of custom-made eye tracking algorithms were used to assess recorded eye-movements. Results The iVOMS was analytically valid compared to the reference (ICC2,1 0.85-0.99) in mTBI and controls. The iVOMS outcomes were clinically valid as there were significant differences between groups for convergence, vertical saccades, smooth pursuit, vestibular ocular reflex and visual motion sensitivity outcomes. However, there was no significant relationship between iVOMS outcomes and self-reported symptoms. Conclusion The iVOMS is analytically and clinically valid in mTBI and controls, but further work is required to examine the sensitivity of iVOMS outcomes across the mTBI spectrum. Findings also highlighted that symptom and physiological issue resolution post-mTBI may not coincide and relationships need further examination.

A Comparison of Night Hazard Detection between Younger and Older Drivers under Driving Simulation and Real-World Conditions

''Using eye-tracking technology, this study examined hazard detection at night. Using a 2 (younger versus older) x 2 (simulator versus on road) repeated-measures mixed design, 16 older adults and 17 younger adults drove their own vehicle and on a driving simulator under nighttime conditions wearing eye tracking technology. Both driving conditions had three roadway hazards of pedestrians looking at their cell phone while posed to cross the roadway. Pupil glances were recorded using outcome measures of total fixation duration, number of fixations, and time to first fixation. Results showed older adults detected hazards similarly to younger adults, especially during on-road performance. Night hazard detection was similar across driving conditions except for time to first fixation, which was faster on-road for both age groups. Results support potential use of driving simulators as a proxy for on-road with night driving needed for research and practice.

Eye tracking to assess concussions: an intra-rater reliability study with healthy youth and adult athletes of selected contact and collision team sports

Eye movements that are dependent on cognition hold promise in assessing sports-related concussions but research on reliability of eye tracking measurements in athletic cohorts is very limited. This observational test-retest study aimed to establish whether eye tracking technology is a reliable tool for assessing sports-related concussions in youth and adult athletes partaking in contact and collision team sports. Forty-three youth (15.4 ± 2.2 years) and 27 adult (22.2 ± 2.9 years) Rugby Union and soccer players completed the study. Eye movements were recorded using SMIRED250mobile while participants completed a test battery twice, with a 1-week interval that included self-paced saccade (SPS), fixation stability, memory-guided sequence (MGS), smooth pursuit (SP), and antisaccades (AS) tasks. Intra-class correlation coefficient (ICC), measurement error (SEM) and smallest real difference (SRD) were calculated for 47 variables. Seventeen variables achieved an ICC > 0.50. In the adults, saccade count in SPS had good reliability (ICC = 0.86, SRD = 146.6 saccades). In the youth, the average blink duration in MGS had excellent reliability (ICC = 0.99, SRD = 59.4 ms); directional errors in AS tasks and gain of diagonal SP had good reliability (ICC = 0.78 and 0.77, SRD = 25.3 and 395.1%, respectively). Four metrics were found in this study to be reliable candidates for further biomarker validity research in contact and collision sport cohorts. Many variables failed to present a sufficient level of robustness for a practical diagnostic tool; possibly, because athletic cohorts have higher homogeneity, along with latent adverse effects of undetected concussions and repetitive head impacts. Since reliability of a measure can influence type II error, effect sizes, and confidence intervals, it is strongly advocated to conduct dedicated reliability evaluations prior to any validity studies.

The Measurement of Eye Movements in Mild Traumatic Brain Injury: A Structured Review of an Emerging Area

Mild traumatic brain injury (mTBI), or concussion, occurs following a direct or indirect force to the head that causes a change in brain function. Many neurological signs and symptoms of mTBI can be subtle and transient, and some can persist beyond the usual recovery timeframe, such as balance, cognitive or sensory disturbance that may pre-dispose to further injury in the future. There is currently no accepted definition or diagnostic criteria for mTBI and therefore no single assessment has been developed or accepted as being able to identify those with an mTBI. Eye-movement assessment may be useful, as specific eye-movements and their metrics can be attributed to specific brain regions or functions, and eye-movement involves a multitude of brain regions. Recently, research has focused on quantitative eye-movement assessments using eye-tracking technology for diagnosis and monitoring symptoms of an mTBI. However, the approaches taken to objectively measure eye-movements varies with respect to instrumentation, protocols and recognition of factors that may influence results, such as cognitive function or basic visual function. This review aimed to examine previous work that has measured eye-movements within those with mTBI to inform the development of robust or standardized testing protocols. Medline/PubMed, CINAHL, PsychInfo and Scopus databases were searched. Twenty-two articles met inclusion/exclusion criteria and were reviewed, which examined saccades, smooth pursuits, fixations and nystagmus in mTBI compared to controls. Current methodologies for data collection, analysis and interpretation from eye-tracking technology in individuals following an mTBI are discussed. In brief, a wide range of eye-movement instruments and outcome measures were reported, but validity and reliability of devices and metrics were insufficiently reported across studies. Interpretation of outcomes was complicated by poor study reporting of demographics, mTBI-related features (e.g., time since injury), and few studies considered the influence that cognitive or visual functions may have on eye-movements. The reviewed evidence suggests that eye-movements are impaired in mTBI, but future research is required to accurately and robustly establish findings. Standardization and reporting of eye-movement instruments, data collection procedures, processing algorithms and analysis methods are required. Recommendations also include comprehensive reporting of demographics, mTBI-related features, and confounding variables.

Aging and eye tracking: in the quest for objective biomarkers

Recent applications of eye tracking for diagnosis, prognosis and follow-up of therapy in age-related neurological or psychological deficits have been reviewed. The review is focused on active aging, neurodegeneration and cognitive impairments. The potential impacts and current limitations of using characterizing features of eye movements and pupillary responses (oculometrics) as objective biomarkers in the context of aging are discussed. A closer look into the findings, especially with respect to cognitive impairments, suggests that eye tracking is an invaluable technique to study hidden aspects of aging that have not been revealed using any other noninvasive tool. Future research should involve a wider variety of oculometrics, in addition to saccadic metrics and pupillary responses, including nonlinear and combinatorial features as well as blink- and fixation-related metrics to develop biomarkers to trace age-related irregularities associated with cognitive and neural deficits.

Validation of a velocity-based algorithm to quantify saccades during walking and turning in mild traumatic brain injury and healthy controls

OBJECTIVE

Saccadic (fast) eye movements are a routine aspect of neurological examination and are a potential biomarker of mild traumatic brain injury (mTBI). Objective measurement of saccades has become a prominent focus of mTBI research, as eye movements may be a useful assessment tool for deficits in neural structures or processes. However, saccadic measurement within mobile infra-red (IR) eye-tracker raw data requires a valid algorithm. The objective of this study was to validate a velocity-based algorithm for saccade detection in IR eye-tracking raw data during walking (straight ahead and while turning) in people with mTBI and healthy controls.

APPROACH

Eye-tracking via a mobile IR Tobii Pro Glasses 2 eye-tracker (100 Hz) was performed in people with mTBI (n  =  10) and healthy controls (n  =  10). Participants completed two walking tasks: straight walking (walking back and forth for 1 min over a 10 m distance), and walking and turning (turns course included 45°, 90° and 135° turns). Five trials per subject, for one-hundred total trials, were completed. A previously reported velocity-based saccade detection algorithm was adapted and validated by assessing agreement between algorithm saccade detections and the number of correct saccade detections determined from manual video inspection (ground truth reference).

MAIN RESULTS

Compared with video inspection, the IR algorithm detected ~97% (n  =  4888) and ~95% (n  =  3699) of saccades made by people with mTBI and controls, respectively, with excellent agreement to the ground truth (intra-class correlation coefficient_2,1  =  .979 to .999).

SIGNIFICANCE

This study provides a simple yet highly robust algorithm for the processing of mobile eye-tracker raw data in mTBI and controls. Future studies may consider validating this algorithm with other IR eye-trackers and populations.

Symmetric Evaluation of Multimodal Human–Robot Interaction with Gaze and Standard Control

Control of robot arms is often required in engineering and can be performed by using different methods. This study examined and symmetrically compared the use of a controller, eye gaze tracker and a combination thereof in a multimodal setup for control of a robot arm. Tasks of different complexities were defined and twenty participants completed an experiment using these interaction modalities to solve the tasks. More specifically, there were three tasks: the first was to navigate a chess piece from a square to another pre-specified square; the second was the same as the first task, but required more moves to complete; and the third task was to move multiple pieces to reach a solution to a pre-defined arrangement of the pieces. Further, while gaze control has the potential to be more intuitive than a hand controller, it suffers from limitations with regard to spatial accuracy and target selection. The multimodal setup aimed to mitigate the weaknesses of the eye gaze tracker, creating a superior system without simply relying on the controller. The experiment shows that the multimodal setup improves performance over the eye gaze tracker alone ( p < 0.05 ) and was competitive with the controller only setup, although did not outperform it ( p > 0.05 ).

Saccade frequency response to visual cues during gait in Parkinson's disease: the selective role of attention

Gait impairment is a core feature of Parkinson's disease (PD) with implications for falls risk. Visual cues improve gait in PD, but the underlying mechanisms are unclear. Evidence suggests that attention and vision play an important role; however, the relative contribution from each is unclear. Measurement of visual exploration (specifically saccade frequency) during gait allows for real-time measurement of attention and vision. Understanding how visual cues influence visual exploration may allow inferences of the underlying mechanisms to response which could help to develop effective therapeutics. This study aimed to examine saccade frequency during gait in response to a visual cue in PD and older adults and investigate the roles of attention and vision in visual cue response in PD. A mobile eye-tracker measured saccade frequency during gait in 55 people with PD and 32 age-matched controls. Participants walked in a straight line with and without a visual cue (50 cm transverse lines) presented under single task and dual-task (concurrent digit span recall). Saccade frequency was reduced when walking in PD compared to controls; however, visual cues ameliorated saccadic deficit. Visual cues significantly increased saccade frequency in both PD and controls under both single task and dual-task. Attention rather than visual function was central to saccade frequency and gait response to visual cues in PD. In conclusion, this study highlights the impact of visual cues on visual exploration when walking and the important role of attention in PD. Understanding these complex features will help inform intervention development.

Patients with chronic peripheral vestibular hypofunction compared to healthy subjects exhibit differences in gaze and gait behaviour when walking on stairs and ramps

Objective

The aim of this study was to compare gaze behaviour during stair and ramp walking between patients with chronic peripheral vestibular hypofunction and healthy human subjects.

Methods

Twenty four (24) patients with chronic peripheral vestibular hypofunction (14 unilateral and 10 bilateral) and 24 healthy subjects performed stair and ramp up and down walks at self-selected speed. The walks were repeated five times. A mobile eye tracker was used to record gaze behaviour (defined as time directed to pre-defined areas) and an insole measurement device assessed gait (speed, step time, step length). During each walk gaze behaviour relative to i) detection of first transition area “First TA”, ii) detection of steps of the mid-staircase area and the handrail “Structure”, iii) detection of second transition area “Second TA”, and iv) looking elsewhere “Elsewhere” was assessed and expressed as a percentage of the walk duration. For all variables, a one-way ANOVA followed by contrast tests was conducted.

Results

Patients looked significantly longer at the “Structure” (p<0.001) and “Elsewhere” (p<0.001) while walking upstairs compared to walking downstairs (p<0.013). Patients looked significantly longer at the “Structure” (p<0.001) and “Elsewhere” (p<0.001) while walking upstairs compared to walking downstairs (p<0.013). No differences between groups were observed for the transition areas with exception of stair ascending. Patients were also slower going downstairs (p = 0.002) and presented with an increased step time (p = 0.003). Patients were walking faster up the ramp (p = 0.014) with longer step length (p = 0.008) compared to walking down the ramp (p = 0.050) with shorter step length (p = 0.024).

Conclusions

Patients with chronic peripheral vestibular hypofunction differed in time directed to pre-defined areas during stair and ramp walking and looked longer at stair and ramp areas of interest during walking compared to healthy subjects. Patients did not differ in time directed to pre-defined areas during the stair-floor transition area while going downstairs, an area where accidents may frequently occur.

Do you see what I see? Mobile eye-tracker contextual analysis and inter-rater reliability

Mobile eye-trackers are currently used during real-world tasks (e.g. gait) to monitor visual and cognitive processes, particularly in ageing and Parkinson’s disease (PD). However, contextual analysis involving fixation locations during such tasks is rarely performed due to its complexity. This study adapted a validated algorithm and developed a classification method to semi-automate contextual analysis of mobile eye-tracking data. We further assessed inter-rater reliability of the proposed classification method. A mobile eye-tracker recorded eye-movements during walking in five healthy older adult controls (HC) and five people with PD. Fixations were identified using a previously validated algorithm, which was adapted to provide still images of fixation locations (n = 116). The fixation location was manually identified by two raters (DH, JN), who classified the locations. Cohen’s kappa correlation coefficients determined the inter-rater reliability. The algorithm successfully provided still images for each fixation, allowing manual contextual analysis to be performed. The inter-rater reliability for classifying the fixation location was high for both PD (kappa = 0.80, 95% agreement) and HC groups (kappa = 0.80, 91% agreement), which indicated a reliable classification method. This study developed a reliable semi-automated contextual analysis method for gait studies in HC and PD. Future studies could adapt this methodology for various gait-related eye-tracking studies.

Direct and indirect effects of attention and visual function on gait impairment in Parkinson's disease: influence of task and turning

Gait impairment is a core feature of Parkinson's disease (PD) which has been linked to cognitive and visual deficits, but interactions between these features are poorly understood. Monitoring saccades allows investigation of real-time cognitive and visual processes and their impact on gait when walking. This study explored: (i) saccade frequency when walking under different attentional manipulations of turning and dual-task; and (ii) direct and indirect relationships between saccades, gait impairment, vision and attention. Saccade frequency (number of fast eye movements per-second) was measured during gait in 60 PD and 40 age-matched control participants using a mobile eye-tracker. Saccade frequency was significantly reduced in PD compared to controls during all conditions. However, saccade frequency increased with a turn and decreased under dual-task for both groups. Poorer attention directly related to saccade frequency, visual function and gait impairment in PD, but not controls. Saccade frequency did not directly relate to gait in PD, but did in controls. Instead, saccade frequency and visual function deficit indirectly impacted gait impairment in PD, which was underpinned by their relationship with attention. In conclusion, our results suggest a vital role for attention with direct and indirect influences on gait impairment in PD. Attention directly impacted saccade frequency, visual function and gait impairment in PD, with connotations for falls. It also underpinned indirect impact of visual and saccadic impairment on gait. Attention therefore represents a key therapeutic target that should be considered in future research.

iTrack: instrumented mobile electrooculography (EOG) eye-tracking in older adults and Parkinson's disease

Detection of saccades (fast eye-movements) within raw mobile electrooculography (EOG) data involves complex algorithms which typically process data acquired during seated static tasks only. Processing of data during dynamic tasks such as walking is relatively rare and complex, particularly in older adults or people with Parkinson's disease (PD). Development of algorithms that can be easily implemented to detect saccades is required. This study aimed to develop an algorithm for the detection and measurement of saccades in EOG data during static (sitting) and dynamic (walking) tasks, in older adults and PD. Eye-tracking via mobile EOG and infra-red (IR) eye-tracker (with video) was performed with a group of older adults (n  =  10) and PD participants (n  =  10) (⩾50 years). Horizontal saccades made between targets set 5°, 10° and 15° apart were first measured while seated. Horizontal saccades were then measured while a participant walked and executed a 40° turn left and right. The EOG algorithm was evaluated by comparing the number of correct saccade detections and agreement (ICC_2,1) between output from visual inspection of eye-tracker videos and IR eye-tracker. The EOG algorithm detected 75-92% of saccades compared to video inspection and IR output during static testing, with fair to excellent agreement (ICC_2,1 0.49-0.93). However, during walking EOG saccade detection reduced to 42-88% compared to video inspection or IR output, with poor to excellent (ICC_2,1 0.13-0.88) agreement between methodologies. The algorithm was robust during seated testing but less so during walking, which was likely due to increased measurement and analysis error with a dynamic task. Future studies may consider a combination of EOG and IR for comprehensive measurement.