EEG & Eye Tracking User Experiments for Spatial Memory Task on Maps

Exploring Horizontally Flipped Interaction in Virtual Reality for Improving Spatial Ability

Virtual reality (VR) is a high-fidelity medium that can offer experiences that are close to real-life. Spatial ability plays an important role in human life, including academic achievement and advancement in work settings. Spatial ability is known to be improved by practicing relevant tasks. Mental rotation and spatial perception are among such tasks that improve spatial skills. In this research, we investigated a "mirror-reversed" interaction technique in a cup stacking task in VR and looked into its effects on spatial ability, brain activity regarding spatial processing and attention (measured with EEG), performance, and user experience in male participants. Participants stacked cups according to given patterns using direct manipulation with horizontally flipped controls, similar to looking in a mirror while performing object manipulation in real life. In a between-subjects user study, we compared this novel interaction with a baseline where the participants completed the same task with regular controls. Although there was no significant main effect of group on the mental rotation and perspective taking/spatial orientation tests scores, within-group analysis indicated a trend toward an improvement in the mirror-reversed group in spatial orientation, while both groups showed a trend toward improvement in mental rotation. Participants in both groups got better at the task over time (their task completion durations decreased). EEG data revealed significant theta band power increase in the mirror-reversed group whereas there was no difference in the alpha band power between the two groups. Our results are encouraging for exploring spatially challenging interactions in VR for spatial skills training. We share the implementation and user study results, and discuss the implications.

Differences in thinking flexibility between novices and experts based on eye tracking

The influence of thinking flexibility on design is often underestimated by researchers in the field of design education. In this study, morphological analysis was used as a tool to develop design proposals and eye tracking technology was applied to track the attention. The feature of thinking activities in problem-solving between two groups (novice and expert) was analyzed by Heat map and Gaze plot in qualitative, and measured by indicators such as fixation and saccade in quantitative. Findings suggested that, i) Experts showed more fixation duration and fixation numbers in thinking activities, and the attention containing more AOIs was positively related to the rationality of the scheme. ii) Saccades with greater amplitude were more beneficial to the novelty of scheme. iii) Experts considered the information of each block in a balanced way, while novices tended to ignore unimportant blocks. These results will have a far-reaching impact on the development of designers’ thinking and help novices to exercise creative thinking and produce high-quality designs.

Assessing Deception in Questionnaire Surveys With Eye-Tracking

Deceit often occurs in questionnaire surveys, which leads to the misreporting of data and poor reliability. The purpose of this study is to explore whether eye-tracking could contribute to the detection of deception in questionnaire surveys, and whether the eye behaviors that appeared in instructed lying still exist in spontaneous lying. Two studies were conducted to explore eye movement behaviors in instructed and spontaneous lying conditions. The results showed that pupil size and fixation behaviors are both reliable indicators to detect lies in questionnaire surveys. Blink and saccade behaviors do not seem to predict deception. Deception resulted in increased pupil size, fixation count and duration. Meanwhile, respondents focused on different areas of the questionnaire when lying versus telling the truth. Furthermore, in the actual deception situation, the linear support vector machine (SVM) deception classifier achieved an accuracy of 74.09%. In sum, this study indicates the eye-tracking signatures of lying are not restricted to instructed deception, demonstrates the potential of using eye-tracking to detect deception in questionnaire surveys, and contributes to the questionnaire surveys of sensitive issues.

Visual predictions, neural oscillations and naïve physics

Prediction is a core function of the human visual system. Contemporary research suggests the brain builds predictive internal models of the world to facilitate interactions with our dynamic environment. Here, we wanted to examine the behavioural and neurological consequences of disrupting a core property of peoples’ internal models, using naturalistic stimuli. We had people view videos of basketball and asked them to track the moving ball and predict jump shot outcomes, all while we recorded eye movements and brain activity. To disrupt people’s predictive internal models, we inverted footage on half the trials, so dynamics were inconsistent with how movements should be shaped by gravity. When viewing upright videos people were better at predicting shot outcomes, at tracking the ball position, and they had enhanced alpha-band oscillatory activity in occipital brain regions. The advantage for predicting upright shot outcomes scaled with improvements in ball tracking and occipital alpha-band activity. Occipital alpha-band activity has been linked to selective attention and spatially-mapped inhibitions of visual brain activity. We propose that when people have a more accurate predictive model of the environment, they can more easily parse what is relevant, allowing them to better target irrelevant positions for suppression—resulting in both better predictive performance and in neural markers of inhibited information processing.

Exploring the Cognitive Load of Expert and Novice Map Users Using EEG and Eye Tracking

The main objective of this research is to explore the cognitive processes of expert and novice map users during the retrieval of map-related information, within varying difficulty levels (i.e., easy, moderate, hard), by using eye tracking and electroencephalogram (EEG). In this context, we present a spatial memory experiment consisting of a large number of stimuli to study the effect of task difficulty on map users’ behavior through cognitive load measurements. Next to the reaction time and success rate, we used fixation and saccade related eye tracking metrics (i.e., average fixation duration, the number of fixations per second, saccade amplitude and saccade velocity), and EEG power spectrum (i.e., event-related changes in alpha and theta frequency bands) to identify the cognitive load. While fixation metrics indicated no statistically significant difference between experts and novices, saccade metrics proved the otherwise. EEG power spectral density analysis, on the other side, suggested an increase in theta (i.e., event-related synchronization) and a decrease in alpha (except moderate tasks) (i.e., event-related desynchronization) at all difficulty levels of the task for both experts and novices, which is an indicator of cognitive load. Although no significant difference emerged between two groups, we found a significant difference in their overall performances when the participants were classified as good and relatively bad learners. Triangulating EEG results with the recorded eye tracking data and the qualitative analysis of focus maps indeed provided a detailed insight on the differences of the individuals’ cognitive processes during this spatial memory task.

What Went Wrong for Bad Solvers during Thematic Map Analysis? Lessons Learned from an Eye-Tracking Study

Thematic map analysis is a complex and challenging task that might result in map user failure for many reasons. In the study reported here, we wanted to search for differences between successful and unsuccessful map users, focusing—unlike many similar studies—on strategies applied by users who give incorrect answers. In the eye-tracking study, followed by a questionnaire survey, we collected data from 39 participants. The eye-tracking data were analyzed both qualitatively and quantitatively to compare participants’ strategies from various perspectives. Unlike the results of some other studies, it turned out that unsuccessful participants show some similarities that are consistent across most analyzed tasks. The main issues that characterize bad solvers relate to improper use of the thematic legend, the inability to focus on relevant map layout elements, as well as on adequate map content. Moreover, they differed in the general problem-solving approach used as they, for example, tended to choose fast, less cautious, strategies. Based on the collected results, we developed tips that could help prevent unsuccessful participants ending with an incorrect answer and therefore be beneficial in map use education.