Assessing how visual search entropy and engagement predict performance in a multiple-objects tracking air traffic control task

Evidence of elevated situational awareness for active duty soldiers during navigation of a virtual environment

U.S. service members maintain constant situational awareness (SA) due to training and experience operating in dynamic and complex environments. Work examining how military experience impacts SA during visual search of a complex naturalistic environment, is limited. Here, we compare Active Duty service members and Civilians' physiological behavior during a navigational visual search task in an open-world virtual environment (VE) while cognitive load was manipulated. We measured eye-tracking and electroencephalogram (EEG) outcomes from Active Duty (N = 21) and Civilians (N = 15) while they navigated a desktop VE at a self-regulated pace. Participants searched and counted targets (N = 15) presented among distractors, while cognitive load was manipulated with an auditory Math Task. Results showed Active Duty participants reported significantly greater/closer to the correct number of targets compared to Civilians. Overall, Active Duty participants scanned the VE with faster peak saccade velocities and greater average saccade magnitudes compared to Civilians. Convolutional Neural Network (CNN) response (EEG P-300) was significantly weighted more to initial fixations for the Active Duty group, showing reduced attentional resources on object refixations compared to Civilians. There were no group differences in fixation outcomes or overall CNN response when comparing targets versus distractor objects. When cognitive load was manipulated, only Civilians significantly decreased their average dwell time on each object and the Active Duty group had significantly fewer numbers of correct answers on the Math Task. Overall, the Active Duty group explored the VE with increased scanning speed and distance and reduced cognitive re-processing on objects, employing a different, perhaps expert, visual search strategy indicative of increased SA. The Active Duty group maintained SA in the main visual search task and did not appear to shift focus to the secondary Math Task. Future work could compare how a stress inducing environment impacts these groups' physiological or cognitive markers and performance for these groups.

Neuroadaptive LBS: towards human-, context-, and task-adaptive mobile geographic information displays to support spatial learning for pedestrian navigation

Well-designed, neuroadaptive mobile geographic information displays (namGIDs) could improve the lives of millions of mobile citizens of the mostly urban information society who daily need to make time critical and societally relevant decisions while navigating. What are the basic perceptual and neurocognitive processes with which individuals make movement decisions when guided by human- and context-adaptive namGIDs? How can we study this in an ecologically valid way, also outside of the highly controlled laboratory? We report first ideas and results from our unique neuroadaptive research agenda that brings us closer to answering this fundamental empirical question. We present our first implemented methodological solutions of novel ambulatory evaluation methods to study and improve Location-based System (LBS) displays, by critical examination of how perceptual, neurocognitive, psychophysiological, and display design factors might influence decision-making and spatial learning in pedestrian mobility across broad ranges of users and mobility contexts.

Air traffic control forgetting prediction based on eye movement information and hybrid neural network

Control forgetting accounts for most of the current unsafe incidents. In the research field of radar surveillance control, how to avoid control forgetting to ensure the safety of flights is becoming a hot issue which attracts more and more attention. Meanwhile, aviation safety is substantially influenced by the way of eye movement. The exact relation of control forgetting with eye movement, however, still remains puzzling. Motivated by this, a control forgetting prediction method is proposed based on the combination of Convolutional Neural Networks and Long-Short Term Memory (CNN-LSTM). In this model, the eye movement characteristics are classified in terms of whether they are time-related, and then regulatory forgetting can be predicted by virtue of CNN-LSTM. The effectiveness of the method is verified by carrying out simulation experiments of eye movement during flight control. Results show that the prediction accuracy of this method is up to 79.2%, which is substantially higher than that of Binary Logistic Regression, CNN and LSTM (71.3%, 74.6%, and 75.1% respectively). This work tries to explore an innovative way to associate control forgetting with eye movement, so as to guarantee the safety of civil aviation.

Multimodal Augmented Reality Applications for Training of Traffic Procedures in Aviation

Mid-air collision is one of the top safety risks in general aviation. This study addresses the description and experimental assessment of multimodal Augmented Reality (AR) applications for training of traffic procedures in accordance with Visual Flight Rules (VFR). AR has the potential to complement the conventional flight instruction by bridging the gap between theory and practice, and by releasing students’ time and performance pressure associated with a limited simulator time. However, it is critical to assess the impact of AR in the specific domain and to identify any potential negative learning transfer. Multimodal AR applications were developed to address various areas of training: guidance and feedback for the correct scanning pattern, estimation if an encountering traffic is on collision course and application of the relevant rules. The AR applications also provided performance feedback for collision detection, avoidance and priority decisions. The experimental assessment was conducted with 59 trainees (28 women, 31 men) assigned to an experimental group (AR training) and a control group (simulator training). The results of tests without AR in the flight simulator show that the group that trained with AR obtained similar levels of performance like the control group. There was no negative training effect of AR on trainees’ performance, workload, situational awareness, emotion nor motivation. After training the tasks were perceived as less challenging, the accuracy of collision detection has improved, and the trainees reported less intense negative emotions and fear of failure. Furthermore, a scanning pattern test in AR showed that the AR training group performed the scanning pattern significantly better than the control group. In addition, there was a significant gender effect on emotion, motivation and preferences for AR features, but not on performance. Women liked the voice interaction with AR and the compass hologram more than men. Men liked the traffic holograms and the AR projection field more than women. These results are important because they provide experimental evidence for the benefits of multimodal AR applications that could be used complementary to the flight simulator training.