Uncovering expertise-related differences in troubleshooting performance: combining eye movement and concurrent verbal protocol data

Ambient and focal attention during complex problem-solving: preliminary evidence from real-world eye movement data

Time course analysis of eye movements during free exploration of real-world scenes often reveals an increase in fixation durations together with a decrease in saccade amplitudes, which has been explained within the two visual systems approach, i.e., a transition from ambient to focal. Short fixations and long saccades during early viewing periods are classified as ambient mode of vision, which is concerned with spatial orientation and is related to simple visual properties such as motion, contrast, and location. Longer fixations and shorter saccades during later viewing periods are classified as focal mode of vision, which is concentrated in the foveal projection and is capable of object identification and its semantic categorization. While these findings are mainly obtained in the context of image exploration, the present study endeavors to investigate whether the same pattern of interplay between ambient and focal visual attention is deployed when people work on complex real-world tasks-and if so, when? Based on a re-analysis of existing data that integrates concurrent think aloud and eye tracking protocols, the present study correlated participants' internal thinking models to the parameters of their eye movements when they planned solutions to an open-ended design problem in a real-world setting. We hypothesize that switching between ambient and focal attentional processing is useful when solvers encounter difficulty compelling them to shift their conceptual direction to adjust the solution path. Individuals may prefer different attentional strategies for information-seeking behavior, such as ambient-to-focal or focal-to-ambient. The observed increase in fixation durations and decrease in saccade amplitudes during the periods around shifts in conceptual direction lends support to the postulation of the ambient-to-focal processing; however, focal-to-ambient processing is not evident. Furthermore, our data demonstrate that the beginning of a shift in conceptual direction is observable in eye movement behavior with a significant prolongation of fixation. Our findings add to the conclusions drawn from laboratory settings by providing preliminary evidence for ambient and focal processing characteristics in real-world problem-solving.

Impacts of Cues on Learning and Attention in Immersive 360-Degree Video: An Eye-Tracking Study

Immersive 360-degree video has become a new learning resource because of its immersive sensory experience. This study examined the effects of textual and visual cues on learning and attention in immersive 360-degree video by using eye-tracking equipment integrated in a virtual reality head-mounted display. Participants (n = 110) were randomly assigned to one of four conditions: (1) no cues, (2) textual cues in the initial field of view (FOV), (3) textual cues outside the initial FOV, and (4) textual cues outside the initial FOV + visual cues. The results showed that the cues (annotations or annotations + arrows) helped learners achieve better learning outcomes and spend more time focusing on the areas with cues. In addition, the study found a serious imbalance in the distribution of learners' attention in each region of the video. The attention directed to textual cues in the initial FOV is much higher than the attention directed to textual cues outside the initial FOV. Adding visual cues can effectively direct attention to textual cues outside the initial FOV and alleviate the imbalance of attention distribution. Consequently, adding cues to immersive 360-degree video can be an appropriate approach to promote learning and guide attention in immersive 360-degree video learning environments. This study provided new insights into the design and development of immersive 360-degree video instructional resources.

The Effects of Language Barriers and Time Constraints on Online Learning Performance: An Eye-Tracking Study

OBJECTIVE

The goal of this study is to model the effect of language use and time pressure on English as a first language (EFL) and English as a second language (ESL) students by measuring their eye movements in an on-screen, self-directed learning environment.

BACKGROUND

Online learning is becoming integrated into learners' daily lives due to the flexibility in scheduling and location that it offers. However, in many cases, the online learners often have no interaction with one another or their instructors, making it difficult to determine how the learners are reading the materials and whether they are learning effectively. Furthermore, online learning may pose challenges to those who face language barriers or are under time pressure.

METHOD

The effects of two factors, language use (EFL vs. ESL) and time constraints (high vs. low time pressure), were investigated during the presentation of online materials. The effects were analyzed based on eye movement measures (eye fixation rate-the total number of eye fixations divided by the task duration and gaze entropy) and behavioral measures (correct rate and task completion time).

RESULTS

The results show that the ESL students had higher eye fixation rates and longer task completion times than the EFL students. Moreover, high time pressure resulted in high fixation rates, short task completion time, low correct rates, and high gaze entropy.

CONCLUSION AND APPLICATION

The results suggest the possibility of using unobtrusive eye movement measures to develop ways to better assist those who struggle with learning in the online environment.

Synchronized eye movements predict test scores in online video education

Experienced teachers pay close attention to their students, adjusting their teaching when students seem lost. This dynamic interaction is missing in online education. We hypothesized that attentive students follow videos similarly with their eyes. Thus, attention to instructional videos could be assessed remotely by tracking eye movements. Here we show that intersubject correlation of eye movements during video presentation is substantially higher for attentive students and that synchronized eye movements are predictive of individual test scores on the material presented in the video. These findings replicate for videos in a variety of production styles, for incidental and intentional learning and for recall and comprehension questions alike. We reproduce the result using standard web cameras to capture eye movements in a classroom setting and with over 1,000 participants at home without the need to transmit user data. Our results suggest that online education could be made adaptive to a student's level of attention in real time.

Airport Security Screener Problem-Solving Knowledge and Implications

OBJECTIVE

This research investigates security screeners' knowledge and the effect that differences in knowledge have on the performance of problem-solving activities. We argue that the development of problem-solving knowledge enables security screeners to perform effective problem-solving activity, which assists search and decision-making processes.

BACKGROUND

Airport security screening research has investigated the many variables that affect security screeners' search and decision making during simulated threat-detection tasks. Although search and decision making are essential aspects of security screening, few studies have investigated the problem-solving knowledge and activities that support security screening task performance.

METHOD

Sixteen more-experienced and 24 less-experienced security screeners were observed as they performed x-ray screening in the field at an Australian international airport's departure security checkpoint. Participants wore eye-tracking glasses and delivered concurrent verbal protocol.

RESULTS

When interacting with other security screeners, more-experienced screeners demonstrated situational knowledge more than less-experienced screeners, whereas less-experienced screeners experienced more insufficient knowledge. Lag-sequential analysis using combined data from both screener groups showed that situational knowledge facilitated effective problem-solving activity to support search and decision making. Insufficient knowledge led screeners to seek assistance and defer decision making.

CONCLUSION

This study expands current understandings of airport security screening. It demonstrates that security screeners develop knowledge that is specific to problem solving. This knowledge assists effective problem-solving activity to support search and decision making, and to mitigate uncertainty during the x-ray screening task.

APPLICATION

Findings can inform future security screening processes, screener training, and technology support tools. Furthermore, findings are potentially transferable to other domains.

Sculptors, Architects, and Painters Conceive of Depicted Spaces Differently

Sculptors, architects, and painters are three professional groups that require a comprehensive understanding of how to manipulate spatial structures. While it has been speculated that they may differ in the way they conceive of space due to the different professional demands, this has not been empirically tested. To achieve this, we asked architects, painters, sculptors, and a control group questions about spatially complex pictures. Verbalizations elicited were examined using cognitive discourse analysis. We found significant differences between each group. Only painters shifted consistently between 2D and 3D concepts, architects were concerned with paths and spatial physical boundedness, and sculptors produced responses that fell between architects and painters. All three differed from controls, whose verbalizations were generally less elaborate and detailed. Thus, for the case of sculptors, architects, and painters, profession appears to relate to a different spatial conceptualization manifested through a systematically contrasting way of talking about space.

video abstract: https://youtu.be/w7dsVjWRitI

Touchscreen Tablets: Coordinating Action and Perception for Mathematical Cognition

Proportional reasoning is important and yet difficult for many students, who often use additive strategies, where multiplicative strategies are better suited. In our research we explore the potential of an interactive touchscreen tablet application to promote proportional reasoning by creating conditions that steer students toward multiplicative strategies. The design of this application (Mathematical Imagery Trainer) was inspired by arguments from embodied-cognition theory that mathematical understanding is grounded in sensorimotor schemes. This study draws on a corpus of previously treated data of 9-11 year-old students, who participated individually in semi-structured clinical interviews, in which they solved a manipulation task that required moving two vertical bars at a constant ratio of heights (1:2). Qualitative analyses revealed the frequent emergence of visual attention to the screen location halfway along the bar that was twice as high as the short bar. The hypothesis arose that students used so-called "attentional anchors" (AAs)-psychological constructions of new perceptual structures in the environment that people invent spontaneously as their heuristic means of guiding effective manual actions for managing an otherwise overwhelming task, in this case keeping vertical bars at the same proportion while moving them. We assumed that students' AAs on the mathematically relevant points were crucial in progressing from additive to multiplicative strategies. Here we seek farther to promote this line of research by reanalyzing data from 38 students (aged 9-11). We ask: (1) What quantitative evidence is there for the emergence of AAs?; and (2) How does the transition from additive to multiplicative reasoning take place when solving embodied proportions tasks in interaction with the touchscreen tablet app? We found that: (a) AAs appeared for all students; (b) the AA-types were few across the students; (c) the AAs were mathematically relevant (top of the bars and halfway along the tall bar); (d) interacting with the tablet was crucial for the AAs' emergence; and (e) the vast majority of students progressed from additive to multiplicative strategies (as corroborated with oral utterances). We conclude that touchscreen applications have the potential to create interaction conditions for coordinating action and perception into mathematical cognition.

Combining principles of Cognitive Load Theory and diagnostic error analysis for designing job aids: Effects on motivation and diagnostic performance in a process control task

Two studies are presented in which the design of a procedural aid and the impact of an additional decision aid for process control were assessed. In Study 1, a procedural aid was developed that avoids imposing unnecessary extraneous cognitive load on novices when controlling a complex technical system. This newly designed procedural aid positively affected germane load, attention, satisfaction, motivation, knowledge acquisition and diagnostic speed for novel faults. In Study 2, the effect of a decision aid for use before the procedural aid was investigated, which was developed based on an analysis of diagnostic errors committed in Study 1. Results showed that novices were able to diagnose both novel faults and practised faults, and were even faster at diagnosing novel faults. This research contributes to the question of how to optimally support novices in dealing with technical faults in process control.

Sherlock Holmes - an expert's view of expertise

In recent years, there has been an intense research effort to understand the cognitive processes and structures underlying expert behaviour. Work in different fields, including scientific domains, sports, games and mnemonics, has shown that there are vast differences in perceptual abilities between experts and novices, and that these differences may underpin other cognitive differences in learning, memory and problem solving. In this article, we evaluate the progress made in the last years through the eyes of an outstanding, albeit fictional, expert: Sherlock Holmes. We first use the Sherlock Holmes character to illustrate expert processes as described by current research and theories. In particular, the role of perception, as well as the nature and influence of expert knowledge, are all present in the description of Conan Doyle's hero. In the second part of the article, we discuss a number of issues that current research on expertise has barely addressed. These gaps include, for example, several forms of reasoning, the influence of emotions on cognition, and the effect of age on experts' knowledge and cognitive processes. Thus, although nearly 120-year-old, Conan Doyle's books show remarkable illustrations of expert behaviour, including the coverage of themes that have mostly been overlooked by current research.

Principles for Designing Effective and Efficient Training of Complex Cognitive Skills

This chapter describes principles for the design of training of complex cognitive tasks that pose relatively great challenges to the cognitive capacity of the trainee. Using the theoretical framework of cognitive load, we argue that the structures that constitute human cognitive architecture need to be taken into account in the design of effective and efficient training. For training complex cognitive tasks, this means that cognitive load needs to be controlled by optimizing the relationship between (a) the intrinsic load imposed by the complexity of the training task and (b) the effective (germane) and ineffective (extraneous) loads imposed by the design of the training tasks. Design principles for training tasks, for sequences of training tasks in fixed training programs (in which all trainees receive the same sequence), and for ways to create adaptive or personalized training programs (in which each trainee receives a personalized sequence) are discussed. Simple to complex sequencing of training tasks, decreasing support, increasing contextual interference, and spacing are discussed as effective principles for fostering learning and transfer in fixed training programs. System-controlled, shared-responsibility, and advisory models for task selection are discussed as possible models to be used to foster learning and transfer in adaptive training programs.

Uncovering the problem-solving process: cued retrospective reporting versus concurrent and retrospective reporting

This study investigated the amounts of problem-solving process information ("action," "why," "how," and "metacognitive") elicited by means of concurrent, retrospective, and cued retrospective reporting. In a within-participants design, 26 participants completed electrical circuit troubleshooting tasks under different reporting conditions. The method of cued retrospective reporting used the original computer-based task and a superimposed record of the participant's eye fixations and mouse-keyboard operations as a cue for retrospection. Cued retrospective reporting (with the exception of why information) and concurrent reporting (with the exception of metacognitive information) resulted in a higher number of codes on the different types of information than did retrospective reporting.

Head-mounted versus remote eye tracking of radiologists searching for breast cancer: a comparison

PURPOSE

We compared performance and visual search parameters of radiologists detecting masses on mammograms by using both a head-mounted (HDMT) and a remote (REM) eye tracker.

MATERIALS AND METHODS

Five experienced radiologists read twice a case set of 20 one-view (medial-lateral oblique) mammograms, of which 12 contained a malignant mass and eight were lesion-free. For each observer, one trial used an HDMT eye-tracking system and the other used an REM system. Trials were separated on average by 2 months. Time to hit the location of the mass, dwell, and number of fixations in the location of the mass were measured. The same parameters were measured on a per-trial basis to determine whether there were memory effects from the previous trial.

RESULTS

Dwell times in the location of true-positive, false-positive, and false-negative results were significantly shorter (P < .05) using the HDMT (median, 0.395 seconds) than REM (median, 0.482 seconds) systems, but the number of fixations in the location of the response was smaller using the REM system (median, 4.33 versus 5.0 for the HDMT). The observed differences did not seem to be caused by a memory effect. In addition, the relative lack of head mobility using the REM system caused observers to report neck strain.

CONCLUSION

Overall, radiologists' visual search behavior was very similar using both types of eye-tracking device. However, because the REM system did not contain a magnetic head tracker, radiologists were allowed very limited head movements when using it, which made them uncomfortable during the experiment.