An Attentive Blank Stare Under Simulator-induced Spatial Disorientation Events

OBJECTIVE

This study investigated the effect of the spatial disorientation (SD) events on an attentive blank stare in the cockpit scene and demonstrated how much the flight task and visual delayed discrimination task were competing for the pilots' attention.

BACKGROUND

SD in flight is the leading cause of human error-related aircraft accidents in the military, general and commercial aviation, and has been an unsolved problem since the inception of flight. In-flight safety research, visually scanning cockpit instruments, and detecting changes are critical countermeasures against SD.

METHOD

Thirty male military pilots were performing a dual task involving piloting a flight simulator and visual change detection, while eye movements were obtained using an eye tracker.

RESULTS

Pilots made more flight errors and spent less time gazing at the area of change in SD-conflict than in non-conflict flights. The vestibular origin SD-conflict led not only to deteriorated piloting and visual scanning but also to problems coordinating overt and covert attention, resulting in lower noticeability of visual changes despite gazing at them.

CONCLUSION

Our study shows that looking at a given area in space is not a sufficient condition for effective covert attention allocation and the correct response to a visual stimulus. It seems to be important to make pilots aware of this during SD training.

APPLICATION

To reduce change blindness, some strategies, such as reducing the number of secondary tasks is extremely valuable. Particular efforts should also be focused on improving the design of the aircraft cockpit by increasing the conspicuousness of critical information.

A Midbrain Inspired Recurrent Neural Network Model for Robust Change Detection

We present a biologically inspired recurrent neural network (RNN) that efficiently detects changes in natural images. The model features sparse, topographic connectivity (st-RNN), closely modeled on the circuit architecture of a "midbrain attention network." We deployed the st-RNN in a challenging change blindness task, in which changes must be detected in a discontinuous sequence of images. Compared with a conventional RNN, the st-RNN learned 9x faster and achieved state-of-the-art performance with 15x fewer connections. An analysis of low-dimensional dynamics revealed putative circuit mechanisms, including a critical role for a global inhibitory (GI) motif, for successful change detection. The model reproduced key experimental phenomena, including midbrain neurons' sensitivity to dynamic stimuli, neural signatures of stimulus competition, as well as hallmark behavioral effects of midbrain microstimulation. Finally, the model accurately predicted human gaze fixations in a change blindness experiment, surpassing state-of-the-art saliency-based methods. The st-RNN provides a novel deep learning model for linking neural computations underlying change detection with psychophysical mechanisms.SIGNIFICANCE STATEMENT For adaptive survival, our brains must be able to accurately and rapidly detect changing aspects of our visual world. We present a novel deep learning model, a sparse, topographic recurrent neural network (st-RNN), that mimics the neuroanatomy of an evolutionarily conserved "midbrain attention network." The st-RNN achieved robust change detection in challenging change blindness tasks, outperforming conventional RNN architectures. The model also reproduced hallmark experimental phenomena, both neural and behavioral, reported in seminal midbrain studies. Lastly, the st-RNN outperformed state-of-the-art models at predicting human gaze fixations in a laboratory change blindness experiment. Our deep learning model may provide important clues about key mechanisms by which the brain efficiently detects changes.

Neurally-constrained modeling of human gaze strategies in a change blindness task

Despite possessing the capacity for selective attention, we often fail to notice the obvious. We investigated participants’ (n = 39) failures to detect salient changes in a change blindness experiment. Surprisingly, change detection success varied by over two-fold across participants. These variations could not be readily explained by differences in scan paths or fixated visual features. Yet, two simple gaze metrics–mean duration of fixations and the variance of saccade amplitudes–systematically predicted change detection success. We explored the mechanistic underpinnings of these results with a neurally-constrained model based on the Bayesian framework of sequential probability ratio testing, with a posterior odds-ratio rule for shifting gaze. The model’s gaze strategies and success rates closely mimicked human data. Moreover, the model outperformed a state-of-the-art deep neural network (DeepGaze II) with predicting human gaze patterns in this change blindness task. Our mechanistic model reveals putative rational observer search strategies for change detection during change blindness, with critical real-world implications.

Our brain has the remarkable capacity to pay attention, selectively, to important objects in the world around us. Yet, sometimes, we fail spectacularly to notice even the most salient events. We tested this phenomenon in the laboratory with a change-blindness experiment, by having participants freely scan and detect changes across discontinuous image pairs. Participants varied widely in their ability to detect these changes. Surprisingly, two low-level gaze metrics—fixation durations and saccade amplitudes—strongly predicted success in this task. We present a novel, computational model of eye movements, incorporating neural constraints on stimulus encoding, that links these gaze metrics with change detection success. Our model is relevant for a mechanistic understanding of human gaze strategies in dynamic visual environments.

Disentangling presentation and processing times in the brain

Visual object recognition seems to occur almost instantaneously. However, not only does it require hundreds of milliseconds of processing, but our eyes also typically fixate the object for hundreds of milliseconds. Consequently, information reaching our eyes at different moments is processed in the brain together. Moreover, information received at different moments during fixation is likely to be processed differently, notably because different features might be selectively attended at different moments. Here, we introduce a novel reverse correlation paradigm that allows us to uncover with millisecond precision the processing time course of specific information received on the retina at specific moments. Using faces as stimuli, we observed that processing at several electrodes and latencies was different depending on the moment at which information was received. Some of these variations were caused by a disruption occurring 160-200 ​ms after the face onset, suggesting a role of the N170 ERP component in gating information processing; others hinted at temporal compression and integration mechanisms. Importantly, the observed differences were not explained by simple adaptation or repetition priming, they were modulated by the task, and they were correlated with differences in behavior. These results suggest that top-down routines of information sampling are applied to the continuous visual input, even within a single eye fixation.

Dimension-based attention in visual short-term memory

We investigated how dimension-based attention influences visual short-term memory (VSTM). This was done through examining the effects of cueing a feature dimension in two perceptual comparison tasks (change detection and sameness detection). In both tasks, a memory array and a test array consisting of a number of colored shapes were presented successively, interleaved by a blank interstimulus interval (ISI). In Experiment 1 (change detection), the critical event was a feature change in one item across the memory and test arrays. In Experiment 2 (sameness detection), the critical event was the absence of a feature change in one item across the two arrays. Auditory cues indicated the feature dimension (color or shape) of the critical event with 80 % validity; the cues were presented either prior to the memory array, during the ISI, or simultaneously with the test array. In Experiment 1, the cue validity influenced sensitivity only when the cue was given at the earliest position; in Experiment 2, the cue validity influenced sensitivity at all three cue positions. We attributed the greater effectiveness of top-down guidance by cues in the sameness detection task to the more active nature of the comparison process required to detect sameness events (Hyun, Woodman, Vogel, Hollingworth, & Luck, Journal of Experimental Psychology: Human Perception and Performance, 35; 1140-1160, 2009).

Assessing attentional prioritization of front-of-pack nutrition labels using change detection

We used a change detection method to evaluate attentional prioritization of nutrition information that appears in the traditional "Nutrition Facts Panel" and in front-of-pack nutrition labels. Results provide compelling evidence that front-of-pack labels attract attention more readily than the Nutrition Facts Panel, even when participants are not specifically tasked with searching for nutrition information. Further, color-coding the relative nutritional value of key nutrients within the front-of-pack label resulted in increased attentional prioritization of nutrition information, but coding using facial icons did not significantly increase attention to the label. Finally, the general pattern of attentional prioritization across front-of-pack designs was consistent across a diverse sample of participants. Our results indicate that color-coded, front-of-pack nutrition labels increase attention to the nutrition information of packaged food, a finding that has implications for current policy discussions regarding labeling change.

Tallman lettering as a strategy for differentiation in look-alike, sound-alike drug names: the role of familiarity in differentiating drug doppelgangers

Tallman lettering, capitalizing the dissimilar portions of easily confused drug names, is one strategy for reducing medication errors. We assessed the efficacy of Tallman lettering in a visually complex environment using a change detection method with healthcare providers and laypeople. In addition, the effect of familiarity with the drug name was assessed using a subset of responses collected from healthcare providers. Both healthcare providers and laypeople detected changes in confusable pairs of drug names more often (P < 0.0001) and more quickly (P < 0.05) when changes were presented in Tallman lettering, though the benefits were more pronounced for healthcare providers (p < 0.05). Familiarity with both drug names in a confusable pair mitigated the benefit of Tallman lettering. Results are discussed in terms of bottom-up and top-down attentional systems for processing of information in the context of the varied healthcare environments.

Simon effects in change detection and change blindness

Responses to centrally presented target stimuli are faster when they are accompanied by a task-irrelevant lateral accessory stimulus that corresponds spatially with the response hand (accessory variant of the Simon effect). In four experiments, we tested whether this effect depends on the awareness of the accessory stimulus. In a change blindness task, participants were asked to respond to a central letter that was accompanied by a lateral background change on some trials. Change blindness describes the phenomenon that even large changes may remain unnoticed when they occur simultaneously with another visual disruption, e.g., a blank screen. In a series of four experiments, a significant Simon effect was observed both when the accessory stimulus reached awareness and when it remained unnoticed. These results indicate that, based on the spatial location of an accessory stimulus, a spatial code is generated. This code interferes with the response code on the response-selection stage.

Task probability and report of feature information: what you know about what you 'see' depends on what you expect to need

We investigated the influence of dimensional set on report of object feature information using an immediate memory probe task. Participants viewed displays containing up to 36 coloured geometric shapes which were presented for several hundred milliseconds before one item was abruptly occluded by a probe. A cue presented simultaneously with the probe instructed participants to report either about the colour or shape of the probe item. A dimensional set towards the colour or shape of the presented items was induced by manipulating task probability - the relative probability with which the two feature dimensions required report. This was done across two participant groups: One group was given trials where there was a higher report probability of colour, the other a higher report probability of shape. Two experiments showed that features were reported most accurately when they were of high task probability, though in both cases the effect was largely driven by the colour dimension. Importantly the task probability effect did not interact with display set size. This is interpreted as tentative evidence that this manipulation influences feature processing in a global manner and at a stage prior to visual short term memory.

Nonexplicit change detection in complex dynamic settings: what eye movements reveal

OBJECTIVE

We employed a computer-controlled command-and-control (C2) simulation and recorded eye movements to examine the extent and nature of the inability to detect critical changes in dynamic displays when change detection is implicit (i.e., requires no explicit report) to the operator's task.

BACKGROUND

Change blindness-the failure to notice significant changes to a visual scene-may have dire consequences on performance in C2 and surveillance operations.

METHOD

Participants performed a radar-based risk-assessment task involving multiple subtasks. Although participants were not required to explicitly report critical changes to the operational display, change detection was critical in informing decision making. Participants' eye movements were used as an index of visual attention across the display.

RESULTS

Nonfixated (i.e., unattended) changes were more likely to be missed than were fixated (i.e., attended) changes, supporting the idea that focused attention is necessary for conscious change detection. The finding of significant pupil dilation for changes undetected but fixated suggests that attended changes can nonetheless be missed because of a failure of attentional processes.

CONCLUSION

Change blindness in complex dynamic displays takes the form of failures in establishing task-appropriate patterns of attentional allocation.

APPLICATION

These findings have implications in the design of change-detection support tools for dynamic displays and work procedure in C2 and surveillance.

Misdirection, attention and awareness: Inattentional blindness reveals temporal relationship between eye movements and visual awareness

We designed a magic trick that could be used to investigate how misdirection can prevent people from perceiving a visually salient event, thus offering a novel paradigm to examine inattentional blindness. We demonstrate that participants’ verbal reports reflect what they saw rather than inferences about how they thought the trick was done and thus provide a reliable index of conscious perception. Eye movements revealed that for a subset of participants their conscious perception was not related to where they were looking at the time of the event and thus demonstrate how overt and covert attention can be spatially dissociated. However, detection of the event resulted in rapid shifts of eye movements towards the detected event, thus indicating a strong temporal link between overt and covert attention, and that covert attention can be allocated at least 2 or 3 saccade targets ahead of where people are fixating.

The scope of social attention deficits in autism: prioritized orienting to people and animals in static natural scenes

A central feature of autism spectrum disorder (ASD) is an impairment in 'social attention'--the prioritized processing of socially relevant information, e.g. the eyes and face. Socially relevant stimuli are also preferentially attended in a broader categorical sense, however: observers orient preferentially to people and animals (compared to inanimate objects) in complex natural scenes. To measure the scope of social attention deficits in autism, observers viewed alternating versions of a natural scene on each trial, and had to 'spot the difference' between them--where the difference involved either an animate or inanimate object. Change detection performance was measured as an index of attentional prioritization. Individuals with ASD showed the same prioritized social attention for animate categories as did control participants. This could not be explained by lower level visual factors, since the effects disappeared when using blurred or inverted images. These results suggest that social attention - and its impairment in autism - may not be a unitary phenomenon: impairments in visual processing of specific social cues may occur despite intact categorical prioritization of social agents.

Attentional filtering of transients allows for a recovery from change blindness

Previous reports suggest that introducing distracting visual transients during a change-detection task can result in change blindness. In four experiments, we found that presenting the distracting transients repeatedly prior to the change produces a recovery from change blindness. This recovery from change blindness is not due solely to low-level neural adaptation of transient detectors, but instead seems to be based on attentional filtering of the distracting transient signals. This attentional filtering can be object-based rather than location-based. In addition, we found that the ability to achieve this attentional filtering depends critically on presenting the to-be-ignored transient signals prior to the time of the change.