The Elephant in the Road: Auditory Perceptual Load Affects Driver Perception and Awareness

The Effects of Auditory Working Memory Task on Situation Awareness in Complex Dynamic Environments: An Eye-movement Study

OBJECTIVE

This study investigated the effect of auditory working memory task on situation awareness (SA) and eye-movement patterns in complex dynamic environments.

BACKGROUND

Many human errors in aviation are caused by a lack of SA, and distraction from auditory secondary tasks is a serious threat to SA. However, it remains unclear how auditory working memory tasks affect SA and eye-movement patterns.

METHOD

Participants (n = 28) were randomly allocated to two groups and received different periods of visual search training (short versus long). They subsequently completed a situation awareness measurement task in three auditory secondary task conditions (without secondary task, auditory calculation task, and auditory 2-back task). Eye-movement data were collected during the situation awareness measurement task.

RESULTS

The auditory 2-back task significantly reduced overall SA, Level 1 SA, dwell times, and total percentage of fixation time on task-related areas of interests in the SA measurement task. Overall SA and Level 3 SA were not reduced by the auditory 2-back task in individuals in the longer visual search training time condition.

CONCLUSION

Auditory working memory load impairs SA in the perception and projection stage; however, greater experience can overcome impairment of SA in the projection stage.

APPLICATION

This study provided possible approaches to preventing loss of SA: (1) improving crew members' communication skills to ensure the accurate and clear transmission of information, reducing the difficulty of processing information, and (2) providing targeted cognitive training tailored to each pilot's level of experience.

Effects of Temporal Characteristics on Pilots Perceiving Audiovisual Warning Signals Under Different Perceptual Loads

Our research aimed to investigate the effectiveness of auditory, visual, and audiovisual warning signals for capturing the attention of the pilot, and how stimulus onset asynchronies (SOA) in audiovisual stimuli affect pilots perceiving the bimodal warning signals under different perceptual load conditions. In experiment 1 of the low perceptual load condition, participants discriminated the location (right vs. left) of visual targets preceded by five different types of warning signals. In experiment 2 of high perceptual load, participants completed the location task identical to a low load condition and a digit detection task in a rapid serial visual presentation (RSVP) stream. The main effect of warning signals in two experiments showed that visual and auditory cues presented simultaneously (AV) could effectively and efficiently arouse the attention of the pilots in high and low load conditions. Specifically, auditory (A), AV, and visual preceding auditory stimulus by 100 ms (VA100) increased the spatial orientation to a valid position in low load conditions. With the increase in visual perceptual load, auditory preceding the visual stimulus by 100 ms (AV100) and A warning signals had stronger spatial orientation. The results are expected to theoretically support the optimization design of the cockpit display interface, contributing to immediate flight crew awareness.

Affective sounds entering the peripersonal space influence the whole-body action preparation

The peripersonal space (PPS), the space surrounding us, is found to have enhanced multisensory-motor representation in the brain. In this study, we investigate how approaching sounds stopping at different distances within the peripersonal space, and carrying emotional content (positive, negative, and neutral), modulate the preparation of action as performing a Step. Premotor reaction times were measured by means of anticipatory forces and muscular activations to capture action preparation, the kinematics of stepping was considered for defining action performance, and for each stimulus, the individual perceived level of arousal and valence was evaluated. In general, we found a prompter premotor reaction for closer sounds compared to the farther ones and the fastest reactions detected for the neutral sound at each distance. We interpreted this time facilitation for neutral sound due to the large frequency spectrum of the stimuli and the absence of affective component and semantical content to decode. Interestingly, while at the close distance, none difference was found between positive and negative emotional stimuli, at the far distance faster reactions were present for negative compared to the positive sounds indicating that when arousal is less enhanced individuals are able to differentiate the emotional content of a sound. The kinematics observed after action initiation sustained the anticipatory results by showing that larger steps were performed when reacting to close compared to far sounds, being perceived as more arousing, and this happened particularly for neutral and negative sounds. Altogether, the results showed that action preparation is influenced by the vicinity and by the valence carried by looming auditory stimuli. For discriminating the stimuli valence, a certain distance, still within the PPS, is necessary; when instead stimuli are too close to the body valence discrimination is not performed.

Predicting Cognitive Load and Operational Performance in a Simulated Marksmanship Task

Modern operational environments can place significant demands on a service member's cognitive resources, increasing the risk of errors or mishaps due to overburden. The ability to monitor cognitive burden and associated performance within operational environments is critical to improving mission readiness. As a key step toward a field-ready system, we developed a simulated marksmanship scenario with an embedded working memory task in an immersive virtual reality environment. As participants performed the marksmanship task, they were instructed to remember numbered targets and recall the sequence of those targets at the end of the trial. Low and high cognitive load conditions were defined as the recall of three- and six-digit strings, respectively. Physiological and behavioral signals recorded included speech, heart rate, breathing rate, and body movement. These features were input into a random forest classifier that significantly discriminated between the low- and high-cognitive load conditions (AUC = 0.94). Behavioral features of gait were the most informative, followed by features of speech. We also showed the capability to predict performance on the digit recall (AUC = 0.71) and marksmanship (AUC = 0.58) tasks. The experimental framework can be leveraged in future studies to quantify the interaction of other types of stressors and their impact on operational cognitive and physical performance.

Effects of perceptual load on eyewitness memory are moderated by individual differences in cognitive ability

Previous research has identified perceptual load - the amount of perceptual processing required by a scene - as a potentially important factor in eyewitness memory. Here, we investigated whether effects of perceptual load on eyewitness memory for a simulated crime are moderated by individual differences in cognitive ability. We presented participants with a video of a simulated crime that imposed either high or low perceptual load, followed by a written narrative which contained a mixture of neutral and misleading descriptions of critical details. We examined the effects of three cognitive variables (working memory capacity, verbal cognitive ability and analytical reasoning) on participants' ability to accurately recall details of the video in both the misinformation and control conditions, and to detect a change in the clothing of a main character. General cognitive ability was associated with improved global memory for details of the event, and enhanced change detection under load. Greater WM capacity and a tendency towards analytical reasoning were both associated with resistance to misinformation under conditions of high perceptual load. We conclude that higher levels of cognitive ability, assessed across a number of domains, may enable eyewitnesses to withstand the effects of perceptual load and preserve the accuracy of their memories.

Attentional resources contribute to the perceptual learning of talker idiosyncrasies in audiovisual speech

To recognize audiovisual speech, listeners evaluate and combine information obtained from the auditory and visual modalities. Listeners also use information from one modality to adjust their phonetic categories to a talker's idiosyncrasy encountered in the other modality. In this study, we examined whether the outcome of this cross-modal recalibration relies on attentional resources. In a standard recalibration experiment in Experiment 1, participants heard an ambiguous sound, disambiguated by the accompanying visual speech as either /p/ or /t/. Participants' primary task was to attend to the audiovisual speech while either monitoring a tone sequence for a target tone or ignoring the tones. Listeners subsequently categorized the steps of an auditory /p/-/t/ continuum more often in line with their exposure. The aftereffect of phonetic recalibration was reduced, but not eliminated, by attentional load during exposure. In Experiment 2, participants saw an ambiguous visual speech gesture that was disambiguated auditorily as either /p/ or /t/. At test, listeners categorized the steps of a visual /p/-/t/ continuum more often in line with the prior exposure. Imposing load in the auditory modality during exposure did not reduce the aftereffect of this type of cross-modal phonetic recalibration. Together, these results suggest that auditory attentional resources are needed for the processing of auditory speech and/or for the shifting of auditory phonetic category boundaries. Listeners thus need to dedicate attentional resources in order to accommodate talker idiosyncrasies in audiovisual speech.

Inattentional Blindness During Driving in Younger and Older Adults

Age-related changes to perceptual and cognitive abilities have been implicated in an increased risk of collision in older adults. This may be due, in part, to their reduced ability to attend to potentially relevant aspects of their driving environment. An associated general phenomenon of inattentional blindness involves a failure to notice visually presented objects or events when attention is directed elsewhere. Previous studies of inattentional blindness using computer paradigms report higher incidence of this effect in older compared to younger adults. However, little is known about whether these age-related effects are observed during more complex, realistic, everyday tasks, such as driving. Therefore, the goal of this study was to explore whether younger and older adults differ in their awareness of objects in their driving environment when their attention is directed toward another primary driving task. This study took place in a high-fidelity, full field of view, driving simulator. Thirty-two younger (M age = 25.41) and 32 older (M age = 73.41) adults drove through 19 short scenarios and were asked to first judge whether their vehicle could fit between two rows of vehicles parked on either side of the road and then to perform the associated driving maneuver (i.e., drive through or drive around). On four critical trials, objects were placed on the side of the road that differed in terms of animacy. Specifically, animate objects consisted of 3D humans standing by a bus shelter and inanimate objects consisted of photographs of the same individuals on a bus shelter advertisement. Inattentional blindness was measured via a post-drive, tablet-based recognition task immediately following the critical trials. Results revealed high rates of inattentional blindness across both age groups, with significantly lower levels of awareness for inanimate objects compared to animate objects. Further, whereas younger adults demonstrated reduced inattentional blindness following the first critical trial, older adults did not show this immediate improvement in recognition performance. Overall, this study provides unique insights into the factors associated with age-related changes to attention and how they may affect important driving-related outcomes.