Assessing the Automaticity of the Exogenous Orienting of Tactile Attention

Establishing gaze markers of perceptual load during multi-target visual search

Highly-automated technologies are increasingly incorporated into existing systems, for instance in advanced car models. Although highly automated modes permit non-driving activities (e.g. internet browsing), drivers are expected to reassume control upon a 'take over' signal from the automation. To assess a person's readiness for takeover, non-invasive eye tracking can indicate their attentive state based on properties of their gaze. Perceptual load is a well-established determinant of attention and perception, however, the effects of perceptual load on a person's ability to respond to a takeover signal and the related gaze indicators are not yet known. Here we examined how load-induced attentional state affects detection of a takeover-signal proxy, as well as the gaze properties that change with attentional state, in an ongoing task with no overt behaviour beyond eye movements (responding by lingering the gaze). Participants performed a multi-target visual search of either low perceptual load (shape targets) or high perceptual load (targets were two separate conjunctions of colour and shape), while also detecting occasional auditory tones (the proxy takeover signal). Across two experiments, we found that high perceptual load was associated with poorer search performance, slower detection of cross-modal stimuli, and longer fixation durations, while saccade amplitude did not consistently change with load. Using machine learning, we were able to predict the load condition from fixation duration alone. These results suggest monitoring fixation duration may be useful in the design of systems to track users' attentional states and predict impaired user responses to stimuli outside of the focus of attention.

Are auditory cues special? Evidence from cross-modal distractor-induced blindness

A target that shares features with preceding distractor stimuli is less likely to be detected due to a distractor-driven activation of a negative attentional set. This transient impairment in perceiving the target (distractor-induced blindness/deafness) can be found within vision and audition. Recently, the phenomenon was observed in a cross-modal setting involving an auditory target and additional task-relevant visual information (cross-modal distractor-induced deafness). In the current study, consisting of three behavioral experiments, a visual target, indicated by an auditory cue, had to be detected despite the presence of visual distractors. Multiple distractors consistently led to reduced target detection if cue and target appeared in close temporal proximity, confirming cross-modal distractor-induced blindness. However, the effect on target detection was reduced compared to the effect of cross-modal distractor-induced deafness previously observed for reversed modalities. The physical features defining cue and target could not account for the diminished distractor effect in the current cross-modal task. Instead, this finding may be attributed to the auditory cue acting as an especially efficient release signal of the distractor-induced inhibition. Additionally, a multisensory enhancement of visual target detection by the concurrent auditory signal might have contributed to the reduced distractor effect.

Digital haptics improve speed of visual search performance in a dual-task setting

Dashboard-mounted touchscreen tablets are now common in vehicles. Screen/phone use in cars likely shifts drivers’ attention away from the road and contributes to risk of accidents. Nevertheless, vision is subject to multisensory influences from other senses. Haptics may help maintain or even increase visual attention to the road, while still allowing for reliable dashboard control. Here, we provide a proof-of-concept for the effectiveness of digital haptic technologies (hereafter digital haptics), which use ultrasonic vibrations on a tablet screen to render haptic perceptions. Healthy human participants (N = 25) completed a divided-attention paradigm. The primary task was a centrally-presented visual conjunction search task, and the secondary task entailed control of laterally-presented sliders on the tablet. Sliders were presented visually, haptically, or visuo-haptically and were vertical, horizontal or circular. We reasoned that the primary task would be performed best when the secondary task was haptic-only. Reaction times (RTs) on the visual search task were fastest when the tablet task was haptic-only. This was not due to a speed-accuracy trade-off; there was no evidence for modulation of VST accuracy according to modality of the tablet task. These results provide the first quantitative support for introducing digital haptics into vehicle and similar contexts.

Effects of mental workload on involuntary attention: A somatosensory ERP study

Previous psychophysiological assessments of mental workload have relied on the addition of visual or auditory stimuli. This study investigated the tactile ERP and EEG spectral power correlates of mental workload by relating limited-capacity involuntary attention allocation to changes in late positive potential (LPP) amplitude, alpha, and theta powers. We examined whether mental workload (high-level cognitive control) can be evaluated using somatosensory stimuli. Sixteen participants all performed three tasks of varying difficulty. Two dual n-back tasks (n = 1 and 2) were used to investigate the degree to which mental workload affected the LPP amplitudes and EEG spectral powers evoked by ignoring salient tactile stimuli. In control trials, tactile vibrations were applied at random without dual n-back tasks. Subjective mental workload of each task was rated using the NASA Task Load Index. LPP amplitudes at Pz were significantly smaller in the dual-2-back trials compared to control and dual-1-back trials. Significantly increased theta power at Fz and reduced alpha power at Pz were found in the dual-2-back condition compared to control and dual-1-back condition. There was no significant difference between control and dual-1-back trials. The same pattern was found for subjective ratings of cognitive workload. These results indicate that the dual-2-back task imposed a significantly greater mental workload, causing impaired cognitive-control functions. Our findings support the notion that selective attention mechanisms necessary for effectively allocating and modulating attentional resources are temporarily impaired during the mentally overloaded state.

Out of touch? Visual load induces inattentional numbness

It is now well known that the absence of attention can leave people unaware of both visual and auditory stimuli (e.g., Dalton & Fraenkel, 2012; Mack & Rock, 1998). However, the possibility of similar effects within the tactile domain has received much less research. Here, we introduce a new tactile inattention paradigm and use it to test whether tactile awareness depends on the level of perceptual load in a concurrent visual task. Participants performed a visual search task of either low or high perceptual load, as well as responding to the presence or absence of a brief vibration delivered simultaneously to either the left or the right hand (50% of trials). Detection sensitivity to the clearly noticeable tactile stimulus was reduced under high (vs. low) visual perceptual load. These findings provide the first robust demonstration of “inattentional numbness,” as well as demonstrating that this phenomenon can be induced by concurrent visual perceptual load.

Forced to remember: when memory is biased by salient information

The last decades have seen a rapid growing in the attempt to understand the key factors involved in the internal memory representation of the external world. Visual salience have been found to provide a major contribution in predicting the probability for an item/object embedded in a complex setting (i.e., a natural scene) to be encoded and then remembered later on. Here I review the existing literature highlighting the impact of perceptual- (based on low-level sensory features) and semantics-related salience (based on high-level knowledge) on short-term memory representation, along with the neural mechanisms underpinning the interplay between these factors. The available evidence reveal that both perceptual- and semantics-related factors affect attention selection mechanisms during the encoding of natural scenes. Biasing internal memory representation, both perceptual and semantics factors increase the probability to remember high- to the detriment of low-saliency items. The available evidence also highlight an interplay between these factors, with a reduced impact of perceptual-related salience in biasing memory representation as a function of the increasing availability of semantics-related salient information. The neural mechanisms underpinning this interplay involve the activation of different portions of the frontoparietal attention control network. Ventral regions support the assignment of selection/encoding priorities based on high-level semantics, while the involvement of dorsal regions reflects priorities assignment based on low-level sensory features.

Independent effects of endogenous and exogenous attention in touch

Endogenous and exogenous attention in touch have typically been investigated separately. Here we use a double-cueing paradigm manipulating both types of orienting in each trial. Bilateral endogenous cues induced long-lasting facilitation of endogenous attention up to 2 s. However, the exogenous cue only elicited an effect at short intervals. Our results favour a supramodal account of attention and this study provides new insight into how endogenous and exogenous attention operates in the tactile modality.

Cross-Modal Distraction by Deviance

Unexpected task-irrelevant changes in the auditory or visual sensory channels have been shown to capture attention in an ineluctable manner and distract participants away from ongoing auditory or visual categorization tasks. We extend the study of this phenomenon by reporting the first within-participant comparison of deviance distraction in the tactile and auditory modalities. Using vibro-tactile-visual and auditory-visual cross-modal oddball tasks, we found that unexpected changes in the tactile and auditory modalities produced a number of functional similarities: A negative impact of distracter deviance on performance in the ongoing visual task, distraction on the subsequent trial (post-deviance distraction), and a similar decrease – but not the disappearance – of these effects across blocks. Despite these functional similarities, deviance distraction only correlated between the auditory and tactile modalities for the accuracy-based measure of deviance distraction and not for response latencies. Post-deviance distraction showed no correlation between modalities. Overall, the results suggest that behavioral deviance distraction may be underpinned by both modality-specific and multimodal mechanisms, while post-deviance distraction may predominantly relate to modality-specific processes.

Reflexive attention in touch: an investigation of event related potentials and behavioural responses

Exogenous attention has been extensively studied in vision but little is known about its behavioural and neural correlates in touch. To investigate this, non-informative tactile cues were followed after 800 ms by tactile targets and participants either detected targets or discriminated their location. Responses were slowed for targets at cued compared to uncued locations (i.e. inhibition of return (IOR)) only in the detection task. Concurrently recorded ERPs showed enhanced negativity for targets at uncued compared to cued locations at the N80 component and this modulation overlapped with the P100 component but only for the detection task indicating IOR may, if anything, be linked to attentional modulations at the P100. Further, cue-target interval analysis showed an enhanced anterior negativity contralateral to the cue side in both tasks, analogous to the anterior directed attention negativity (ADAN) previously only reported during endogenous orienting.

An attentional approach to study mental representations of different parts of the hand

The aim of this study is to investigate whether the fingers are represented separately from the palm. An exogenous spatial orientation paradigm was used where participants had to detect a tactile stimulus that could appear on the palm, the middle finger or the ring finger of the left hand. The tactile target was preceded by a non-predictive cue using different stimulus-onset asynchronies (SOA). We observed a Facilitation Effect in the palm and inhibition of return (IOR) for fingers using a short cue-target SOA, whereas the IOR was found in fingers and palm in long cue-target SOA. Also we observed a 'Cue above Target' effect (facilitation effect when the Cue had appeared distal to the target location in a vertical line) at the long SOA. Together, we suggest that the general pattern of results supports the proposed hypothesis about the different mental representation of fingers and palms, but with a considerable and hierarchical interrelation between them.

Capturing spatial attention with multisensory cues: a review

The last 30 years have seen numerous studies demonstrating unimodal and crossmodal spatial cuing effects. However, surprisingly few studies have attempted to investigate whether multisensory cues might be any more effective in capturing a person's spatial attention than unimodal cues. Indeed, until very recently, the consensus view was that multisensory cues were, in fact, no more effective. However, the results of several recent studies have overturned this conclusion, by showing that multisensory cues retain their attention-capturing ability under conditions of perceptual load (i.e., when participants are simultaneously engaged in a concurrent attention-demanding task) while their constituent signals (when presented unimodally) do not. Here we review the empirical literature on multisensory spatial cuing effects and highlight the implications that this research has for the design of more effective warning signals in applied settings.

Tactile and Multisensory Spatial Warning Signals for Drivers

The last few years have seen many exciting developments in the area of tactile and multisensory interface design. One of the most rapidly-moving practical application areas for these findings is in the development of warning signals and information displays for drivers. For instance, tactile displays can be used to awaken sleepy drivers, to capture the attention of distracted drivers, and even to present more complex information to drivers who may be visually-overloaded. This review highlights the most important potential costs and benefits associated with the use of tactile and multisensory information displays in a vehicular setting. Multisensory displays that are based on the latest cognitive neuroscience research findings can capture driver attention significantly more effective than their unimodal (i.e., tactile) counterparts. Multisensory displays can also be used to transmit information more efficiently, as well as to reduce driver workload. Finally, we highlight the key research questions currently awaiting further research, including questions such as: Are tactile warning signals really intuitive? Are there certain regions of the body (or in the space surrounding the body) where tactile/multisensory warning signals are particularly effective? To what extent is the spatial coincidence and temporal synchrony of the individual sensory signals critical to determining the effectiveness of multisensory displays? And, finally, how does the issue of compliance vs. reliance (or the 'cry wolf' phenomenon associated with the presentation of signals that are perceived as false alarms) influence the effectiveness of tactile and/or multisensory warning signals?

Investigating the time course of tactile reflexive attention using a non-spatial discrimination task

Peripheral cues are thought to facilitate responses to stimuli presented at the same location because they lead to exogenous attention shifts. Facilitation has been observed in numerous studies of visual and auditory attention, but there have been only four demonstrations of tactile facilitation, all in studies with potential confounds. Three studies used a spatial (finger versus thumb) discrimination task, where the cue could have provided a spatial framework that might have assisted the discrimination of subsequent targets presented on the same side as the cue. The final study circumvented this problem by using a non-spatial discrimination; however, the cues were informative and interspersed with visual cues which may have affected the attentional effects observed. In the current study, therefore, we used a non-spatial tactile frequency discrimination task following a non-informative tactile white noise cue. When the target was presented 150 ms after the cue, we observed faster discrimination responses to targets presented on the same side compared to the opposite side as the cue; by 1000 ms, responses were significantly faster to targets presented on the opposite side to the cue. Thus, we demonstrated that tactile attentional facilitation can be observed in a non-spatial discrimination task, under unimodal conditions and with entirely non-predictive cues. Furthermore, we provide the first demonstration of significant tactile facilitation and tactile inhibition of return within a single experiment.

Multisensory integration affects ERP components elicited by exogenous cues

Previous studies have shown that the amplitude of event related brain potentials (ERPs) elicited by a combined audiovisual stimulus is larger than the sum of a single auditory and visual stimulus. This enlargement is thought to reflect multisensory integration. Based on these data, it may be hypothesized that the speeding up of responses, due to exogenous orienting effects induced by bimodal cues, exceeds the sum of single unimodal cues. Behavioral data, however, typically revealed no increased orienting effect following bimodal as compared to unimodal cues, which could be due to a failure of multisensory integration of the cues. To examine this possibility, we computed ERPs elicited by both bimodal (audiovisual) and unimodal (either auditory or visual) cues, and determined their exogenous orienting effects on responses to a to-be-discriminated visual target. Interestingly, the posterior P1 component elicited by bimodal cues was larger than the sum of the P1 components elicited by a single auditory and visual cue (i.e., a superadditive effect), but no enhanced orienting effect was found on response speed. The latter result suggests that multisensory integration elicited by our bimodal cues plays no special role for spatial orienting, at least in the present setting.

Perceptual load affects exogenous spatial orienting while working memory load does not

We examined whether or not increasing visual perceptual load or visual working memory (WM) load would affect the exogenous orienting of visuo-spatial attention, in order to assess whether or not exogenous orienting is genuinely automatic. In Experiment 1, we manipulated visual perceptual load by means of a central morphing shape that in some trials morphed into a particular target shape (a rectangle) that participants had to detect. In Experiment 2, the possibility that the presentation of any changing stimulus at fixation would eliminate exogenous orienting was ruled out, by presenting two alternating letters at fixation. In Experiment 3, we manipulated visual WM load by means of arrays consisting of three (low-load) or five (high-load) randomly located coloured squares. The participants had to remember these items in order to judge whether a cued square had been presented in the same or different colour at the end of each trial. In all the experiments, exogenous visuo-spatial attentional orienting was measured by means of an orthogonal spatial cuing task, in which the participants had to discriminate the elevation (up vs. down) of a visual target previously cued by a spatially nonpredictive visual cue. The results showed that increasing the perceptual load of the task eliminated the exogenous orienting of visuo-spatial attention. By contrast, increasing the WM load had no effect on spatial orienting. These results are discussed in terms of the light that they shed on claims regarding the automaticity of visuo-spatial exogenous orienting.