Finding the balance between capture and control: Oculomotor selection in early deaf adults

Research of visual attention networks in deaf individuals: a systematic review

The impact of deafness on visual attention has been widely discussed in previous research. It has been noted that deficiencies and strengths of previous research can be attributed to temporal or spatial aspects of attention, as well as variations in development and clinical characteristics. Visual attention is categorized into three networks: orienting (exogenous and endogenous), alerting (phasic and tonic), and executive control. This study aims to contribute new neuroscientific evidence supporting this hypothesis. This paper presents a systematic review of the international literature from the past 15 years focused on visual attention in the deaf population. The final review included 24 articles. The function of the orienting network is found to be enhanced in deaf adults and children, primarily observed in native signers without cochlear implants, while endogenous orienting is observed only in the context of gaze cues in children, with no differences found in adults. Results regarding alerting and executive function vary depending on clinical characteristics and paradigms used. Implications for future research on visual attention in the deaf population are discussed.

The effects of eccentricity on attentional capture

Visual attention may be captured by an irrelevant yet salient distractor, thereby slowing search for a relevant target. This phenomenon has been widely studied using the additional singleton paradigm in which search items are typically all presented at one and the same eccentricity. Yet, differences in eccentricity may well bias the competition between target and distractor. Here we investigate how attentional capture is affected by the relative eccentricities of a target and a distractor. Participants searched for a shape-defined target in a grid of homogeneous nontargets of the same color. On 75% of trials, one of the nontarget items was replaced by a salient color-defined distractor. Crucially, target and distractor eccentricities were independently manipulated across three levels of eccentricity (i.e., near, middle, and far). Replicating previous work, we show that the presence of a distractor slows down search. Interestingly, capture as measured by manual reaction times was not affected by target and distractor eccentricity, whereas capture as measured by the eyes was: items close to fixation were more likely to be selected than items presented further away. Furthermore, the effects of target and distractor eccentricity were largely additive, suggesting that the competition between saliency- and relevance-driven selection was modulated by an independent eccentricity-based spatial component. Implications of the dissociation between manual and oculomotor responses are also discussed.

Cognition Assessment Technologies on Deaf People

In recent years there has been a growing interest in research about the different ways of processing and consolidating cognition in deaf people. It is known that hearing loss can lead to differences in some executive functions like control inhibitory or working memory. This literature review describes executive functions in deaf people and how they could be evaluated through technological devices complementing traditional assessments, like neuropsychological batteries. We identified biometric devices, digital and physical interfaces, and software from the literature, whose goal is to design or adapt technology to assess some cognition domains in several ways. The results of the review suggest the need to understand the cognitive phenomenon that significantly impacts the context of deaf people; moreover, it becomes relevant as a line of research in the Cognitive Science of Hearing. Using technologies to measure them and gain a better understanding of cognition in deaf people may provide possibilities for designing or adapting targeted educational or therapeutic strategies.

Effects of exogenous and endogenous cues on attentional orienting in deaf adults

Adults who are deaf have been shown to have better visual attentional orienting than those with typical hearing, especially when the target is located in the periphery of the visual field. However, most studies in this population have assessed exogenous visual attention orienting (bottom-up processing of external cues) rather than endogenous visual attention orienting (top-down processing of internal cues). We used a target detection task to assess both types of visual attention orienting. A modified cue-target paradigm was adopted to assess the facilitation effects of exogenous and endogenous cues during short and long inter-stimulus intervals (ISI), using a 2 (Group: deaf/typically hearing) * 2 (Location: central/peripheral) * 2 (Cue Type: exogenous/endogenous) mixed factorial design. ANOVAs showed that both exogenous cues and endogenous cues can facilitate deaf adults’ visual attentional orienting, and the facilitation effect of exogenous cues on attention orienting was significantly stronger for deaf participants than hearing participants. When the ISI was long, the effect was significantly stronger when the exogenous cue appeared in the periphery of the visual field. In the periphery, deaf adults benefited most from exogenous cues, whereas hearing adults benefited most from endogenous cues. The results suggest that not only exogenous cues but also endogenous cues can facilitate deaf adults’ visual attentional orienting. However, the effect of exogenous cues appears to be greater, especially when the stimulus appears in the peripheral visual field.

An attentional limbo: Saccades become momentarily non-selective in between saliency-driven and relevance-driven selection

Human vision involves selectively directing the eyes to potential objects of interest. According to most prominent theories, selection is the quantal outcome of an ongoing competition between saliency-driven signals on the one hand, and relevance-driven signals on the other, with both types of signals continuously and concurrently projecting onto a common priority map. Here, we challenge this view. We asked participants to make a speeded eye movement towards a target orientation, which was presented together with a non-target of opposing tilt. In addition to the difference in relevance, the target and non-target also differed in saliency, with the target being either more or less salient than the non-target. We demonstrate that saliency- and relevance-driven eye movements have highly idiosyncratic temporal profiles, with saliency-driven eye movements occurring rapidly after display onset while relevance-driven eye movements occur only later. Remarkably, these types of eye movements can be fully separated in time: We find that around 250 ms after display onset, eye movements are no longer driven by saliency differences between potential targets, but also not yet driven by relevance information, resulting in a period of non-selectivity, which we refer to as the attentional limbo. Binomial modeling further confirmed that visual selection is not necessarily the outcome of a direct battle between saliency- and relevance-driven signals. Instead, selection reflects the dynamic changes in the underlying saliency- and relevance-driven processes themselves, and the time at which an action is initiated then determines which of the two will emerge as the driving force of behavior.

Food Captures Attention, but Not the Eyes: An Eye-Tracking Study on Mindset and BMI’s Impact on Attentional Capture by High-Caloric Visual Food Stimuli

Obesity is a worldwide pandemic and theories propose that attentional bias (AB) for food triggers craving and overeating, especially for people with obesity. However, empirical evidence is inconsistent, which may be due to methodological diversity and the double-sided nature of high-caloric palatable foods. That is, these foods simultaneously have a high hedonic and a low health value. So, depending on context and/or emotional state, people’s mindset while viewing foods may alternate between hedonic (taste) and health (calories) values, possibly affecting AB for food in opposite directions. This study tests how mindset and BMI (Body Mass Index) influences AB and food intake. We expect greater AB for food and more food intake in the hedonic compared to the health mindset, especially for people with obesity. Mindsets were induced using short video-clips in two sessions in counterbalanced order. Participants (35 with a healthy-weight-category BMI, 31 with obesity) performed a modified Additional Singleton paradigm where they searched for a neutral target among neutral fillers. On 90% of the trials, either a food or a neutral distractor appeared. Response latencies to the target and eye-movements to the distractor were recorded. Dependent variables included: response latencies, and eye-movement variables on the distractor: fixations (%), 1st fixation duration, dwell-time. Food intake was assessed in a bogus taste test. No significant effects emerged from the eye-movements analysis, whereas the analysis of response latencies showed an AB for food, not significantly moderated by BMI or mindset. Food intake was affected by mindset partly as expected, as participants ate more in the hedonic than in the health mindset when the hedonic mindset was induced in the second session. One AB measure (fixations) correlated positively with food intake. Finally, food captured attention – but not the eyes – and mindset affects food intake partly as expected.

Attention capture by brief abrupt-onset cues in deaf individuals

Auditory loss in deaf individuals has been associated with an enhancement in the visual modality. Visual attention is one domain where such plasticity-induced changes have been observed, although which specific attentional mechanisms are improved is still not clear. Using a modified spatial cueing paradigm, we examined attention capture in deaf and normal-hearing participants. Brief abrupt-onset cues were presented for 16 ms either in attended or ignored locations. The to-be-attended locations for each trial were indicated by a horizontal or a vertical bar at the centre of the screen. These were presented either in vertical- or horizontal-only blocks or mixed together. We observed greater negative cueing effects in the NH group compared to deaf. Additionally, people with deafness showed greater capture by cues at ignored locations in the slower responses. These findings shed further light on orienting mechanisms in deaf and help in understanding the specificity of the differences in visual processing between deaf and normal-hearing individuals.

Congenital Deafness Leads to Altered Overt Oculomotor Behaviors

The human brain is highly cross-modal, and sensory information may affect a wide range of behaviors. In particular, there is evidence that auditory functions are implicated in oculomotor behaviors. Considering this apparent auditory-oculomotor link, one might wonder how the loss of auditory input from birth might have an influence on these motor behaviors. Eye movement tracking enables to extract several components, including saccades and smooth pursuit. One study suggested that deafness can alter saccades processing. Oculomotor behaviors have not been examined further in the deaf. The main goal of this study was to examine smooth pursuit following deafness. A pursuit task paradigm was used in this experiment. Participants were instructed to move their eyes to follow a target as it moved. The target movements have a possibility of four different trajectories (horizontal, vertical, elliptic clockwise, and elliptic counter-clockwise). Results indicate a significant reduction in the ability to track a target in both elliptical conditions showing that more complex motion processing differs in deaf individuals. The data also revealed significantly more saccades per trial in the vertical, anti-clockwise, and, to a lesser extent, the clockwise elliptic condition. This suggests that auditory deprivation from birth leads to altered overt oculomotor behaviors.

The sampling precision of research in five major areas of psychology

After obtaining a sample of published, peer-reviewed articles from journals with high and low impact factors in social, cognitive, neuro-, developmental, and clinical psychology, we used a priori equations recently derived by Trafimow (Educational and Psychological Measurement, 77, 831-854, 2017; Trafimow & MacDonald in Educational and Psychological Measurement, 77, 204-219, 2017) to compute the articles' median levels of precision. Our findings indicate that developmental research performs best with respect to precision, whereas cognitive research performs the worst; however, none of the psychology subfields excelled. In addition, we found important differences in precision between journals in the upper versus lower echelons with respect to impact factors in cognitive, neuro-, and clinical psychology, whereas the difference was dramatically attenuated for social and developmental psychology. Implications are discussed.

The Cross-Modal Effects of Sensory Deprivation on Spatial and Temporal Processes in Vision and Audition: A Systematic Review on Behavioral and Neuroimaging Research since 2000

One of the most significant effects of neural plasticity manifests in the case of sensory deprivation when cortical areas that were originally specialized for the functions of the deprived sense take over the processing of another modality. Vision and audition represent two important senses needed to navigate through space and time. Therefore, the current systematic review discusses the cross-modal behavioral and neural consequences of deafness and blindness by focusing on spatial and temporal processing abilities, respectively. In addition, movement processing is evaluated as compiling both spatial and temporal information. We examine whether the sense that is not primarily affected changes in its own properties or in the properties of the deprived modality (i.e., temporal processing as the main specialization of audition and spatial processing as the main specialization of vision). References to the metamodal organization, supramodal functioning, and the revised neural recycling theory are made to address global brain organization and plasticity principles. Generally, according to the reviewed studies, behavioral performance is enhanced in those aspects for which both the deprived and the overtaking senses provide adequate processing resources. Furthermore, the behavioral enhancements observed in the overtaking sense (i.e., vision in the case of deafness and audition in the case of blindness) are clearly limited by the processing resources of the overtaking modality. Thus, the brain regions that were previously recruited during the behavioral performance of the deprived sense now support a similar behavioral performance for the overtaking sense. This finding suggests a more input-unspecific and processing principle-based organization of the brain. Finally, we highlight the importance of controlling for and stating factors that might impact neural plasticity and the need for further research into visual temporal processing in deaf subjects.

Children flexibly seek visual information to support signed and spoken language comprehension

During grounded language comprehension, listeners must link the incoming linguistic signal to the visual world despite uncertainty in the input. Information gathered through visual fixations can facilitate understanding. But do listeners flexibly seek supportive visual information? Here, we propose that even young children can adapt their gaze and actively gather information for the goal of language comprehension. We present 2 studies of eye movements during real-time language processing, where the value of fixating on a social partner varies across different contexts. First, compared with children learning spoken English (n = 80), young American Sign Language (ASL) learners (n = 30) delayed gaze shifts away from a language source and produced a higher proportion of language-consistent eye movements. This result provides evidence that ASL learners adapt their gaze to effectively divide attention between language and referents, which both compete for processing via the visual channel. Second, English-speaking preschoolers (n = 39) and adults (n = 31) fixated longer on a speaker's face while processing language in a noisy auditory environment. Critically, like the ASL learners in Experiment 1, this delay resulted in gathering more visual information and a higher proportion of language-consistent gaze shifts. Taken together, these studies suggest that young listeners can adapt their gaze to seek visual information from social partners to support real-time language comprehension. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Increased overt attention to objects in early deaf adults: An eye-tracking study of complex naturalistic scenes

The study of selective attention in people with profound deafness has repeatedly documented enhanced attention to the peripheral regions of the visual field compared to hearing controls. This finding emerged from covert attention studies (i.e., without eye-movements) involving extremely simplified visual scenes and comprising few visual items. In this study, we aimed to test whether this key finding extends also to overt attention, using a more ecologically valid experimental context in which complex naturalistic images were presented for 3 s. In Experiment 1 (N = 35), all images contained a single central object superimposed on a congruent naturalistic background (e.g., a tiger in the woods). At the end of the visual exploration phase, an incidental memory task probed the participants' recollection of the seen central objects and image backgrounds. Results showed that hearing controls explored and remembered the image backgrounds more than deaf participants, who lingered on the central object to a greater extent. In Experiment 2 we aimed to disentangle if this behaviour of deaf participants reflected a bias in overt space-based attention towards the centre of the image, or instead, enhanced object-centred attention. We tested new participants (N = 42) in the visual exploration task adding images with lateralized objects, as well as images with multiple object or images without any object. Results confirmed increased exploration of objects in deaf participants. Taken together our novel findings show limitations of the well-known peripheral attention bias of deaf people and suggest that visual object-centred attention may also change after prolonged auditory deprivation.

Sign language experience redistributes attentional resources to the inferior visual field

While a substantial body of work has suggested that deafness brings about an increased allocation of visual attention to the periphery there has been much less work on how using a signed language may also influence this attentional allocation. Signed languages are visual-gestural and produced using the body and perceived via the human visual system. Signers fixate upon the face of interlocutors and do not directly look at the hands moving in the inferior visual field. It is therefore reasonable to predict that signed languages require a redistribution of covert visual attention to the inferior visual field. Here we report a prospective and statistically powered assessment of the spatial distribution of attention to inferior and superior visual fields in signers - both deaf and hearing - in a visual search task. Using a Bayesian Hierarchical Drift Diffusion Model, we estimated decision making parameters for the superior and inferior visual field in deaf signers, hearing signers and hearing non-signers. Results indicated a greater attentional redistribution toward the inferior visual field in adult signers (both deaf and hearing) than in hearing sign-naïve adults. The effect was smaller for hearing signers than for deaf signers, suggestive of either a role for extent of exposure or greater plasticity of the visual system in the deaf. The data provide support for a process by which the demands of linguistic processing can influence the human attentional system.

Multisensory Interference in Early Deaf Adults

Multisensory interactions in deaf cognition are largely unexplored. Unisensory studies suggest that behavioral/neural changes may be more prominent for visual compared to tactile processing in early deaf adults. Here we test whether such an asymmetry results in increased saliency of vision over touch during visuo-tactile interactions. About 23 early deaf and 25 hearing adults performed two consecutive visuo-tactile spatial interference tasks. Participants responded either to the elevation of the tactile target while ignoring a concurrent visual distractor at central or peripheral locations (respond to touch/ignore vision), or they performed the opposite task (respond to vision/ignore touch). Multisensory spatial interference emerged in both tasks for both groups. Crucially, deaf participants showed increased interference compared to hearing adults when they attempted to respond to tactile targets and ignore visual distractors, with enhanced difficulties with ipsilateral visual distractors. Analyses on task-order revealed that in deaf adults, interference of visual distractors on tactile targets was much stronger when this task followed the task in which vision was behaviorally relevant (respond to vision/ignore touch). These novel results suggest that behavioral/neural changes related to early deafness determine enhanced visual dominance during visuo-tactile multisensory conflict.

Conditional control in visual selection

Attention and eye movements provide a window into the selective processing of visual information. Evidence suggests that selection is influenced by various factors and is not always under the strategic control of the observer. The aims of this tutorial review are to give a brief introduction to eye movements and attention and to outline the conditions that help determine control. Evidence suggests that the ability to establish control depends on the complexity of the display as well as the point in time at which selection occurs. Stimulus-driven selection is more probable in simple displays than in complex natural scenes, but it critically depends on the timing of the response: Salience determines selection only when responses are triggered quickly following display presentation, and plays no role in longer-latency responses. The time course of selection is also important for the relationship between attention and eye movements. Specifically, attention and eye movements appear to act independently when oculomotor selection is quick, whereas attentional processes are able to influence oculomotor control when saccades are triggered only later in time. This relationship may also be modulated by whether the eye movement is controlled in a voluntary or an involuntary manner. To conclude, we present evidence that shows that visual control is limited in flexibility and that the mechanisms of selection are constrained by context and time. The outcome of visual selection changes with the situational context, and knowing the constraints of control is necessary to understanding when and how visual selection is truly controlled by the observer.