Attention in natural scenes: contrast affects rapid visual processing and fixations alike

Foothold selection during locomotion in uneven terrain: Results from the integration of eye tracking, motion capture, and photogrammetry

Relatively little is known about the way vision is used to guide locomotion in the natural world. What visual features are used to choose paths in natural complex terrain? To answer this question, we measured eye and body movements while participants walked in natural outdoor environments. We incorporated measurements of the 3D terrain structure into our analyses and reconstructed the terrain along the walker's path, applying photogrammetry techniques to the eyetracker's scene camera videos. Combining these reconstructions with the walker's body movements, we demonstrate that walkers take terrain structure into account when selecting paths through an environment. We find that they change direction to avoid taking steeper steps that involve large height changes, instead of choosing more circuitous, relatively flat paths. Our data suggest walkers plan the location of individual footholds and plan ahead to select flatter paths. These results provide evidence that locomotor behavior in natural environments is controlled by decision mechanisms that account for multiple factors, including sensory and motor information, costs, and path planning.

Validation of the open biological negative image set for a Portuguese population: Comparing Japanese and Portuguese samples and an exploration of low-order visual properties of the stimuli

Recently, Shirai and Watanabe Royal Society Open Science, 9(1), 211128 (2022) developed OBNIS (Open Biological Negative Image Set), a comprehensive database containing images (primarily animals but also fruits, mushrooms, and vegetables) that visually elicit disgust, fear, or neither. OBNIS was initially validated for a Japanese population. In this article, we validated the color version of OBNIS for a Portuguese population. In study 1, the methodology of the original article was used. This allowed direct comparisons between the Portuguese and Japanese populations. Aside from a few emotional classification mismatches between disgust, fear, or neither-related images, we found that arousal and valence relate distinctively in both populations. In contrast to the Japanese sample, the Portuguese reported increased arousal for more positive valenced stimuli, suggesting that OBNIS images elicit positive emotions in the Portuguese population. These results showed important cross-cultural differences regarding OBNIS. In study 2, a methodological change was introduced: instead of the three classification options used originally (fear, disgust, or neither), six basic emotions were used (fear, disgust, sadness, surprise, anger, happiness), and a "neither" option, to confirm whether some of the originally "neither-related" images are associated with positive emotions (happiness). Additionally, the low-order visual properties of images (luminosity, contrast, chromatic complexity, and spatial frequency distribution) were explored due to their important role in emotion-related research. A fourth image group associated with happiness was found in the Portuguese sample. Moreover, image groups present differences regarding the low-order visual characteristics, which are correlated with arousal and valence ratings, highlighting the importance of controlling such characteristics in emotion-related research.

Predicting the attention of others

As social animals, people are highly sensitive to the attention of others. Seeing someone else gaze at an object automatically draws one's own attention to that object. Monitoring the attention of others aids in reconstructing their emotions, beliefs, and intentions and may play a crucial role in social alignment. Recently, however, it has been suggested that the human brain constructs a predictive model of other people's attention that is far more involved than a moment-by-moment monitoring of gaze direction. The hypothesized model learns the statistical patterns in other people's attention and extrapolates how attention is likely to move. Here, we tested the hypothesis of a predictive model of attention. Subjects saw movies of attention displayed as a bright spot shifting around a scene. Subjects were able to correctly distinguish natural attention sequences (based on eye tracking of prior participants) from altered sequences (e.g., played backward or in a scrambled order). Even when the attention spot moved around a blank background, subjects could distinguish natural from scrambled sequences, suggesting a sensitivity to the spatial-temporal statistics of attention. Subjects also showed an ability to recognize the attention patterns of different individuals. These results suggest that people possess a sophisticated model of the normal statistics of attention and can identify deviations from the model. Monitoring attention is therefore more than simply registering where someone else's eyes are pointing. It involves predictive modeling, which may contribute to our remarkable social ability to predict the mind states and behavior of others.

Assessment of the Effect of Cleanliness on the Visual Inspection of Aircraft Engine Blades: An Eye Tracking Study

Background-The visual inspection of aircraft parts such as engine blades is crucial to ensure safe aircraft operation. There is a need to understand the reliability of such inspections and the factors that affect the results. In this study, the factor 'cleanliness' was analysed among other factors. Method-Fifty industry practitioners of three expertise levels inspected 24 images of parts with a variety of defects in clean and dirty conditions, resulting in a total of N = 1200 observations. The data were analysed statistically to evaluate the relationships between cleanliness and inspection performance. Eye tracking was applied to understand the search strategies of different levels of expertise for various part conditions. Results-The results show an inspection accuracy of 86.8% and 66.8% for clean and dirty blades, respectively. The statistical analysis showed that cleanliness and defect type influenced the inspection accuracy, while expertise was surprisingly not a significant factor. In contrast, inspection time was affected by expertise along with other factors, including cleanliness, defect type and visual acuity. Eye tracking revealed that inspectors (experts) apply a more structured and systematic search with less fixations and revisits compared to other groups. Conclusions-Cleaning prior to inspection leads to better results. Eye tracking revealed that inspectors used an underlying search strategy characterised by edge detection and differentiation between surface deposits and other types of damage, which contributed to better performance.

Hemodynamic Response to Three Types of Urban Spaces before and after Lockdown during the COVID-19 Pandemic

(1) Background: Prolonged lockdowns with stay-at-home orders have been introduced in many countries since the outbreak of the COVID-19 pandemic. They have caused a drastic change in the everyday lives of people living in urbanized areas, and are considered to contribute to a modified perception of the public space. As research related to the impact of COVID-19 restrictions on mental health and well-being emerges, the associated longitudinal changes of brain hemodynamics in healthy adults remain largely unknown. (2) Methods: this study examined the hemodynamic activation patterns of the prefrontal and occipital cortices of 12 participants (5 male, Mage = 47.80, SDage = 17.79, range 25 to 74, and 7 female, Mage = 39.00, SDage = 18.18, range 21 to 65) passively viewing videos from three urban sites in Singapore (Urban Park, Neighborhood Landscape and City Center) at two different time points-T1, before the COVID-19 pandemic and T2, soon after the lockdown was over. (3) Results: We observed a significant and marginally significant decrease in average oxyhemoglobin (Oxy-Hb) over time for each of the visual conditions. For both green spaces (Urban Park and Neighborhood Landscape), the decrease was in the visual cortex, while for the City Center with no green elements, the marginal decrease was observed in the visual cortex and the frontal eye fields. (4) Conclusions: The results suggest that the COVID-19-related lockdown experienced by urban inhabitants may have contributed to decreased brain hemodynamics, which are further related to a heightened risk of mental health disorders, such as depression or a decline in cognitive functions. Moreover, the busy City Center scenes induced a hemodynamic pattern associated with stress and anxiety, while urban green spaces did not cause such an effect. Urban green scenes can be an important factor to offset the negative neuropsychological impact of busy urban environments post-pandemic.

Effects of COVID-19-related stay-at-home order on neuropsychophysiological response to urban spaces: Beneficial role of exposure to nature?

Background

The COVID-19 pandemic has had severe negative effects on populations worldwide. The seriousness of the pandemic necessitated local and even national lockdowns. In Singapore a national lockdown with aStay-at-Home Order (SHO) lasted for over 7 weeks. The impact of the pandemic and of the long SHO period on neuropsychophysiological functioning remains unknown. Studies prior to the pandemic highlighted the beneficial role of nature exposure on mental health and well-being, although this has not yet been explored in the post-pandemic world. This is the first study to investigate the longitudinal changes in (1) brain frontal alpha asymmetry (FAA; neuroelectric marker of approach-related motivation), (2) depressive mood and (3) symptoms, and (4) emotional response to videos of various urban spaces from before COVID-19 to immediately after the SHO in Singapore was over. Finally, we examined whether higher vs lower exposure to nature during the SHO moderated changes over time.

Methods

The sample included 25 healthy adult Singaporeans (56% female, M_age = 40.4 y, SD = 17.8), who attended two electroencephalography (EEG) lab sessions, within a year before the COVID-19 pandemic (T1) and immediately following the SHO (T2). The participants viewed 9 fixed-frame videos, filmed before the pandemic, from 3 urban public spaces (Busy Downtown, Residential Green, Lush Garden) on the roll-up screen. They rated their emotional response (arousal, valence) after each video and completed Becks Depression Inventory-II (BDI-II) as a measure of depressive symptoms, Profile of Mood Scale (POMS), as a measure of momentary mood, and self-reported the frequency and duration of their nature visits during the SHO.

Results

Linear mixed models were fitted to examine changes over time, and effect moderation by amount of nature exposure during the SHO. The results showed decrease in FAA (p < 0.001), increase in depressive symptoms (p = 0.046), and a trend for marginal increase in momentary mood disturbance (p = 0.097) after the SHO. Importantly, people with high nature exposure during SHO had greater decrease in FAA over time (p = 0.005) than those with low nature exposure, FAA scores decreased the most for Residential Green. Valence and Arousal did not change over time, but Arousal towards Busy Downtown decreased among high nature exposure individuals (p = 0.002).

Conclusions

Post SHO, brain activity and responsiveness to landscapes changed, and showed a general reduction in positive emotions and increased depressive symptoms among participants. The higher nature exposure during the SHO did not help mitigate this depressive symptoms, as previous research would suggest. This can be due to the modified quality of nature exposure during lockdown, which highlights the importance of high quality nature experience in cities and the provision of diversified visual exposures. Potential neuropsychophysiological consequences of SHO should be considered by policy makers in the post-COVID-19 world.

Magnocellular and Parvocellular Mediated Luminance Contrast Discrimination in Severe Alcohol Use Disorder

BACKGROUND

Severe alcohol use disorder (SAUD) is associated with widespread cognitive impairments, including low-level visual processing deficits that persist after prolonged abstinence. However, the extent and characteristics of these visual deficits remain largely undetermined, impeding the identification of their underlying mechanisms and influence on higher-order processing. In particular, little work has been conducted to assess the integrity of the magnocellular (MC) and parvocellular (PC) visual pathways, namely the 2 main visual streams that convey information from the retina up to striate, extrastriate, and dorsal/ventral cerebral regions.

METHODS

We investigated achromatic luminance contrast processing mediated by inferred MC and PC pathways in 33 patients with SAUD and 32 matched healthy controls using 2 psychophysical pedestal contrast discrimination tasks that promote responses of inferred MC or PC pathways. We relied on a staircase procedure to assess participants' ability to detect small changes in luminance within an array of 4 gray squares that were either continuously presented (steady pedestal, MC-biased) or briefly flashed (pulsed pedestal, PC-biased).

RESULTS

We replicated the expected pattern of MC and PC contrast responses in healthy controls. We found preserved dissociation of MC and PC contrast signatures in SAUD but higher MC-mediated mean contrast discrimination thresholds combined with a steeper PC-mediated contrast discrimination slope compared with healthy controls.

CONCLUSION

These findings indicate altered MC-mediated contrast sensitivity and PC-mediated contrast gain, confirming the presence of early sensory disturbances in individuals with SAUD. Such low-level deficits, while usually overlooked, might influence higher-order abilities (e.g., memory, executive functions) in SAUD by disturbing the "coarse-to-fine" tuning of the visual system, which relies on the distinct functional properties of MC and PC pathways and ensures proper and efficient monitoring of the environment.

Gaze and the Control of Foot Placement When Walking in Natural Terrain

Human locomotion through natural environments requires precise coordination between the biomechanics of the bipedal gait cycle and the eye movements that gather the information needed to guide foot placement. However, little is known about how the visual and locomotor systems work together to support movement through the world. We developed a system to simultaneously record gaze and full-body kinematics during locomotion over different outdoor terrains. We found that not only do walkers tune their gaze behavior to the specific information needed to traverse paths of varying complexity but that they do so while maintaining a constant temporal look-ahead window across all terrains. This strategy allows walkers to use gaze to tailor their energetically optimal preferred gait cycle to the upcoming path in order to balance between the drive to move efficiently and the need to place the feet in stable locations. Eye movements and locomotion are intimately linked in a way that reflects the integration of energetic costs, environmental uncertainty, and momentary informational demands of the locomotor task. Thus, the relationship between gaze and gait reveals the structure of the sensorimotor decisions that support successful performance in the face of the varying demands of the natural world.

Motivational Objects in Natural Scenes (MONS): A Database of >800 Objects

In daily life, we are surrounded by objects with pre-existing motivational associations. However, these are rarely controlled for in experiments with natural stimuli. Research on natural stimuli would therefore benefit from stimuli with well-defined motivational properties; in turn, such stimuli also open new paths in research on motivation. Here we introduce a database of Motivational Objects in Natural Scenes (MONS). The database consists of 107 scenes. Each scene contains 2 to 7 objects placed at approximately equal distance from the scene center. Each scene was photographed creating 3 versions, with one object (“critical object”) being replaced to vary the overall motivational value of the scene (appetitive, aversive, and neutral), while maintaining high visual similarity between the three versions. Ratings on motivation, valence, arousal and recognizability were obtained using internet-based questionnaires. Since the main objective was to provide stimuli of well-defined motivational value, three motivation scales were used: (1) Desire to own the object; (2) Approach/Avoid; (3) Desire to interact with the object. Three sets of ratings were obtained in independent sets of observers: for all 805 objects presented on a neutral background, for 321 critical objects presented in their scene context, and for the entire scenes. On the basis of the motivational ratings, objects were subdivided into aversive, neutral, and appetitive categories. The MONS database will provide a standardized basis for future studies on motivational value under realistic conditions.

Direct and indirect effects of attention and visual function on gait impairment in Parkinson's disease: influence of task and turning

Gait impairment is a core feature of Parkinson's disease (PD) which has been linked to cognitive and visual deficits, but interactions between these features are poorly understood. Monitoring saccades allows investigation of real-time cognitive and visual processes and their impact on gait when walking. This study explored: (i) saccade frequency when walking under different attentional manipulations of turning and dual-task; and (ii) direct and indirect relationships between saccades, gait impairment, vision and attention. Saccade frequency (number of fast eye movements per-second) was measured during gait in 60 PD and 40 age-matched control participants using a mobile eye-tracker. Saccade frequency was significantly reduced in PD compared to controls during all conditions. However, saccade frequency increased with a turn and decreased under dual-task for both groups. Poorer attention directly related to saccade frequency, visual function and gait impairment in PD, but not controls. Saccade frequency did not directly relate to gait in PD, but did in controls. Instead, saccade frequency and visual function deficit indirectly impacted gait impairment in PD, which was underpinned by their relationship with attention. In conclusion, our results suggest a vital role for attention with direct and indirect influences on gait impairment in PD. Attention directly impacted saccade frequency, visual function and gait impairment in PD, with connotations for falls. It also underpinned indirect impact of visual and saccadic impairment on gait. Attention therefore represents a key therapeutic target that should be considered in future research.

Changing What You See by Changing What You Know: The Role of Attention

Attending is a cognitive process that incorporates a person's knowledge, goals, and expectations. What we perceive when we attend to one thing is different from what we perceive when we attend to something else. Yet, it is often argued that attentional effects do not count as evidence that perception is influenced by cognition. I investigate two arguments often given to justify excluding attention. The first is arguing that attention is a post-perceptual process reflecting selection between fully constructed perceptual representations. The second is arguing that attention as a pre-perceptual process that simply changes the input to encapsulated perceptual systems. Both of these arguments are highly problematic. Although some attentional effects can indeed be construed as post-perceptual, others operate by changing perceptual content across the entire visual hierarchy. Although there is a natural analogy between spatial attention and a change of input, the analogy falls apart when we consider other forms of attention. After dispelling these arguments, I make a case for thinking of attention not as a confound, but as one of the mechanisms by which cognitive states affect perception by going through cases in which the same or similar visual inputs are perceived differently depending on the observer's cognitive state, and instances where cuing an observer using language affects what one sees. Lastly, I provide two compelling counter-examples to the critique that although cognitive influences on perception can be demonstrated in the laboratory, it is impossible to really experience them for oneself in a phenomenologically compelling way. Taken together, the current evidence strongly supports the thesis that what we know routinely influences what we see, that the same sensory input can be perceived differently depending on the current cognitive state of the viewer, and that phenomenologically salient demonstrations are possible if certain conditions are met.

Attention in natural scenes: Affective-motivational factors guide gaze independently of visual salience

In addition to low-level stimulus characteristics and current goals, our previous experience with stimuli can also guide attentional deployment. It remains unclear, however, if such effects act independently or whether they interact in guiding attention. In the current study, we presented natural scenes including every-day objects that differed in affective-motivational impact. In the first free-viewing experiment, we presented visually-matched triads of scenes in which one critical object was replaced that varied mainly in terms of motivational value, but also in terms of valence and arousal, as confirmed by ratings by a large set of observers. Treating motivation as a categorical factor, we found that it affected gaze. A linear-effect model showed that arousal, valence, and motivation predicted fixations above and beyond visual characteristics, like object size, eccentricity, or visual salience. In a second experiment, we experimentally investigated whether the effects of emotion and motivation could be modulated by visual salience. In a medium-salience condition, we presented the same unmodified scenes as in the first experiment. In a high-salience condition, we retained the saturation of the critical object in the scene, and decreased the saturation of the background, and in a low-salience condition, we desaturated the critical object while retaining the original saturation of the background. We found that highly salient objects guided gaze, but still found additional additive effects of arousal, valence and motivation, confirming that higher-level factors can also guide attention, as measured by fixations towards objects in natural scenes.

Five Factors that Guide Attention in Visual Search

How do we find what we are looking for? Fundamental limits on visual processing mean that even when the desired target is in our field of view, we often need to search, because it is impossible to recognize everything at once. Searching involves directing attention to objects that might be the target. This deployment of attention is not random. It is guided to the most promising items and locations by five factors discussed here: Bottom-up salience, top-down feature guidance, scene structure and meaning, the previous history of search over time scales from msec to years, and the relative value of the targets and distractors. Modern theories of search need to specify how all five factors combine to shape search behavior. An understanding of the rules of guidance can be used to improve the accuracy and efficiency of socially-important search tasks, from security screening to medical image perception.

Accuracy and re-test reliability of mobile eye-tracking in Parkinson's disease and older adults

Mobile eye-tracking is important for understanding the role of vision during real-world tasks in older adults (OA) and people with Parkinson's disease (PD). However, accuracy and reliability of such devices have not been established in these populations. We used a novel protocol to quantify accuracy and reliability of a mobile eye-tracker in OA and PD. A mobile eye-tracker (Dikablis) measured the saccade amplitudes of 20 OA and 14 PD on two occasions. Participants made saccades between targets placed 5°, 10° and 15° apart. Impact of visual correction (glasses) on saccadic amplitude measurement was also investigated in 10 OA. Saccade amplitude accuracy (median bias) was -1.21° but a wide range of bias (-7.73° to 5.81°) was seen in OA and PD, with large vertical saccades (15°) being least accurate. Reliability assessment showed a median difference between sessions of <1° for both groups, with poor to good relative agreement (Spearman rho: 0.14 to 0.85). Greater accuracy and reliability was observed in people without visual correction. Saccade amplitude can be measured with variable accuracy and reliability using a mobile eye-tracker in OA and PD. Human, technological and study-specific protocol factors may introduce error and are discussed along with methodological recommendations.

Normal discrimination of spatial frequency and contrast across visual hemifields in left-onset Parkinson's disease: evidence against perceptual hemifield biases

Individuals with Parkinson's disease (PD) with symptom onset on the left side of the body (LPD) show a mild type of left-sided visuospatial neglect, whereas those with right-onset (RPD) generally do not. The functional mechanisms underlying these observations are unknown. Two hypotheses are that the representation of left-space in LPD is either compressed or reduced in salience. We tested these hypotheses psychophysically. Participants were 31 non-demented adults with PD (15 LPD, 16 RPD) and 17 normal control adults (NC). The spatial compression hypothesis was tested by showing two sinusoidal gratings, side by side. One grating's spatial frequency (SF) was varied across trials, following a staircase procedure, whereas the comparison grating was held at a constant SF. While fixating on a central target, participants estimated the point at which they perceived the two gratings to be equal in SF. The reduced salience hypothesis was tested in a similar way, but by manipulating the contrast of the test grating rather than its SF. There were no significant differences between groups in the degree of bias across hemifields for SF discrimination or for contrast discrimination. Results did not support either the spatial compression hypothesis or the reduced salience hypothesis. Instead, they suggest that at this perceptual level, LPD do not have a systematically biased way of representing space in the left hemifield that differs from healthy individuals, nor do they perceive stimuli on the left as less salient than stimuli on the right. Neglect-like syndrome in LPD instead presumably arises from dysfunction of higher-order attention.

A high-capacity model for one shot association learning in the brain

We present a high-capacity model for one-shot association learning (hetero-associative memory) in sparse networks. We assume that basic patterns are pre-learned in networks and associations between two patterns are presented only once and have to be learned immediately. The model is a combination of an Amit-Fusi like network sparsely connected to a Willshaw type network. The learning procedure is palimpsest and comes from earlier work on one-shot pattern learning. However, in our setup we can enhance the capacity of the network by iterative retrieval. This yields a model for sparse brain-like networks in which populations of a few thousand neurons are capable of learning hundreds of associations even if they are presented only once. The analysis of the model is based on a novel result by Janson et al. on bootstrap percolation in random graphs.

Influence of local scene color on fixation position in visual search

Where observers concentrate their gaze during visual search depends on several factors. The aim here was to determine how much of the variance in observers' fixations in natural scenes can be explained by local scene color and how that variance is related to viewing bias. Fixation data were taken from an experiment in which observers searched images of 20 natural rural and urban scenes for a small target. The proportion R2 of the variance explained in a regression on local color properties (lightness and the red-green and yellow-blue chromatic components) ranged from 1% to 85%, depending mainly on how well those properties were consistent with observers' viewing bias. When viewing bias was included in the regression, values of R2 increased, ranging from 62% to 96%. By comparison, local lightness and local lightness contrast, edge density, and entropy each explained less variance than local color properties. Local scene color may have a much stronger influence on gaze position than is generally recognized, capturing significant aspects of scene structure on target search behavior.

Attentional selection in visual perception, memory and action: a quest for cross-domain integration

For decades, the cognitive and neural sciences have benefitted greatly from a separation of mind and brain into distinct functional domains. The tremendous success of this approach notwithstanding, it is self-evident that such a view is incomplete. Goal-directed behaviour of an organism requires the joint functioning of perception, memory and sensorimotor control. A prime candidate for achieving integration across these functional domains are attentional processes. Consequently, this Theme Issue brings together studies of attentional selection from many fields, both experimental and theoretical, that are united in their quest to find overreaching integrative principles of attention between perception, memory and action. In all domains, attention is understood as combination of competition and priority control ('bias'), with the task as a decisive driving factor to ensure coherent goal-directed behaviour and cognition. Using vision as the predominant model system for attentional selection, many studies of this Theme Issue focus special emphasis on eye movements as a selection process that is both a fundamental action and serves a key function in perception. The Theme Issue spans a wide range of methods, from measuring human behaviour in the real word to recordings of single neurons in the non-human primate brain. We firmly believe that combining such a breadth in approaches is necessary not only for attentional selection, but also to take the next decisive step in all of the cognitive and neural sciences: to understand cognition and behaviour beyond isolated domains.

Fixations on objects in natural scenes: dissociating importance from salience

The relation of selective attention to understanding of natural scenes has been subject to intense behavioral research and computational modeling, and gaze is often used as a proxy for such attention. The probability of an image region to be fixated typically correlates with its contrast. However, this relation does not imply a causal role of contrast. Rather, contrast may relate to an object's "importance" for a scene, which in turn drives attention. Here we operationalize importance by the probability that an observer names the object as characteristic for a scene. We modify luminance contrast of either a frequently named ("common"/"important") or a rarely named ("rare"/"unimportant") object, track the observers' eye movements during scene viewing and ask them to provide keywords describing the scene immediately after. When no object is modified relative to the background, important objects draw more fixations than unimportant ones. Increases of contrast make an object more likely to be fixated, irrespective of whether it was important for the original scene, while decreases in contrast have little effect on fixations. Any contrast modification makes originally unimportant objects more important for the scene. Finally, important objects are fixated more centrally than unimportant objects, irrespective of contrast. Our data suggest a dissociation between object importance (relevance for the scene) and salience (relevance for attention). If an object obeys natural scene statistics, important objects are also salient. However, when natural scene statistics are violated, importance and salience are differentially affected. Object salience is modulated by the expectation about object properties (e.g., formed by context or gist), and importance by the violation of such expectations. In addition, the dependence of fixated locations within an object on the object's importance suggests an analogy to the effects of word frequency on landing positions in reading.