Modeling attention to salient proto-objects

Properties of visually guided saccadic behavior and bottom-up attention in marmoset, macaque, and human

Saccades are stereotypic behaviors whose investigation improves our understanding of how primate brains implement precise motor control. Furthermore, saccades offer an important window into the cognitive and attentional state of the brain. Historically, saccade studies have largely relied on macaques. However, the cortical network giving rise to the saccadic command is difficult to study in macaques because relevant cortical areas lie in deep sulci and are difficult to access. Recently, a New World monkey. the marmoset, has garnered attention as an alternative to macaques because of advantages including its smooth cortical surface. However, adoption of the marmoset for oculomotor research has been limited due to a lack of in-depth descriptions of marmoset saccade kinematics and their ability to perform psychophysical tasks. Here, we directly compare free-viewing and visually guided behavior of marmoset, macaque, and human engaged in identical tasks under similar conditions. In the video free-viewing task, all species exhibited qualitatively similar saccade kinematics up to 25° in amplitude although with different parameters. Furthermore, the conventional bottom-up saliency model predicted gaze targets at similar rates for all species. We further verified their visually guided behavior by training them with step and gap saccade tasks. In the step paradigm, marmosets did not show shorter saccade reaction time for upward saccades whereas macaques and humans did. In the gap paradigm, all species showed similar gap effect and express saccades. Our results suggest that the marmoset can serve as a model for oculomotor, attentional, and cognitive research while we need to be aware of their difference from macaque or human.NEW & NOTEWORTHY We directly compared the results of a video free-viewing task and visually guided saccade tasks (step and gap) among three different species: marmoset, macaque, and human. We found that all species exhibit qualitatively similar saccadic kinematics and saliency-driven saccadic behavior albeit with different parameters. Our results suggest that the marmoset possesses similar neural mechanisms to macaque and human for saccadic control, and it is an appropriate model to study neural mechanisms for active vision and attention.

Spatially Filtered Emotional Faces Dominate during Binocular Rivalry

The present investigation explores the role of bottom-up and top-down factors in the recognition of emotional facial expressions during binocular rivalry. We manipulated spatial frequencies (SF) and emotive features and asked subjects to indicate whether the emotional or the neutral expression was dominant during binocular rivalry. Controlling the bottom-up saliency with a computational model, physically comparable happy and fearful faces were presented dichoptically with neutral faces. The results showed the dominance of emotional faces over neutral ones. In particular, happy faces were reported more frequently as the first dominant percept even in the presence of coarse information (at a low SF level: 2-6 cycle/degree). Following current theories of emotion processing, the results provide further support for the influence of positive compared to negative meaning on binocular rivalry and, for the first time, showed that individuals perceive the affective quality of happiness even in the absence of details in the visual display. Furthermore, our findings represent an advance in knowledge regarding the association between the high- and low-level mechanisms behind binocular rivalry.

Comparable search efficiency for human and animal targets in the context of natural scenes

In a previous series of studies, we have shown that search for human targets in the context of natural scenes is more efficient than search for mechanical targets. Here we asked whether this search advantage extends to other categories of biological objects. We used videos of natural scenes to directly contrast search efficiency for animal and human targets among biological or nonbiological distractors. In visual search arrays consisting of two, four, six, or eight videos, observers searched for animal targets among machine distractors, and vice versa (Exp. 1). Another group searched for animal targets among human distractors, and vice versa (Exp. 2). We measured search slope as a proxy for search efficiency, and complemented the slope with eye movement measurements (fixation duration on the target, as well as the proportion of first fixations landing on the target). In both experiments, we observed no differences in search slopes or proportions of first fixations between any of the target-distractor category pairs. With respect to fixation durations, we found shorter on-target fixations only for animal targets as compared to machine targets (Exp. 1). In summary, we did not find that the search advantage for human targets over mechanical targets extends to other biological objects. We also found no search advantage for detecting humans as compared to other biological objects. Overall, our pattern of findings suggests that search efficiency in natural scenes, as elsewhere, depends crucially on the specific target-distractor categories.

Salience-based object prioritization during active viewing of naturalistic scenes in young and older adults

Whether fixation selection in real-world scenes is guided by image salience or by objects has been a matter of scientific debate. To contrast the two views, we compared effects of location-based and object-based visual salience in young and older (65 + years) adults. Generalized linear mixed models were used to assess the unique contribution of salience to fixation selection in scenes. When analysing fixation guidance without recurrence to objects, visual salience predicted whether image patches were fixated or not. This effect was reduced for the elderly, replicating an earlier finding. When using objects as the unit of analysis, we found that highly salient objects were more frequently selected for fixation than objects with low visual salience. Interestingly, this effect was larger for older adults. We also analysed where viewers fixate within objects, once they are selected. A preferred viewing location close to the centre of the object was found for both age groups. The results support the view that objects are important units of saccadic selection. Reconciling the salience view with the object view, we suggest that visual salience contributes to prioritization among objects. Moreover, the data point towards an increasing relevance of object-bound information with increasing age.

Reorganization in the representation of face-race categories from 6 to 9 months of age: Behavioral and computational evidence

Prior research has reported developmental change in how infants represent categories of other-race faces (Developmental Science 19 (2016) 362-371). In particular, Caucasian 6-month-olds were shown to represent African versus Asian face categories, whereas Caucasian 9 month-olds represented different classes of other-race faces in one category, inclusive of African and Asian faces but exclusive of Caucasian faces. The current investigation sought to provide stronger evidence that is convergent with these findings by asking whether infants will generalize looking-time responsiveness from one to another other-race category. In Experiment 1, an experimental group of Caucasian 6-month-olds was familiarized with African (or Asian) faces and then given a novel category preference test with an Asian (or African) face versus a Caucasian face, while a control group of Caucasian 6-month-olds viewed the test faces without prior familiarization. Infants in the experimental group divided attention between the test faces and infants in the control group did not manifest a spontaneous preference. Experiment 2 used the same procedure, but was conducted with Caucasian 9-month-olds. Infants in the experimental group displayed a robust preference for Caucasian faces when considered against the finding that infants in the control group displayed a spontaneous preference for other-race faces. The results offer confirmation that between 6 and 9 months, infants transition to representing own-race versus other-race face categories, with the latter inclusive of multiple other-race face classes with clear perceptual differences. Computational modeling of infant responding suggests that the developmental change is rooted in the statistics of experience with majority versus minority group faces.

Viewing of figurative paintings affects pseudoneglect as measured by line bisection

Neurologically intact individuals usually show a leftward bias in spatial attention, known as pseudoneglect, likely reflecting a right-hemisphere dominance in the control of spatial attention. A leftward bias also seems to manifest when individuals are asked to provide aesthetic judgments about visual stimuli, like artworks. However, whether artwork perception affects the allocation of spatial attention has never been directly investigated. Here, we assessed whether viewing figurative paintings affects hemispheric imbalance in the control of spatial attention by asking participants to bisect a series of lines presented on a grey background, on figurative paintings or on non-artistic photographs of real-world scenes, while either simply observing or explicitly evaluating each image. In line with previous evidence, participants showed a leftward bisection bias in the baseline condition, reflecting pseudoneglect. Critically, the presence of a painting in the background (irrespective of whether an aesthetic evaluation was required or not) significantly shifted the bias further to the left compared to when lines were bisected over a grey background (baseline) or a photographed scene. This finding suggests that perception of visual art may affect the control of spatial attention, possibly tapping on right-hemisphere resources related to visuospatial exploration, regardless of reward apprehension mechanisms (at least when images do not evoke strong emotional reactions leading to polarized like/dislike judgements).

Look duration at the face as a developmental endophenotype: elucidating pathways to autism and ADHD

Identifying developmental endophenotypes on the pathway between genetics and behavior is critical to uncovering the mechanisms underlying neurodevelopmental conditions. In this proof-of-principle study, we explored whether early disruptions in visual attention are a unique or shared candidate endophenotype of autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder (ADHD). We calculated the duration of the longest look (i.e., peak look) to faces in an array-based eye-tracking task for 335 14-month-old infants with and without first-degree relatives with ASD and/or ADHD. We leveraged parent-report and genotype data available for a proportion of these infants to evaluate the relation of looking behavior to familial (n = 285) and genetic liability (using polygenic scores, n = 185) as well as ASD and ADHD-relevant temperament traits at 2 years of age (shyness and inhibitory control, respectively, n = 272) and ASD and ADHD clinical traits at 6 years of age (n = 94).

Results showed that longer peak looks at the face were associated with elevated polygenic scores for ADHD (β = 0.078, p = .023), but not ASD (β = 0.002, p = .944), and with elevated ADHD traits in mid-childhood (F(1,88) = 6.401, p = .013, $\eta _p^2$=0.068; ASD: F (1,88) = 3.218, p = .076), but not in toddlerhood (ps > 0.2). This pattern of results did not emerge when considering mean peak look duration across face and nonface stimuli. Thus, alterations in attention to faces during spontaneous visual exploration may be more consistent with a developmental endophenotype of ADHD than ASD. Our work shows that dissecting paths to neurodevelopmental conditions requires longitudinal data incorporating polygenic contribution, early neurocognitive function, and clinical phenotypic variation.

Social anxiety and attention to dynamic disgust facial expression: No more attention captured from evidence of N2pc

In this study we investigated the role of stimulus-driven attention in the attentional bias in individuals with social anxiety using electrophysiological technique. For this purpose, we employed static and dynamic facial expressions as stimuli in a dot-probe task. The results revealed that in behavior high socially anxious (HSA) group had longer response time in incongruent trials than congruent trials, and showed higher trial level-bias score variability than low socially anxious (LSA) group. In electrophysiology, we found that compared to LSA group, HSA group showed an enhanced N2pc amplitude to disgust facial expressions in static condition, whereas this difference was eliminated in dynamic facial expression condition. The results suggest the faces with peak emotion intensity are more likely to capture initial attention of individuals with social anxiety.

Proto-Object Based Saliency Model With Texture Detection Channel

The amount of visual information projected from the retina to the brain exceeds the information processing capacity of the latter. Attention, therefore, functions as a filter to highlight important information at multiple stages of the visual pathway that requires further and more detailed analysis. Among other functions, this determines where to fixate since only the fovea allows for high resolution imaging. Visual saliency modeling, i.e. understanding how the brain selects important information to analyze further and to determine where to fixate next, is an important research topic in computational neuroscience and computer vision. Most existing bottom-up saliency models use low-level features such as intensity and color, while some models employ high-level features, like faces. However, little consideration has been given to mid-level features, such as texture, for visual saliency models. In this paper, we extend a biologically plausible proto-object based saliency model by adding simple texture channels which employ nonlinear operations that mimic the processing performed by primate visual cortex. The extended model shows statistically significant improved performance in predicting human fixations compared to the previous model. Comparing the performance of our model with others on publicly available benchmarking datasets, we find that our biologically plausible model matches the performance of other models, even though those were designed entirely for maximal performance with little regard to biological realism.

The anger superiority effect revisited: a visual crowding task

Visual search studies have shown that threatening facial expressions are more efficiently detected among a crowd of distractor faces than nonthreatening expressions, known as the anger superiority effect (ASE). However, the opposite finding has also been documented. The present study investigated the ASE in the visual periphery with a visual crowding task. In the study, the target face either appeared alone (uncrowded condition) or was crowded by four neutral or emotional faces (crowded condition). Participants were instructed to determine whether the target face was happy or angry. Experiment 1 showed an ASE when crowded by neutral faces. Intriguingly, this superiority vanished when the target face was crowded by emotional faces that had a different expression from the target as well as when the target face was presented alone. Experiment 2 replicated this result in an independent sample of East Asians (vs. Caucasians in Experiment 1) and thus demonstrated the robustness and cross-cultural consistency of our findings. Together, these results suggest that the ASE in the visual periphery is contingent on task demands induced by visual crowding.

Frontal, Parietal, and Temporal Brain Areas Are Differentially Activated When Disambiguating Potential Objects of Joint Attention

Humans establish joint attention with others by following the other's gaze. Previous work has suggested that a cortical patch (gaze-following patch, GFP) close to the posterior superior temporal sulcus (pSTS) may serve as a link between the extraction of the other's gaze direction and the resulting shifts of attention, mediated by human lateral intraparietal area (hLIP). However, it is not clear how the brain copes with situations in which information on gaze direction alone is insufficient to identify the target object because more than one may lie along the gaze vector. In this fMRI study, we tested human subjects on a paradigm that allowed the identification of a target object based on the integration of the other's gaze direction and information provided by an auditory cue on the relevant object category. Whereas the GFP activity turned out to be fully determined by the use of gaze direction, activity in hLIP reflected the total information needed to pinpoint the target. Moreover, in an exploratory analysis, we found that a region in the inferior frontal junction (IFJ) was sensitive to the total information on the target. An examination of the BOLD time courses in the three identified areas suggests functionally complementary roles. Although the GFP seems to primarily process directional information stemming from the other's gaze, the IFJ may help to analyze the scene when gaze direction and auditory information are not sufficient to pinpoint the target. Finally, hLIP integrates both streams of information to shift attention to distinct spatial locations.

Dynamic Advances in Emotion Processing: Differential Attention towards the Critical Features of Dynamic Emotional Expressions in 7-Month-Old Infants

Infants' visual processing of emotion undergoes significant development across the first year of life, yet our knowledge regarding the mechanisms underlying these advances is limited. Additionally, infant emotion processing is commonly examined using static faces, which do not accurately depict real-world emotional displays. The goal of this study was to characterize 7-month-olds' visual scanning strategies when passively viewing dynamic emotional expressions to examine whether infants modify their scanning patterns depending on the emotion. Eye-tracking measures revealed differential attention towards the critical features (eyes, mouth) of expressions. The eyes captured the greatest attention for angry and neutral faces, and the mouth captured the greatest attention for happy faces. A time-course analysis further elucidated at what point during the trial differential scanning patterns emerged. The current results suggest that 7-month-olds are sensitive to the critical features of emotional expressions and scan them differently depending on the emotion. The scanning patterns presented in this study may serve as a link to understanding how infants begin to differentiate between expressions in the context of emotion recognition.

Differential sensitivity to species- and race-based information in the development of attention orienting and attention holding face biases in infancy

Experience-based biases in face processing can reflect both attention orienting biases that support efficient selection of faces from competing stimuli and attention holding biases that allow for detailed encoding of selected faces. It is well established that infants demonstrate both species- and race-based biases in attention holding. Fewer studies have found species-based, but not race-based, orienting biases in infancy but these studies examined species- and race-based biases separately and measured overall orienting without examining attention to distractors. The present study directly compared 6- and 11-month-old infants' species- and race-based biases in attention holding and orienting to faces. We measured infants' duration of looking and frequency/speed of orienting to own-race, other-race, and monkey faces in multi-item search arrays, and their frequency of orienting to faces and distractors during search. Infants showed expected species- and race-based biases in attention holding but only a species-based bias in overall orienting. However, they also showed reduced orienting to salient distractors in the context of own-race faces. These results suggest that orienting mechanisms mediating face selection are robustly driven by species information while orienting to faces versus distractors during search may also reflect prior learning about frequently experienced own-race faces.

A neurodynamic model of the interaction between color perception and color memory

The memory color effect and Spanish castle illusion have been taken as evidence of the cognitive penetrability of vision. In the same manner, the successful decoding of color-related brain signals in functional neuroimaging studies suggests the retrieval of memory colors associated with a perceived gray object. Here, we offer an alternative account of these findings based on the design principles of adaptive resonance theory (ART). In ART, conscious perception is a consequence of a resonant state. Resonance emerges in a recurrent cortical circuit when a bottom-up spatial pattern agrees with the top-down expectation. When they do not agree, a special control mechanism is activated that resets the network and clears off erroneous expectation, thus allowing the bottom-up activity to always dominate in perception. We developed a color ART circuit and evaluated its behavior in computer simulations. The model helps to explain how traces of erroneous expectations about incoming color are eventually removed from the color perception, although their transient effect may be visible in behavioral responses or in brain imaging. Our results suggest that the color ART circuit, as a predictive computational system, is almost never penetrable, because it is equipped with computational mechanisms designed to constrain the impact of the top-down predictions on ongoing perceptual processing.

Partial Observer Decision Process Model for Crane-Robot Action

The most common use of robots is to effectively decrease the human’s effort with desirable output. In the human-robot interaction, it is essential for both parties to predict subsequent actions based on their present actions so as to well complete the cooperative work. A lot of effort has been devoted in order to attain cooperative work between human and robot precisely. In case of decision making , it is observed from the previous studies that short-term or midterm forecasting have long time horizon to adjust and react. To address this problem, we suggested a new vision-based interaction model. The suggested model reduces the error amplification problem by applying the prior inputs through their features, which are repossessed by a deep belief network (DBN) though Boltzmann machine (BM) mechanism. Additionally, we present a mechanism to decide the possible outcome (accept or reject). The said mechanism evaluates the model on several datasets. Hence, the systems would be able to capture the related information using the motion of the objects. And it updates this information for verification, tracking, acquisition, and extractions of images in order to adapt the situation. Furthermore, we have suggested an intelligent purifier filter (IPF) and learning algorithm based on vision theories in order to make the proposed approach stronger. Experiments show the higher performance of the proposed model compared to the state-of-the-art methods.

What first drives visual attention during the recognition of object-directed actions? The role of kinematics and goal information

The recognition of others' object-directed actions is known to involve the decoding of both the visual kinematics of the action and the action goal. Yet whether action recognition is first guided by the processing of visual kinematics or by a prediction about the goal of the actor remains debated. In order to provide experimental evidence to this issue, the present study aimed at investigating whether visual attention would be preferentially captured by visual kinematics or by action goal information when processing others' actions. In a visual search task, participants were asked to find correct actions (e.g., drinking from glass) among distractor actions. Distractors actions contained grip and/or goal violations and could therefore share the correct goal and/or the correct grip with the target. The time course of fixation proportion on each distractor action has been taken as an indicator of visual attention allocation. Results show that visual attention is first captured by the distractor action with similar goal. Then the withdrawal of visual attention from the action distractor with similar goal suggests a later attentional capture by the action distractor with similar grip. Overall, results are in line with predictive approaches of action understanding, which assume that observers first make a prediction about the actor's goal before verifying this prediction using the visual kinematics of the action.

Extra-foveal Processing of Object Semantics Guides Early Overt Attention During Visual Search

Eye-tracking studies using arrays of objects have demonstrated that some high-level processing of object semantics can occur in extra-foveal vision, but its role on the allocation of early overt attention is still unclear. This eye-tracking visual search study contributes novel findings by examining the role of object-to-object semantic relatedness and visual saliency on search responses and eye-movement behaviour across arrays of increasing size (3, 5, 7). Our data show that a critical object was looked at earlier and for longer when it was semantically unrelated than related to the other objects in the display, both when it was the search target (target-present trials) and when it was a target's semantically related competitor (target-absent trials). Semantic relatedness effects manifested already during the very first fixation after array onset, were consistently found for increasing set sizes, and were independent of low-level visual saliency, which did not play any role. We conclude that object semantics can be extracted early in extra-foveal vision and capture overt attention from the very first fixation. These findings pose a challenge to models of visual attention which assume that overt attention is guided by the visual appearance of stimuli, rather than by their semantics.

Caucasian Infants' Attentional Orienting to Own- and Other-Race Faces

Infants show preferential attention toward faces and detect faces embedded within complex naturalistic scenes. Newborn infants are insensitive to race, but rapidly develop differential processing of own- and other-race faces. In the present study, we investigated the development of attentional orienting toward own- and other-race faces embedded within naturalistic scenes. Infants aged six-, nine- and twelve-months did not show differences in the speed of orienting to own- and other race faces, but other-race faces held infants' visual attention for longer. We also found a clear developmental progression in attentional capture and holding, with older infants orienting to faces faster and fixating them for longer. Results are interpreted within the context of the two-process model of face processing.

Temporal Limitations of the Standard Leaky Integrate and Fire Model

Itti and Koch's Saliency Model has been used extensively to simulate fixation selection in a variety of tasks from visual search to simple reaction times. Although the Saliency Model has been tested for its spatial prediction of fixations in visual salience, it has not been well tested for their temporal accuracy. Visual tasks, like search, invariably result in a positively skewed distribution of saccadic reaction times over large numbers of samples, yet we show that the leaky integrate and fire (LIF) neuronal model included in the classic implementation of the model tends to produce a distribution shifted to shorter fixations (in comparison with human data). Further, while parameter optimization using a genetic algorithm and Nelder-Mead method does improve the fit of the resulting distribution, it is still unable to match temporal distributions of human responses in a visual task. Analysis of times for individual images reveal that the LIF algorithm produces initial fixation durations that are fixed instead of a sample from a distribution (as in the human case). Only by aggregating responses over many input images do they result in a distribution, although the form of this distribution still depends on the input images used to create it and not on internal model variability.

Ventromedial Frontal Lobe Damage Alters how Specific Attributes are Weighed in Subjective Valuation

The concept of subjective value is central to current neurobiological views of economic decision-making. Much of this work has focused on signals in the ventromedial frontal lobe (VMF) that correlate with the subjective value of a variety of stimuli (e.g., food, monetary gambles), and are thought to support decision-making. However, the neural processes involved in assessing and integrating value information from the attributes of such complex options remain to be defined. Here, we tested the necessary role of VMF in weighting attributes of naturalistic stimuli during value judgments. We asked how distinct attributes of visual artworks influenced the subjective value ratings of subjects with VMF damage, compared to healthy participants and a frontal lobe damaged control group. Subjects with VMF damage were less influenced by the energy (emotion, complexity) and color radiance (warmth, saturation) of the artwork, while they were similar to control groups in considering saliency, balance and concreteness. These dissociations argue that VMF is critical for allowing certain affective content to influence subjective value, while sparing the influence of perceptual or representational information. These distinctions are important for better defining the often-underspecified concept of subjective value and developing more detailed models of the brain mechanisms underlying decision behavior.

Salience Models: A Computational Cognitive Neuroscience Review

The seminal model by Laurent Itti and Cristoph Koch demonstrated that we can compute the entire flow of visual processing from input to resulting fixations. Despite many replications and follow-ups, few have matched the impact of the original model-so what made this model so groundbreaking? We have selected five key contributions that distinguish the original salience model by Itti and Koch; namely, its contribution to our theoretical, neural, and computational understanding of visual processing, as well as the spatial and temporal predictions for fixation distributions. During the last 20 years, advances in the field have brought up various techniques and approaches to salience modelling, many of which tried to improve or add to the initial Itti and Koch model. One of the most recent trends has been to adopt the computational power of deep learning neural networks; however, this has also shifted their primary focus to spatial classification. We present a review of recent approaches to modelling salience, starting from direct variations of the Itti and Koch salience model to sophisticated deep-learning architectures, and discuss the models from the point of view of their contribution to computational cognitive neuroscience.

Relationships between depressive symptoms and brain responses during emotional movie viewing emerge in adolescence

Affective disorders such as major depression are common but serious illnesses characterized by altered processing of emotional information. Although the frequency and severity of depressive symptoms increase dramatically over the course of childhood and adolescence, much of our understanding of their neurobiological bases comes from work characterizing adults' responses to static emotional information. As a consequence, relationships between depressive brain phenotypes and naturalistic emotional processing, as well as the manner in which these associations emerge over the lifespan, remain poorly understood. Here, we apply static and dynamic inter-subject correlation analyses to examine how brain function is associated with clinical and non-clinical depressive symptom severity in 112 children and adolescents (7-21 years old) who viewed an emotionally evocative clip from the film Despicable Me during functional MRI. Our results reveal that adolescents with greater depressive symptom severity exhibit atypical fMRI responses during movie viewing, and that this effect is stronger during less emotional moments of the movie. Furthermore, adolescents with more similar item-level depressive symptom profiles showed more similar brain responses during movie viewing. In contrast, children's depressive symptom severity and profiles were unrelated to their brain response typicality or similarity. Together, these results indicate a developmental change in the relationships between brain function and depressive symptoms from childhood through adolescence. Our findings suggest that depressive symptoms may shape how the brain responds to complex emotional information in a dynamic manner sensitive to both developmental stage and affective context.

Dual-target hazard perception: Could identifying one hazard hinder a driver's capacity to find a second?

Low-level cognitive processes like visual search are crucial for hazard detection. In dual-target searches, subsequent search misses (SSMs) are known to occur when the identification of one target impedes detection of another that is concurrently presented. Despite the high likelihood of concurrent hazards in busy driving environments, SSMs have not been empirically investigated in driving. In three studies, participants were asked to identify safety-related target(s) in simulated traffic scenes that contained zero, one, or two target(s) of low or high perceptual saliency. These targets were defined as objects or events that would have prevented safe travel in the direction indicated by an arrow preceding the traffic scene. Findings from the pilot study (n = 20) and Experiment 1 (n = 29) demonstrated that detecting one target hindered drivers' abilities to find a second from the same scene. In Experiment 2 (n = 30), explicit instructions regarding the level of risk were manipulated. It was found that search times were affected by the instructions, though SSMs persisted. Implications of SSMs in understanding the causes of some crashes are discussed, as well as future directions to improve ecological and criterion validity and to explore the roles of expertise and cognitive capabilities in multi-hazard detection.

Seeing Beyond Salience and Guidance: The Role of Bias and Decision in Visual Search

Visual search is a popular tool for studying a range of questions about perception and attention, thanks to the ease with which the basic paradigm can be controlled and manipulated. While often thought of as a sub-field of vision science, search tasks are significantly more complex than most other perceptual tasks, with strategy and decision playing an essential, but neglected, role. In this review, we briefly describe some of the important theoretical advances about perception and attention that have been gained from studying visual search within the signal detection and guided search frameworks. Under most circumstances, search also involves executing a series of eye movements. We argue that understanding the contribution of biases, routines and strategies to visual search performance over multiple fixations will lead to new insights about these decision-related processes and how they interact with perception and attention. We also highlight the neglected potential for variability, both within and between searchers, to contribute to our understanding of visual search. The exciting challenge will be to account for variations in search performance caused by these numerous factors and their interactions. We conclude the review with some recommendations for ways future research can tackle these challenges to move the field forward.

Face detection in 2- to 6-month-old infants is influenced by gaze direction and species

Humans detect faces efficiently from a young age. Face detection is critical for infants to identify and learn from relevant social stimuli in their environments. Faces with eye contact are an especially salient stimulus, and attention to the eyes in infancy is linked to the emergence of later sociality. Despite the importance of both of these early social skills-attending to faces and attending to the eyes-surprisingly little is known about how they interact. We used eye tracking to explore whether eye contact influences infants' face detection. Longitudinally, we examined 2-, 4-, and 6-month-olds' (N = 65) visual scanning of complex image arrays with human and animal faces varying in eye contact and head orientation. Across all ages, infants displayed superior detection of faces with eye contact; however, this effect varied as a function of species and head orientation. Infants were more attentive to human than animal faces and were more sensitive to eye and head orientation for human faces compared to animal faces. Unexpectedly, human faces with both averted heads and eyes received the most attention. This pattern may reflect the early emergence of gaze following-the ability to look where another individual looks-which begins to develop around this age. Infants may be especially interested in averted gaze faces, providing early scaffolding for joint attention. This study represents the first investigation to document infants' attention patterns to faces systematically varying in their attentional states. Together, these findings suggest that infants develop early, specialized functional conspecific face detection.

Attention Based Visual Analysis for Fast Grasp Planning With a Multi-Fingered Robotic Hand

We present an attention based visual analysis framework to compute grasp-relevant information which helps to guide grasp planning using a multi-fingered robotic hand. Our approach uses a computational visual attention model to locate regions of interest in a scene and employ a deep convolutional neural network to detect grasp type and grasp attention point for a sub-region of the object in a region of interest. We demonstrate the proposed framework with object grasping tasks, in which the information generated from the proposed framework is used as prior information to guide grasp planning. The effectiveness of the proposed approach is evaluated in both simulation experiments and real-world experiments. Experimental results show that the proposed framework can not only speed up grasp planning with more stable configurations, but also handle unknown objects. Furthermore, our framework can handle cluttered scenarios. A new Grasp Type Dataset (GTD) which includes six commonly used grasp types and covers 12 household objects is also presented.

The face-in-the-crowd effect: Threat detection versus iso-feature suppression and collinear facilitation

Are people biologically prepared for the rapid detection of threat posed by an angry facial expression, even when it is conveyed in the form of a schematic line drawing? Based on visual search times, the current literature would suggest that the answer is yes. But are there low-level explanations for this effect? Here, we present visual search results for schematic faces using current best practice, based on a concentric search array and set size manipulation. Using this approach, we replicate the classic search advantage for angry over happy faces. However, we also report a comparable effect when abstract plus- and square-shaped stimuli-derived from the angry and happy schematic faces respectively-are used within the same paradigm. We then go on to demonstrate that, while reduced, the effect remains after removal of the circular surround, bringing us closer to the source of the effect. We explore the possibility that the source of this search asymmetry could be the iso-feature suppression and collinear facilitation model proposed in Li's (1999a, 1999b, and 2002) bottom-up model of saliency. Simulations with this model using the abstract stimuli align with the corresponding behavioral results (i.e., the plus shape was found to be more salient than the square). Given the deliberate similarities between these abstract shapes and the respective face stimuli, we propose that the underlying cause for the asymmetries typically found using schematic faces, may be more related to early visual processing of line orientation than threat detection.

The Changing Landscape: High-Level Influences on Eye Movement Guidance in Scenes

The use of eye movements to explore scene processing has exploded over the last decade. Eye movements provide distinct advantages when examining scene processing because they are both fast and spatially measurable. By using eye movements, researchers have investigated many questions about scene processing. Our review will focus on research performed in the last decade examining: (1) attention and eye movements; (2) where you look; (3) influence of task; (4) memory and scene representations; and (5) dynamic scenes and eye movements. Although typically addressed as separate issues, we argue that these distinctions are now holding back research progress. Instead, it is time to examine the intersections of these seemingly separate influences and examine the intersectionality of how these influences interact to more completely understand what eye movements can tell us about scene processing.

A neural signature of pattern separation in the monkey hippocampus

The CA3 and dentate gyrus (DG) regions of the hippocampus are considered key for disambiguating sensory inputs from similar experiences in memory, a process termed pattern separation. The neural mechanisms underlying pattern separation, however, have been difficult to compare across species: rodents offer robust recording methods with less human-centric tasks, while humans provide complex behavior with less recording potential. To overcome these limitations, we trained monkeys to perform a visual pattern separation task similar to those used in humans while recording activity from single CA3/DG neurons. We find that, when animals discriminate recently seen novel images from similar (lure) images, behavior indicative of pattern separation, CA3/DG neurons respond to lure images more like novel than repeat images. Using a population of these neurons, we are able to classify novel, lure, and repeat images from each other using this pattern of firing rates. Notably, one subpopulation of these neurons is more responsible for distinguishing lures and repeats-the key discrimination indicative of pattern separation.

Unique objects attract attention even when faint

Locally contrasting objects, e.g. a red apple surrounded by green apples, attract attention. Does this generalize to differences in feature space? That is, do unique objects-regardless of their location-stand out from a collection of objects that are similar to one another, even when the unique object has lower local contrast with the background than the other objects? Behavioral data show indeed a preference for unique items but previous experiments enabled viewers to anticipate what response they were "supposed" to give. We developed a new experimental paradigm that minimizes such top-down effects. Pitting local contrast against global uniqueness, we show that unique stimuli attract attention even in not-anticipated, never-seen images, and even when the unique stimuli are faint (low contrast). A computational model explains how competition between objects in feature space favors dissimilar objects over those with similar features. The model explains how humans select unique objects, without a loss of performance on natural scenes.

Infants rapidly detect human faces in complex naturalistic visual scenes

Infants respond preferentially to faces and face-like stimuli from birth, but past research has typically presented faces in isolation or amongst an artificial array of competing objects. In the current study infants aged 3- to 12-months viewed a series of complex visual scenes; half of the scenes contained a person, the other half did not. Infants rapidly detected and oriented to faces in scenes even when they were not visually salient. Although a clear developmental improvement was observed in face detection and interest, all infants displayed sensitivity to the presence of a person in a scene, by displaying eye movements that differed quantifiably across a range of measures when viewing scenes that either did or did not contain a person. We argue that infant's face detection capabilities are ostensibly "better" with naturalistic stimuli and artificial array presentations used in previous studies have underestimated performance.

Mid-level feature contributions to category-specific gaze guidance

Our research has previously shown that scene categories can be predicted from observers' eye movements when they view photographs of real-world scenes. The time course of category predictions reveals the differential influences of bottom-up and top-down information. Here we used these known differences to determine to what extent image features at different representational levels contribute toward guiding gaze in a category-specific manner. Participants viewed grayscale photographs and line drawings of real-world scenes while their gaze was tracked. Scene categories could be predicted from fixation density at all times over a 2-s time course in both photographs and line drawings. We replicated the shape of the prediction curve found previously, with an initial steep decrease in prediction accuracy from 300 to 500 ms, representing the contribution of bottom-up information, followed by a steady increase, representing top-down knowledge of category-specific information. We then computed the low-level features (luminance contrasts and orientation statistics), mid-level features (local symmetry and contour junctions), and Deep Gaze II output from the images, and used that information as a reference in our category predictions in order to assess their respective contributions to category-specific guidance of gaze. We observed that, as expected, low-level salience contributes mostly to the initial bottom-up peak of gaze guidance. Conversely, the mid-level features that describe scene structure (i.e., local symmetry and junctions) split their contributions between bottom-up and top-down attentional guidance, with symmetry contributing to both bottom-up and top-down guidance, while junctions play a more prominent role in the top-down guidance of gaze.

Ultra-rapid object categorization in real-world scenes with top-down manipulations

Humans are able to achieve visual object recognition rapidly and effortlessly. Object categorization is commonly believed to be achieved by interaction between bottom-up and top-down cognitive processing. In the ultra-rapid categorization scenario where the stimuli appear briefly and response time is limited, it is assumed that a first sweep of feedforward information is sufficient to discriminate whether or not an object is present in a scene. However, whether and how feedback/top-down processing is involved in such a brief duration remains an open question. To this end, here, we would like to examine how different top-down manipulations, such as category level, category type and real-world size, interact in ultra-rapid categorization. We have constructed a dataset comprising real-world scene images with a built-in measurement of target object display size. Based on this set of images, we have measured ultra-rapid object categorization performance by human subjects. Standard feedforward computational models representing scene features and a state-of-the-art object detection model were employed for auxiliary investigation. The results showed the influences from 1) animacy (animal, vehicle, food), 2) level of abstraction (people, sport), and 3) real-world size (four target size levels) on ultra-rapid categorization processes. This had an impact to support the involvement of top-down processing when rapidly categorizing certain objects, such as sport at a fine grained level. Our work on human vs. model comparisons also shed light on possible collaboration and integration of the two that may be of interest to both experimental and computational vision researches. All the collected images and behavioral data as well as code and models are publicly available at https://osf.io/mqwjz/.

The Role of Eye Gaze During Natural Social Interactions in Typical and Autistic People

Social interactions involve complex exchanges of a variety of social signals, such as gaze, facial expressions, speech and gestures. Focusing on the dual function of eye gaze, this review explores how the presence of an audience, communicative purpose and temporal dynamics of gaze allow interacting partners to achieve successful communication. First, we focus on how being watched modulates social cognition and behavior. We then show that the study of interpersonal gaze processing, particularly gaze temporal dynamics, can provide valuable understanding of social behavior in real interactions. We propose that the Interpersonal Gaze Processing model, which combines both sensing and signaling functions of eye gaze, provides a framework to make sense of gaze patterns in live interactions. Finally, we discuss how autistic individuals process the belief in being watched and interpersonal dynamics of gaze, and suggest that systematic manipulation of factors modulating gaze signaling can reveal which aspects of social eye gaze are challenging in autism.

What Do Different Evaluation Metrics Tell Us About Saliency Models?

How best to evaluate a saliency model's ability to predict where humans look in images is an open research question. The choice of evaluation metric depends on how saliency is defined and how the ground truth is represented. Metrics differ in how they rank saliency models, and this results from how false positives and false negatives are treated, whether viewing biases are accounted for, whether spatial deviations are factored in, and how the saliency maps are pre-processed. In this paper, we provide an analysis of 8 different evaluation metrics and their properties. With the help of systematic experiments and visualizations of metric computations, we add interpretability to saliency scores and more transparency to the evaluation of saliency models. Building off the differences in metric properties and behaviors, we make recommendations for metric selections under specific assumptions and for specific applications.

Saccade Velocity Driven Oscillatory Network Model of Grid Cells

Grid cells and place cells are believed to be cellular substrates for the spatial navigation functions of hippocampus as experimental animals physically navigated in 2D and 3D spaces. However, a recent saccade study on head fixated monkey has also reported grid-like representations on saccadic trajectory while the animal scanned the images on a computer screen. We present two computational models that explain the formation of grid patterns on saccadic trajectory formed on the novel Images. The first model named Saccade Velocity Driven Oscillatory Network -Direct PCA (SVDON—DPCA) explains how grid patterns can be generated on saccadic space using Principal Component Analysis (PCA) like learning rule. The model adopts a hierarchical architecture. We extend this to a network model viz. Saccade Velocity Driven Oscillatory Network—Network PCA (SVDON-NPCA) where the direct PCA stage is replaced by a neural network that can implement PCA using a neurally plausible algorithm. This gives the leverage to study the formation of grid cells at a network level. Saccade trajectory for both models is generated based on an attention model which attends to the salient location by computing the saliency maps of the images. Both models capture the spatial characteristics of grid cells such as grid scale variation on the dorso-ventral axis of Medial Entorhinal cortex. Adding one more layer of LAHN over the SVDON-NPCA model predicts the Place cells in saccadic space, which are yet to be discovered experimentally. To the best of our knowledge, this is the first attempt to model grid cells and place cells from saccade trajectory.

Indoor Scene and Position Recognition Based on Visual Landmarks Obtained from Visual Saliency without Human Effect

Numerous autonomous robots are used not only for factory automation as labor saving devices, but also for interaction and communication with humans in our daily life. Although superior compatibility for semantic recognition of generic objects provides wide applications in a practical use, it is still a challenging task to create an extraction method that includes robustness and stability against environmental changes. This paper proposes a novel method of scene and position recognition based on visual landmarks (VLs) used for an autonomous mobile robot in an environment living with humans. The proposed method provides a mask image of human regions using histograms of oriented gradients (HOG). The VL features are described with accelerated KAZE (AKAZE) after extracting conspicuous regions obtained using saliency maps (SMs). The experimentally obtained results using leave-one-out cross validation (LOOCV) revealed that recognition accuracy of high-saliency feature points was higher than that of low-saliency feature points. We created our original benchmark datasets using a mobile robot. The recognition accuracy evaluated using LOOCV reveals 49.9% for our method, which is 3.2 percentage points higher than the accuracy of the comparison method without HOG detectors. The analysis of false recognition using a confusion matrix examines false recognition occurring in neighboring zones. This trend is reduced according to zone separations.

Affectively Biased Competition: Sustained Attention is Tuned to Rewarding Expressions and is Not Modulated by Norepinephrine Receptor Gene Variant

It is well established that emotionally salient stimuli evoke greater visual cortex activation than neutral ones, and can distract attention from competing tasks. Yet less is known about underlying neurobiological processes. As a proxy of population level biased competition, EEG steady-state visual evoked potentials are sensitive to competition effects from salient stimuli. Here we wished to examine whether individual differences in norepinephrine activity play a role in emotionally-biased competition.

Our previous research has found robust effects of a common variation in the ADRA2B gene, coding for alpha2B norepinephrine (NE) receptors, on emotional modulation of attention and memory. In the present study, EEG was collected while 87 carriers of the ADRA2B deletion variant and 95 non-carriers (final sample) performed a change detection task in which target gratings (gabor patches) were superimposed directly over angry, happy, and neutral faces. Participants indicated the number of phase changes (0–3) in the target. Overlapping targets and distractors were flickered at a distinct driving frequencies. Relative EEG power for faces vs. targets at the driving frequency served as an index of cortical resources allocated to each of the competing stimuli. Deletion carriers and non-carriers were randomly assigned to Discovery and Replication samples and reliability of results across samples was assessed before the groups were combined for greater power.

Overall happy faces evoked higher competition than angry or neutral faces; however, we observed no hypothesized effects of ADRA2B. Increased competition from happy faces was not due to the effect of low-level visual features or individuals low in social anxiety. Our results indicate that emotionally biased competition during sustained attention, while reliably observed in young adults, is not influenced by commonly observed individual differences linked to NE receptor function. They further indicate an overall pattern of affectively-biased competition for happy faces, which we interpret in relation to previously observed boundary conditions.

Deep Synthesis of Realistic Medical Images: A Novel Tool in Clinical Research and Training

Making clinical decisions based on medical images is fundamentally an exercise in statistical decision-making. This is because in this case, the decision-maker must distinguish between image features that are clinically diagnostic (i.e., signal) from a large amount of non-diagnostic features. (i.e., noise). To perform this task, the decision-maker must have learned the underlying statistical distributions of the signal and noise to begin with. The same is true for machine learning algorithms that perform a given diagnostic task. In order to train and test human experts or expert machine systems in any diagnostic or analytical task, it is advisable to use large sets of images, so as to capture the underlying statistical distributions adequately. Large numbers of images are also useful in clinical and scientific research about the underlying diagnostic process, which remains poorly understood. Unfortunately, it is often difficult to obtain medical images of given specific descriptions in sufficiently large numbers. This represents a significant barrier to progress in the arenas of clinical care, education, and research. Here we describe a novel methodology that helps overcome this barrier. This method leverages the burgeoning technologies of deep learning (DL) and deep synthesis (DS) to synthesize medical images de novo. We provide a proof-of-principle of this approach using mammograms as an illustrative case. During the initial, prerequisite DL phase of the study, we trained a publicly available deep learning neural network (DNN), using open-sourced, radiologically vetted mammograms as labeled examples. During the subsequent DS phase of the study, the fully trained DNN was made to synthesize, de novo, images that capture the image statistics of a given input image. The resulting images indicated that our DNN was able to faithfully capture the image statistics of visually diverse sets of mammograms. We also briefly outline rigorous psychophysical testing methods to measure the extent to which synthesized mammography were sufficiently alike their original counterparts to human experts. These tests reveal that mammography experts fail to distinguish synthesized mammograms from their original counterparts at a statistically significant level, suggesting that the synthesized images were sufficiently realistic. Taken together, these results demonstrate that deep synthesis has the potential to be impactful in all fields in which medical images play a key role, most notably in radiology and pathology.