The spatial resolution of visual attention

What is visible across the visual field?

It is sometimes claimed that because the resolution and sensitivity of visual perception are better in the fovea than in the periphery, peripheral vision cannot support the same kinds of colour and sharpness percepts as foveal vision. The fact that a scene nevertheless seems colourful and sharp throughout the visual field then poses a puzzle. In this study, I use a detailed model of human spatial vision to estimate the visibility of certain properties of natural scenes, including aspects of colourfulness, sharpness, and blurriness, across the visual field. The model is constructed to reproduce basic aspects of human contrast and colour sensitivity over a range of retinal eccentricities. I apply the model to colourful, complex natural scene images, and estimate the degree to which colour and edge information are present in the model's representation of the scenes. I find that, aside from the intrinsic drift in the spatial scale of the representation, there are not large qualitative differences between foveal and peripheral representations of 'colourfulness' and 'sharpness'.

Individual differences in crowding predict visual search performance

Visual search is an integral part of human behavior and has proven important to understanding mechanisms of perception, attention, memory, and oculomotor control. Thus far, the dominant theoretical framework posits that search is mainly limited by covert attentional mechanisms, comprising a central bottleneck in visual processing. A different class of theories seeks the cause in the inherent limitations of peripheral vision, with search being constrained by what is known as the functional viewing field (FVF). One of the major factors limiting peripheral vision, and thus the FVF, is crowding. We adopted an individual differences approach to test the prediction from FVF theories that visual search performance is determined by the efficacy of peripheral vision, in particular crowding. Forty-four participants were assessed with regard to their sensitivity to crowding (as measured by critical spacing) and their search efficiency (as indicated by manual responses and eye movements). This revealed substantial correlations between the two tasks, as stronger susceptibility to crowding was predictive of slower search, more eye movements, and longer fixation durations. Our results support FVF theories in showing that peripheral vision is an important determinant of visual search efficiency.

A critical systematic review of the Neurotracker perceptual-cognitive training tool

In this systematic review, we evaluate the scientific evidence behind “Neurotracker,” one of the most popular perceptual-cognitive training tools in sports. The tool, which is also used in rehabilitation and aging research to examine cognitive abilities, uses a 3D multiple object-tracking (MOT) task. In this review, we examine Neurotracker from both a sport science and a basic science perspective. We first summarize the sport science debate regarding the value of general cognitive skill training, based on tools such as Neurotracker, versus sport-specific skill training. We then consider the several hundred MOT publications in cognitive and vision science from the last 30 years that have investigated cognitive functions and object tracking processes. This literature suggests that the abilities underlying object tracking are not those advertised by the Neurotracker manufacturers. With a systematic literature search, we scrutinize the evidence for whether general cognitive skills can be tested and trained with Neurotracker and whether these trained skills transfer to other domains. The literature has major limitations, for example a total absence of preregistered studies, which makes the evidence for improvements for working memory and sustained attention very weak. For other skills as well, the effects are mixed. Only three studies investigated far transfer to ecologically valid tasks, two of which did not find any effect. We provide recommendations for future Neurotracker research to improve the evidence base and for making better use of sport and basic science findings.

Forecasting People Trajectories and Head Poses by Jointly Reasoning on Tracklets and Vislets

In this article, we explore the correlation between people trajectories and their head orientations. We argue that people trajectory and head pose forecasting can be modelled as a joint problem. Recent approaches on trajectory forecasting leverage short-term trajectories (aka tracklets) of pedestrians to predict their future paths. In addition, sociological cues, such as expected destination or pedestrian interaction, are often combined with tracklets. In this article, we propose MiXing-LSTM (MX-LSTM) to capture the interplay between positions and head orientations (vislets) thanks to a joint unconstrained optimization of full covariance matrices during the LSTM backpropagation. We additionally exploit the head orientations as a proxy for the visual attention, when modeling social interactions. MX-LSTM predicts future pedestrians location and head pose, increasing the standard capabilities of the current approaches on long-term trajectory forecasting. Compared to the state-of-the-art, our approach shows better performances on an extensive set of public benchmarks. MX-LSTM is particularly effective when people move slowly, i.e., the most challenging scenario for all other models. The proposed approach also allows for accurate predictions on a longer time horizon.

Adaptive target enhancement determines levels of guidability in Multiple Object Tracking

Flexible resource models suggest that attentional deployment in Multiple Object Tracking either shows in target enhancement or distractor inhibition or both. In order to gauge the influence of these processes we manipulated distinctiveness of targets and distractors along a single dimension by finely grading gray values of objects. We established that a quantitative increase in distinctiveness results in a quantitative increase in tracking performance thus demonstrating a new finding. Further increases of distinctiveness beyond a certain degree produced no further improvements in tracking. This effect, however, was moderated by levels of task difficulty thus providing evidence for the relativity of this effect. With higher task difficulty higher degrees of distinctiveness still resulted in a significant performance gain (Experiment 1). A follow up experiment generalized this finding. By manipulating speed of objects, a situation with a yet higher task difficulty could be established showing that an increase of an already striking distinctiveness can produce an even further performance gain - provided higher task difficulty (Experiment 2). Finally we tested the hypothesis that target enhancement and distractor inhibition are influenced by feature distinctiveness. Although we found possible indications for distractor inhibition, we could not demonstrate an effect of feature distinctiveness on distractor inhibition. Target enhancement, however, was substantially influenced by variation of feature distinctiveness (Experiment 3).

Binocular summation is affected by crowding and tagging

In perceptual crowding, a letter easily recognized on its own, becomes unrecognizable if it is surrounded by other letters, an effect that confers a limit on the visual processing. Models assume that crowding is a hallmark of the periphery but that it is almost absent in the fovea. However, recently it was shown that crowding occurs in the fovea of people with an abnormal development of functional vision (amblyopia), when the stimulus is presented for a very short time. When targets and flankers are dissimilar, the crowding is reduced (tagging). Since a combination of binocular inputs increases the processing load, we investigated whether color tagging the target reduces crowding in the fovea of subjects with normal vision and determined how crowding is combined with binocular vision. The crowding effect at the fovea was significantly reduced by tagging with a color target. Interestingly, whereas binocular summation for a single letter was expected to be about 40%, it was significantly reduced and almost absent under crowding conditions. Our results are consistent with the notion that the crowding effect produces a high processing load on visual processing, which interferes with other processes such as binocular summation. We assume that the tagging effect in our experiment improved the subject's abilities (sensitivity and RT) by creating a "segmentation", i.e., a visual simulated separation between the target letter and the background. Interestingly, tagging the target with a distinct color can eliminate or reduce the crowding effect and consequently, binocular summation recovers.

The Focal Attention Window Size Explains Letter Substitution Errors in Reading

Acquired Neglect Dyslexia is often associated with right-hemisphere brain damage and is mainly characterized by omissions and substitutions in reading single words. Martelli et al. proposed in 2011 that these two types of error are due to different mechanisms. Omissions should depend on neglect plus an oculomotor deficit, whilst substitutions on the difficulty with which the letters are perceptually segregated from each other (i.e., crowding phenomenon). In this study, we hypothesized that a deficit of focal attention could determine a pathological crowding effect, leading to imprecise letter identification and consequently substitution errors. In Experiment 1, three brain-damaged patients, suffering from peripheral dyslexia, mainly characterized by substitutions, underwent an assessment of error distribution in reading pseudowords and a T detection task as a function of cue size and timing, in order to measure focal attention. Each patient, when compared to a control group, showed a deficit in adjusting the attentional focus. In Experiment 2, a group of 17 right-brain-damaged patients were asked to perform the focal attention task and to read single words and pseudowords as a function of inter-letter spacing. The results allowed us to confirm a more general association between substitution-type reading errors and the performance in the focal attention task.

Mixture model investigation of the inner-outer asymmetry in visual crowding reveals a heavier weight towards the visual periphery

Crowding, the failure to identify a peripheral item in clutter, is an essential bottleneck in visual information processing. A hallmark characteristic of crowding is the inner-outer asymmetry in which the outer flanker (more eccentric) produces stronger interference than the inner one (closer to the fovea). We tested the contribution of the inner-outer asymmetry to the pattern of crowding errors in a typical radial crowding display in which both flankers are presented simultaneously on the horizontal meridian. In two experiments, observers were asked to estimate the orientation of a Gabor target. Instead of the target, observers reported the outer flanker much more frequently than the inner one. When the target was the outer Gabor, crowding was reduced. Furthermore, when there were four flankers, two on each side of the target, observers misreported the outer flanker adjacent to the target, not the outermost flanker. Model comparisons suggested that orientation crowding reflects sampling over a weighted sum of the represented features, in which the outer flanker is more heavily weighted compared to the inner one. Our findings reveal a counterintuitive phenomenon: in a radial arrangement of orientation crowding, within a region of selection, the outer item dominates appearance more than the inner one.

Redundancy masking: The loss of repeated items in crowded peripheral vision

Crowding is the deterioration of target identification in the presence of neighboring objects. Recent studies using appearance-based methods showed that the perceived number of target elements is often diminished in crowding. Here we introduce a related type of diminishment in repeating patterns (sets of parallel lines), which we term “redundancy masking.” In four experiments, observers were presented with arrays of small numbers of lines centered at 10° eccentricity. The task was to indicate the number of lines. In Experiment 1, spatial characteristics of redundancy masking were examined by varying the inter-line spacing. We found that redundancy masking decreased with increasing inter-line spacing and ceased at spacings of approximately 0.25 times the eccentricity. In Experiment 2, we assessed whether the strength of redundancy masking differed between radial and tangential arrangements of elements as it does in crowding. Redundancy masking was strong with radially arranged lines (horizontally arranged vertical lines), and absent with tangentially arranged lines (vertically arranged horizontal lines). In Experiment 3, we investigated whether target size (line width and length) modulated redundancy masking. There was an effect of width: Thinner lines yielded stronger redundancy masking. We did not find any differences between the tested line lengths. In Experiment 4, we varied the regularity of the line arrays by vertically or horizontally jittering the positions of the lines. Redundancy masking was strongest with regular spacings and weakened with decreasing regularity. Our experiments show under which conditions whole items are lost in crowded displays, and how this redundancy masking resembles—and partly diverges from—crowded identification. We suggest that redundancy masking is a contributor to the deterioration of performance in crowded displays with redundant patterns.

Attentional tracking takes place over perceived rather than veridical positions

Illusions can induce striking differences between perception and retinal input. For instance, a static Gabor with a moving internal texture appears to be shifted in the direction of its internal motion, a shift that increases dramatically when the Gabor itself is also in motion. Here, we ask whether attention operates on the perceptual or physical location of this stimulus. To do so, we generated an attentional tracking task where participants (N = 15) had to keep track of a single target among three Gabors that rotated around a common center in the periphery. During tracking, the illusion was used to make three Gabors appear either shifted away from or toward one another while maintaining the same physical separation. Because tracking performance depends in part on target to distractor spacing, if attention selects targets from perceived positions, performance should be better when the Gabors appear further apart and worse when they appear closer together. We find that tracking performance is superior with greater perceived separation, implying that attentional tracking operates over perceived rather than physical positions.

On the contrast dependence of crowding

Visual clutter affects our ability to see. Objects that would be identifiable on their own may become unrecognizable when presented close together ("crowding"), but the psychophysical characteristics of crowding have resisted simplification. Image properties initially thought to produce crowding have paradoxically yielded unexpected results; for example, adding flanking objects can ameliorate crowding (Manassi, Sayim, & Herzog, 2012; Herzog, Sayim, Chcherov, & Manassi, 2015; Pachai, Doerig, & Herzog, 2016). The resulting theory revisions have been sufficiently complex and specialized as to make it difficult to discern what principles may underlie the observed phenomena. Here, a generalized formulation of simple visual contrast energy is presented, arising from straightforward analyses of center and surround neurons in the early visual stream. Extant contrast measures, such as root mean square contrast, are easily shown to fall out as reduced special cases. The new generalized contrast energy metric surprisingly predicts the principal findings of a broad range of crowding studies. These early crowding phenomena may thus be said to arise predominantly from contrast or are, at least, severely confounded by contrast effects. Note that these findings may be distinct from accounts of other, likely downstream, "configural" or "semantic" instances of crowding, suggesting at least two separate forms of crowding that may resist unification. The new fundamental contrast energy formulation provides a candidate explanatory framework that addresses multiple psychophysical phenomena beyond crowding.

Mesh Saliency via Weakly Supervised Classification-for-Saliency CNN

Recently, effort has been made to apply deep learning to the detection of mesh saliency. However, one major barrier is to collect a large amount of vertex-level annotation as saliency ground truth for training the neural networks. Quite a few pilot studies showed that this task is difficult. In this work, we solve this problem by developing a novel network trained in a weakly supervised manner. The training is end-to-end and does not require any saliency ground truth but only the class membership of meshes. Our Classification-for-Saliency CNN (CfS-CNN) employs a multi-view setup and contains a newly designed two-channel structure which integrates view-based features of both classification and saliency. It essentially transfers knowledge from 3D object classification to mesh saliency. Our approach significantly outperforms the existing state-of-the-art methods according to extensive experimental results. Also, the CfS-CNN can be directly used for scene saliency. We showcase two novel applications based on scene saliency to demonstrate its utility.

Superior Parietal Lobule: A Role in Relative Localization of Multiple Different Elements

Simultanagnosia is an impairment in processing multiple visual elements simultaneously consecutive to bilateral posterior parietal damage, and neuroimaging data have specifically implicated the superior parietal lobule (SPL) in multiple element processing. We previously reported that a patient with focal and bilateral lesions of the SPL performed slower than controls in visual search but only for stimuli consisting of separable lines. Here, we further explored this patient's visual processing of plain object (colored disk) versus object consisting of separable lines (letter), presented in isolation (single object) versus in triplets. Identification of objects was normal in isolation but dropped to chance level when surrounded by distracters, irrespective of eccentricity and spacing. We speculate that this poor performance reflects a deficit in processing objects' relative locations within the triplet (for colored disks), aggravated by a deficit in processing the relative location of each separable line (for letters). Confirming this, performance improved when the patient just had to detect the presence of a specific colored disk within the triplets (visual search instruction), while the inability to identify the middle letter was alleviated when the distracters were identical letters that could be grouped, thereby reducing the number of ways individual lines could be bound.

Incidental encoding of visual information in temporal reference frames in working memory

Visual events are structured in space and time, yet models of visual working memory (VWM) have largely relied on tasks emphasizing spatial aspects. Here, we show that temporal properties of visual events are incidentally encoded along with spatial properties. In five experiments, participants performed change-detection tasks, in which items had unique spatial and temporal coordinates at encoding. Crucially, neither space nor time was task-relevant. The key manipulation concerned the retrieval context: The test array was identical to the memory array either in its entire spatiotemporal structure, or only its spatial or temporal structure. Removing spatial or temporal information at retrieval resulted in costs, indicating that memory relied on both spatial and temporal context in which items were initially perceived. Encoding of spatiotemporal structure occurred incidentally, not strategically, as it was robust even when the retrieval context was perfectly predictable. However, spatial and temporal inter-item spacings influenced the weighting of spatial and temporal information: It favoured the domain in which items were more widely spaced, facilitating their individuation and, likely, access to representations. Across individuals, the weighting of spatial and temporal information varied substantially, but it remained consistent across sessions, suggesting stable preferences for coding in the spatial or temporal domain. No comparable incidental encoding occurred for other task-irrelevant feature dimensions (size or colour). We propose that temporal structure serves as fundamental a function in VWM as spatial structure, scaffolding events that unfold over time.

Crowding and attention in a framework of neural network model

In this article, I present a framework that would accommodate the classic ideas of visual information processing together with more recent computational approaches. I used the current knowledge about visual crowding, capacity limitations, attention, and saliency to place these phenomena within a standard neural network model. I suggest some revisions to traditional mechanisms of attention and feature integration that are required to fit better into this framework. The results allow us to explain some apparent theoretical controversies in vision research, suggesting a rationale for the limited spatial extent of crowding, a role of saliency in crowding experiments, and several amendments to the feature integration theory. The scheme can be elaborated or modified by future research.

Number of flankers influences foveal crowding and contour interaction differently

Nearby flanking objects degrade visual resolution. If the flankers are similar to the acuity target, this influence is called crowding (CW), whereas if the flanking stimuli are simple bars then the phenomenon is known as contour interaction (CI). The aim of this study was to compare the influence of the number and position of flankers on foveal CW and CI to investigate possible differences in mechanism of these two effects. Five normal observers viewed single, foveally presented Sloan letters surrounded by 1, 2 or 4 flankers (either a Sloan letter or one-stroke-width bars), presented at several edge-to-edge separations. Single flankers were presented in the right, left, top or bottom position, 2 flankers were placed equally to the right and left or top and bottom of the central target, and 4 flankers were equally spaced in all four directions. Percent correct letter identification was determined for each type, number, position and separation of flankers and confusion matrices were constructed for separations equal to 20% and 100% letter width. Increasing the number of flankers caused an increase in the magnitude of both phenomena. CW showed a greater magnitude than CI for higher numbers of flankers. Analysis of confusion matrices suggests that in addition to the edge-to-edge interaction that appears to mediate CI, letter substitution and feature pooling contribute significantly to CW when higher numbers of flankers are presented. Foveal CW is more strongly influenced by an increase in the number of flankers than CI, which can be explained by the presence of additional interaction effects.

Anatomic and functional asymmetries interactively shape human early visual cortex responses

Early visual processing is surprisingly flexible even in the adult brain. This flexibility involves both long-term structural plasticity and online adaptations conveyed by top-down feedback. Although this view is supported by rich evidence from both human behavioral studies and invasive electrophysiology in nonhuman models, it has proven difficult to close the gap between species. In particular, it remains debated whether noninvasive measures of neural activity can capture top-down modulations of the earliest stages of processing in the human visual cortex. We previously reported modulations of retinotopic C1, the earliest component of the human visual evoked potential. However, these effects were selectively observed in the upper visual field (UVF). Here we test whether this asymmetry is linked to an interaction between differences in spatial resolution across the visual field and the specific stimuli used in previous studies. We measured visual evoked potentials in response to task-irrelevant, high-contrast textures of different densities in a comparatively large sample of healthy volunteers (N = 31) using high-density electroencephalogram. Our results show differential response profiles for upper and lower hemifields, with UVF responses saturating at higher stimulus densities. In contrast, lower visual field responses did not increase, and even showed a tendency toward a decrease at the highest density tested. We propose that these findings reflect feature- and task-specific pooling of signals from retinotopic regions with different sensitivity profiles. Such complex interactions between anatomic and functional asymmetries need to be considered to resolve whether human early visual cortex activity is modulated by top-down factors.

Attention as a Unitary Concept

In this paper, I discuss attention in terms of selecting visual information and acting on it. Selection has been taken as a bedrock concept in attention research since James (1890). Selective attention guides action by privileging some things at the expense of others. I formalize this notion with models which capture the relationship between input and output under the control of spatial and temporal attention, by attenuating or discarding certain inputs and by weighing energetic costs, speed, and accuracy in meeting pre-chosen goals. Examples are given from everyday visually guided actions, and from modeling data obtained from visual searches through temporal and spatial arrays and related research. The relation between selection, as defined here, and other forms of attention is discussed at the end.

The anger superiority effect revisited: a visual crowding task

Visual search studies have shown that threatening facial expressions are more efficiently detected among a crowd of distractor faces than nonthreatening expressions, known as the anger superiority effect (ASE). However, the opposite finding has also been documented. The present study investigated the ASE in the visual periphery with a visual crowding task. In the study, the target face either appeared alone (uncrowded condition) or was crowded by four neutral or emotional faces (crowded condition). Participants were instructed to determine whether the target face was happy or angry. Experiment 1 showed an ASE when crowded by neutral faces. Intriguingly, this superiority vanished when the target face was crowded by emotional faces that had a different expression from the target as well as when the target face was presented alone. Experiment 2 replicated this result in an independent sample of East Asians (vs. Caucasians in Experiment 1) and thus demonstrated the robustness and cross-cultural consistency of our findings. Together, these results suggest that the ASE in the visual periphery is contingent on task demands induced by visual crowding.

Target-flanker similarity effects reflect image segmentation not perceptual grouping

When responding to the identity of a visual target, nearby stimuli (flankers) that are associated with the same response as the target cause faster and more accurate responding than flankers that are associated with different responses. Because this flanker-congruence effect (FCE) decreases with increasing target-flanker separation, it was thought to reflect limited precision of spatial selection mechanisms. Later studies, however, showed that FCEs are larger when the target and flankers are the same color compared to when they are different colors. This led to the group selection hypothesis, which states that flankers are perceptually grouped with the target and are obligatorily selected along with it, regardless of spatial separation. An alternative hypothesis, the image segmentation hypothesis, states that feature differences facilitate the segmentation of visual information into relevant and irrelevant parts, thereby mitigating the limitations of spatial precision of selection mechanisms. We test between these hypotheses using a design in which targets and flankers are grouped or not grouped, while holding feature differences in the stimulus constant. Contrary to earlier results, we found that same-colored flankers do not yield larger FCEs than different-colored flankers when feature differences are held constant. We conclude that similarity effects on the FCE reflect differential support for image segmentation, on which selection depends, rather than the obligatory selection of perceptually grouped flankers and targets.

Excessive visual crowding effects in developmental dyscalculia

Visual crowding refers to the inability to identify objects when surrounded by other similar items. Crowding-like mechanisms are thought to play a key role in numerical perception by determining the sensory mechanisms through which ensembles are perceived. Enhanced visual crowding might hence prevent the normal development of a system involved in segregating and perceiving discrete numbers of items and ultimately the acquisition of more abstract numerical skills. Here, we investigated whether excessive crowding occurs in developmental dyscalculia (DD), a neurodevelopmental disorder characterized by difficulty in learning the most basic numerical and arithmetical concepts, and whether it is found independently of associated major reading and attentional difficulties. We measured spatial crowding in two groups of adult individuals with DD and control subjects. In separate experiments, participants were asked to discriminate the orientation of a Gabor patch either in isolation or under spatial crowding. Orientation discrimination thresholds were comparable across groups when stimuli were shown in isolation, yet they were much higher for the DD group with respect to the control group when the target was crowded by closely neighbouring flanking gratings. The difficulty in discriminating orientation (as reflected by the combination of accuracy and reaction times) in the DD compared to the control group persisted over several larger target flanker distances. Finally, we found that the degree of such spatial crowding correlated with impairments in mathematical abilities even when controlling for visual attention and reading skills. These results suggest that excessive crowding effects might be a characteristic of DD, independent of other associated neurodevelopmental disorders.

Changes in eye movement parameters in the presence of an artificial central scotoma

BACKGROUND

Central vision loss, such as in the case of age-related macular degeneration (AMD), has a a major negative impact on patients' quality of life. However, some patients have shown spontaneous adaptive strategies development, mostly relying on their peripheral vision.

OBJECTIVE

This study assesses eye movement and eccentric visual function adaptive behaviors of a healthy population in the presence of simulated central vision loss. We wished to determine how central vision loss affects eye movements, specifically the foveal-target alignment.

METHODS

Fifteen healthy participants (7 females, M = 21.69, SD = 2.13) discriminated the orientation of a Gabor relative to the vertical located at 12 deg of eccentricity to the right of fixation, in the presence of a gaze-contingent artificial central scotoma either visible or invisible. The artificial central scotoma was 4° diameter in order to simulate an earlier stage of degenerative disease while still impairing foveal vision. The target's orientation varied between 10° counter-clockwise and 10° clockwise. Each participant performed four blocks of 75 trials each per day over 10 days, the first day being a baseline without scotoma.

RESULTS

We found changes in the endpoints of the 1st saccade over the practice days. The most common pattern was a gradual upward shift. We also observed a significant increase in discrimination performance over the 9 days of practice. We did not find any difference linked to the scotoma types.

CONCLUSIONS

These findings suggest that the presence of an artificial central scotoma combined with a challenging discrimination task induces both changes in saccade planning mechanisms, resulting in a new eccentric-target alignment, and improvements in eccentric visual functions. This demonstrates the potential of this research paradigm to understand and potentially improve visual function in patients with central vision loss.

Labor division in joint tasks: Humans maximize use of their individual attentional capacities

In daily life, humans frequently perform visuospatial tasks together (e.g., visual search) and distribute the labor in such tasks. Previous research has shown that humans prefer a left and right labor division in a joint multiple object tracking (MOT) task. Yet, findings from studies investigating individuals' tracking ability suggest attentional capacities may be more maximally used with a top and bottom labor division. We investigated whether co-actors' labor division preference is influenced by how they are seated (neighboring vs. opposite of each other) or how the MOT task is displayed (portrait vs. landscape). We find that pairs attain a higher performance using a top and bottom labor division and preferred this labor division compared to a left and right division. This preference was unaffected by the seating arrangement. For the landscape display, however, we find that participants no longer attain a higher performance for the top and bottom labor division and accordingly participants' preference for this labor division was greatly reduced as well. Overall, we propose that co-actors are sensitive to changes within their environment, which allows them to choose a labor division that maximizes use of their individual attentional capacities.

Examining the roles of working memory and visual attention in multiple object tracking expertise

When tracking multiple moving targets among visually similar distractors, human observers are capable of distributing attention over several spatial locations. It is unclear, however, whether capacity limitations or perceptual-cognitive abilities are responsible for the development of expertise in multiple object tracking. Across two experiments, we examined the role of working memory and visual attention in tracking expertise. In Experiment 1, individuals who regularly engaged in object tracking sports (soccer and rugby) displayed improved tracking performance, relative to non-tracking sports (swimming, rowing, running) (p = 0.02, ηp2 = 0.163), but no differences in gaze strategy (ps > 0.31). In Experiment 2, participants trained on an adaptive object tracking task showed improved tracking performance (p = 0.005, d = 0.817), but no changes in gaze strategy (ps > 0.07). They did, however, show significant improvement in a working memory transfer task (p < 0.001, d = 0.970). These findings indicate that the development of tracking expertise is more closely linked to processing capacity limits than perceptual-cognitive strategies.

Crowding in humans is unlike that in convolutional neural networks

Object recognition is a primary function of the human visual system. It has recently been claimed that the highly successful ability to recognise objects in a set of emergent computer vision systems-Deep Convolutional Neural Networks (DCNNs)-can form a useful guide to recognition in humans. To test this assertion, we systematically evaluated visual crowding, a dramatic breakdown of recognition in clutter, in DCNNs and compared their performance to extant research in humans. We examined crowding in three architectures of DCNNs with the same methodology as that used among humans. We manipulated multiple stimulus factors including inter-letter spacing, letter colour, size, and flanker location to assess the extent and shape of crowding in DCNNs. We found that crowding followed a predictable pattern across architectures that was different from that in humans. Some characteristic hallmarks of human crowding, such as invariance to size, the effect of target-flanker similarity, and confusions between target and flanker identities, were completely missing, minimised or even reversed. These data show that DCNNs, while proficient in object recognition, likely achieve this competence through a set of mechanisms that are distinct from those in humans. They are not necessarily equivalent models of human or primate object recognition and caution must be exercised when inferring mechanisms derived from their operation.

Multiple-target tracking in human and machine vision

Humans are able to track multiple objects at any given time in their daily activities—for example, we can drive a car while monitoring obstacles, pedestrians, and other vehicles. Several past studies have examined how humans track targets simultaneously and what underlying behavioral and neural mechanisms they use. At the same time, computer-vision researchers have proposed different algorithms to track multiple targets automatically. These algorithms are useful for video surveillance, team-sport analysis, video analysis, video summarization, and human–computer interaction. Although there are several efficient biologically inspired algorithms in artificial intelligence, the human multiple-target tracking (MTT) ability is rarely imitated in computer-vision algorithms. In this paper, we review MTT studies in neuroscience and biologically inspired MTT methods in computer vision and discuss the ways in which they can be seen as complementary.

Multiple-target tracking (MTT) is a challenging task vital for both a human’s daily life and for many artificial intelligent systems, such as those used for urban traffic control. Neuroscientists are interested in discovering the underlying neural mechanisms that successfully exploit cognitive resources, e.g., spatial attention or memory, during MTT. Computer-vision specialists aim to develop powerful MTT algorithms based on advanced models or data-driven computational methods. In this paper, we review MTT studies from both communities and discuss how findings from cognitive studies can inspire developers to construct higher performing MTT algorithms. Moreover, some directions have been proposed through which MTT algorithms could raise new questions in the cognitive science domain, and answering them can shed light on neural processes underlying MTT.

Flocking in complex environments-Attention trade-offs in collective information processing

The ability of biological and artificial collectives to outperform solitary individuals in a wide variety of tasks depends crucially on the efficient processing of social and environmental information at the level of the collective. Here, we model collective behavior in complex environments with many potentially distracting cues. Counter-intuitively, large-scale coordination in such environments can be maximized by strongly limiting the cognitive capacity of individuals, where due to self-organized dynamics the collective self-isolates from disrupting information. We observe a fundamental trade-off between coordination and collective responsiveness to environmental cues. Our results offer important insights into possible evolutionary trade-offs in collective behavior in biology and suggests novel principles for design of artificial swarms exploiting attentional bottlenecks.

Understanding how consensus is reached and information is processed within a collective is fundamental to many aspects of social dynamics in animals and humans. It is widely accepted that high connectivity among individuals facilitates group consensus, and being in a group provides benefits to individuals through social information about the environment provided by other group members. We show that this does not hold for collectives in complex environments: Limited attention capacity, that severely reduces connectivity among individuals, is highly beneficial for global coordination. However, this comes at a price: Collectives outperform isolated individuals in responding to the environment only at sufficiently high attention capacities, where global coordination breaks down. Thus, we demonstrate a fundamental trade-off in collective behavior between social coordination and responsiveness to environmental cues. Our work demonstrates the importance of sensory and cognitive limitations for the emergence and function of animal collectives, and poses fundamental questions about co-evolution of social behavior and individual attention capacity. The observed trade-off in collective information processing has implications for human social systems and for the design of robotic swarms operating in complex environments.

Temporal predictability does not impact attentional blink performance: effects of fixed vs. random inter-trial intervals

Background

Does the inclusion of a randomized inter-trial interval (ITI) impact performance on an Attentional Blink (AB) task? The AB phenomenon is often used as a test of transient attention (Dux & Marois, 2009); however, it is unclear whether incorporating aspects of sustained attention, by implementing a randomized ITI, would impact task performance. The current research sought to investigate this, by contrasting a standard version of the AB task with a random ITI version to determine whether performance changed, reflecting a change in difficulty, engagement, or motivation.

Method

Thirty university students (21 female; age range 18–57, M_age= 21.5, SD = 7.4) completed both versions of the task, in counterbalanced order.

Results

No significant difference in performance was found between the standard AB task and the AB task with the random ITI. Bayesian analyses suggested moderate evidence for the null.

Conclusion

Temporal unpredictability did not appear to impact task performance. This suggests that the standard AB task has cognitive properties with regards to task difficulty, engagement, and motivation, that are inherently similar to tasks that employ a randomized ITI to measure sustained attention (e.g., the Psychomotor Vigilance Task; PVT; Dinges & Powell, 1985). This finding provides important support for future research which may seek to obtain a more detailed understanding of attention through the comparison of performance on transient and sustained attention tasks.

The eye that binds: Feature integration is not disrupted by saccadic eye movements

Feature integration theory proposes that visual features, such as shape and color, can only be combined into a unified object when spatial attention is directed to their location in retinotopic maps. Eye movements cause dramatic changes on our retinae, and are associated with obligatory shifts in spatial attention. In two experiments, we measured the prevalence of conjunction errors (that is, reporting an object as having an attribute that belonged to another object), for brief stimulus presentation before, during, and after a saccade. Planning and executing a saccade did not itself disrupt feature integration. Motion did disrupt feature integration, leading to an increase in conjunction errors. However, retinal motion of an equal extent but caused by saccadic eye movements is spared this disruption, and showed similar rates of conjunction errors as a condition with static stimuli presented to a static eye. The results suggest that extra-retinal signals are able to compensate for the motion caused by saccadic eye movements, thereby preserving the integrity of objects across saccades and preventing their features from mixing or mis-binding.

Distinct Neural Mechanisms Meet Challenges in Dynamic Visual Attention due to Either Load or Object Spacing

When engaged in dynamic visuospatial tasks, the brain copes with perceptual and cognitive processing challenges. During multiple-object tracking (MOT), the number of objects to be tracked (i.e., load) imposes attentional demands, but so does spatial interference from irrelevant objects (i.e., close encounters). Presently, it is not clear whether the effect of load on accuracy solely depends on the number of close encounters. If so, the same cognitive and physiological mechanisms deal with increasing load by preparing for and dealing with spatial interference. However, this has never been directly tested. Such knowledge is important to understand the neurophysiology of dynamic visual attention and resolve conflicting views within visual cognition concerning sources of capacity limitations. We varied the processing challenge in MOT task in two ways: the number of targets and the minimum spatial proximity between targets and distractors. In a first experiment, we measured task-induced pupil dilations and saccades during MOT. In a separate cohort, we measured fMRI activity. In both cohorts, increased load and close encounters (i.e., close spatial proximity) led to reduced accuracy in an additive manner. Load was associated with pupil dilations, whereas close encounters were not. Activity in dorsal attentional areas and frequency of saccades were proportionally larger both with higher levels of load and close encounters. Close encounters recruited additionally ventral attentional areas that may reflect orienting mechanisms. The activity in two brainstem nuclei, ventral tegmental area/substantia nigra and locus coeruleus, showed clearly dissociated patterns. Our results constitute convergent evidence indicating that different mechanisms underlie processing challenges due to load and object spacing.

Quantifying the resolution of spatial and temporal representation in children with 22q11.2 deletion syndrome

OBJECTIVES

Our ability to generate mental representation of magnitude from sensory information affects how we perceive and experience the world. Reduced resolution of the mental representations formed from sensory inputs may generate impairment in the proximal and distal information processes that utilize these representations. Impairment of spatial and temporal information processing likely underpins the non-verbal cognitive impairments observed in 22q11.2 deletion syndrome (22q11DS). The present study builds on prior research by seeking to quantify the resolution of spatial and temporal representation in children with 22q11DS, sex chromosome aneuploidy (SCA), and a typically developing (TD) control group.

PARTICIPANTS AND METHODS

Children (22q11DS = 70, SCA = 49, TD = 46) responded to visual or auditory stimuli with varying difference ratios. The participant's task was to identify which of two sequentially presented stimuli was of larger magnitude in terms of, size, duration, or auditory frequency. Detection threshold was calculated as the minimum difference ratio between the "standard" and the "target" stimuli required to achieve 75% accuracy in detecting that the two stimuli were different.

RESULTS

Children with 22q11DS required larger magnitude difference between spatial stimuli for accurate identification compared with both the SCA and TD groups (% difference from standard: 22q11DS = 14; SCA = 8; TD: 7; F  = 8.42, p < 0.001). Temporal detection threshold was also higher for the 22q11DS group to both visual (% difference from standard: 22q11DS = 14; SCA = 8; TD = 7; F  = 8.33, p < 0.001) and auditory (% difference from standard: 22q11DS = 23; SCA = 12; TD: 8; F  = 8.99, p < 0.001) stimuli compared with both the SCA and TD groups, while the SCA and TD groups displayed equivalent performance on these measures (p's > 0.05). Pitch detection threshold did not differ among the groups (p's > 0.05).

CONCLUSIONS

The observation of higher detection thresholds to spatial and temporal stimuli indicates further evidence for reduced resolution in both spatial and temporal magnitude representation in 22q11DS, that does not extend to frequency magnitude representation (pitch detection), and which is not explained by generalized cognitive impairment alone. These findings generate further support for the hypothesis that spatiotemporal hypergranularity of mental representations contributes to the non-verbal cognitive impairment seen in 22q11DS.

Why Does Space Feel the Way it Does? Towards a Principled Account of Spatial Experience

There must be a reason why an experience feels the way it does. A good place to begin addressing this question is spatial experience, because it may be more penetrable by introspection than other qualities of consciousness such as color or pain. Moreover, much of experience is spatial, from that of our body to the visual world, which appears as if painted on an extended canvas in front of our eyes. Because it is ‘right there’, we usually take space for granted and overlook its qualitative properties. However, we should realize that a great number of phenomenal distinctions and relations are required for the canvas of space to feel ‘extended’. Here we argue that, to be experienced as extended, the canvas of space must be composed of countless spots, here and there, small and large, and these spots must be related to each other in a characteristic manner through connection, fusion, and inclusion. Other aspects of the structure of spatial experience follow from extendedness: every spot can be experienced as enclosing a particular region, with its particular location, size, boundary, and distance from other spots. We then propose an account of the phenomenal properties of spatial experiences based on integrated information theory (IIT). The theory provides a principled approach for characterizing both the quantity and quality of experience by unfolding the cause-effect structure of a physical substrate. Specifically, we show that a simple simulated substrate of units connected in a grid-like manner yields a cause-effect structure whose properties can account for the main properties of spatial experience. These results uphold the hypothesis that our experience of space is supported by brain areas whose units are linked by a grid-like connectivity. They also predict that changes in connectivity, even in the absence of changes in activity, should lead to a warping of experienced space. To the extent that this approach provides an initial account of phenomenal space, it may also serve as a starting point for investigating other aspects of the quality of experience and their physical correspondents.

Location-cued visual selection-Placeholder dots improve target identification

Visual cues help to select a target and attract attention to it. In the present study, a 50-ms exogenous cue was presented to select one of 80 tilted lines, and attention effects at various delays were measured as the time observers needed to identify this target. Like in earlier detection studies, there was a transient cuing effect; targets presented soon after the cue (delays of 50-300 ms) were identified particularly fast. This benefit was followed by a continuous decay of performance toward longer delays (measured up to 5 s), at which the necessary presentation time to identify the target was strongly increased. The decay was substantially reduced when placeholder dots were shown during the delay, at subsequent line positions. The simple presentation of a structured background in the form of random dots did not have this effect. When the presentation times for constant performance were taken to compute the presumed strength of underlying neural responses, the effect of placeholders was seen as a nearly constant addition to the cued target signals, with an additional transient peak about 100 ms after cue (and placeholders) onset.

The spread of presaccadic attention depends on the spatial configuration of the visual scene

When preparing a saccade, attentional resources are focused at the saccade target and its immediate vicinity. Here we show that this does not hold true when saccades are prepared toward a recently extinguished target. We obtained detailed maps of orientation sensitivity when participants prepared a saccade toward a target that either remained on the screen or disappeared before the eyes moved. We found that attention was mainly focused on the immediate surround of the visible target and spread to more peripheral locations as a function of the distance from the cue and the delay between the target's disappearance and the saccade. Interestingly, this spread was not accompanied with a spread of the saccade endpoint. These results suggest that presaccadic attention and saccade programming are two distinct processes that can be dissociated as a function of their interaction with the spatial configuration of the visual scene.

Object categorization in visual periphery is modulated by delayed foveal noise

Behavioral studies in humans indicate that peripheral vision can do object recognition to some extent. Moreover, recent studies have shown that some information from brain regions retinotopic to visual periphery is somehow fed back to regions retinotopic to the fovea and disrupting this feedback impairs object recognition in human. However, it is unclear to what extent the information in visual periphery contributes to human object categorization. Here, we designed two series of rapid object categorization tasks to first investigate the performance of human peripheral vision in categorizing natural object images at different eccentricities and abstraction levels (superordinate, basic, and subordinate). Then, using a delayed foveal noise mask, we studied how modulating the foveal representation impacts peripheral object categorization at any of the abstraction levels. We found that peripheral vision can quickly and accurately accomplish superordinate categorization, while its performance in finer categorization levels dramatically drops as the object presents further in the periphery. Also, we found that a 300-ms delayed foveal noise mask can significantly disturb categorization performance in basic and subordinate levels, while it has no effect on the superordinate level. Our results suggest that human peripheral vision can easily process objects at high abstraction levels, and the information is fed back to foveal vision to prime foveal cortex for finer categorizations when a saccade is made toward the target object.

Attention, Perception, and Action in a Simulated Decision-Making Task

Over the last decade, research on the visual focus of attention has become increasingly popular in psychological science. The focus of attention has been shown to be important in fast team-sport games. The authors developed a method that measures the extent of the attentional focus and perceptual capabilities during performance of a sport-specific task. The participants were required to judge different player configurations on their left and right sides with varying visual angles between the stimuli. In keeping with the notion that the focus of attention is smaller than the visual field, attentional performance was poorest at the wider viewing angles compared with perceptual performance. Moreover, the team-sport players were better able to enlarge their attentional focus and make correct decisions more frequently than individual athletes, particularly when a motor response was required. The findings provide a new perspective, dissociating the attentional and perceptual processes that affect decision making under various response modes.

What limits search for conjunctions of simple visual features?

Despite of decades of research, we still do not know for sure the roles of internal noise, attention, and crowding in search for conjunctions of simple visual features. In this study, we tried several modifications to the classic design of conjunction-search experiments. In order to match exactly the proportions of simple features, two different targets were presented in target-present trials-vertical red and horizontal blue bars among vertical blue and horizontal red distractors. Both the length of the bars and the number of objects in a display were varied. Positions of objects were selected for minimal crowding effects. Exposure duration was 60 ms, and proportion correct was used as the measure of performance. For conjunction search, the results rejected the unlimited-capacity model and were consistent with limited-capacity attentional processing, and with the Naka-Rushton transform of the target-distractor difference. Qualitatively the same results were obtained when bar length was fixed, and fine orientation difference was used to manipulate target-distractor discriminability. An experiment of feature (orientation) search produced results close to the unlimited-capacity model.

Goal-directed unequal attention allocation during multiple object tracking

In standard multiple object tracking (MOT) tasks the relative importance of the targets being tracked is equal. This is atypical of everyday situations in which an individual may need to prioritize one target relative to another and so allocate attention unequally. We report three experiments that examined whether participants could unequally split attention using a modified MOT task in which target priority was manipulated. Specifically, we examined the effect of priority on participants' magnitude of error and used a distribution mixture analysis to investigate how priority affected both participants' probability of losing an item and tracking precision. Experiment 1 (trajectory tracking) revealed a higher magnitude of error and higher proportion of guessing for low- compared with high-priority targets. Experiments 2 (trajectory tracking) and 3 (position tracking) examined how fine-grained this ability is by manipulating target priority at finer increments. In line with Experiment 1, results from both these experiments indicated that participants could split attention unequally. There was some evidence that participants could allocate attention unequally at fine increments, but this was less conclusive. Taken together, these experiments demonstrate participants' ability to distribute attention unequally across multiple moving objects but suggest some limitation with the flexibility of attention allocation.

Challenges to pooling models of crowding: Implications for visual mechanisms

A set of phenomena known as crowding reveal peripheral vision's vulnerability in the face of clutter. Crowding is important both because of its ubiquity, making it relevant for many real-world tasks and stimuli, and because of the window it provides onto mechanisms of visual processing. Here we focus on models of the underlying mechanisms. This review centers on a popular class of models known as pooling models, as well as the phenomenology that appears to challenge a pooling account. Using a candidate high-dimensional pooling model, we gain intuitions about whether a pooling model suffices and reexamine the logic behind the pooling challenges. We show that pooling mechanisms can yield substitution phenomena and therefore predict better performance judging the properties of a set versus a particular item. Pooling models can also exhibit some similarity effects without requiring mechanisms that pool at multiple levels of processing, and without constraining pooling to a particular perceptual group. Moreover, we argue that other similarity effects may in part be due to noncrowding influences like cuing. Unlike low-dimensional straw-man pooling models, high-dimensional pooling preserves rich information about the stimulus, which may be sufficient to support high-level processing. To gain insights into the implications for pooling mechanisms, one needs a candidate high-dimensional pooling model and cannot rely on intuitions from low-dimensional models. Furthermore, to uncover the mechanisms of crowding, experiments need to separate encoding from decision effects. While future work must quantitatively examine all of the challenges to a high-dimensional pooling account, insights from a candidate model allow us to conclude that a high-dimensional pooling mechanism remains viable as a model of the loss of information leading to crowding.

Is excessive visual crowding causally linked to developmental dyslexia?

For about 10% of children reading acquisition is extremely difficult because they are affected by a heritable neurobiological disorder called developmental dyslexia (DD), mainly associated to an auditory-phonological disorder. Visual crowding is a universal phenomenon that impairs the recognition of stimuli in clutter, such as a letter in a word or a word in a text. Several studies have shown an excessive crowding in individuals with DD, but the causal link between excessive crowding and DD is not yet clearly established. An excessive crowding might be, indeed, a simple effect of DD due to reduced reading experience. The results of five experiments in 181 children reveal that: (i) an excessive crowding only at unattended locations characterizes an unselected group of children with DD (Experiment 1); (ii) an extra-large spaced text increases reading accuracy by reducing crowding in an unselected group of children with DD (Experiment 2); (iii) efficient attentional action video game trainings reduce crowding and accelerate reading speed in two unselected groups of children with DD (Experiment 3 and 4), and; (iv) pre-reading crowding longitudinally predicts future poor readers (Experiment 5). Our results show multiple causal links between visual crowding and learning to read. These findings provide new insights for a more efficient remediation and prevention for DD.

Attentional Modulation of Visual Spatial Integration: Psychophysical Evidence Supported by Population Coding Modeling

Two prominent strategies that the human visual system uses to reduce incoming information are spatial integration and selective attention. Whereas spatial integration summarizes and combines information over the visual field, selective attention can single it out for scrutiny. The way in which these well-known mechanisms-with rather opposing effects-interact remains largely unknown. To address this, we had observers perform a gaze-contingent search task that nudged them to deploy either spatial or feature-based attention to maximize performance. We found that, depending on the type of attention employed, visual spatial integration strength changed either in a strong and localized or a more modest and global manner compared with a baseline condition. Population code modeling revealed that a single mechanism can account for both observations: Attention acts beyond the neuronal encoding stage to tune the spatial integration weights of neural populations. Our study shows how attention and integration interact to optimize the information flow through the brain.

Emergent features in the crowding zone: When target-flanker grouping surmounts crowding

Crowding is the impairment of target identification when the target is surrounded by nearby flankers. Two hallmarks of crowding are that it is stronger when the flankers are close to the target and when the target strongly groups with the flankers. Here we show the opposite of both. A chevron target (pointing up or down) was presented at 8° eccentricity in the right visual field. It was surrounded by four flankers. Three of the flankers varied (pointing left or right). The fourth, the critical flanker (CF), was fixed in one orientation (left, right, up, down), yielding different configurations with the target. The CF's distance to the target was varied. Target identification depended strongly on the distance and the orientation of the CF. Remarkably, when the target and the CF grouped into a good configuration and elicited an emergent feature, performance was high if the CF was close to the target. This effect was particularly strong when participants were informed about the different CF-target configurations before the experiment. Reducing crowding and grouping by asynchronous presentation of the CF and the other items abolished the effect. When participants reported the entire configuration of the CF and the target, performance rapidly decreased with increasing spacing when the CF and the target were different but not when they were the same, indicating different spatial extents of the corresponding grouping processes. Our results show that the features emerging from the configurations of the target and a flanker strongly modulate crowding. Strong target-flanker grouping can benefit performance.