Analyzing vision at the complexity level

Tracking the Misallocation and Reallocation of Spatial Attention toward Auditory Stimuli

Completely ignoring a salient distractor presented concurrently with a target is difficult, and sometimes attention is involuntarily attracted to the distractor's location (attentional capture). Employing the N2ac component as a marker of attention allocation toward sounds, in this study we investigate the spatiotemporal dynamics of auditory attention across two experiments. Human participants (male and female) performed an auditory search task, where the target was accompanied by a distractor in two-third of the trials. For a distractor more salient than the target (Experiment 1), we observe not only a distractor N2ac (indicating attentional capture) but the full chain of attentional dynamics implied by the notion of attentional capture, namely, (1) the distractor captures attention before the target is attended, (2) allocation of attention to the target is delayed by distractor presence, and (3) the target is attended after the distractor. Conversely, for a distractor less salient than the target (Experiment 2), although responses were delayed, no attentional capture was observed. Together, these findings reveal two types of spatial attentional dynamics in the auditory modality (distraction with and without attentional capture).

Learnable PM diffusion coefficients and reformative coordinate attention network for low dose CT denoising

Objective.Various deep learning methods have recently been used for low dose CT (LDCT) denoising. Aggressive denoising may destroy the edge and fine anatomical structures of CT images. Therefore a key issue in LDCT denoising tasks is the difficulty of balancing noise/artifact suppression and edge/structure preservation.Approach.We proposed an LDCT denoising network based on the encoder-decoder structure, namely the Learnable PM diffusion coefficient and efficient attention network (PMA-Net). First, using the powerful feature modeling capability of partial differential equations, we constructed a multiple learnable edge module to generate precise edge information, incorporating the anisotropic image processing idea of Perona-Malik (PM) model into the neural network. Second, a multiscale reformative coordinate attention module was designed to extract multiscale information. Non-overlapping dilated convolution capturing abundant contextual content was combined with coordinate attention which could embed the spatial location information of important features into the channel attention map. Finally, we imposed additional constraints on the edge information using edge-enhanced multiscale perceptual loss to avoid structure loss and over-smoothing.Main results.Experiments are conducted on simulated and real datasets. The quantitative and qualitative results show that the proposed method has better performance in suppressing noise/artifacts and preserving edges/structures.Significance.This work proposes a novel edge feature extraction method that unfolds partial differential equation into neural networks, which contributes to the interpretability and clinical application value of neural network.

DARMF-UNet: A dual-branch attention-guided refinement network with multi-scale features fusion U-Net for gland segmentation

Accurate gland segmentation is critical in determining adenocarcinoma. Automatic gland segmentation methods currently suffer from challenges such as less accurate edge segmentation, easy mis-segmentation, and incomplete segmentation. To solve these problems, this paper proposes a novel gland segmentation network Dual-branch Attention-guided Refinement and Multi-scale Features Fusion U-Net (DARMF-UNet), which fuses multi-scale features using deep supervision. At the first three layers of feature concatenation, a Coordinate Parallel Attention (CPA) is proposed to guide the network to focus on the key regions. A Dense Atrous Convolution (DAC) block is used in the fourth layer of feature concatenation to perform multi-scale features extraction and obtain global information. A hybrid loss function is adopted to calculate the loss of each segmentation result of the network to achieve deep supervision and improve the accuracy of segmentation. Finally, the segmentation results at different scales in each part of the network are fused to obtain the final gland segmentation result. The experimental results on the gland datasets Warwick-QU and Crag show that the network improves in terms of the evaluation metrics of F1 Score, Object Dice, Object Hausdorff, and the segmentation effect is better than the state-of-the-art network models.

A cortical zoom-in operation underlies covert shifts of visual spatial attention

Shifting the focus of attention without moving the eyes poses challenges for signal coding in visual cortex in terms of spatial resolution, signal routing, and cross-talk. Little is known how these problems are solved during focus shifts. Here, we analyze the spatiotemporal dynamic of neuromagnetic activity in human visual cortex as a function of the size and number of focus shifts in visual search. We find that large shifts elicit activity modulations progressing from highest (IT) through mid-level (V4) to lowest hierarchical levels (V1). Smaller shifts cause those modulations to start at lower levels in the hierarchy. Successive shifts involve repeated backward progressions through the hierarchy. We conclude that covert focus shifts arise from a cortical coarse-to-fine process progressing from retinotopic areas with larger toward areas with smaller receptive fields. This process localizes the target and increases the spatial resolution of selection, which resolves the above issues of cortical coding.

Wider and Stronger Inhibitory Ring of the Attentional Focus in Schizophrenia

Anomalies of attentional selection have been repeatedly described in individuals with schizophrenia spectrum disorders. However, a precise analysis of their ability to inhibit irrelevant visual information during attentional selection is not documented. Recent behavioral as well as neurophysiological and computational evidence showed that attentional search among different competing stimuli elicits an area of suppression in the immediate surrounding of the attentional focus. In the present study, the strength and spatial extension of this surround suppression were tested in individuals with schizophrenia and neurotypical controls. Participants were asked to report the orientation of a visual "pop-out" target, which appeared in different positions within a peripheral array of non-target stimuli. In half of the trials, after the target appeared, a probe circle circumscribed a non-target stimulus at various target-to-probe distances; in this case, participants were asked to report the probe orientation instead. Results suggest that, as compared to neurotypical controls, individuals with schizophrenia showed stronger and spatially more extended filtering of visual information in the areas surrounding their attentional focus. This increased filtering of visual information outside the focus of attention might potentially hamper their ability to integrate different elements into coherent percepts and influence higher order behavioral, affective, and cognitive domains.

When We Study the Ability to Attend, What Exactly Are We Trying to Understand?

When we study the human ability to attend, what exactly do we seek to understand? It is not clear what the answer might be to this question. There is still so much to know, while acknowledging the tremendous progress of past decades of research. It is as if each new study adds a tile to the mosaic that, when viewed from a distance, we hope will reveal the big picture of attention. However, there is no map as to how each tile might be placed nor any guide as to what the overall picture might be. It is like digging up bits of mosaic tile at an ancient archeological site with no key as to where to look and then not only having to decide which picture it belongs to but also where exactly in that puzzle it should be placed. I argue that, although the unearthing of puzzle pieces is very important, so is their placement, but this seems much less emphasized. We have mostly unearthed a treasure trove of puzzle pieces but they are all waiting for cleaning and reassembly. It is an activity that is scientifically far riskier, but with great risk comes a greater reward. Here, I will look into two areas of broad agreement, specifically regarding visual attention, and dig deeper into their more nuanced meanings, in the hope of sketching a starting point for the guide to the attention mosaic. The goal is to situate visual attention as a purely computational problem and not as a data explanation task; it may become easier to place the puzzle pieces once you understand why they exist in the first place.

Task-independent metrics of computational hardness predict human cognitive performance

The survival of human organisms depends on our ability to solve complex tasks in the face of limited cognitive resources. However, little is known about the factors that drive the complexity of those tasks. Here, building on insights from computational complexity theory, we quantify the computational hardness of cognitive tasks using a set of task-independent metrics related to the computational resource requirements of individual instances of a task. We then examine the relation between those metrics and human behavior and find that they predict both time spent on a task as well as accuracy in three canonical cognitive tasks. Our findings demonstrate that performance in cognitive tasks can be predicted based on generic metrics of their inherent computational hardness.

Attention-Setting and Human Mental Function

This article provides an introduction to experimental research on top-down human attention in complex scenes, written for cognitive scientists in general. We emphasize the major effects of goals and intention on mental function, measured with behavioral experiments. We describe top-down attention as an open category of mental actions that initiates particular task sets, which are assembled from a wide range of mental processes. We call this attention-setting. Experiments on visual search, task switching, and temporal attention are described and extended to the important human time scale of seconds.

Event-driven proto-object based saliency in 3D space to attract a robot's attention

To interact with its environment, a robot working in 3D space needs to organise its visual input in terms of objects or their perceptual precursors, proto-objects. Among other visual cues, depth is a submodality used to direct attention to visual features and objects. Current depth-based proto-object attention models have been implemented for standard RGB-D cameras that produce synchronous frames. In contrast, event cameras are neuromorphic sensors that loosely mimic the function of the human retina by asynchronously encoding per-pixel brightness changes at very high temporal resolution, thereby providing advantages like high dynamic range, efficiency (thanks to their high degree of signal compression), and low latency. We propose a bio-inspired bottom-up attention model that exploits event-driven sensing to generate depth-based saliency maps that allow a robot to interact with complex visual input. We use event-cameras mounted in the eyes of the iCub humanoid robot to directly extract edge, disparity and motion information. Real-world experiments demonstrate that our system robustly selects salient objects near the robot in the presence of clutter and dynamic scene changes, for the benefit of downstream applications like object segmentation, tracking and robot interaction with external objects.

LPMP: A Bio-Inspired Model for Visual Localization in Challenging Environments

Autonomous vehicles require precise and reliable self-localization to cope with dynamic environments. The field of visual place recognition (VPR) aims to solve this challenge by relying on the visual modality to recognize a place despite changes in the appearance of the perceived visual scene. In this paper, we propose to tackle the VPR problem following a neuro-cybernetic approach. To this end, the Log-Polar Max-Pi (LPMP) model is introduced. This bio-inspired neural network allows building a neural representation of the environment via an unsupervised one-shot learning. Inspired by the spatial cognition of mammals, visual information in the LPMP model are processed through two distinct pathways: a “what” pathway that extracts and learns the local visual signatures (landmarks) of a visual scene and a “where” pathway that computes their azimuth. These two pieces of information are then merged to build a visuospatial code that is characteristic of the place where the visual scene was perceived. Three main contributions are presented in this article: 1) the LPMP model is studied and compared with NetVLAD and CoHog, two state-of-the-art VPR models; 2) a test benchmark for the evaluation of VPR models according to the type of environment traveled is proposed based on the Oxford car dataset; and 3) the impact of the use of a novel detector leading to an uneven paving of an environment is evaluated in terms of the localization performance and compared to a regular paving. Our experiments show that the LPMP model can achieve comparable or better localization performance than NetVLAD and CoHog.

FCAU-Net for the Semantic Segmentation of Fine-Resolution Remotely Sensed Images

The semantic segmentation of fine-resolution remotely sensed images is an urgent issue in satellite image processing. Solving this problem can help overcome various obstacles in urban planning, land cover classification, and environmental protection, paving the way for scene-level landscape pattern analysis and decision making. Encoder-decoder structures based on attention mechanisms have been frequently used for fine-resolution image segmentation. In this paper, we incorporate a coordinate attention (CA) mechanism, adopt an asymmetric convolution block (ACB), and design a refinement fusion block (RFB), forming a network named the fusion coordinate and asymmetry-based U-Net (FCAU-Net). Furthermore, we propose novel convolutional neural network (CNN) architecture to fully capture long-term dependencies and fine-grained details in fine-resolution remotely sensed imagery. This approach has the following advantages: (1) the CA mechanism embeds position information into a channel attention mechanism to enhance the feature representations produced by the network while effectively capturing position information and channel relationships; (2) the ACB enhances the feature representation ability of the standard convolution layer and captures and refines the feature information in each layer of the encoder; and (3) the RFB effectively integrates low-level spatial information and high-level abstract features to eliminate background noise when extracting feature information, reduces the fitting residuals of the fused features, and improves the ability of the network to capture information flows. Extensive experiments conducted on two public datasets (ZY-3 and DeepGlobe) demonstrate the effectiveness of the FCAU-Net. The proposed FCAU-Net transcends U-Net, Attention U-Net, the pyramid scene parsing network (PSPNet), DeepLab v3+, the multistage attention residual U-Net (MAResU-Net), MACU-Net, and the Transformer U-Net (TransUNet). Specifically, the FCAU-Net achieves a 97.97% (95.05%) pixel accuracy (PA), a 98.53% (91.27%) mean PA (mPA), a 95.17% (85.54%) mean intersection over union (mIoU), and a 96.07% (90.74%) frequency-weighted IoU (FWIoU) on the ZY-3 (DeepGlobe) dataset.

Multi-Scale Feature Aggregation Network for Water Area Segmentation

Water area segmentation is an important branch of remote sensing image segmentation, but in reality, most water area images have complex and diverse backgrounds. Traditional detection methods cannot accurately identify small tributaries due to incomplete mining and insufficient utilization of semantic information, and the edge information of segmentation is rough. To solve the above problems, we propose a multi-scale feature aggregation network. In order to improve the ability of the network to process boundary information, we design a deep feature extraction module using a multi-scale pyramid to extract features, combined with the designed attention mechanism and strip convolution, extraction of multi-scale deep semantic information and enhancement of spatial and location information. Then, the multi-branch aggregation module is used to interact with different scale features to enhance the positioning information of the pixels. Finally, the two high-performance branches designed in the Feature Fusion Upsample module are used to deeply extract the semantic information of the image, and the deep information is fused with the shallow information generated by the multi-branch module to improve the ability of the network. Global and local features are used to determine the location distribution of each image category. The experimental results show that the accuracy of the segmentation method in this paper is better than that in the previous detection methods, and has important practical significance for the actual water area segmentation.

On the control of attentional processes in vision

The study of attentional processing in vision has a long and deep history. Recently, several papers have presented insightful perspectives into how the coordination of multiple attentional functions in the brain might occur. These begin with experimental observations and the authors propose structures, processes, and computations that might explain those observations. Here, we consider a perspective that past works have not, as a complementary approach to the experimentally-grounded ones. We approach the same problem as past authors but from the other end of the computational spectrum, from the problem nature, as Marr's Computational Level would prescribe. What problem must the brain solve when orchestrating attentional processes in order to successfully complete one of the myriad possible visuospatial tasks at which we as humans excel? The hope, of course, is for the approaches to eventually meet and thus form a complete theory, but this is likely not soon. We make the first steps towards this by addressing the necessity of attentional control, examining the breadth and computational difficulty of the visuospatial and attentional tasks seen in human behavior, and suggesting a sketch of how attentional control might arise in the brain. The key conclusions of this paper are that an executive controller is necessary for human attentional function in vision, and that there is a 'first principles' computational approach to its understanding that is complementary to the previous approaches that focus on modelling or learning from experimental observations directly.

Context isn't everything: Search performance is influenced by the nature of the task but not the background

AbstractIt has been demonstrated in the literature that cues in the environment that are predictive of how a task ought to be performed can influence performance. In an extension of this general notion, Cosman and Vecera (Journal of Experimental Psychology: Human Perception and Performance, 39(3), 836-848, 2013) reported that simply performing singleton and feature search tasks when irrelevant scenes were displayed in the background automatically modulated the search strategies adopted by participants when these scenes were reinstated at a later time. While intriguing, this result was also somewhat surprising given that an adaptive system (like the human brain) should disregard irrelevant information so task competencies generalize across environments. To investigate this issue further, we replicated the experimental procedures of Cosman and Vecera, while varying whether the test phase was either a singleton search (Experiments 1 and 3) or a feature search (Experiment 2) task. While it was observed that the nature of the search task varied whether a color singleton distractor influenced performance, there was no evidence that background scenes modulated the search strategies adopted by participants, contrasting the results of Cosman and Vecera. Overall, the findings here support the conclusion that the visual system prioritizes task-relevant information while disregarding irrelevant background information.

Descriptive Complexity, Computational Tractability, and the Logical and Cognitive Foundations of Mathematics

In computational complexity theory, decision problems are divided into complexity classes based on the amount of computational resources it takes for algorithms to solve them. In theoretical computer science, it is commonly accepted that only functions for solving problems in the complexity class P, solvable by a deterministic Turing machine in polynomial time, are considered to be tractable. In cognitive science and philosophy, this tractability result has been used to argue that only functions in P can feasibly work as computational models of human cognitive capacities. One interesting area of computational complexity theory is descriptive complexity, which connects the expressive strength of systems of logic with the computational complexity classes. In descriptive complexity theory, it is established that only first-order (classical) systems are connected to P, or one of its subclasses. Consequently, second-order systems of logic are considered to be computationally intractable, and may therefore seem to be unfit to model human cognitive capacities. This would be problematic when we think of the role of logic as the foundations of mathematics. In order to express many important mathematical concepts and systematically prove theorems involving them, we need to have a system of logic stronger than classical first-order logic. But if such a system is considered to be intractable, it means that the logical foundation of mathematics can be prohibitively complex for human cognition. In this paper I will argue, however, that this problem is the result of an unjustified direct use of computational complexity classes in cognitive modelling. Placing my account in the recent literature on the topic, I argue that the problem can be solved by considering computational complexity for humanly relevant problem solving algorithms and input sizes.

Demystifying visual awareness: Peripheral encoding plus limited decision complexity resolve the paradox of rich visual experience and curious perceptual failures

Human beings subjectively experience a rich visual percept. However, when behavioral experiments probe the details of that percept, observers perform poorly, suggesting that vision is impoverished. What can explain this awareness puzzle? Is the rich percept a mere illusion? How does vision work as well as it does? This paper argues for two important pieces of the solution. First, peripheral vision encodes its inputs using a scheme that preserves a great deal of useful information, while losing the information necessary to perform certain tasks. The tasks rendered difficult by the peripheral encoding include many of those used to probe the details of visual experience. Second, many tasks used to probe attentional and working memory limits are, arguably, inherently difficult, and poor performance on these tasks may indicate limits on decision complexity. Two assumptions are critical to making sense of this hypothesis: (1) All visual perception, conscious or not, results from performing some visual task; and (2) all visual tasks face the same limit on decision complexity. Together, peripheral encoding plus decision complexity can explain a wide variety of phenomena, including vision's marvelous successes, its quirky failures, and our rich subjective impression of the visual world.

Geobia Achievements and Spatial Opportunities in the Era of Big Earth Observation Data

The primary goal of collecting Earth observation (EO) imagery is to map, analyze, and contribute to an understanding of the status and dynamics of geographic phenomena. In geographic information science (GIScience), the term object-based image analysis (OBIA) was tentatively introduced in 2006. When it was re-formulated in 2008 as geographic object-based image analysis (GEOBIA), the primary focus was on integrating multiscale EO data with GIScience and computer vision (CV) solutions to cope with the increasing spatial and temporal resolution of EO imagery. Building on recent trends in the context of big EO data analytics as well as major achievements in CV, the objective of this article is to review the role of spatial concepts in the understanding of image objects as the primary analytical units in semantic EO image analysis, and to identify opportunities where GEOBIA may support multi-source remote sensing analysis in the era of big EO data analytics. We (re-)emphasize the spatial paradigm as a key requisite for an image understanding system capable to deal with and exploit the massive data streams we are currently facing; a system which encompasses a combined physical and statistical model-based inference engine, a well-structured CV system design based on a convergence of spatial and colour evidence, semantic content-based image retrieval capacities, and the full integration of spatio-temporal aspects of the studied geographical phenomena.

A computational account of threat-related attentional bias

Visual selective attention acts as a filter on perceptual information, facilitating learning and inference about important events in an agent's environment. A role for visual attention in reward-based decisions has previously been demonstrated, but it remains unclear how visual attention is recruited during aversive learning, particularly when learning about multiple stimuli concurrently. This question is of particular importance in psychopathology, where enhanced attention to threat is a putative feature of pathological anxiety. Using an aversive reversal learning task that required subjects to learn, and exploit, predictions about multiple stimuli, we show that the allocation of visual attention is influenced significantly by aversive value but not by uncertainty. Moreover, this relationship is bidirectional in that attention biases value updates for attended stimuli, resulting in heightened value estimates. Our findings have implications for understanding biased attention in psychopathology and support a role for learning in the expression of threat-related attentional biases in anxiety.

Feed-forward visual processing suffices for coarse localization but fine-grained localization in an attention-demanding context needs feedback processing

It is well known that simple visual tasks, such as object detection or categorization, can be performed within a short period of time, suggesting the sufficiency of feed-forward visual processing. However, more complex visual tasks, such as fine-grained localization may require high-resolution information available at the early processing levels in the visual hierarchy. To access this information using a top-down approach, feedback processing would need to traverse several stages in the visual hierarchy and each step in this traversal takes processing time. In the present study, we compared the processing time required to complete object categorization and localization by varying presentation duration and complexity of natural scene stimuli. We hypothesized that performance would be asymptotic at shorter presentation durations when feed-forward processing suffices for visual tasks, whereas performance would gradually improve as images are presented longer if the tasks rely on feedback processing. In Experiment 1, where simple images were presented, both object categorization and localization performance sharply improved until 100 ms of presentation then it leveled off. These results are a replication of previously reported rapid categorization effects but they do not support the role of feedback processing in localization tasks, indicating that feed-forward processing enables coarse localization in relatively simple visual scenes. In Experiment 2, the same tasks were performed but more attention-demanding and ecologically valid images were used as stimuli. Unlike in Experiment 1, both object categorization performance and localization precision gradually improved as stimulus presentation duration became longer. This finding suggests that complex visual tasks that require visual scrutiny call for top-down feedback processing.

Long-range neural inhibition and stimulus competition in the archerfish optic tectum

The archerfish, which is unique in its ability to hunt insects above the water level by shooting a jet of water at its prey, operates in a complex visual environment. The fish needs to quickly select one object from among many others. In animals other than the archerfish, long-range inhibition is considered to drive selection. As a result of long-range inhibition, a potential target outside a neuron's receptive field suppresses the activity elicited by another potential target within the receptive field. We tested whether a similar mechanism operates in the archerfish by recording the activity of neurons in the optic tectum while presenting a target stimulus inside the receptive field and a competing stimulus outside the receptive field. We held the features of the target constant while varying the size, speed, and distance of the competing stimulus. We found cells that exhibit long-range inhibition; i.e., inhibition that extends to a significant part of the entire visual field of the animal. The competing stimulus depressed the firing rate. In some neurons, this effect was dependent on the features of the competing stimulus. These findings suggest that long-range inhibition may play a crucial role in the target selection process in the archerfish.

Unique objects attract attention even when faint

Locally contrasting objects, e.g. a red apple surrounded by green apples, attract attention. Does this generalize to differences in feature space? That is, do unique objects-regardless of their location-stand out from a collection of objects that are similar to one another, even when the unique object has lower local contrast with the background than the other objects? Behavioral data show indeed a preference for unique items but previous experiments enabled viewers to anticipate what response they were "supposed" to give. We developed a new experimental paradigm that minimizes such top-down effects. Pitting local contrast against global uniqueness, we show that unique stimuli attract attention even in not-anticipated, never-seen images, and even when the unique stimuli are faint (low contrast). A computational model explains how competition between objects in feature space favors dissimilar objects over those with similar features. The model explains how humans select unique objects, without a loss of performance on natural scenes.

What drives prioritized visual processing? A motivational relevance account

Emotion is fundamental to our being, and an essential aspect guiding behavior when rapid responding is required. This includes whether we approach or avoid a stimulus, and the accompanying physiological responses. A common tenet is that threat-related content drives stimulus processing and biases visual attention, so that rapid responding can be initiated. In this paper, it will be argued instead that prioritization of threatening stimuli should be encompassed within a motivational relevance framework. To more fully understand what is, or is not, prioritized for visual processing one must, however, additionally consider: (i) stimulus ambiguity and perceptual saliency; (ii) task demands, including both perceptual load and cognitive load; and (iii) endogenous/affective states of the individual. Combined with motivational relevance, this then leads to a multifactorial approach to understanding the drivers of prioritized visual processing. This accords with current recognition that the brain basis allowing for visual prioritization is also multifactorial, including transient, dynamic and overlapping networks. Taken together, the paper provides a reconceptualization of how "emotional" information prioritizes visual processing.

Attention to colors induces surround suppression at category boundaries

We investigated how attention to a visual feature modulates representations of other features. The feature-similarity gain model predicts a graded modulation, whereas an alternative model asserts an inhibitory surround in feature space. Although evidence for both types of modulations can be found, a consensus has not emerged in the literature. Here, we aimed to reconcile these different views by systematically measuring how attention modulates color perception. Based on previous literature, we also predicted that color categories would impact attentional modulation. Our results showed that both surround suppression and feature-similarity gain modulate perception of colors but they operate on different similarity scales. Furthermore, the region of the suppressive surround coincided with the color category boundary, suggesting a categorical sharpening effect. We implemented a neural population coding model to explain the observed behavioral effects, which revealed a hitherto unknown connection between neural tuning shift and surround suppression.

AutoCloud+, a “Universal” Physical and Statistical Model-Based 2D Spatial Topology-Preserving Software for Cloud/Cloud–Shadow Detection in Multi-Sensor Single-Date Earth Observation Multi-Spectral Imagery—Part 1: Systematic ESA EO Level 2 Product Generation at the Ground Segment as Broad Context

The European Space Agency (ESA) defines Earth observation (EO) Level 2 information product the stack of: (i) a single-date multi-spectral (MS) image, radiometrically corrected for atmospheric, adjacency and topographic effects, with (ii) its data-derived scene classification map (SCM), whose thematic map legend includes quality layers cloud and cloud–shadow. Never accomplished to date in an operating mode by any EO data provider at the ground segment, systematic ESA EO Level 2 product generation is an inherently ill-posed computer vision (CV) problem (chicken-and-egg dilemma) in the multi-disciplinary domain of cognitive science, encompassing CV as subset-of artificial general intelligence (AI). In such a broad context, the goal of our work is the research and technological development (RTD) of a “universal” AutoCloud+ software system in operating mode, capable of systematic cloud and cloud–shadow quality layers detection in multi-sensor, multi-temporal and multi-angular EO big data cubes characterized by the five Vs, namely, volume, variety, veracity, velocity and value. For the sake of readability, this paper is divided in two. Part 1 highlights why AutoCloud+ is important in a broad context of systematic ESA EO Level 2 product generation at the ground segment. The main conclusions of Part 1 are that ESA EO Level 2 information product is regarded as: (I) necessary-but-not-sufficient pre-condition for the yet-unaccomplished dependent problems of semantic content-based image retrieval (SCBIR) and semantics-enabled information/knowledge discovery (SEIKD) in multi-source EO big data cubes, where SCBIR and SEIKD are part-of the GEO-CEOS visionary goal of a yet-unaccomplished Global EO System of Systems (GEOSS). (II) State-of-the-art definition of EO Analysis Ready Data (ARD) format. (III) Horizontal policy, the goal of which is background developments, in a “seamless chain of innovation” needed for a new era of Space Economy 4.0. In the subsequent Part 2, the AutoCloud+ software system requirements specification, information/knowledge representation, system design, algorithm, implementation and preliminary experimental results are presented and discussed.

The Attentional Suppressive Surround: Eccentricity, Location-Based and Feature-Based Effects and Interactions

The Selective Tuning model of visual attention (Tsotsos, 1990) has proposed that the focus of attention is surrounded by an inhibitory zone, eliciting a center-surround attentional distribution. This attentional suppressive surround inhibits irrelevant information which is located close to attended information in physical space (e.g., Cutzu and Tsotsos, 2003; Hopf et al., 2010) or in feature space (e.g., Tombu and Tsotsos, 2008; Störmer and Alvarez, 2014; Bartsch et al., 2017). In Experiment 1, we investigate the interaction between location-based and feature-based surround suppression and hypothesize that the attentional surround suppression would be maximized when spatially adjacent stimuli are also represented closely within a feature map. Our results demonstrate that perceptual discrimination is worst when two similar orientations are presented in proximity to each other, suggesting the interplay of the two surround suppression mechanisms. The Selective Tuning model also predicts that the size of the attentional suppressive surround is determined by the receptive field size of the neuron which optimally processes the attended information. The receptive field size of the processing neurons is tightly associated with stimulus size and eccentricity. Therefore, Experiment 2 tested the hypothesis that the size of the attentional suppressive surround would become larger as stimulus size and eccentricity increase, corresponding to an increase in the neuron's receptive field size. We show that stimulus eccentricity but not stimulus size modulates the size of the attentional suppressive surround. These results are consistent for both low- and high-level features (e.g., orientation and human faces). Overall, the present study supports the existence of the attentional suppressive surround and reveals new properties of this selection mechanism.

Fundamental computational constraints on the time course of perception and action

A cognitive system faced with contingent events that cause rapid changes in sensory data may (i) incrementally incorporate new data into the ongoing perceptual and motor processing; or (ii) restart processing on each new event; or (iii) sample the data and hold onto the sample until its processing is complete, while disregarding any contingent changes. We offer a set of computational first-principles arguments for a hypothesis, according to which any system that contends with certain classes of perception and behavioral control tasks must include the sample-and-hold option (possibly alongside the other two, which may be useful in other tasks). This hypothesis has implications for understanding the dynamics of perception and action. In particular, a sample-and-hold channel necessarily processes sensory data on some kind of cycle (which does not imply precise periodicity). Further, being prepared to face the world at all times requires that the sampling that initiates each cycle be triggered by every significant action on part of the agent itself, such as saccades. We survey a range of evidence for the sample-and-hold functionality, touching upon diverse phenomena such as attentional blink and backward masking, the yoking of olfaction to respiration, thalamocortical interactions, and metastable brain dynamics in perception and consciousness.

GEO-CEOS stage 4 validation of the Satellite Image Automatic Mapper lightweight computer program for ESA Earth observation level 2 product generation - Part 1: Theory

ESA defines as Earth Observation (EO) Level 2 information product a single-date multi-spectral (MS) image corrected for atmospheric, adjacency and topographic effects, stacked with its data-derived scene classification map (SCM), whose legend includes quality layers cloud and cloud-shadow. No ESA EO Level 2 product has ever been systematically generated at the ground segment. To fill the information gap from EO big data to ESA EO Level 2 product in compliance with the GEO-CEOS stage 4 validation (Val) guidelines, an off-the-shelf Satellite Image Automatic Mapper (SIAM) lightweight computer program was validated by independent means on an annual 30 m resolution Web-Enabled Landsat Data (WELD) image composite time-series of the conterminous U.S. (CONUS) for the years 2006-2009. The SIAM core is a prior knowledge-based decision tree for MS reflectance space hyperpolyhedralization into static color names. Typically, a vocabulary of MS color names in a MS data (hyper)cube and a dictionary of land cover (LC) class names in the scene-domain do not coincide and must be harmonized (reconciled). The present Part 1-Theory provides the multidisciplinary background of a priori color naming. The subsequent Part 2-Validation accomplishes a GEO-CEOS stage 4 Val of the test SIAM-WELD annual map time-series in comparison with a reference 30 m resolution 16-class USGS National Land Cover Data 2006 map, based on an original protocol for wall-to-wall thematic map quality assessment without sampling, where the test and reference maps feature the same spatial resolution and spatial extent, but whose legends differ and must be harmonized.

Parameterized Complexity of Theory of Mind Reasoning in Dynamic Epistemic Logic

Theory of mind refers to the human capacity for reasoning about others' mental states based on observations of their actions and unfolding events. This type of reasoning is notorious in the cognitive science literature for its presumed computational intractability. A possible reason could be that it may involve higher-order thinking (e.g., 'you believe that I believe that you believe'). To investigate this we formalize theory of mind reasoning as updating of beliefs about beliefs using dynamic epistemic logic, as this formalism allows to parameterize 'order of thinking.' We prove that theory of mind reasoning, so formalized, indeed is intractable (specifically, PSPACE-complete). Using parameterized complexity we prove, however, that the 'order parameter' is not a source of intractability. We furthermore consider a set of alternative parameters and investigate which of them are sources of intractability. We discuss the implications of these results for the understanding of theory of mind.

Short and Long-Term Attentional Firing Rates Can Be Explained by ST-Neuron Dynamics

Attention modulates neural selectivity and optimizes the allocation of cortical resources during visual tasks. A large number of experimental studies in primates and humans provide ample evidence. As an underlying principle of visual attention, some theoretical models suggested the existence of a gain element that enhances contrast of the attended stimuli. In contrast, the Selective Tuning model of attention (ST) proposes an attentional mechanism based on suppression of irrelevant signals. In this paper, we present an updated characterization of the ST-neuron proposed by the Selective Tuning model, and suggest that the inclusion of adaptation currents (Ih) to ST-neurons may explain the temporal profiles of the firing rates recorded in single V4 cells during attentional tasks. Furthermore, using the model we show that the interaction between stimulus-selectivity of a neuron and attention shapes the profile of the firing rate, and is enough to explain its fast modulation and other discontinuities observed, when the neuron responds to a sudden switch of stimulus, or when one stimulus is added to another during a visual task.

Weak surround suppression of the attentional focus characterizes visual selection in the ventral stream in autism

Neurophysiological findings in the typical population demonstrate that spatial scrutiny for visual selection determines a center-surround profile of the attentional focus, which is the result of recurrent processing in the visual system. Individuals with autism spectrum disorder (ASD) manifest several anomalies in their visual selection, with strengths in detail-oriented tasks, but also difficulties in distractor inhibition tasks. Here, we asked whether contradictory aspects of perception in ASD might be due to a different center-surround profile of their attentional focus. In two experiments, we tested two independent samples of children with ASD, comparing them with typically developing (TD) peers. In Experiment 1, we used a psychophysical task that mapped the entire spatial profile of the attentional focus. In Experiment 2, we used dense-array electroencephalography (EEG) to explore its neurophysiological underpinnings. Experiment 1 results showed that the suppression, surrounding the attentional focus, was markedly reduced in children with ASD. Experiment 2 showed that the center-surround profile in TD children resulted in a modulation of the posterior N2 ERP component, with cortical sources in the lateral-occipital and medial/inferior temporal areas. In contrast, children with ASD did not show modulation of the N2 and related activations in the ventral visual stream. Furthermore, behavioural and neurophysiological measures of weaker suppression predicted more severe autistic symptomatology. The present findings, showing an altered center-surround profile during attentional selection, give an important insight to understand superior visual processing in autism as well as the experiencing of sensory overload.

Perceptual Color Space Representations in the Oculomotor System Are Modulated by Surround Suppression and Biased Selection

The oculomotor system utilizes color extensively for planning saccades. Therefore, we examined how the oculomotor system actually encodes color and several factors that modulate these representations: attention-based surround suppression and inherent biases in selecting and encoding color categories. We measured saccade trajectories while human participants performed a memory-guided saccade task with color targets and distractors and examined whether oculomotor target selection processing was functionally related to the CIE (x,y) color space distances between color stimuli and whether there were hierarchical differences between color categories in the strength and speed of encoding potential saccade goals. We observed that saccade planning was modulated by the CIE (x,y) distances between stimuli thus demonstrating that color is encoded in perceptual color space by the oculomotor system. Furthermore, these representations were modulated by (1) cueing attention to a particular color thereby eliciting surround suppression in oculomotor color space and (2) inherent selection and encoding biases based on color category independent of cueing and perceptual discriminability. Since surround suppression emerges from recurrent feedback attenuation of sensory projections, observing oculomotor surround suppression suggested that oculomotor encoding of behavioral relevance results from integrating sensory and cognitive signals that are pre-attenuated based on task demands and that the oculomotor system therefore does not functionally contribute to this process. Second, although perceptual discriminability did partially account for oculomotor processing differences between color categories, we also observed preferential processing of the red color category across various behavioral metrics. This is consistent with numerous previous studies and could not be simply explained by perceptual discriminability. Since we utilized a memory-guided saccade task, this indicates that the biased processing of the red color category does not rely on sustained sensory input and must therefore involve cortical areas associated with the highest levels of visual processing involved in visual working memory.

Complexity Level Analysis Revisited: What Can 30 Years of Hindsight Tell Us about How the Brain Might Represent Visual Information?

Much has been written about how the biological brain might represent and process visual information, and how this might inspire and inform machine vision systems. Indeed, tremendous progress has been made, and especially during the last decade in the latter area. However, a key question seems too often, if not mostly, be ignored. This question is simply: do proposed solutions scale with the reality of the brain's resources? This scaling question applies equally to brain and to machine solutions. A number of papers have examined the inherent computational difficulty of visual information processing using theoretical and empirical methods. The main goal of this activity had three components: to understand the deep nature of the computational problem of visual information processing; to discover how well the computational difficulty of vision matches to the fixed resources of biological seeing systems; and, to abstract from the matching exercise the key principles that lead to the observed characteristics of biological visual performance. This set of components was termed complexity level analysis in Tsotsos (1987) and was proposed as an important complement to Marr's three levels of analysis. This paper revisits that work with the advantage that decades of hindsight can provide.

The trickle-down effect of predictability: Secondary task performance benefits from predictability in the primary task

Predictions optimize processing by reducing attentional resources allocation to expected or predictable sensory data. Our study demonstrates that these saved processing resources can be then used on concurrent stimuli, and in consequence improve their processing and encoding. We illustrate this "trickle-down" effect with a dual task, where the primary task varied in terms of predictability. The primary task involved detection of a pre-specified symbol that appeared at some point of a short video of a dot moving along a random, semi-predictable or predictable trajectory. The concurrent secondary task involved memorization of photographs representing either emotionally neutral or non-neutral (social or threatening) content. Performance in the secondary task was measured by a memory test. We found that participants allocated more attention to unpredictable (random and semi-predictable) stimuli than to predictable stimuli. Additionally, when the stimuli in the primary task were more predictable, participants performed better in the secondary task, as evidenced by higher sensitivity in the memory test. Finally, social or threatening stimuli were allocated more "looking time" and a larger number of saccades than neutral stimuli. This effect was stronger for the threatening stimuli than social stimuli. Thus, predictability of environmental input is used in optimizing the allocation of attentional resources, which trickles-down and benefits the processing of concurrent stimuli.

Sequential sampling of visual objects during sustained attention

In a crowded visual scene, attention must be distributed efficiently and flexibly over time and space to accommodate different contexts. It is well established that selective attention enhances the corresponding neural responses, presumably implying that attention would persistently dwell on the task-relevant item. Meanwhile, recent studies, mostly in divided attentional contexts, suggest that attention does not remain stationary but samples objects alternately over time, suggesting a rhythmic view of attention. However, it remains unknown whether the dynamic mechanism essentially mediates attentional processes at a general level. Importantly, there is also a complete lack of direct neural evidence reflecting whether and how the brain rhythmically samples multiple visual objects during stimulus processing. To address these issues, in this study, we employed electroencephalography (EEG) and a temporal response function (TRF) approach, which can dissociate responses that exclusively represent a single object from the overall neuronal activity, to examine the spatiotemporal characteristics of attention in various attentional contexts. First, attention, which is characterized by inhibitory alpha-band (approximately 10 Hz) activity in TRFs, switches between attended and unattended objects every approximately 200 ms, suggesting a sequential sampling even when attention is required to mostly stay on the attended object. Second, the attentional spatiotemporal pattern is modulated by the task context, such that alpha-mediated switching becomes increasingly prominent as the task requires a more uniform distribution of attention. Finally, the switching pattern correlates with attentional behavioral performance. Our work provides direct neural evidence supporting a generally central role of temporal organization mechanism in attention, such that multiple objects are sequentially sorted according to their priority in attentional contexts. The results suggest that selective attention, in addition to the classically posited attentional "focus," involves a dynamic mechanism for monitoring all objects outside of the focus. Our findings also suggest that attention implements a space (object)-to-time transformation by acting as a series of concatenating attentional chunks that operate on 1 object at a time.

“I am not dead yet!” – The Item responds to Hulleman & Olivers

The item can only be dispensed within artificial tasks that, although useful in the lab, do not reflect the real world. There, the attended item is the goal of search. Hulleman & Olivers' (H&O's) model can ignore the item only by reducing search to the question of whether a patch of 0s (distractors) contains a 1 (target).

Competitive Selection and Age-Related Changes in Visual Attention

Healthy aging entails selective losses in visual attention, including the ability to filter clutter, divide attention between inputs, and search for configurations or conjunctions of features. A model of attention as a competition to influence neurons in the visual brain provides a framework for understanding these effects. Under the model, competition is necessary to disambiguate neural responses and resolve object details when multiple stimuli fall within the same visual receptive fields. A pattern of perceptual interference between attended stimuli in close spatial proximity with one another appears to be a psychophysical marker of this competition. Studies of divided visual attention in young and older adults show pronounced age-related increases in the strength of spatial interference between attended items, but only in the presence of clutter. Results suggest that inefficient competition for selection contributes to older adults’ visual attentional difficulties, compromising the ability to resolve details of multiple stimuli within small regions of the visual field. The conceptualization of attention as a competition for selection may thus provide a framework for understanding and assessing age-related attention losses.