The centroid paradigm: Quantifying feature-based attention in terms of attention filters

Characterizing motion prediction in small autonomous swarms

The use of robotic swarms has become increasingly common in research, industrial, and military domains for tasks such as collective exploration, coordinated movement, and collective localization. Despite the expanded use of robotic swarms, little is known about how swarms are perceived by human operators. To characterize human-swarm interactions, we evaluate how operators perceive swarm characteristics, including movement patterns, control schemes, and occlusion. In a series of experiments manipulating movement patterns and control schemes, participants tracked swarms on a computer screen until they were occluded from view, at which point participants were instructed to estimate the spatiotemporal dynamics of the occluded swarm by mouse click. In addition to capturing mouse click responses, eye tracking was used to capture participants eye movements while visually tracking swarms. We observed that manipulating control schemes had minimal impact on the perception of swarms, and that swarms are easier to track when they are visible compared to when they were occluded. Regarding swarm movements, a complex pattern of data emerged. For example, eye tracking indicates that participants more closely track a swarm in an arc pattern compared to sinusoid and linear movement patterns. When evaluating behavioral click-responses, data show that time is underestimated, and that spatial accuracy is reduced in complex patterns. Results suggest that measures of performance may capture different patterns of behavior, underscoring the need for multiple measures to accurately characterize performance. In addition, the lack of generalizable data across different movement patterns highlights the complexity involved in the perception of swarms of objects.

On the flexibility of strategies for center estimation

When subjects are asked to indicate the center of a spatially distributed stimulus, the features that control their responses tend to vary (1) across subjects and (2) as stimulus properties are altered. Here we ask: can subjects bring these different response tendencies under top-down control? In each of three tasks, all using briefly displayed, Gaussian dot-clouds, subjects were trained to perform different center-estimation responses. In the "mass task," the target was the centroid of the dots. In the "hull task," the target was the centroid of the region circumscribed by the convex hull of the dot-cloud. In the "hull-vertex task," the target was the centroid of the vertices of the convex hull. Subjects were able to perform each of the mass- and hull-tasks accurately and reliably. However, they found the hull-vertex task more difficult; errors were substantially larger in this task, and responses tended to be closer to both of the hull- and mass-task centers. The finding that subjects can intentionally target either the centroid of the dots in the stimulus or the centroid of the stimulus convex hull suggests that individual differences in feedback-free experiments may reflect idiosyncratic decisions by different subjects about what combination of these statistics to use in responding.

The Weighted Average Illusion: Biases in Perceived Mean Position in Scatterplots

Scatterplots can encode a third dimension by using additional channels like size or color (e.g. bubble charts). We explore a potential misinterpretation of trivariate scatterplots, which we call the weighted average illusion, where locations of larger and darker points are given more weight toward x- and y-mean estimates. This systematic bias is sensitive to a designer's choice of size or lightness ranges mapped onto the data. In this paper, we quantify this bias against varying size/lightness ranges and data correlations. We discuss possible explanations for its cause by measuring attention given to individual data points using a vision science technique called the centroid method. Our work illustrates how ensemble processing mechanisms and mental shortcuts can significantly distort visual summaries of data, and can lead to misjudgments like the demonstrated weighted average illusion.

The density effect in centroid estimation is blind to contrast polarity

Human vision is highly efficient in estimating the centroids of spatially scattered items. However, the processes underlying this remarkable skill remain poorly understood. A salient fact is that in estimating the centroids of dot-clouds, observers underweight densely packed dots relative to isolated dots; thus, when an observer estimates the centroid of a dot cloud, the weight exerted on the subject's response by a given dot tends to be suppressed by other dots near it. The current experiment sought to determine whether dots of contrast polarity equal vs. opposite to a given dot differ in how they alter the weight it exerts. Six observers were tested in a task that used brief (180 ms), Gaussian clouds that mixed 9 white and 9 black dots on a gray background. On each trial, the observer strove to mouse-click the centroid of the stimulus cloud weighting all dots equally. The model used to describe the results allows the weight exerted on the subject's response by a given dot to depend on its peripherality in the stimulus cloud as well as on the density of same- and opposite-polarity dots surrounding it. For four observers, peripheral dots exerted lower influence than central dots on responses; the other two showed little effect of peripherality. For all observers, dots in high-density regions exerted less weight on responses than dots in low-density regions. Concerning the primary research question: dots of opposite vs. the same polarity as a given dot suppressed the weight it exerted with equal effectiveness. This suggests that the site of the interaction producing the density effect is a neural population that registers positive and negative contrast polarities in the same way.

An explicit investigation of the roles that feature distributions play in rapid visual categorization

Ensemble representations are often described as efficient tools when summarizing features of multiple similar objects as a group. However, it can sometimes be more useful not to compute a single summary description for all of the objects if they are substantially different, for example when they belong to entirely different categories. It was proposed that the visual system can efficiently use the distributional information of ensembles to decide whether simultaneously displayed items belong to single or several different categories. Here we directly tested how the feature distribution of items in a visual array affects an ability to discriminate individual items (Experiment 1) and sets (Experiments 2-3) when participants were instructed explicitly to categorize individual objects based on the median of size distribution. We varied the width (narrow or fat) as well as the shape (smooth or two-peaked) of distributions in order to manipulate the ease of ensemble extraction from the items. We found that observers unintentionally relied on the grand mean as a natural categorical boundary and that their categorization accuracy increased as a function of the size differences among individual items and a function of their separation from the grand mean. For ensembles drawn from two-peaked size distributions, participants showed better categorization performance. They were more accurate at judging within-category ensemble properties in other dimensions (centroid and orientation) and less biased by superset statistics. This finding corroborates the idea that the two-peaked feature distributions support the "segmentability" of spatially intermixed sets of objects. Our results emphasize important roles of ensemble statistics (mean, range, distribution shape) in explicit visual categorization.

The anisotropic field of ensemble coding

Human observers can accurately estimate statistical summaries from an ensemble of multiple stimuli, including the average size, hue, and direction of motion. The efficiency and speed with which statistical summaries are extracted suggest an automatic mechanism of ensemble coding that operates beyond the capacity limits of attention and memory. However, the extent to which ensemble coding reflects a truly parallel and holistic mode of processing or a non-uniform and biased integration of multiple items is still under debate. In the present work, we used a technique, based on a Spatial Weighted Average Model (SWM), to recover the spatial profile of weights with which individual stimuli contribute to the estimated average during mean size adjustment tasks. In a series of experiments, we derived two-dimensional SWM maps for ensembles presented at different retinal locations, with different degrees of dispersion and under different attentional demands. Our findings revealed strong spatial anisotropies and leftward biases in ensemble coding that were organized in retinotopic reference frames and persisted under attentional manipulations. These results demonstrate an anisotropic spatial contribution to ensemble coding that could be mediated by the differential activation of the two hemispheres during spatial processing and scene encoding.

Multiple concurrent centroid judgments imply multiple within-group salience maps

Subjects viewed a brief flash of 8-24 dots of either two or three colors randomly arrayed. Their task was to move a mouse cursor to the centroid (center-of-gravity) of each color in a pre-designated order. Conventional and idea-detector analyses show that subjects accurately judged all three centroids utilizing an astounding 13/24 stimulus dots, with only a modest loss of accuracy compared to judging a single-predesignated color centroid. The ability to concurrently compute three centroids is important because it is believed that centroid judgments are made on salience maps that record only salience and are ignorant of the features that produced the salience. Our explanation, instantiated in a computational model of salience processing, is that subjects have three salience maps. Dots are initially segregated into three groups according to color, then each color-group is recorded on a different salience map to compute a centroid. In Part 2, the data are analyzed in terms of Attention Operating Characteristics to characterize impairments in subjects' color-attention filters (mostly insignificant) and encoding efficiency (20% drop for the hardest task) in making multiple versus single centroid judgments. A new, more sensitive analysis measured five sources of subject error variance, four independent, additive sources of error variance: imperfect color-attention filters; a Bayesian-like bias towards a central tendency; storage, retrieval, and cursor misplacement error; a large residual error due mostly to inefficient encoding; and fifth, an interactive source - error in all four components that increases when multiple centroid judgments versus a single centroid judgment are required on each trial. SIGNIFICANCE STATEMENT: An important brain process is a salience map, a representation of the relative importance (salience) of the locations of visual space. It is needed to guide where to look next, for computing the center (technically "centroid") of a cluster of items, and for other important computations. Here we show that in a brief flash of dots of three different colors, randomly interleaved, subjects can compute all three centroids. As a single salience map cannot discriminate dots of different colors, accurately reporting three centroids demonstrates that subjects have not just one, as is commonly believed, but at least three salience maps.

A direct comparison of central tendency recall and temporal integration in the successive field iconic memory task

The ensemble coding literature suggests the existence of a fast, automatic formation of some ensemble codes. Can statistical representations, such as memory for the central tendency along a particular visual feature dimension, be extracted from information held in the sensory register? Furthermore, can knowledge of early, iconic memory processes be used to determine how central tendency is extracted? We focused on the potential role of visible persistence mechanisms that support temporal integration. We tested whether mean orientation could be accurately recalled from brief visual displays using the successive field task. On separate blocks of trials, participants were asked to report the location of a split element (requiring differentiation of frames), a missing element (requiring integration across frames), and the average orientation of elements pooled across both frames (central tendency recall). Results replicate the expected tradeoff between differentiation and integration performance across inter-frame interval (IFI). In contrast, precision of mean estimates was high and invariant across IFIs. A manipulation of within-frame distributional similarity coupled with simulations using 12 models supported 2-item subsampling. The results argue against the “strategic” interpretation of subsampling since 2-item readout was predicted by information theoretic estimates of STM encoding rate: the 2 items were not from a superset in STM. Most crucially, the results argue against the various early “preattentive/parallel/global pooling” accounts and instead suggest that non-selective readout of information from iconic memory supplies a relatively small amount of item information to STM, and it is only at this point that the computation of ensemble averages begins.

Independent and parallel visual processing of ensemble statistics: Evidence from dual tasks

The visual system can represent multiple objects in a compressed form of ensemble summary statistics (such as object numerosity, mean, and feature variance/range). Yet the relationships between the different types of visual statistics remain relatively unclear. Here, we tested whether two summaries (mean and numerosity, or mean and range) are calculated independently from each other and in parallel. Our participants performed dual tasks requiring a report about two summaries in each trial, and single tasks requiring a report about one of the summaries. We estimated trial-by-trial correlations between the precision of reports as well as correlations across observers. Both analyses showed the absence of correlations between different types of ensemble statistics, suggesting their independence. We also found no decrement (except that related to the order of report explained by memory retrieval) in performance in dual compared to single tasks, which suggests that two statistics of one ensemble can be processed in parallel.

Central tendency representation and exemplar matching in visual short-term memory

I address a recent extension of the generalized context model (GCM), a model which excludes prototypes, to the visual short-term memory (VSTM) literature, which is currently deluged with prototype effects. The paper includes a brief review whose aim is to discuss the background and key findings suggesting that prototypes have an obligatory influence on visual short-term memory responses in the same VSTM task that the GCM's random walk extension, EBRW, was extended to account for: Sternberg scanning. I present a new model that incorporates such "central tendency representations" in memory, as well as several other regularities of the literature, and compare its prediction and postdictions to those of the GCM on some unpublished Sternberg scanning data. The GCM cannot account for the pattern in those data without post hoc modifications but the pattern is predicted nicely by the central tendency representation model. Although the new model is certainly wrong, the review and modeling exercise suggest a reconsideration of prototype models may be warranted, at least in the VSTM literature.

What is a preattentive feature?

The concept of a preattentive feature has been central to vision and attention research for about half a century. A preattentive feature is a feature that guides attention in visual search and that cannot be decomposed into simpler features. While that definition seems straightforward, there is no simple diagnostic test that infallibly identifies a preattentive feature. This paper briefly reviews the criteria that have been proposed and illustrates some of the difficulties of definition.

Variation in target and distractor heterogeneity impacts performance in the centroid task

In a selective centroid task, the participant views a brief cloud of items of different types-some of which are targets, the others distractors-and strives to mouse-click the centroid of the target items, ignoring the distractors. Advantages of the centroid task are that multiple target types can appear in the same display and that influence functions, which estimate the weight of each stimulus type in the cloud on the perceived centroid for each participant, can be obtained easily and efficiently. Here we document the strong, negative impact on performance that results when the participant is instructed to attend to target dots that consist of two or more levels of a single feature dimension, even when those levels differ categorically from those of the distractor dots. The results also show a smaller, but still observable decrement in performance that results when there is heterogeneity in the distractor dots.

The features that control discrimination of an isodipole texture pair

Visual features such as edges and corners are carried by high-order statistics. Previous analysis of discrimination of isodipole textures, which isolate specific high-order statistics, demonstrates visual sensitivity to these statistics but stops short of analyzing the underlying computations. Here we use a new texture centroid paradigm to probe these computations. We focus on two canonical isodipole textures, the even and odd textures: any 2 × 2 block of even (odd) texture contains an even (odd) number of black (and white) checks. Each stimulus comprised a spatially random array of black-and-white texture-disks (background = mean gray) that varied in their fourth-order statistics. In the Even (Odd) condition, disks varied along the continuum between random coinflip texture and pure (highly structured) even (odd) target texture. The task was to mouse-click the centroid of the disk array, weighting each disk location by the target structure level of the disk-texture (ranging from 0 for coinflip to 1 for even or odd). For each of block-sizes S=2×2, 2 × 3, 2 × 4 and 3 × 3, a linear model was used to estimate the weight exerted on the subject's responses by the differently patterned blocks of size S. Only the results with 2 × 4 and 3 × 3 blocks were consistent with the data. In the Even condition, homogeneous blocks exerted the most weight; in the odd condition, block-pattern symmetry was important. These findings show that visual mechanisms sensitive to four-point correlations do not compute evenness or oddness per se, but rather are activated selectively by features whose frequency varies across isodipole textures.

High-capacity preconscious processing in concurrent groupings of colored dots

Grouping is a perceptual process in which a subset of stimulus components (a group) is selected for a subsequent-typically implicit-perceptual computation. Grouping is a critical precursor to segmenting objects from the background and ultimately to object recognition. Here, we study grouping by color. We present subjects with 300-ms exposures of 12 dots colored with the same but unknown identical color interspersed among 14 dots of seven different colors. To indicate grouping, subjects point-click the remembered centroid ("center of gravity") of the set of homogeneous dots, of heterogeneous dots, or of all dots. Subjects accurately judge all of these centroids. Furthermore, after a single stimulus exposure, subjects can judge both the heterogeneous and homogeneous centroids, that is, subjects simultaneously group by similarity and by dissimilarity. The centroid paradigm reveals the relative weight of each dot among targets and distractors to the underlying grouping process, offering a more detailed, quantitative description of grouping than was previously possible. A change detection experiment reveals that conscious memory contains less than two dots and their locations, whereas an ideal detector would have to perfectly process at least 15 of 26 dots to match the subjects' centroid judgments-indicating an extraordinary capacity for preconscious grouping. A different color set yielded identical results. Grouping theories that rely on predefined feature maps would fail to explain these results. Rather, the results indicate that preconscious grouping is automatic, flexible, and rapid, and a far more complex process than previously believed.

Decision making on spatially continuous scales

A new diffusion model of decision making in continuous space is presented and tested. The model is a sequential sampling model in which both spatially continuously distributed evidence and noise are accumulated up to a decision criterion (a 1 dimensional [1D] line or a 2 dimensional [2D] plane). There are two major advances represented in this research. The first is to use spatially continuously distributed Gaussian noise in the decision process (Gaussian process or Gaussian random field noise) which allows the model to represent truly spatially continuous processes. The second is a series of experiments that collect data from a variety of tasks and response modes to provide the basis for testing the model. The model accounts for the distributions of responses over position and response time distributions for the choices. The model applies to tasks in which the stimulus and the response coincide (moving eyes or fingers to brightened areas in a field of pixels) and ones in which they do not (color, motion, and direction identification). The model also applies to tasks in which the response is made with eye movements, finger movements, or mouse movements. This modeling offers a wide potential scope of applications including application to any device or scale in which responses are made on a 1D continuous scale or in a 2D spatial field. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Human attention filters for single colors

The visual images in the eyes contain much more information than the brain can process. An important selection mechanism is feature-based attention (FBA). FBA is best described by attention filters that specify precisely the extent to which items containing attended features are selectively processed and the extent to which items that do not contain the attended features are attenuated. The centroid-judgment paradigm enables quick, precise measurements of such human perceptual attention filters, analogous to transmission measurements of photographic color filters. Subjects use a mouse to locate the centroid-the center of gravity-of a briefly displayed cloud of dots and receive precise feedback. A subset of dots is distinguished by some characteristic, such as a different color, and subjects judge the centroid of only the distinguished subset (e.g., dots of a particular color). The analysis efficiently determines the precise weight in the judged centroid of dots of every color in the display (i.e., the attention filter for the particular attended color in that context). We report 32 attention filters for single colors. Attention filters that discriminate one saturated hue from among seven other equiluminant distractor hues are extraordinarily selective, achieving attended/unattended weight ratios >20:1. Attention filters for selecting a color that differs in saturation or lightness from distractors are much less selective than attention filters for hue (given equal discriminability of the colors), and their filter selectivities are proportional to the discriminability distance of neighboring colors, whereas in the same range hue attention-filter selectivity is virtually independent of discriminabilty.