Neural synergy in visual grouping: when good continuation meets common fate

Evidence for a role of synchrony but not common fate in the perception of biological group movements

Extensive research has shown that observers are able to efficiently extract summary information from groups of people. However, little is known about the cues that determine whether multiple people are represented as a social group or as independent individuals. Initial research on this topic has primarily focused on the role of static cues. Here, we instead investigate the role of dynamic cues. In two experiments with male and female human participants, we use EEG frequency tagging to investigate the influence of two fundamental Gestalt principles - synchrony and common fate - on the grouping of biological movements. In Experiment 1, we find that brain responses coupled to four point-light figures walking together are enhanced when they move in sync vs. out of sync, but only when they are presented upright. In contrast, we found no effect of movement direction (i.e., common fate). In Experiment 2, we rule out that synchrony takes precedence over common fate by replicating the null effect of movement direction while keeping synchrony constant. These results suggest that synchrony plays an important role in the processing of biological group movements. In contrast, the role of common fate is less clear and will require further research.

Contribution of higher-order structure to perception of mirror symmetry: Role of shapes and corners

Visual mirror symmetry is a global feature that is dependent on specific low-level relationships between component elements. Initially conceptualized as virtual lines between paired elements, it has been suggested that higher-order structure between pairs of symmetric elements forming virtual four cornered shapes may also be important for strengthening the percept of mirror symmetry. We utilize corner elements, formed by joining two Gabor elements along a central midline creating vertices with variable internal angles, in a temporal integration paradigm. This allows us to specifically manipulate the presence and type of higher-order versus lower-order structure in patterns with symmetrically placed elements. We show a significant contribution of higher-order structure to the salience of visual symmetries compared with patterns with only lower-order structures. We also find that although we are more sensitive to patterns with higher-order structure, there is no difference in the temporal processing of higher-order versus lower-order patterns. These findings have important implications for existing spatial filter models of symmetry perception that rely on lower-order structures alone and reinforces the need for elaborated models that can more readily capture the complexities of real-world symmetries.

Reduced Monocular Luminance Increases Monocular Temporal Synchrony Threshold in Human Adults

Purpose

The purpose of this study was to present our investigation of the influence of reduced monocular luminance on monocular and dichoptic temporal synchrony processing in healthy adults.

Methods

Ten adults with normal or corrected to normal visual acuity participated in our psychophysical study. The temporal synchrony threshold in dichoptic (experiment 1), monocular (experiment 2), and binocular (experiment 3) viewing configurations was obtained from each observer. Four flickering Gaussian dots (one synchronous and one asynchronous pair of two dots) were displayed, from which the observers were asked to identify the asynchronous pair. The temporal phase lag in the signal pair (asynchronous) but not in the reference pair (synchronous) was varied. In addition, a neutral density (ND) filter of various intensities (1.3 and 2.0 log units) was placed before the dominant eye throughout the behavioral measurement. In the end, dichoptic, monocular, and binocular thresholds were measured for each observer.

Results

With decreasing monocular luminance, the dichoptic threshold (2 ND vs. 0 ND, P < 0.001; 2 ND vs. 1.3 ND P = 0.001) and monocular threshold (2 ND vs. 0 ND, P < 0.001; 2 ND vs. 1.3 ND, P = 0.003) increased; however, the bincoular threshold remained unaffected (P = 0.576).

Conclusions

Reduced luminance induces delay and disturbs the discrimination of temporal synchrony. Our findings have clinical implications in visual disorders.

Common Fate for Animated Transitions in Visualization

The Law of Common Fate from Gestalt psychology states that visual objects moving with the same velocity along parallel trajectories will be perceived by a human observer as grouped. However, the concept of common fate is much broader than mere velocity; in this paper we explore how common fate results from coordinated changes in luminance and size. We present results from a crowdsourced graphical perception study where we asked workers to make perceptual judgments on a series of trials involving four graphical objects under the influence of conflicting static and dynamic visual factors (position, size and luminance) used in conjunction. Our results yield the following rankings for visual grouping: motion > (dynamic luminance, size, luminance); dynamic size > (dynamic luminance, position); and dynamic luminance > size. We also conducted a follow-up experiment to evaluate the three dynamic visual factors in a more ecologically valid setting, using both a Gapminder-like animated scatterplot and a thematic map of election data. The results indicate that in practice the relative grouping strengths of these factors may depend on various parameters including the visualization characteristics and the underlying data. We discuss design implications for animated transitions in data visualization.

Configuration-Specific Attentional Modulation of Flanker – Target Lateral Interactions

Elements of a contour are often easier to detect when they possess collinearity, with their local orientations matching the global orientation of the contour. We recently reported attentional modulation of such lateral interactions between a central near-threshold target Gabor patch and flanking high-contrast patches (Freeman et al, 2001 Nature Neuroscience4 1032–1036). Here, we examined whether such attentional effects reflect specific modulation of mechanisms sensitive to collinear configurations, or instead more general modulation of sensitivity to either the global or local orientation-components of the stimulus. Thresholds for detecting a central Gabor target were measured, while observers also judged the Vernier alignment between one pair of flankers and ignored a second flanker pair (when present). Target contrast-thresholds were facilitated only when attending collinear flankers. There was no facilitation when attending flankers that shared only local orientation with the target, or flankers that fell on a global axis aligned with target orientation but having orthogonal local orientation. Ignored collinear flankers had no effect on target thresholds. These results demonstrate strong and specific attentional modulation of contour-integration mechanisms in early vision sensitive to collinear configurations.

Does cognitive perception have access to brief temporal events?

To determine whether conscious perception has access to brief temporal event, we asked subjects in an odd-man out paradigm to determine which of the four Gaussian blobs was flickering asynchronously in time. We measure synchrony thresholds as a function of the base temporal frequency for spatially scaled stimuli in foveal and peripheral vision. The results are consistent with a time delay of around 67 milliseconds (ms) for foveal vision and 91 ms for peripheral vision. We conclude that conscious perception has access to only relatively long (∼67 ms) time events.

Automatic computation of an image's statistical surprise predicts performance of human observers on a natural image detection task

To understand the neural mechanisms underlying humans' exquisite ability at processing briefly flashed visual scenes, we present a computer model that predicts human performance in a Rapid Serial Visual Presentation (RSVP) task. The model processes streams of natural scene images presented at a rate of 20Hz to human observers, and attempts to predict when subjects will correctly detect if one of the presented images contains an animal (target). We find that metrics of Bayesian surprise, which models both spatial and temporal aspects of human attention, differ significantly between RSVP sequences on which subjects will detect the target (easy) and those on which subjects miss the target (hard). Extending beyond previous studies, we here assess the contribution of individual image features including color opponencies and Gabor edges. We also investigate the effects of the spatial location of surprise in the visual field, rather than only using a single aggregate measure. A physiologically plausible feed-forward system, which optimally combines spatial and temporal surprise metrics for all features, predicts performance in 79.5% of human trials correctly. This is significantly better than a baseline maximum likelihood Bayesian model (71.7%). We can see that attention as measured by surprise, accounts for a large proportion of observer performance in RSVP. The time course of surprise in different feature types (channels) provides additional quantitative insight in rapid bottom-up processes of human visual attention and recognition, and illuminates the phenomenon of attentional blink and lag-1 sparing. Surprise also reveals classical Type-B like masking effects intrinsic in natural image RSVP sequences. We summarize these with the discussion of a multistage model of visual attention.

Distinct perceptual grouping pathways revealed by temporal carriers and envelopes

S. E. Guttman, L. A. Gilroy, and R. Blake (2005) investigated whether observers could perform temporal grouping in multi-element displays where each local element was stochastically modulated over time along one of several potential dimensions--or "messenger types"--such as contrast, position, orientation, or spatial scale. Guttman et al.'s data revealed that grouping discards messenger type and therefore support a single-pathway model that groups elements with similar temporal waveforms. In the current study, we carried out three experiments in which temporal-grouping information resided either in the carrier, the envelope, or the combined carrier and envelope of each messenger's timecourse. Results revealed that grouping is highly specific for messenger type if carrier envelopes lack grouping information but largely messenger nonspecific if carrier envelopes contain grouping information. These imply that temporal grouping is mediated by several messenger-specific carrier pathways as well as by a messenger-nonspecific envelope pathways. Findings also challenge simple temporal-filtering accounts of perceptual grouping (E. H. Adelson & H. Farid, 1999).

Threshold recognition of phantom-contour objects requires constant contrast velocity

Recognition of phantom objects--those with contours defined by rapid contrast reversal of adjacent fields of dark and light random dots--was investigated under conditions of abrupt or ramped onset and offset. Discrimination contrast thresholds were determined for a random-dot phantom letter in four possible orientations. For abrupt onset or offset, thresholds were almost independent of the duration of presentation time, over a range that varied tenfold, from 34-340 msec. However, when the onset and offset were shaped by a triangular envelope, thresholds were raised, so that form blindness occurred even when peak d ot contrasts exceeded 60%. Also under ramped onset and offset conditions, threshold contrast varied strictly linearly with stimulus duration in all subjects, suggesting a new construct--contrast velocity, the rate of change of contrast critical for phantom-object recognition.

Spatial grouping in human vision: temporal structure trumps temporal synchrony

Temporal information promotes visual grouping of local image features into global spatial form. However, experiments demonstrating time-based grouping typically confound two potential sources of information: temporal synchrony (precise timing of changes) and temporal structure (pattern of changes over time). Here, we show that observers prefer temporal structure for determining perceptual organization. That is, human vision groups elements that change according to the same global pattern, even if the changes themselves are not synchronous. This finding prompts an important, testable prediction concerning the neural mechanisms of binding: patterns of neural spiking over time may be more important than absolute spike synchrony.

The role of temporal structure in human vision

Gestalt psychologists identified several stimulus properties thought to underlie visual grouping and figure/ground segmentation, and among those properties was common fate: the tendency to group together individual objects that move together in the same direction at the same speed. Recent years have witnessed an upsurge of interest in visual grouping based on other time-dependent sources of visual information, including synchronized changes in luminance, in motion direction, and in figure/ ground relations. These various sources of temporal grouping information can be subsumed under the rubric temporal structure. In this article, the authors review evidence bearing on the effectiveness of temporal structure in visual grouping. They start with an overview of evidence bearing on temporal acuity of human vision, covering studies dealing with temporal integration and temporal differentiation. They then summarize psychophysical studies dealing with figure/ground segregation based on temporal phase differences in deterministic and stochastic events. The authors conclude with a brief discussion of neurophysiological implications of these results.

Computational modeling and exploration of contour integration for visual saliency

We propose a computational model of contour integration for visual saliency. The model uses biologically plausible devices to simulate how the representations of elements aligned collinearly along a contour in an image are enhanced. Our model adds such devices as a dopamine-like fast plasticity, local GABAergic inhibition and multi-scale processing of images. The fast plasticity addresses the problem of how neurons in visual cortex seem to be able to influence neurons they are not directly connected to, for instance, as observed in contour closure effect. Local GABAergic inhibition is used to control gain in the system without using global mechanisms which may be non-plausible given the limited reach of axonal arbors in visual cortex. The model is then used to explore not only its validity in real and artificial images, but to discover some of the mechanisms involved in processing of complex visual features such as junctions and end-stops as well as contours. We present evidence for the validity of our model in several phases, starting with local enhancement of only a few collinear elements. We then test our model on more complex contour integration images with a large number of Gabor elements. Sections of the model are also extracted and used to discover how the model might relate contour integration neurons to neurons that process end-stops and junctions. Finally, we present results from real world images. Results from the model suggest that it is a good current approximation of contour integration in human vision. As well, it suggests that contour integration mechanisms may be strongly related to mechanisms for detecting end-stops and junction points. Additionally, a contour integration mechanism may be involved in finding features for objects such as faces. This suggests that visual cortex may be more information efficient and that neural regions may have multiple roles.

A spatial explanation for synchrony biases in perceptual grouping: consequences for the temporal-binding hypothesis

If two images are shown in rapid sequential order, they are perceived as a single, fused image. Despite this, recent studies have revealed that fundamental perceptual processes are influenced by extremely brief temporal offsets in stimulus presentation. Some researchers have suggested that this is due to the action of a cortical temporal-binding mechanism, which would serve to keep multiple mental representations of one object distinct from those of other objects. There is now gathering evidence that these studies should be reassessed. This article describes evidence for sensitivity to fixational eye and head movements, which provides a purely spatial explanation for the earlier results. Taken in conjunction with other studies, the work serves to undermine the current body of behavioral evidence for a temporal-binding mechanism.

Task-dependent modulation of target-flanker lateral interactions in vision

Visibility of a central target Gabor element often improves in the presence of collinear flankers. Such lateral interactions may reflect fundamental mechanisms underlying the perceptual integration of contours in early vision. We recently reported (Freeman, Sagi, & Driver, 2001) attentional modulation of these interactions. Here, we test whether this modulation is task dependent. Subjects had to detect a near-threshold central target while performing a secondary discrimination task on one pair of flankers that could appear with another distractor pair (one pair collinear with the target, the other orthogonal). Central target thresholds were lowered when collinear flankers were judged for the secondary task, but only when this task concerned the global spatial relationship between these flankers (discrimination of their Vernier offset or global orientation). Other secondary tasks involving discriminating the local orientations, contrasts, or colors of the relevant flanker pair produced no such attentional modulation. However, this task-dependent modulation was observed only when two flanker pairs were present, not for displays with only a single flanker pair. Top-down modulation of lateral interactions may function to select between overlapping potential contours whenever the global spatial properties of one are task relevant.

Mixed messengers, unified message: spatial grouping from temporal structure

In dynamic visual environments, objects can differ from their backgrounds in terms of their associated temporal structure--the time course of changes in some stimulus property defining object and background. In a series of experiments, we investigated whether different "messengers" of temporal structure group into coherent spatial forms. Observers viewed arrays of Gabor patches in which different temporal structures designated figure and ground regions; extracting the figure required grouping across synchronized orientation, spatial frequency, phase, and/or contrast changes. Observers were able to extract spatial form from temporal structure even when information had to be combined across different messengers. Further, mixing messengers of temporal structure proved cost-free: task performance when grouping across messengers approximated performance when all information resided within a single messenger. Thus, the visual system can abstract temporal structure regardless of the messenger of the dynamic event; a coherent spatial structure emerges from this abstracted temporal structure.

Pop-out from abrupt visual onsets

We report a novel psychophysical paradigm that distinguishes the information present in abrupt stimulus onset from that in the following display. The task is to pick the one odd item from a set added to a pre-existing background of similar items. When all new items are added simultaneously, observers are impaired even at distinguishing one red item amongst several green ones. An asynchrony of about 40 ms between target and distracter items restores performance, with evidence that it is cortical, rather than stimulus timing difference that is significant. The results are consistent with a role for neural synchrony in dynamic grouping.

Grouping and segmentation in binocular rivalry

Dichoptic presentation of dot arrays produces binocular rivalry if the arrays are of opposite contrast relative to background. Rivalry can occur even if individual dots in one eye's array do not overlap with the dots in the contralateral eye's array. The amount of unitary perception of only one array is a measure of the probability that the stimuli rival as textured surfaces rather than as portions of arrays or as individual dot elements. In accordance with Gestalt grouping principles, arrays of uniform brightness or color produced more unitary perception than mixed arrays. However, experiments with parametric variation of dot motion coherence suggested that segmentation mechanisms based on detection of collinearity can also influence perceptual selection and suppression in binocular rivalry.

Good continuation with kinetic edges

We examined the perceptual formation of extended contours from second-order kinetic-edges created by motion discontinuities. Paths were formed by spatially separate kinetic-edge elements, aligned along smooth paths, and embedded in randomly oriented noise elements. Path detection was severely degraded when the sign of motion contrast alternated along the path compared to when the same sign was used, or if random motion direction was assigned to each edge element, or if alternating opposite motion directions was used along the paths. Performance increased monotonically with the length of the path. Irrespectively of path curvature a fast temporal summation occurs within the first 200-400 ms and then levels off. Hence, the kinetic-edge grouping is relatively fast and a pure second-order process that senses whether the motion is globally in the same phase and direction along extended contours.

A symbolic-connectionist theory of relational inference and generalization

The authors present a theory of how relational inference and generalization can be accomplished within a cognitive architecture that is psychologically and neurally realistic. Their proposal is a form of symbolic connectionism: a connectionist system based on distributed representations of concept meanings, using temporal synchrony to bind fillers and roles into relational structures. The authors present a specific instantiation of their theory in the form of a computer simulation model, Learning and Inference with Schemas and Analogies (LISA). By using a kind of self-supervised learning, LISA can make specific inferences and form new relational generalizations and can hence acquire new schemas by induction from examples. The authors demonstrate the sufficiency of the model by using it to simulate a body of empirical phenomena concerning analogical inference and relational generalization.

Perceptual learning of temporal structure

We investigated the extent to which the ability to perceive spatial form from temporal structure (TS) improves with practice. Observers trained monocularly for a number of consecutive days on a shape discrimination task, with one group of observers judging shape defined by luminance contrast between target and background elements and another group judging shape defined by correlated TS (synchronized changes in motion direction between target and background elements). Substantial learning was found for both shape tasks, with complete interocular transfer of training. Observers trained on TS showed no transfer of learning to the luminance condition, but observers trained using the luminance display with incidental synchronized changes did show transfer to the TS task. Possible underlying neural changes are discussed.

Visual coherence of moving and stationary image changes

Detection thresholds were compared for moving and stationary oscillations with equivalent contrast changes. Motion was more detectable than stationary oscillation, and the difference increased with size of the feature (a Gaussian blob). Phase discriminations between a center and two flanking features were much better for motion than for stationary oscillation. Motion phase discriminations were similar to motion detection and were robust over increases in spatial separation and temporal frequency, but not so for stationary oscillations. Separate visual motion signals were positively correlated, but visual signals for stationary oscillation were negatively correlated. Evidently, motion produces visually coherent changes in image structure, but stationary contrast oscillation does not.