Loss of positional information when tracking multiple moving dots: The role of visual memory

The perceptual timescape: Perceptual history on the sub-second scale

There is a high-capacity store of brief time span (∼1000 ms) which information enters from perceptual processing, often called iconic memory or sensory memory. It is proposed that a main function of this store is to hold recent perceptual information in a temporally segregated representation, named the perceptual timescape. The perceptual timescape is a continually active representation of change and continuity over time that endows the perceived present with a perceived history. This is accomplished primarily by two kinds of time marking information: time distance information, which marks all items of information in the perceptual timescape according to how far in the past they occurred, and ordinal temporal information, which organises items of information in terms of their temporal order. Added to that is information about connectivity of perceptual objects over time. These kinds of information connect individual items over a brief span of time so as to represent change, persistence, and continuity over time. It is argued that there is a one-way street of information flow from perceptual processing either to the perceived present or directly into the perceptual timescape, and thence to working memory. Consistent with that, the information structure of the perceptual timescape supports postdictive reinterpretations of recent perceptual information. Temporal integration on a time scale of hundreds of milliseconds takes place in perceptual processing and does not draw on information in the perceptual timescape, which is concerned with temporal segregation, not integration.

Perception of Happening: How the Brain Deals with the No-History Problem

In physics, the temporal dimension has units of infinitesimally brief duration. Given this, how is it possible to perceive things, such as motion, music, and vibrotactile stimulation, that involve extension across many units of time? To address this problem, it is proposed that there is what is termed an "information construct of happening" (ICOH), a simultaneous representation of recent, temporally differentiated perceptual information on the millisecond time scale. The main features of the ICOH are (i) time marking, semantic labeling of all information in the ICOH with ordinal temporal information and distance from what is informationally identified as the present moment, (ii) vector informational features that specify kind, direction, and rate of change for every feature in a percept, and (iii) connectives, information relating vector informational features at adjacent temporal locations in the ICOH. The ICOH integrates products of perceptual processing with recent historical information in sensory memory on the subsecond time scale. Perceptual information about happening in informational sensory memory is encoded in semantic form that preserves connected semantic trails of vector and timing information. The basic properties of the ICOH must be supported by a general and widespread timing mechanism that generates ordinal and interval timing information and it is suggested that state-dependent networks may suffice for that purpose. Happening, therefore, is perceived at a moment and is constituted by an information structure of connected recent historical information.

Sensory Memory Is Allocated Exclusively to the Current Event-Segment

The Atkinson-Shiffrin modal model forms the foundation of our understanding of human memory. It consists of three stores (Sensory Memory (SM), also called iconic memory, Short-Term Memory (STM), and Long-Term Memory (LTM)), each tuned to a different time-scale. Since its inception, the STM and LTM components of the modal model have undergone significant modifications, while SM has remained largely unchanged, representing a large capacity system funneling information into STM. In the laboratory, visual memory is usually tested by presenting a brief static stimulus and, after a delay, asking observers to report some aspect of the stimulus. However, under ecological viewing conditions, our visual system receives a continuous stream of inputs, which is segmented into distinct spatio-temporal segments, called events. Events are further segmented into event-segments. Here we show that SM is not an unspecific general funnel to STM but is allocated exclusively to the current event-segment. We used a Multiple-Object Tracking (MOT) paradigm in which observers were presented with disks moving in different directions, along bi-linear trajectories, i.e., linear trajectories, with a single deviation in direction at the mid-point of each trajectory. The synchronized deviation of all of the trajectories produced an event stimulus consisting of two event-segments. Observers reported the pre-deviation or the post-deviation directions of the trajectories. By analyzing observers' responses in partial- and full-report conditions, we investigated the involvement of SM for the two event-segments. The hallmarks of SM hold only for the current event segment. As the large capacity SM stores only items involved in the current event-segment, the need for event-tagging in SM is eliminated, speeding up processing in active vision. By characterizing how memory systems are interfaced with ecological events, this new model extends the Atkinson-Shiffrin model by specifying how events are stored in the first stage of multi-store memory systems.

Preattentive and Predictive Processing of Visual Motion

Interaction with the environment requires fast and reliable sensory processing. The visual system is confronted with a continuous flow of high-dimensional input (e.g. orientation, color, motion). From a theoretical point of view, it would be advantageous if critical information was processed independent of attentional load, i.e. preattentively. Here, we hypothesized that visual motion is such a critical signal and aimed for a neural signature of its preattentive encoding. Furthermore, we were interested in the neural correlates of predictability of linear motion trajectories based on the presence or absence of preceding motion. We presented a visual oddball paradigm and studied event-related potentials (ERPs). Stimuli were linearly moving Gabor patches that disappeared behind an occluder. The difference between deviant and standard trials was a trajectory change which happened behind the occluder in deviant trials only, inducing a prediction error. As hypothesized, we found a visual mismatch negativity-component over parietal and occipital electrodes. In a further condition, trials without preceding motion were presented in which the patch just appeared from behind the occluder and, hence, was not predictable. We found larger ERP-components for unpredictable stimuli. In summary, our results provide evidence for a preattentive and predictive processing of linear trajectories of visual motion.

The reference frame for encoding and retention of motion depends on stimulus set size

The goal of this study was to investigate the reference frames used in perceptual encoding and storage of visual motion information. In our experiments, observers viewed multiple moving objects and reported the direction of motion of a randomly selected item. Using a vector-decomposition technique, we computed performance during smooth pursuit with respect to a spatiotopic (nonretinotopic) and to a retinotopic component and compared them with performance during fixation, which served as the baseline. For the stimulus encoding stage, which precedes memory, we found that the reference frame depends on the stimulus set size. For a single moving target, the spatiotopic reference frame had the most significant contribution with some additional contribution from the retinotopic reference frame. When the number of items increased (Set Sizes 3 to 7), the spatiotopic reference frame was able to account for the performance. Finally, when the number of items became larger than 7, the distinction between reference frames vanished. We interpret this finding as a switch to a more abstract nonmetric encoding of motion direction. We found that the retinotopic reference frame was not used in memory. Taken together with other studies, our results suggest that, whereas a retinotopic reference frame may be employed for controlling eye movements, perception and memory use primarily nonretinotopic reference frames. Furthermore, the use of nonretinotopic reference frames appears to be capacity limited. In the case of complex stimuli, the visual system may use perceptual grouping in order to simplify the complexity of stimuli or resort to a nonmetric abstract coding of motion information.

Bottlenecks of motion processing during a visual glance: the leaky flask model

Where do the bottlenecks for information and attention lie when our visual system processes incoming stimuli? The human visual system encodes the incoming stimulus and transfers its contents into three major memory systems with increasing time scales, viz., sensory (or iconic) memory, visual short-term memory (VSTM), and long-term memory (LTM). It is commonly believed that the major bottleneck of information processing resides in VSTM. In contrast to this view, we show major bottlenecks for motion processing prior to VSTM. In the first experiment, we examined bottlenecks at the stimulus encoding stage through a partial-report technique by delivering the cue immediately at the end of the stimulus presentation. In the second experiment, we varied the cue delay to investigate sensory memory and VSTM. Performance decayed exponentially as a function of cue delay and we used the time-constant of the exponential-decay to demarcate sensory memory from VSTM. We then decomposed performance in terms of quality and quantity measures to analyze bottlenecks along these dimensions. In terms of the quality of information, two thirds to three quarters of the motion-processing bottleneck occurs in stimulus encoding rather than memory stages. In terms of the quantity of information, the motion-processing bottleneck is distributed, with the stimulus-encoding stage accounting for one third of the bottleneck. The bottleneck for the stimulus-encoding stage is dominated by the selection compared to the filtering function of attention. We also found that the filtering function of attention is operating mainly at the sensory memory stage in a specific manner, i.e., influencing only quantity and sparing quality. These results provide a novel and more complete understanding of information processing and storage bottlenecks for motion processing.

Comparison of Blue-Yellow Opponent Color Contrast Sensitivity Function between Female Badminton Players and Non-athletes

Purpose

To compare the chromatic contrast sensitivity function (CSF) for the blue-yellow opponent channel (BYOC) between female badminton players and non-athlete controls.

Methods

We recruited 40 young females (18-25 years old) who played badminton for at least 5 consecutive years as the test group, and 30 age-matched female controls who had no history of regular physical activity. The Pattern Generator™ system was used to test the CSF for the BYOC which was performed at three spatial frequencies (SFs) of 2 cycles per degree (cpd), 5 cpd, and 25 cpd.

Results

Comparison of BYOC thresholds showed significantly better results in the test group for all three SFs (P<0.001). Band pass shift (better CSF in the middle SF) was seen in the test group. The control group had low pass (better CSF in the low SF). Ocular motility (heterophoria, fusional convergence and divergence at far and near distances, and near point of convergence) was better in the test group, but the inter-group difference was not significant.

Conclusions

The BYOC threshold results for badminton players indicated a better visual performance which may be a result of enhanced performance of the parallel processing of the parvocellular and magnocellular systems. This may be inherent and/or acquired in badminton players. In addition, badminton players appear to have developed sensory-motor programmed activities. Testing the CSF for BYOC may be useful for athlete selection in different levels and/or used as a criterion for screening players in the field of badminton.

Changing target trajectories influences tracking performance

People have the ability to attentively select and successfully track several moving objects, a process known as multiple-object tracking (MOT; Pylyshyn & Storm Spatial Vision 3: 179-197, 1988). Various factors have been known to influence MOT performance, such as speed, number of distractors, and proximity, while recent work has suggested that object trajectories may also be a factor (Fencsik, Kleiger, & Horowitz Perception and Psychophysics 69: 567-577, 2007). Meanwhile, unexpected changes in motion information have been demonstrated to be a critical facet for attracting attention Howard & Holcombe Attention, Perception & Psychophysics 72: 2087-2095, (2010). Therefore, we suggest that unexpected changes in target trajectories are an important factor in tracking performance. The research presented here controlled for spatial proximity while manipulating the number of instances in which an object changed trajectory. We found that spatial proximity had no effect on tracking performance but, rather, as the number of trajectory changes increased, tracking performance suffered. Results imply that the ability to track multiple moving objects is limited by unexpected changes in direction.

Normal aging delays and compromises early multifocal visual attention during object tracking

Declines in selective attention are one of the sources contributing to age-related impairments in a broad range of cognitive functions. Most previous research on mechanisms underlying older adults' selection deficits has studied the deployment of visual attention to static objects and features. Here we investigate neural correlates of age-related differences in spatial attention to multiple objects as they move. We used a multiple object tracking task, in which younger and older adults were asked to keep track of moving target objects that moved randomly in the visual field among irrelevant distractor objects. By recording the brain's electrophysiological responses during the tracking period, we were able to delineate neural processing for targets and distractors at early stages of visual processing (~100-300 msec). Older adults showed less selective attentional modulation in the early phase of the visual P1 component (100-125 msec) than younger adults, indicating that early selection is compromised in old age. However, with a 25-msec delay relative to younger adults, older adults showed distinct processing of targets (125-150 msec), that is, a delayed yet intact attentional modulation. The magnitude of this delayed attentional modulation was related to tracking performance in older adults. The amplitude of the N1 component (175-210 msec) was smaller in older adults than in younger adults, and the target amplification effect of this component was also smaller in older relative to younger adults. Overall, these results indicate that normal aging affects the efficiency and timing of early visual processing during multiple object tracking.

Misperceptions in the trajectories of objects undergoing curvilinear motion

Trajectory perception is crucial in scene understanding and action. A variety of trajectory misperceptions have been reported in the literature. In this study, we quantify earlier observations that reported distortions in the perceived shape of bilinear trajectories and in the perceived positions of their deviation. Our results show that bilinear trajectories with deviation angles smaller than 90 deg are perceived smoothed while those with deviation angles larger than 90 degrees are perceived sharpened. The sharpening effect is weaker in magnitude than the smoothing effect. We also found a correlation between the distortion of perceived trajectories and the perceived shift of their deviation point. Finally, using a dual-task paradigm, we found that reducing attentional resources allocated to the moving target causes an increase in the perceived shift of the deviation point of the trajectory. We interpret these results in the context of interactions between motion and position systems.

Tracking planets and moons: mechanisms of object tracking revealed with a new paradigm

People can attend to and track multiple moving objects over time. Cognitive theories of this ability emphasize location information and differ on the importance of motion information. Results from several experiments have shown that increasing object speed impairs performance, although speed was confounded with other properties such as proximity of objects to one another. Here, we introduce a new paradigm to study multiple object tracking in which object speed and object proximity were manipulated independently. Like the motion of a planet and moon, each target-distractor pair rotated about both a common local point as well as the center of the screen. Tracking performance was strongly affected by object speed even when proximity was controlled. Additional results suggest that two different mechanisms are used in object tracking--one sensitive to speed and proximity and the other sensitive to the number of distractors. These observations support models of object tracking that include information about object motion and reject models that use location alone.

Direction information in multiple object tracking is limited by a graded resource

Is multiple object tracking (MOT) limited by a fixed set of structures (slots), a limited but divisible resource, or both? Here, we answer this question by measuring the precision of the direction representation for tracked targets. The signature of a limited resource is a decrease in precision as the square root of the tracking load. The signature of fixed slots is a fixed precision. Hybrid models predict a rapid decrease to asymptotic precision. In two experiments, observers tracked moving disks and reported target motion direction by adjusting a probe arrow. We derived the precision of representation of correctly tracked targets using a mixture distribution analysis. Precision declined with target load according to the square-root law up to six targets. This finding is inconsistent with both pure and hybrid slot models. Instead, directional information in MOT appears to be limited by a continuously divisible resource.

The perception of motion smear during eye and head movements

Because the visual system integrates information across time, an image that moves on the retina would be expected to be perceived as smeared. In this article, we summarize the previous evidence that human observers perceive a smaller extent of smear when retinal image motion results from an eye or head movement, compared to when a physically moving target generates comparable image motion while the eyes and head are still. This evidence indicates that the reduction of perceived motion smear is asymmetrical, occurring only for targets that move against the direction of an eye or head movement. In addition, we present new data to show that no reduction of perceived motion smear occurs for targets that move in either direction during a visually-induced perception of self motion. We propose that low-level extra-retinal eye- and head-movement signals are responsible for the reduction of perceived motion smear, by decreasing the duration of the temporal impulse response. Although retinal as well as extra-retinal mechanisms can reduce the extent of perceived motion smear, available evidence suggests that improved visual functioning may occur only when an extra-retinal mechanism reduces the perception of smear.

Aging and visual counting

Background

Much previous work on how normal aging affects visual enumeration has been focused on the response time required to enumerate, with unlimited stimulus duration. There is a fundamental question, not yet addressed, of how many visual items the aging visual system can enumerate in a “single glance”, without the confounding influence of eye movements.

Methodology/Principal Findings

We recruited 104 observers with normal vision across the age span (age 21–85). They were briefly (200 ms) presented with a number of well- separated black dots against a gray background on a monitor screen, and were asked to judge the number of dots. By limiting the stimulus presentation time, we can determine the maximum number of visual items an observer can correctly enumerate at a criterion level of performance (counting threshold, defined as the number of visual items at which ≈63% correct rate on a psychometric curve), without confounding by eye movements. Our findings reveal a 30% decrease in the mean counting threshold of the oldest group (age 61–85: ∼5 dots) when compared with the youngest groups (age 21–40: 7 dots). Surprisingly, despite decreased counting threshold, on average counting accuracy function (defined as the mean number of dots reported for each number tested) is largely unaffected by age, reflecting that the threshold loss can be primarily attributed to increased random errors. We further expanded this interesting finding to show that both young and old adults tend to over-count small numbers, but older observers over-count more.

Conclusion/Significance

Here we show that age reduces the ability to correctly enumerate in a glance, but the accuracy (veridicality), on average, remains unchanged with advancing age. Control experiments indicate that the degraded performance cannot be explained by optical, retinal or other perceptual factors, but is cortical in origin.

Where Did they Go? Recovering Dynamic Objects after Interruptions

The aim of the current study was to identify strategies used in recovering dynamic objects after long interruptions. The primary task in this study required participants to track a set of six target dots amongst eight distracter dots and locate them after a 30 second interruption. In one condition the dots reappeared in their displaced locations (moving condition) while in the other condition they reappeared in their original pre-interruption locations (not-moving condition). Consistent with previous research we found that the not-moving condition had significantly better accuracy than the moving condition. Above chance accuracy was found in both conditions. Reaction time data provided further insight into the strategies used to recover displaced objects. In the end, it was concluded that pre-interruption location is the most salient and easily remembered characteristic. Reaction time data did provide preliminary support for the use of on-line tracking during interruptions, although such abilities seem to be limited in capacity to approximately three targets. The results of this research have wide spread implications to domains requiring constant tracking of objects such as air traffic control.

High-capacity, transient retention of direction-of-motion information for multiple moving objects

The multiple-object tracking paradigm (MOT) has been used extensively for studying dynamic visual attention, but the basic mechanisms which subserve this capability are as yet unknown. Among the unresolved issues surrounding MOT are the relative importance of motion (as opposed to positional) information and the role of various memory mechanisms. We sought to quantify the capacity and dynamics for retention of direction-of-motion information when viewing a multiple-object motion stimulus similar to those used in MOT. Observers viewed three to nine objects in random linear motion and then reported motion direction after motion ended. Using a partial-report paradigm and varying the parameters of set size and time of retention, we found evidence for two complementary memory systems, one transient with high capacity and a second sustained system with low capacity. For the transient high-capacity memory, retention capacity was equally high whether object motion lasted several seconds or a fraction of a second. Also, a graded deterioration in performance with increased set size lends support to a flexible-capacity theory of MOT.

Demand-based dynamic distribution of attention and monitoring of velocities during multiple-object tracking

The ability to track multiple moving objects with attention has been the focus of much research. However, the literature is relatively inconclusive regarding two key aspects of this ability, (1) whether the distribution of attention among the tracked targets is fixed during a period of tracking or is dynamically adjusted, and (2) whether motion information (direction and/or speed) is used to anticipate target locations even when velocities constantly change due to inter-object collisions. These questions were addressed by analyzing target-localization errors. Targets in crowded situations (i.e., those in danger of being lost) were localized more precisely than were uncrowded targets. Furthermore, the response vector (pointing from the target location to the reported location) was tuned to the direction of target motion, and observers with stronger direction tuning localized targets more precisely. Overall, our results provide evidence that multiple-object tracking mechanisms dynamically adjust the spatial distribution of attention in a demand-based manner (allocating more resources to targets in crowded situations) and utilize motion information (especially direction information) to anticipate target locations.