Motion-based analysis of spatial patterns by the human visual system

Flicker adaptation improves acuity for briefly presented stimuli by reducing crowding

Adaptation to flickering/dynamic noise improves visual acuity for briefly presented stimuli (Arnold et al., 2016). Here, we investigate whether such adaptation operates directly on our ability to see detail or by changing fixational eye movements and pupil size or by reducing visual crowding. Following earlier work, visual acuity was measured in observers who were either unadapted or who had adapted to a 60-Hz flickering noise pattern. Participants reported the orientation of a white tumbling-T target (four-alternative forced choice [4AFC], ⊤⊣⊥⊢). The target was presented for 110 ms either in isolation or flanked by randomly oriented T's (e.g., ⊣⊤⊢) followed by an isolated (+) or flanked (+++) mask, respectively. We measured fixation stability (using an infrared eye tracker) while observers performed the task (with and without adaptation). Visual acuity improved modestly (around 8.4%) for flanked optotypes following adaptation to flicker (mean, -0.038 ± 0.063 logMAR; p = 0.015; BF10 = 3.66) but did not when measured with isolated letters (mean, -0.008 ± 0.055 logMAR; p = 0.5; BF10 = 0.29). The magnitude of acuity improvement was associated with individuals' (unadapted) susceptibility to crowding (the ratio of crowded to uncrowded acuity; r = -0.58, p = 0.008, BF10 = 7.70) but to neither fixation stability nor pupil size. Confirming previous reports, flicker improved acuity for briefly presented stimuli, but we show that this was only the case for crowded letters. These improvements likely arise from attenuation of sensitivity to a transient low spatial frequency (SF) image structure (Arnold et al., 2016; Tagoh et al., 2022), which may, for example, reduce masking of high SFs by low SFs. We also suggest that this attenuation could reduce backward masking and so reduce foveal crowding.

Temporal integration characteristics of an image defined by binocular disparity cues

We can correctly recognize the content of an image by presenting all of the elements within a limited time, such as in a slit view or a divided painting image. It is important to clarify how temporally divided information is integrated and perceived to understand the temporal properties of the information-processing mechanism of visual systems. Previous studies related to this topic have often used two-dimensional pictorial stimuli; however, few have considered the temporal integration of binocular disparity for the recognition of objects defined with disparity. In this study, we examined image recognition properties based on the temporal integration of binocular disparity, by comparing that based on the temporal integration of luminance. The effect of element onset asynchrony (the time lag among presented elements) was somewhat similar between disparity and luminance with respect to randomly divided elements. On the other hand, under slit-vision conditions, the tolerance range of spatiotemporal integration for luminance stimuli was much wider than that for disparity stimuli. These results indicate that the temporal integration mechanism in localized areas is common to disparity and luminance, but that for global motion shows differences between the two mechanisms. Thus, we conclude that global motion has little contribution to the temporal integration of binocular disparity information for image recognition.

Direction of Apparent Motion During Smooth Pursuit Is Determined Using a Mixture of Retinal and Objective Proximities

Many studies have investigated various effects of smooth pursuit on visual motion processing, especially the effects related to the additional retinal shifts produced by eye movement. In this article, we show that the perception of apparent motion during smooth pursuit is determined by the interelement proximity in retinal coordinates and also by the proximity in objective world coordinates. In Experiment 1, we investigated the perceived direction of the two-frame apparent motion of a square-wave grating with various displacement sizes under fixation and pursuit viewing conditions. The retinal and objective displacements between the two frames agreed with each other under the fixation condition. However, the displacements differed by 180 degrees in terms of phase shift, under the pursuit condition. The proportions of the reported motion direction between the two viewing conditions did not coincide when they were plotted as a function of either the retinal displacement or of the objective displacement; however, they did coincide when plotted as a function of a mixture of the two. The result from Experiment 2 showed that the perceived jump size of the apparent motion was also dependent on both retinal and objective displacements. Our findings suggest that the detection of the apparent motion during smooth pursuit considers the retinal proximity and also the objective proximity. This mechanism may assist with the selection of a motion path that is more likely to occur in the real world and, therefore, be useful for ensuring perceptual stability during smooth pursuit.

Motion Perception: From Detection to Interpretation

Visual motion processing can be conceptually divided into two levels. In the lower level, local motion signals are detected by spatiotemporal-frequency-selective sensors and then integrated into a motion vector flow. Although the model based on V1-MT physiology provides a good computational framework for this level of processing, it needs to be updated to fully explain psychophysical findings about motion perception, including complex motion signal interactions in the spatiotemporal-frequency and space domains. In the higher level, the velocity map is interpreted. Although there are many motion interpretation processes, we highlight the recent progress in research on the perception of material (e.g., specular reflection, liquid viscosity) and on animacy perception. We then consider possible linking mechanisms of the two levels and propose intrinsic flow decomposition as the key problem. To provide insights into computational mechanisms of motion perception, in addition to psychophysics and neurosciences, we review machine vision studies seeking to solve similar problems.

The reference frame for encoding and retention of motion depends on stimulus set size

The goal of this study was to investigate the reference frames used in perceptual encoding and storage of visual motion information. In our experiments, observers viewed multiple moving objects and reported the direction of motion of a randomly selected item. Using a vector-decomposition technique, we computed performance during smooth pursuit with respect to a spatiotopic (nonretinotopic) and to a retinotopic component and compared them with performance during fixation, which served as the baseline. For the stimulus encoding stage, which precedes memory, we found that the reference frame depends on the stimulus set size. For a single moving target, the spatiotopic reference frame had the most significant contribution with some additional contribution from the retinotopic reference frame. When the number of items increased (Set Sizes 3 to 7), the spatiotopic reference frame was able to account for the performance. Finally, when the number of items became larger than 7, the distinction between reference frames vanished. We interpret this finding as a switch to a more abstract nonmetric encoding of motion direction. We found that the retinotopic reference frame was not used in memory. Taken together with other studies, our results suggest that, whereas a retinotopic reference frame may be employed for controlling eye movements, perception and memory use primarily nonretinotopic reference frames. Furthermore, the use of nonretinotopic reference frames appears to be capacity limited. In the case of complex stimuli, the visual system may use perceptual grouping in order to simplify the complexity of stimuli or resort to a nonmetric abstract coding of motion information.

Motion-based super-resolution in the peripheral visual field

Improvements in foveal acuity for moving targets have been interpreted as evidence for the ability of the visual system to combine information over space and time, in order to reconstruct the image at a higher resolution (super-resolution). Here, we directly test whether this occurs in the peripheral visual field and discuss its potential for improving functional capacity in ocular disease. The effect of motion on visual acuity was first compared under conditions in which performance was limited either by natural undersampling in the retinal periphery or by the presence of overlaid masks with opaque elements to simulate retinal loss. To equate the information content of moving and static sequences, we next manipulated the dynamic properties of the masks. Finally, we determined the dependence of motion-related improvements on the object of motion (target or mask) and its trajectory (smooth or jittered). Motion improved visual acuity for masked but not unmasked peripheral targets. Equating the information content of moving and static conditions removed some but not all of this benefit. Residual motion-related improvements were largest in conditions in which the target moved along a consistent and predictable path. Our results show that motion can improve peripheral acuity in situations in which performance is limited by abnormal undersampling. These findings are consistent with the operation of a super-resolution system and could have important implications for any pathology that alters the regular sampling properties of the retinal mosaic.

Active Vision: Dynamic Reformatting of Visual Information by the Saccade-Drift Cycle

Visual processing depends on rapid parsing of global features followed by analysis of fine detail. A new study suggests that this transformation is enabled by a cycle of saccades and fixational drifts, which reformat visual input to match the spatiotemporal sensitivity of fast and slow neuronal pathways.

Contour Integration over Time: Psychophysical and fMRI Evidence

The brain integrates discrete but collinear stimuli to perceive global contours. Previous contour integration (CI) studies mainly focus on integration over space, and CI is attributed to either V1 long-range connections or contour processing in high-visual areas that top-down modulate V1 responses. Here, we show that CI also occurs over time in a design that minimizes the roles of V1 long-range interactions. We use tilted contours embedded in random orientation noise and moving horizontally behind a fixed vertical slit. Individual contour elements traveling up/down within the slit would be encoded over time by parallel, rather than aligned, V1 neurons. However, we find robust contour detection even when the slit permits only one viewable contour element. Similar to CI over space, CI over time also obeys the rule of collinearity. fMRI evidence shows that while CI over space engages visual areas as early as V1, CI over time mainly engages higher dorsal and ventral visual areas involved in shape processing, as well as posterior parietal regions involved in visual memory that can represent the orientation of temporally integrated contours. These results suggest at least partially dissociable mechanisms for implementing the Gestalt rule of continuity in CI over space and time.

Sharpening vision by adapting to flicker

Human vision is surprisingly malleable. A static stimulus can seem to move after prolonged exposure to movement (the motion aftereffect), and exposure to tilted lines can make vertical lines seem oppositely tilted (the tilt aftereffect). The paradigm used to induce such distortions (adaptation) can provide powerful insights into the computations underlying human visual experience. Previously spatial form and stimulus dynamics were thought to be encoded independently, but here we show that adaptation to stimulus dynamics can sharpen form perception. We find that fast flicker adaptation (FFAd) shifts the tuning of face perception to higher spatial frequencies, enhances the acuity of spatial vision-allowing people to localize inputs with greater precision and to read finer scaled text, and it selectively reduces sensitivity to coarse-scale form signals. These findings are consistent with two interrelated influences: FFAd reduces the responsiveness of magnocellular neurons (which are important for encoding dynamics, but can have poor spatial resolution), and magnocellular responses contribute coarse spatial scale information when the visual system synthesizes form signals. Consequently, when magnocellular responses are mitigated via FFAd, human form perception is transiently sharpened because "blur" signals are mitigated.

A New Conceptualization of Human Visual Sensory-Memory

Memory is an essential component of cognition and disorders of memory have significant individual and societal costs. The Atkinson–Shiffrin “modal model” forms the foundation of our understanding of human memory. It consists of three stores: Sensory Memory (SM), whose visual component is called iconic memory, Short-Term Memory (STM; also called working memory, WM), and Long-Term Memory (LTM). Since its inception, shortcomings of all three components of the modal model have been identified. While the theories of STM and LTM underwent significant modifications to address these shortcomings, models of the iconic memory remained largely unchanged: A high capacity but rapidly decaying store whose contents are encoded in retinotopic coordinates, i.e., according to how the stimulus is projected on the retina. The fundamental shortcoming of iconic memory models is that, because contents are encoded in retinotopic coordinates, the iconic memory cannot hold any useful information under normal viewing conditions when objects or the subject are in motion. Hence, half-century after its formulation, it remains an unresolved problem whether and how the first stage of the modal model serves any useful function and how subsequent stages of the modal model receive inputs from the environment. Here, we propose a new conceptualization of human visual sensory memory by introducing an additional component whose reference-frame consists of motion-grouping based coordinates rather than retinotopic coordinates. We review data supporting this new model and discuss how it offers solutions to the paradoxes of the traditional model of sensory memory.

Field-like interactions between motion-based reference frames

A reference frame is required to specify how motion is perceived. For example, the motion of part of an object is usually perceived relative to the motion of the object itself. Johansson (Psychological Research, 38, 379-393, 1976) proposed that the perceptual system carries out a vector decomposition, which rewsults in common and relative motion percepts. Because vector decomposition is an ill-posed problem, several studies have introduced constraints by means of which the number of solutions can be substantially reduced. Here, we have adopted an alternative approach and studied how, rather than why, a subset of solutions is selected by the visual system. We propose that each retinotopic motion vector creates a reference-frame field in the retinotopic space, and that the fields created by different motion vectors interact in order to determine a motion vector that will serve as the reference frame at a given point and time in space. To test this theory, we performed a set of psychophysical experiments. The field-like influence of motion-based reference frames was manifested by increased nonspatiotopic percepts of the backward motion of a target square with decreasing distance from a drifting grating. We then sought to determine whether these field-like effects of motion-based reference frames can also be extended to stationary landmarks. The results suggest that reference-field interactions occur only between motion-generated fields. Finally, we investigated whether and how different reference fields interact with each other, and found that different reference-field interactions are nonlinear and depend on how the motion vectors are grouped. These findings are discussed from the perspective of the reference-frame metric field (RFMF) theory, according to which perceptual grouping operations play a central and essential role in determining the prevailing reference frames.

Spatiotemporal Form Integration: sequentially presented inducers can lead to representations of stationary and rigidly rotating objects

Objects in the world often are occluded and in motion. The visible fragments of such objects are revealed at different times and locations in space. To form coherent representations of the surfaces of these objects, the visual system must integrate local form information over space and time. We introduce a new illusion in which a rigidly rotating square is perceived on the basis of sequentially presented Pacman inducers. The illusion highlights two fundamental processes that allow us to perceive objects whose form features are revealed over time: Spatiotemporal Form Integration (STFI) and Position Updating. STFI refers to the spatial integration of persistent representations of local form features across time. Position updating of these persistent form representations allows them to be integrated into a rigid global motion percept. We describe three psychophysical experiments designed to identify spatial and temporal constraints that underlie these two processes and a fourth experiment that extends these findings to more ecologically valid stimuli. Our results indicate that although STFI can occur across relatively long delays between successive inducers (i.e., greater than 500 ms), position updating is limited to a more restricted temporal window (i.e., ~300 ms or less), and to a confined range of spatial (mis)alignment. These findings lend insight into the limits of mechanisms underlying the visual system's capacity to integrate transient, piecemeal form information, and support coherent object representations in the ever-changing environment.

Perceptual transparency from image deformation

Human vision has a remarkable ability to perceive two layers at the same retinal locations, a transparent layer in front of a background surface. Critical image cues to perceptual transparency, studied extensively in the past, are changes in luminance or color that could be caused by light absorptions and reflections by the front layer, but such image changes may not be clearly visible when the front layer consists of a pure transparent material such as water. Our daily experiences with transparent materials of this kind suggest that an alternative potential cue of visual transparency is image deformations of a background pattern caused by light refraction. Although previous studies have indicated that these image deformations, at least static ones, play little role in perceptual transparency, here we show that dynamic image deformations of the background pattern, which could be produced by light refraction on a moving liquid's surface, can produce a vivid impression of a transparent liquid layer without the aid of any other visual cues as to the presence of a transparent layer. Furthermore, a transparent liquid layer perceptually emerges even from a randomly generated dynamic image deformation as long as it is similar to real liquid deformations in its spatiotemporal frequency profile. Our findings indicate that the brain can perceptually infer the presence of "invisible" transparent liquids by analyzing the spatiotemporal structure of dynamic image deformation, for which it uses a relatively simple computation that does not require high-level knowledge about the detailed physics of liquid deformation.

Short temporal asynchrony disrupts visual object recognition

Humans can recognize objects and scenes in a small fraction of a second. The cascade of signals underlying rapid recognition might be disrupted by temporally jittering different parts of complex objects. Here we investigated the time course over which shape information can be integrated to allow for recognition of complex objects. We presented fragments of object images in an asynchronous fashion and behaviorally evaluated categorization performance. We observed that visual recognition was significantly disrupted by asynchronies of approximately 30 ms, suggesting that spatiotemporal integration begins to break down with even small deviations from simultaneity. However, moderate temporal asynchrony did not completely obliterate recognition; in fact, integration of visual shape information persisted even with an asynchrony of 100 ms. We describe the data with a concise model based on the dynamic reduction of uncertainty about what image was presented. These results emphasize the importance of timing in visual processing and provide strong constraints for the development of dynamical models of visual shape recognition.

Efficiency of extracting stereo-driven object motions

Most living things and many nonliving things deform as they move, requiring observers to separate object motions from object deformations. When the object is partially occluded, the task becomes more difficult because it is not possible to use two-dimensional (2-D) contour correlations (Cohen, Jain, & Zaidi, 2010). That leaves dynamic depth matching across the unoccluded views as the main possibility. We examined the role of stereo cues in extracting motion of partially occluded and deforming three-dimensional (3-D) objects, simulated by disk-shaped random-dot stereograms set at randomly assigned depths and placed uniformly around a circle. The stereo-disparities of the disks were temporally oscillated to simulate clockwise or counterclockwise rotation of the global shape. To dynamically deform the global shape, random disparity perturbation was added to each disk's depth on each stimulus frame. At low perturbation, observers reported rotation directions consistent with the global shape, even against local motion cues, but performance deteriorated at high perturbation. Using 3-D global shape correlations, we formulated an optimal Bayesian discriminator for rotation direction. Based on rotation discrimination thresholds, human observers were 75% as efficient as the optimal model, demonstrating that global shapes derived from stereo cues facilitate inferences of object motions. To complement reports of stereo and motion integration in extrastriate cortex, our results suggest the possibilities that disparity selectivity and feature tracking are linked, or that global motion selective neurons can be driven purely from disparity cues.

Non-retinotopic feature processing in the absence of retinotopic spatial layout and the construction of perceptual space from motion

The spatial representation of a visual scene in the early visual system is well known. The optics of the eye map the three-dimensional environment onto two-dimensional images on the retina. These retinotopic representations are preserved in the early visual system. Retinotopic representations and processing are among the most prevalent concepts in visual neuroscience. However, it has long been known that a retinotopic representation of the stimulus is neither sufficient nor necessary for perception. Saccadic Stimulus Presentation Paradigm and the Ternus-Pikler displays have been used to investigate non-retinotopic processes with and without eye movements, respectively. However, neither of these paradigms eliminates the retinotopic representation of the spatial layout of the stimulus. Here, we investigated how stimulus features are processed in the absence of a retinotopic layout and in the presence of retinotopic conflict. We used anorthoscopic viewing (slit viewing) and pitted a retinotopic feature-processing hypothesis against a non-retinotopic feature-processing hypothesis. Our results support the predictions of the non-retinotopic feature-processing hypothesis and demonstrate the ability of the visual system to operate non-retinotopically at a fine feature processing level in the absence of a retinotopic spatial layout. Our results suggest that perceptual space is actively constructed from the perceptual dimension of motion. The implications of these findings for normal ecological viewing conditions are discussed.

Nonretinotopic exogenous attention

Attention is crucial for visual perception because it allows the visual system to effectively use its limited resources by selecting behaviorally and cognitively relevant stimuli from the large amount of information impinging on the eyes. Reflexive, stimulus-driven attention is essential for successful interactions with the environment because it can, for example, speed up responses to life-threatening events. It is commonly believed that exogenous attention operates in the retinotopic coordinates of the early visual system. Here, using a novel experimental paradigm [1], we show that a nonretinotopic cue improves both accuracy and reaction times in a visual search task. Furthermore, the influence of the cue is limited both in space and time, a characteristic typical of exogenous cueing. These and other recent findings show that many more aspects of vision are processed nonretinotopically than previously thought.

Barrier effects in non-retinotopic feature attribution

When objects move in the environment, their retinal images can undergo drastic changes and features of different objects can be inter-mixed in the retinal image. Notwithstanding these changes and ambiguities, the visual system is capable of establishing correctly feature-object relationships as well as maintaining individual identities of objects through space and time. Recently, by using a Ternus-Pikler display, we have shown that perceived motion correspondences serve as the medium for non-retinotopic attribution of features to objects. The purpose of the work reported in this manuscript was to assess whether perceived motion correspondences provide a sufficient condition for feature attribution. Our results show that the introduction of a static "barrier" stimulus can interfere with the feature attribution process. Our results also indicate that the barrier stops feature attribution based on interferences related to the feature attribution process itself rather than on mechanisms related to perceived motion.

Motion and tilt aftereffects occur largely in retinal, not in object, coordinates in the Ternus-Pikler display

Recent studies have shown that a variety of aftereffects occurs in a non-retinotopic frame of reference. These findings have been taken as strong evidence that remapping of visual information occurs in a hierarchic manner in the human cortex with an increasing magnitude from early to higher levels. Other studies, however, failed to find non-retinotopic aftereffects. These experiments all relied on paradigms involving eye movements. Recently, we have developed a new paradigm, based on the Ternus-Pikler display, which tests retinotopic vs. non-retinotopic processing without the involvement of eye movements. Using this paradigm, we found strong evidence that attention, form, and motion processing can occur in a non-retinotopic frame of reference. Here, we show that motion and tilt aftereffects are largely retinotopic.

The utility of shape attributes in deciphering movements of non-rigid objects

Most moving objects in the world are non-rigid, changing shape as they move. To disentangle shape changes from movements, computational models either fit shapes to combinations of basis shapes or motion trajectories to combinations of oscillations but are biologically unfeasible in their input requirements. Recent neural models parse shapes into stored examples, which are unlikely to exist for general shapes. We propose that extracting shape attributes, e.g., symmetry, facilitates veridical perception of non-rigid motion. In a new method, identical dots were moved in and out along invisible spokes, to simulate the rotation of dynamically and randomly distorting shapes. Discrimination of rotation direction measured as a function of non-rigidity was 90% as efficient as the optimal Bayesian rotation decoder and ruled out models based on combining the strongest local motions. Remarkably, for non-rigid symmetric shapes, observers outperformed the Bayesian model when perceived rotation could correspond only to rotation of global symmetry, i.e., when tracking of shape contours or local features was uninformative. That extracted symmetry can drive perceived motion suggests that shape attributes may provide links across the dorsal-ventral separation between motion and shape processing. Consequently, the perception of non-rigid object motion could be based on representations that highlight global shape attributes.

Spatiotemporal Rivalry

In human vision, mechanisms specialized for encoding static form can signal the presence of blurred forms trailing behind moving objects. People are typically unaware of these motion-blur signals because other mechanisms signal sharply defined moving forms. When active, these mechanisms can suppress awareness of motion blur. Thus, although discrepant form signals can be produced, human vision usually settles on a single coherent perceptual outcome. Here we report a dramatic exception. We found that, in some circumstances, static motion-blur form signals and moving-form signals can engage in a dynamic competition for perceptual dominance. We refer to the phenomenon as spatiotemporal rivalry (STR). Our data confirm that moving- and static-form mechanisms can generate independent signals, each of which can intermittently dominate perception. STR could therefore be exploited to investigate how these mechanisms contribute to determining the content of visual awareness.

The Geometry of Visual Perception: Retinotopic and Non-retinotopic Representations in the Human Visual System

Geometry is closely linked to visual perception; yet, very little is known about the geometry of visual processing beyond early retinotopic organization. We present a variety of perceptual phenomena showing that a retinotopic representation is neither sufficient nor necessary to support form perception. We discuss the popular "object files" concept as a candidate for non-retinotopic representations and, based on its shortcomings, suggest future directions for research using local manifold representations. We suggest that these manifolds are created by the emergence of dynamic reference-frames that result from motion segmentation. We also suggest that the metric of these manifolds is based on relative motion vectors.

How Motion Signals Are Integrated Across Frequencies: Study on Motion Perception and Ocular Following Responses Using Multiple-Slit Stimuli

Visual motion signals, which are initially extracted in parallel at multiple spatial frequencies, are subsequently integrated into a unified motion percept. Cross-frequency integration plays a crucial role when directional information conflicts across frequencies due to such factors as occlusion. We investigated the human observers' open-loop oculomotor tracking responses (ocular following responses, or OFRs) and the perceived motion direction in an idealized situation of occlusion—multiple-slits viewing (MSV)—in which a moving pattern is visible only through an array of slits. We also tested a more challenging viewing condition, contrast-alternating MSV (CA-MSV), in which the contrast polarity of the moving pattern alternates when it passes the slits. We found that changes in the distribution of the spectral content of the slit stimuli, introduced by variations of both the interval between the slits and the frame rate of the image stream, modulated the OFR and the reported motion direction in a rather complex manner. We show that those complex modulations could be explained by the weighted sum of the motion signal (motion contrast) of each spatiotemporal frequency. The estimated distribution of frequency weights (tuning maps) indicate that the cross-frequency integration of supra-threshold motion signals gives strong weight to low spatial frequency components (<0.25 cpd) for both OFR and motion perception. However, the tuning map estimated with the MSV stimuli were significantly different from those estimated with the CA-MSV (and from those measured in a more direct manner using grating stimuli), suggesting that inter-frequency interactions (e.g., interaction producing speed-dependent tuning) was involved.

Pre-exposure to moving form enhances static form sensitivity

Background

Motion-defined form can seem to persist briefly after motion ceases, before seeming to gradually disappear into the background. Here we investigate if this subjective persistence reflects a signal capable of improving objective measures of sensitivity to static form.

Methodology/Principal Findings

We presented a sinusoidal modulation of luminance, masked by a background noise pattern. The sinusoidal luminance modulation was usually subjectively invisible when static, but visible when moving. We found that drifting then stopping the waveform resulted in a transient subjective persistence of the waveform in the static display. Observers' objective sensitivity to the position of the static waveform was also improved after viewing moving waveforms, compared to viewing static waveforms for a matched duration. This facilitation did not occur simply because movement provided more perspectives of the waveform, since performance following pre-exposure to scrambled animations did not match that following pre-exposure to smooth motion. Observers did not simply remember waveform positions at motion offset, since removing the waveform before testing reduced performance.

Conclusions/Significance

Motion processing therefore interacts with subsequent static visual inputs in a way that can improve performance in objective sensitivity measures. We suggest that the brief subjective persistence of motion-defined forms that can occur after motion offsets is a consequence of the decay of a static form signal that has been transiently enhanced by motion processing.

Shape distortions and Gestalt grouping in anorthoscopic perception

When a figure moves behind a stationary narrow slit, observers often report seeing the figure as a whole, a phenomenon called slit viewing or anorthoscopic perception. Interestingly, in slit viewing, the figure is perceived compressed along the axis of motion. As with other perceptual distortions, it is unclear whether the perceptual space in the vicinity of the slit or the representation of the figure itself undergoes compression. In a psychophysical experiment, we tested these two hypotheses. We found that the percept of a stationary bar, presented within the slit, was not distorted even when at the same time a circle underwent compression by moving through the slit. This result suggests that the compression of form results from figural rather than from space compression. In support of this hypothesis, we found that when the bar was perceptually grouped with the circle, the bar appeared compressed. Our results show that, in slit viewing, the distortion occurs at a non-retinotopic level where grouped objects are jointly represented.

Mobile computation: spatiotemporal integration of the properties of objects in motion

We demonstrate that, as an object moves, color and motion signals from successive, widely spaced locations are integrated, but letter and digit shapes are not. The features that integrate as an object moves match those that integrate when the eyes move but the object is stationary (spatiotopic integration). We suggest that this integration is mediated by large receptive fields gated by attention and that it occurs for surface features (motion and color) that can be summed without precise alignment but not shape features (letters or digits) that require such alignment. Rapidly alternating pairs of colors and motions were presented at several locations around a circle centered at fixation. The same two stimuli alternated at each location with the phase of the alternation reversing from one location to the next. When observers attended to only one location, the stimuli alternated in both retinal coordinates and in the attended stream: feature identification was poor. When the observer's attention shifted around the circle in synchrony with the alternation, the stimuli still alternated at each location in retinal coordinates, but now attention always selected the same color and motion, with the stimulus appearing as a single unchanging object stepping across the locations. The maximum presentation rate at which the color and motion could be reported was twice that for stationary attention, suggesting (as control experiments confirmed) object-based integration of these features. In contrast, the identification of a letter or digit alternating with a mask showed no advantage for moving attention despite the fact that moving attention accessed (within the limits of precision for attentional selection) only the target and never the mask. The masking apparently leaves partial information that cannot be integrated across locations, and we speculate that for spatially defined patterns like letters, integration across large shifts in location may be limited by problems in aligning successive samples. Our results also suggest that as attention moves, the selection of any given location (dwell time) can be as short as 50 ms, far shorter than the typical dwell time for stationary attention. Moving attention can therefore sample a brief instant of a rapidly changing stream if it passes quickly through, giving access to events that are otherwise not seen.

Distinct position assignment mechanisms revealed by cross-order motion

Motion perception influences perceived position. It has been shown that first-order (luminance defined) motion shifts perceived position across a wide range of spatial and temporal frequencies. On the other hand, second-order (contrast defined) motion shifts perceived position over a narrow range of temporal frequencies, regardless of spatial frequency [Bressler, D. W., & Whitney, D. (2006). Second-order motion shifts perceived position. Vision Research, 46(6-7), 1120-1128]. These results suggest the presence of distinct position assignment mechanisms for first- and second-order motion. We investigated whether the first- and second-order systems independently encode and assign the position of a moving stimulus. To measure motion induced position shift we presented two horizontally offset Gabors placed above and below a central fixation point, with sine wave carriers drifting in opposite directions. Subjects judged the position of the top Gabor relative to the bottom one. We used both first-order Gabors (sinusoidal luminance modulation of a dynamic noise carrier enveloped by a static Gaussian) and second-order Gabors (sinusoidal contrast modulation of a dynamic noise carrier enveloped by a static Gaussian). Results showed a strong position shift in the direction of the carrier motion when both Gabors were first-order, a weak position shift when both Gabors were second-order, and no appreciable position shift when one Gabor was first-order and the other was second-order (cross-order motion). The absence of a position shift using cross-order motion supports the hypothesis that the two motion systems independently encode and assign the position of a moving object. These results are consistent with those of experiments investigating global spatial interactions between static first-order and second-order Gabor patches, indicating a commonality in the underlying spatial integration processes.

A theory of moving form perception: Synergy between masking, perceptual grouping, and motion computation in retinotopic and non-retinotopic representations

Because object and self-motion are ubiquitous in natural viewing conditions, understanding how the human visual system achieves a relatively clear perception for moving objects is a fundamental problem in visual perception. Several studies have shown that the visible persistence of a briefly presented stationary stimulus is approximately 120 ms under normal viewing conditions. Based on this duration of visible persistence, we would expect moving objects to appear highly blurred. However, in human vision, objects in motion typically appear relatively sharp and clear. We suggest that clarity of form in dynamic viewing is achieved by a synergy between masking, perceptual grouping, and motion computation across retinotopic and non-retinotopic representations. We also argue that dissociations observed in masking are essential to create and maintain this synergy.

Spatio-temporal interpolation is accomplished by binocular form and motion mechanisms

Spatio-temporal interpolation describes the ability of the visual system to perceive shapes as whole figures (Gestalts), even if they are moving behind narrow apertures, so that only thin slices of them meet the eye at any given point in time. The interpolation process requires registration of the form slices, as well as perception of the shape's global motion, in order to reassemble the slices in the correct order. The commonly proposed mechanism is a spatio-temporal motion detector with a receptive field, for which spatial distance and temporal delays are interchangeable, and which has generally been regarded as monocular. Here we investigate separately the nature of the motion and the form detection involved in spatio-temporal interpolation, using dichoptic masking and interocular presentation tasks. The results clearly demonstrate that the associated mechanisms for both motion and form are binocular rather than monocular. Hence, we question the traditional view according to which spatio-temporal interpolation is achieved by monocular first-order motion-energy detectors in favour of models featuring binocular motion and form detection.

Human Visual System Integrates Color Signals along a Motion Trajectory

Whether fundamental visual attributes, such as color, motion, and shape, are analyzed separately in specialized pathways has been one of the central questions of visual neuroscience. Although recent studies have revealed various forms of cross-attribute interactions, including significant contributions of color signals to motion processing, it is still widely believed that color perception is relatively independent of motion processing. Here, we report a new color illusion, motion-induced color mixing, in which moving bars, the color of each of which alternates between two colors (e.g., red and green), are perceived as the mixed color (e.g., yellow) even though the two colors are never superimposed on the retina. The magnitude of color mixture is significantly stronger than that expected from direction-insensitive spatial integration of color signals. This illusion cannot be ascribed to optical image blurs, including those induced by chromatic aberration, or to involuntary eye movements of the observer. Our findings indicate that color signals are integrated not only at the same retinal location, but also along a motion trajectory. It is possible that this neural mechanism helps us to see veridical colors for moving objects by reducing motion blur, as in the case of luminance-based pattern perception.

Second-order motion without awareness: passive adaptation to second-order motion produces a motion aftereffect

Although second-order motion may be detected by early and automatic mechanisms, some models suggest that perceiving second-order motion requires higher-order processes, such as feature or attentive tracking. These types of attentionally mediated mechanisms could explain the motion aftereffect (MAE) perceived in dynamic displays after adapting to second-order motion. Here we tested whether there is a second-order MAE in the absence of attention or awareness. If awareness of motion, mediated by high-level or top-down mechanisms, is necessary for the second-order MAE, then there should be no measurable MAE if the ability to detect directionality is impaired during adaptation. To eliminate the subject's ability to detect directionality of the adapting stimulus, a second-order drifting Gabor was embedded in a dense array of additional crowding Gabors. We found that a significant MAE was perceived even after adaptation to second-order motion in crowded displays that prevented awareness. The results demonstrate that second-order motion can be passively coded in the absence of awareness and without top-down attentional control.

Independent coding of object motion and position revealed by distinct contingent aftereffects

Despite several findings of perceptual asynchronies between object features, it remains unclear whether independent neuronal populations necessarily code these perceptually unbound properties. To examine this, we investigated the binding between an object's spatial frequency and its rotational motion using contingent motion aftereffects (MAE). Subjects adapted to an oscillating grating whose direction of rotation was paired with a high or low spatial frequency pattern. In separate adaptation conditions, we varied the moment when the spatial frequency change occurred relative to the direction reversal. After adapting to one stimulus, subjects made judgments of either the perceived MAE (rotational movement) or the position shift (instantaneous phase rotation) that accompanied the MAE. To null the spatial frequency-contingent MAE, motion reversals had to physically lag changes in spatial frequency during adaptation. To null the position shift that accompanied the MAE, however, no temporal lag between the attributes was required. This demonstrates that perceived motion and position can be perceptually misbound. Indeed, in certain conditions, subjects perceived the test pattern to drift in one direction while its position appeared shifted in the opposite direction. The dissociation between perceived motion and position of the same test pattern, following identical adaptation, demonstrates that distinguishable neural populations code for these object properties.

Second-order motion shifts perceived position

Many studies have documented that first-order motion influences perceived position. Here, we show that second-order (contrast defined) motion influences the perceived positions of stationary objects as well. We used a Gabor pattern as our second-order stimulus, which consisted of a drifting sinusoidal contrast modulation of a dynamic random-dot background; this second-order carrier was enveloped by a static Gaussian contrast modulation. Two vertically aligned Gabors had carrier motion in opposite directions. Subjects judged the relative positions of the Gabors' static envelopes. The positions of the Gabors appeared shifted in the direction of the carrier motion, but the effect was narrowly tuned to low temporal frequencies across all tested spatial frequencies. In contrast, first-order (luminance defined) motion shifted perceived positions across a wide range of temporal frequencies, and this differential tuning could not be explained by differences in the visibility of the patterns. The results show that second-order motion detection mechanisms contribute to perceived position. Further, the differential spatial and temporal tuning of the illusion supports the idea that there are distinct position assignment mechanisms for first and second-order motion.

The effect of retinal stabilization on anorthoscopic percepts under free-viewing conditions

Outline shapes were moved back and forth at .5 Hz behind a simulated narrow (10' or 15') slit without visible borders. Under free-viewing conditions observers reported spontaneous transitions between periods in which they saw a horizontally moving shape and periods in which they saw only vertically moving contour segments. Eye movements were monitored with a dual-Purkinje image eye-tracker. On half the trials we used retinal stabilization to eliminate any pursuit dependent retinal painting. With no stabilization, we observed low amplitude spontaneous pursuit of the horizontal figure motion when a shape was seen, and occasionally lower amplitude pursuit even when no shape was seen. Pursuit amplitudes increased when the display was stabilized. However, stabilization did not alter either the duration or frequency of the episodes of shape perception. Our findings indicate that under the free-viewing conditions we employed, spontaneous smooth pursuit and pursuit dependent painting did not contribute to the formation or maintenance of anorthoscopic percepts.