Motion and tilt aftereffects occur largely in retinal, not in object, coordinates in the Ternus-Pikler display

Unifying network model links recency and central tendency biases in working memory

The central tendency bias, or contraction bias, is a phenomenon where the judgment of the magnitude of items held in working memory appears to be biased toward the average of past observations. It is assumed to be an optimal strategy by the brain and commonly thought of as an expression of the brain's ability to learn the statistical structure of sensory input. On the other hand, recency biases such as serial dependence are also commonly observed and are thought to reflect the content of working memory. Recent results from an auditory delayed comparison task in rats suggest that both biases may be more related than previously thought: when the posterior parietal cortex (PPC) was silenced, both short-term and contraction biases were reduced. By proposing a model of the circuit that may be involved in generating the behavior, we show that a volatile working memory content susceptible to shifting to the past sensory experience - producing short-term sensory history biases - naturally leads to contraction bias. The errors, occurring at the level of individual trials, are sampled from the full distribution of the stimuli and are not due to a gradual shift of the memory toward the sensory distribution's mean. Our results are consistent with a broad set of behavioral findings and provide predictions of performance across different stimulus distributions and timings, delay intervals, as well as neuronal dynamics in putative working memory areas. Finally, we validate our model by performing a set of human psychophysics experiments of an auditory parametric working memory task.

Spatial correspondence in relative space regulates serial dependence

Our perception is often attracted to what we have seen before, a phenomenon called 'serial dependence.' Serial dependence can help maintain a stable perception of the world, given the statistical regularity in the environment. If serial dependence serves this presumed utility, it should be pronounced when consecutive elements share the same identity when multiple elements spatially shift across successive views. However, such preferential serial dependence between identity-matching elements in dynamic situations has never been empirically tested. Here, we hypothesized that serial dependence between consecutive elements is modulated more effectively by the spatial correspondence in relative space than by that in absolute space because spatial correspondence in relative coordinates can warrant identity matching invariantly to changes in absolute coordinates. To test this hypothesis, we developed a task where two targets change positions in unison between successive views. We found that serial dependence was substantially modulated by the correspondence in relative coordinates, but not by that in absolute coordinates. Moreover, such selective modulation by the correspondence in relative space was also observed even for the serial dependence defined by previous non-target elements. Our findings are consistent with the view that serial dependence subserves object-based perceptual stabilization over time in dynamic situations.

Serial dependence in visual perception: A meta-analysis and review

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Serial dependence in visual perception: A review

How does the visual system represent continuity in the constantly changing visual input? A recent proposal is that vision is serially dependent: Stimuli seen a moment ago influence what we perceive in the present. In line with this, recent frameworks suggest that the visual system anticipates whether an object seen at one moment is the same as the one seen a moment ago, binding visual representations across consecutive perceptual episodes. A growing body of work supports this view, revealing signatures of serial dependence in many diverse visual tasks. Yet, the variety of disparate findings and interpretations calls for a more general picture. Here, we survey the main paradigms and results over the past decade. We also focus on the challenge of finding a relationship between serial dependence and the concept of "object identity," taking centuries-long history of research into account. Among the seemingly contrasting findings on serial dependence, we highlight common patterns that may elucidate the nature of this phenomenon and attempt to identify questions that are unanswered.

The effect of attention on body size adaptation and body dissatisfaction

Attentional bias to low-fat bodies is thought to be associated with body dissatisfaction-a symptom and risk factor of eating disorders. However, the causal nature of this relationship is unclear. In three preregistered experiments, we trained 370 women to attend towards either high- or low-fat body stimuli using an attention training dot probe task. For each experiment, we analysed the effect of the attention training on (i) attention to subsequently presented high- versus low-fat body stimuli, (ii) visual adaptation to body size, and (iii) body dissatisfaction. The attention training had no effect on attention towards high- or low-fat bodies in an online setting (Experiment 1), but did increase attention to high-fat bodies in a laboratory setting (Experiment 2). Neither perceptions of a 'normal' body size nor levels of body dissatisfaction changed as a result of the attention training in either setting. The results in the online setting did not change when we reduced the stimulus onset-asynchrony of the dot probe task from 500 to 100 ms (Experiment 3). Our results provide no evidence that the dot probe training task used here has robust effects on attention to body size, body image disturbance or body dissatisfaction.

Microsaccadic Eye Movements but not Pupillary Dilation Response Characterizes the Crossmodal Freezing Effect

In typical spatial orienting tasks, the perception of crossmodal (e.g., audiovisual) stimuli evokes greater pupil dilation and microsaccade inhibition than unisensory stimuli (e.g., visual). The characteristic pupil dilation and microsaccade inhibition has been observed in response to "salient" events/stimuli. Although the "saliency" account is appealing in the spatial domain, whether this occurs in the temporal context remains largely unknown. Here, in a brief temporal scale (within 1 s) and with the working mechanism of involuntary temporal attention, we investigated how eye metric characteristics reflect the temporal dynamics of perceptual organization, with and without multisensory integration. We adopted the crossmodal freezing paradigm using the classical Ternus apparent motion. Results showed that synchronous beeps biased the perceptual report for group motion and triggered the prolonged sound-induced oculomotor inhibition (OMI), whereas the sound-induced OMI was not obvious in a crossmodal task-free scenario (visual localization without audiovisual integration). A general pupil dilation response was observed in the presence of sounds in both visual Ternus motion categorization and visual localization tasks. This study provides the first empirical account of crossmodal integration by capturing microsaccades within a brief temporal scale; OMI but not pupillary dilation response characterizes task-specific audiovisual integration (shown by the crossmodal freezing effect).

A Bayesian and efficient observer model explains concurrent attractive and repulsive history biases in visual perception

Human perceptual decisions can be repelled away from (repulsive adaptation) or attracted towards recent visual experience (attractive serial dependence). It is currently unclear whether and how these repulsive and attractive biases interact during visual processing and what computational principles underlie these history dependencies. Here we disentangle repulsive and attractive biases by exploring their respective timescales. We find that perceptual decisions are concurrently attracted towards the short-term perceptual history and repelled from stimuli experienced up to minutes into the past. The temporal pattern of short-term attraction and long-term repulsion cannot be captured by an ideal Bayesian observer model alone. Instead, it is well captured by an ideal observer model with efficient encoding and Bayesian decoding of visual information in a slowly changing environment. Concurrent attractive and repulsive history biases in perceptual decisions may thus be the consequence of the need for visual processing to simultaneously satisfy constraints of efficiency and stability.

Effect of spatial attention on spatiotopic visual motion perception

We almost never experience visual instability, despite retinal image instability induced by eye movements. How the stability of visual perception is maintained through spatiotopic representation remains a matter of debate. The discrepancies observed in the findings of existing neuroscience studies regarding spatiotopic representation partly originate from differences in regard to how attention is deployed to stimuli. In this study, we psychophysically examined whether spatial attention is needed to perceive spatiotopic visual motion. For this purpose, we used visual motion priming, which is a phenomenon in which a preceding priming stimulus modulates the perceived moving direction of an ambiguous test stimulus, such as a drifting grating that phase shifts by 180°. To examine the priming effect in different coordinates, participants performed a saccade soon after the offset of a primer. The participants were tasked with judging the direction of a subsequently presented test stimulus. To control the effect of spatial attention, the participants were asked to conduct a concurrent dot contrast-change detection task after the saccade. Positive priming was prominent in spatiotopic conditions, whereas negative priming was dominant in retinotopic conditions. At least a 600-ms interval between the priming and test stimuli was needed to observe positive priming in spatiotopic coordinates. When spatial attention was directed away from the location of the test stimulus, spatiotopic positive motion priming completely disappeared; meanwhile, the spatiotopic positive motion priming at shorter interstimulus intervals was enhanced when spatial attention was directed to the location of the test stimulus. These results provide evidence that an attentional resource is requisite for developing spatiotopic representation more quickly.

A New Conceptualization of Human Visual Sensory-Memory

Memory is an essential component of cognition and disorders of memory have significant individual and societal costs. The Atkinson–Shiffrin “modal model” forms the foundation of our understanding of human memory. It consists of three stores: Sensory Memory (SM), whose visual component is called iconic memory, Short-Term Memory (STM; also called working memory, WM), and Long-Term Memory (LTM). Since its inception, shortcomings of all three components of the modal model have been identified. While the theories of STM and LTM underwent significant modifications to address these shortcomings, models of the iconic memory remained largely unchanged: A high capacity but rapidly decaying store whose contents are encoded in retinotopic coordinates, i.e., according to how the stimulus is projected on the retina. The fundamental shortcoming of iconic memory models is that, because contents are encoded in retinotopic coordinates, the iconic memory cannot hold any useful information under normal viewing conditions when objects or the subject are in motion. Hence, half-century after its formulation, it remains an unresolved problem whether and how the first stage of the modal model serves any useful function and how subsequent stages of the modal model receive inputs from the environment. Here, we propose a new conceptualization of human visual sensory memory by introducing an additional component whose reference-frame consists of motion-grouping based coordinates rather than retinotopic coordinates. We review data supporting this new model and discuss how it offers solutions to the paradoxes of the traditional model of sensory memory.

Putting low-level vision into global context: Why vision cannot be reduced to basic circuits

To cope with the complexity of vision, most models in neuroscience and computer vision are of hierarchical and feedforward nature. Low-level vision, such as edge and motion detection, is explained by basic low-level neural circuits, whose outputs serve as building blocks for more complex circuits computing higher level features such as shape and entire objects. There is an isomorphism between states of the outer world, neural circuits, and perception, inspired by the positivistic philosophy of the mind. Here, we show that although such an approach is conceptually and mathematically appealing, it fails to explain many phenomena including crowding, visual masking, and non-retinotopic processing.

Spatial properties of non-retinotopic reference frames in human vision

Many visual attributes of a target stimulus are computed according to dynamic, non-retinotopic reference frames. For example, the motion trajectory of a reflector on a bicycle wheel is perceived as orbital, even though it is in fact cycloidal in retinal, as well as spatial coordinates. We cannot perceive the cycloidal motion because the linear motion of the bike is discounted for. In other words, the linear motion common to all bicycle components serves as a non-retinotopic reference frame, with respect to which the residual (orbital) motion of the reflector is computed. Very little is known about the underlying mechanisms involved in formation and operation of non-retinotopic reference frames. Here, we investigate spatial properties of non-retinotopic reference frames. We show that reference frames are not restricted within the boundaries of moving stimuli but extend over space. By using a variation of the Ternus-Pikler paradigm, we show that the spatial extent of a non-retinotopic reference frame is independent of the size of the inducing elements and the target position near the object boundary. While dynamic reference-frames interact with each other significantly, a static reference-frame has no effect on a dynamic one. The magnitude of interactions between two neighboring dynamic reference-frames increases as the distance between them reduces. Finally, our results indicate that the reference-frame strength is significantly attenuated if the locus of attention is shifted to the elements of the neighboring reference instead of the main reference. We suggest that these results can be conceptualized as reference frames that act and interact as fields.

Localizing non-retinotopically moving objects

How does the brain determine the position of moving objects? It turns out to be rather complex to answer this question when we realize that the brain has to solve the motion correspondence problem in two kinds of reference frames: Retinotopic and non-retinotopic ones. We show that visual objects are mislocalized along a non-retinotopic motion direction. Observers viewed two successive movie frames each consisting of an outlined square and two target elements inside the square. In the non-retinotopic condition the elements as well as the square moved vertically while two bars also centripetally or centrifugally moved. In the retinotopic condition the vertical movement of them was removed from the stimuli. The task of the observers was to judge a relative position of the elements. Consequently, the elements were mislocalized in the direction of both retinotopic and non-retinotopic motion, although the mislocalization was significantly larger in the retinotopic than in the non-retinotopic conditions. The results suggest that non-retinotopic as well as retinotopic motion processing contributes to the determination of perceived positions of moving objects.

Automatic frame-centered object representation and integration revealed by iconic memory, visual priming, and backward masking

Object identities ("what") and their spatial locations ("where") are processed in distinct pathways in the visual system, raising the question of how the what and where information is integrated. Because of object motions and eye movements, the retina-based representations are unstable, necessitating nonretinotopic representation and integration. A potential mechanism is to code and update objects according to their reference frames (i.e., frame-centered representation and integration). To isolate frame-centered processes, in a frame-to-frame apparent motion configuration, we (a) presented two preceding or trailing objects on the same frame, equidistant from the target on the other frame, to control for object-based (frame-based) effect and space-based effect, and (b) manipulated the target's relative location within its frame to probe frame-centered effect. We show that iconic memory, visual priming, and backward masking depend on objects' relative frame locations, orthogonal of the retinotopic coordinate. These findings not only reveal that iconic memory, visual priming, and backward masking can be nonretinotopic but also demonstrate that these processes are automatically constrained by contextual frames through a frame-centered mechanism. Thus, object representation is robustly and automatically coupled to its reference frame and continuously being updated through a frame-centered, location-specific mechanism. These findings lead to an object cabinet framework, in which objects ("files") within the reference frame ("cabinet") are orderly coded relative to the frame.

Remapping of the line motion illusion across eye movements

Although motion processing in the brain has been classically studied in terms of retinotopically defined receptive fields, recent evidence suggests that motion perception can occur in a spatiotopic reference frame. We investigated the underlying mechanisms of spatiotopic motion perception by examining the role of saccade metrics as well as the capacity of trans-saccadic motion. To this end, we used the line motion illusion (LMI), in which a straight line briefly shown after a high contrast stimulus (inducer) is perceived as expanding away from the inducer position. This illusion provides an interesting test of spatiotopic motion because the neural correlates of this phenomenon have been found early in the visual cortex and the effect does not require focused attention. We measured the strength of LMI both with stable fixation and when participants were asked to perform a 10° saccade during the blank ISI between the inducer and the line. A strong motion illusion was found across saccades in spatiotopic coordinates. When the inducer was presented near in time to the saccade cue, saccadic latencies were longer, saccade amplitudes were shorter, and the strength of reported LMI was consistently reduced. We also measured the capacity of the trans-saccadic LMI by varying the number of inducers. In contrast to a visual-spatial memory task, we found that the LMI was largely eliminated by saccades when two or more inducers were displayed. Together, these results suggest that motion perceived in non-retinotopic coordinates depends on an active, saccade-dependent remapping process with a strictly limited capacity.