Reconstructing a disambiguation sequence that forms perceptual memory of multistable displays via reverse correlation method: Bias onset perception but gently

When multistable displays are presented intermittently with long blank intervals, their onset perception is determined by perceptual memory of multistable displays. We investigated when and how it is formed using a reverse correlation method and bistable kinetic depth effect displays. Each experimental block consisted of interleaved fully ambiguous probe and exogenously disambiguated prime displays. The purpose of the former was to "read out" the perceptual memory, whereas the latter contained purely random disambiguation sequences that were presented at the beginning of the prime display, throughout the entire presentation, or at the beginning and the end of the presentation. For each experiment and condition, we selected a subset of trials with disambiguation sequences that led to a change in perception of either the prime itself (sequences that modified perception) or the following fully ambiguous probe (sequences that modified perceptual memory). We estimated average disambiguation sequences for each participant using additive linear models. We found that an optimal sequence started at the onset with a moderate disambiguation against the previously dominant state (dominant perception for the previous probe) that gradually reduced until the display is fully ambiguous. We also show that the same sequence leads to an altered perception of the prime, indicating that perception and perceptual memory form at the same time. We suggest that perceptual memory is a consequence of an earlier evidence accumulation process and is informative about how the visual system treated ambiguity in the past rather than how it anticipates an uncertain future.

Top-down resolution of visual ambiguity - knowledge from the future or footprints from the past?

Current theories about visual perception assume that our perceptual system weights the a priori incomplete, noisy and ambiguous sensory information with previous, memorized perceptual experiences in order to construct stable and reliable percepts. These theories are supported by numerous experimental findings. Theories about precognition have an opposite point of view. They assume that information from the future can have influence on perception, thoughts, and behavior. Several experimental studies provide evidence for precognition effects, other studies found no such effects. One problem may be that the vast majority of precognition paradigms did not systematically control for potential effects from the perceptual history. In the present study, we presented ambiguous Necker cube stimuli and disambiguated cube variants and systematically tested in two separate experiments whether perception of a currently observed ambiguous Necker cube stimulus can be influenced by a disambiguated cube variant, presented in the immediate perceptual past (perceptual history effects) and/or in the immediate perceptual future (precognition effects). We found perceptual history effects, which partly depended on the length of the perceptual history trace but were independent of the perceptual future. Results from some individual participants suggest on the first glance a precognition pattern, but results from our second experiment make a perceptual history explanation more probable. On the group level, no precognition effects were statistically indicated. The perceptual history effects found in the present study are in confirmation with related studies from the literature. The precognition analysis revealed some interesting individual patterns, which however did not allow for general conclusions. Overall, the present study demonstrates that any future experiment about sensory or extrasensory perception urgently needs to control for potential perceptual history effects and that temporal aspects of stimulus presentation are of high relevance.

Using the perceptual past to predict the perceptual future influences the perceived present - A novel ERP paradigm

The information available through our senses is noisy, incomplete, and to varying degrees ambiguous. The perceptual system must create stable and reliable percepts out of this restricted information. It solves this perceptual inference problem by integrating memories of previous percepts and making predictions about the perceptual future.

Using ambiguous figures and a new experimental approach, we studied whether generating predictions based on regularities in the past affects processing of the present and how this is done. Event-related potentials (ERPs) were measured to investigate whether a highly regular temporal context of either ambiguous or unambiguous stimulus variants differently affects processing of a current stimulus and/or task execution. Further, we tested whether symbolic announcements about the immediate perceptual future can replace the past experience of regularities as a source for making predictions. Both ERP and reaction time varied as a function of stimulus ambiguity in the temporal context of a present stimulus. No such effects were found with symbolic announcements.

Our results indicate that predictions about the future automatically alter processing of the present, even if the predictions are irrelevant for the present percept and task. However, direct experiences of past regularities are necessary for predicting the future whereas symbolic information about the future is not sufficient.

A Bayesian and efficient observer model explains concurrent attractive and repulsive history biases in visual perception

Human perceptual decisions can be repelled away from (repulsive adaptation) or attracted towards recent visual experience (attractive serial dependence). It is currently unclear whether and how these repulsive and attractive biases interact during visual processing and what computational principles underlie these history dependencies. Here we disentangle repulsive and attractive biases by exploring their respective timescales. We find that perceptual decisions are concurrently attracted towards the short-term perceptual history and repelled from stimuli experienced up to minutes into the past. The temporal pattern of short-term attraction and long-term repulsion cannot be captured by an ideal Bayesian observer model alone. Instead, it is well captured by an ideal observer model with efficient encoding and Bayesian decoding of visual information in a slowly changing environment. Concurrent attractive and repulsive history biases in perceptual decisions may thus be the consequence of the need for visual processing to simultaneously satisfy constraints of efficiency and stability.

Large EEG amplitude effects are highly similar across Necker cube, smiley, and abstract stimuli

The information available through our senses is noisy, incomplete, and ambiguous. Our perceptual systems have to resolve this ambiguity to construct stable and reliable percepts. Previous EEG studies found large amplitude differences in two event-related potential (ERP) components 200 and 400 ms after stimulus onset when comparing ambiguous with disambiguated visual information ("ERP Ambiguity Effects"). These effects so far generalized across classical ambiguous figures from different visual categories at lower (geometry, motion) and intermediate (Gestalt perception) levels. The present study aimed to examine whether these ERP Effects are restricted to ambiguous figures or whether they also occur for different degrees of visibility. Smiley faces with low and high visibility of emotional expressions, as well as abstract figures with low and high visibility of a target curvature were presented. We thus compared ambiguity effects in geometric cube stimuli with visibility in emotional faces, and with visibility in abstract figures. ERP Effects were replicated for the geometric stimuli and very similar ERP Effects were found for stimuli with emotional face expressions but also for abstract figures. Conclusively, the ERP amplitude effects generalize across fundamentally different stimulus categories and show highly similar effects for different degrees of stimulus ambiguity and stimulus visibility. We postulate the existence of a high-level/meta-perceptual evaluation instance, beyond sensory details, that estimates the certainty of a perceptual decision. The ERP Effects may reflect differences in evaluation results.

Switch rates for orthogonally oriented kinetic-depth displays are correlated across observers

When continuously viewing multistable displays, which are compatible with several comparably likely interpretations, perception perpetually switches between available alternatives. Prior studies typically report the lack of consistent individual switch rates across different displays. However, this comparison is based on an assumption that neural representations of physically identical displays are consistent across observers. Yet, given how different individuals are already at the level of the retina, it is likely that the difference in other relevant factors might mask the correlation. To address this issue, we compared switch rates in two kinetic-depth displays (KDE) that rotated around orthogonal axes (45° counterclockwise vs. 45° clockwise relative to the vertical). This ensured that dynamics of multistable perception was based on highly similar, but different and independent neural representations. We also included a Necker cube (NC) display as a control. We report that switch rates were correlated between two kinetic-depth effect displays, but not between either of the KDE and NC displays. This demonstrates that the usual lack of correlation may not be evidence for the lack of a shared pacesetter mechanism of multistable perception, but reflect other factors, such as differently modulated inputs to competing representations. In addition, we asked participants to speed-up or slow-down perceptual alternations and found that only the former ability was correlated across different displays. This indicates that these two types of volitional control may differ in their use of attentional resources.

Positive and negative hysteresis effects for the perception of geometric and emotional ambiguities

AIM

The present study utilizes perceptual hysteresis effects to compare the ambiguity of Mona Lisa's emotional face expression (high-level ambiguity) and of geometric cube stimuli (low-level ambiguity).

METHODS

In two experiments we presented series of nine Mona Lisa variants and nine cube variants. Stimulus ambiguity was manipulated by changing Mona Lisa's mouth curvature (Exp. 1) and the cubes' back-layer luminance (Exp. 2). Each experiment consisted of three conditions, two with opposite stimulus presentation sequences with increasing and decreasing degrees of ambiguity, respectively, and a third condition with a random presentation sequence. Participants indicated happy or sad face percepts (Exp. 1) and alternative 3D cube percepts (Exp. 2) by key presses. We studied the influences of a priori perceptual biases (long-term memory) and presentation order (short-term memory) on perception.

RESULTS

Perception followed sigmoidal functions of the stimulus ambiguity morphing parameters. The morphing parameter for the functions' inflection points depended strongly on stimulus presentation order with similar effect sizes but different signs for the two stimulus types (positive hysteresis / priming for the cubes; negative hysteresis / adaptation for Mona Lisa). In the random conditions, the inflection points were located in the middle between those from the two directional conditions for the Mona Lisa stimuli. For the cube stimuli, they were superimposed on one sigmoidal function for the ordered condition.

DISCUSSION

The hysteresis effects reflect the influence of short-term memory during the perceptual disambiguation of ambiguous sensory information. The effects for the two stimulus types are of similar size, explaining up to 34% of the perceptual variance introduced by the paradigm. We explain the qualitative difference between positive and negative hysteresis with adaptation for Mona Lisa and with priming for the cubes. In addition, the hysteresis paradigm allows a quantitative determination of the impact of adaptation and priming during the resolution of perceptual ambiguities. The asymmetric shifts of inflection points in the case of the cube stimuli is likely due to an a priori perceptual bias, reflecting an influence of long-term memory. Whether corresponding influences also exist for the Mona Lisa variants is so far unclear.

Long-Range Temporal Correlations in Alpha Oscillations Stabilize Perception of Ambiguous Visual Stimuli

Ongoing brain dynamics have been proposed as a type of “neuronal noise” that can trigger perceptual switches when viewing an ambiguous, bistable stimulus. However, no prior study has directly quantified how such neuronal noise relates to the rate of percept reversals. Specifically, it has remained unknown whether individual differences in complexity of resting-state oscillations—as reflected in long-range temporal correlations (LRTC)—are associated with perceptual stability. We hypothesized that participants with stronger resting-state LRTC in the alpha band experience more stable percepts, and thereby fewer perceptual switches. Furthermore, we expected that participants who report less discontinuous thoughts during rest, experience less switches. To test this, we recorded electroencephalography (EEG) in 65 healthy volunteers during 5 min Eyes-Closed Rest (ECR), after which they filled in the Amsterdam Resting-State Questionnaire (ARSQ). This was followed by three conditions where participants attended an ambiguous structure-from-motion stimulus—Neutral (passively observe the stimulus), Hold (the percept for as long as possible), and Switch (as often as possible). LRTC of resting-state alpha oscillations predicted the number of switches only in the Hold condition, with stronger LRTC associated with less switches. Contrary to our expectations, there was no association between resting-state Discontinuity of Mind and percept stability. Participants were capable of controlling switching according to task goals, and this was accompanied by increased alpha power during Hold and decreased power during Switch. Fewer switches were associated with stronger task-related alpha LRTC in all conditions. Together, our data suggest that bistable visual perception is to some extent under voluntary control and influenced by LRTC of alpha oscillations.

First, you need a Gestalt: An interaction of bottom-up and top-down streams during the perception of the ambiguously rotating human walker

Our visual system combines sensory evidence with prior knowledge to produce a representation of an outside world. Here, we explored the limits of the feedforward computation using an ambiguously rotating human biological motion. Specifically, we investigated whether an overall rotation, which was added to all the displays used in the study, would be perceived when the point-light walker was presented upside-down, a condition that typically obliterates perception of a human Gestalt. We report that inversion of the point-light walker or the absence of an identifiable Gestalt abolished the perception of an overall rotation. Perception of rotation was restored if the human walker Gestalt could be identified (an upright walker), if observers were informed about the nature of the motion display, or if observers expected to see the rotation of an unknown dynamic object. This implies that a mathematically more complex human motion was accounted for before the remaining motion components could be used to infer an overall rotation. Our results indicate that the perceptual inference does not proceed in a hierarchical manner with the simpler components being identified first. Instead, prior knowledge acts as a starting point for the decomposition of an even relatively simple combination of two motions.

Perception and the strongest sensory memory trace of multi-stable displays both form shortly after the stimulus onset

We investigated the relation between perception and sensory memory of multi-stable structure-from-motion displays. The latter is an implicit visual memory that reflects a recent history of perceptual dominance and influences only the initial perception of multi-stable displays. First, we established the earliest time point when the direction of an illusory rotation can be reversed after the display onset (29-114 ms). Because our display manipulation did not bias perception towards a specific direction of illusory rotation but only signaled the change in motion, this means that the perceptual dominance was established no later than 29-114 ms after the stimulus onset. Second, we used orientation-selectivity of sensory memory to establish which display orientation produced the strongest memory trace and when this orientation was presented during the preceding prime interval (80-140 ms). Surprisingly, both estimates point towards the time interval immediately after the display onset, indicating that both perception and sensory memory form at approximately the same time. This suggests a tighter integration between perception and sensory memory than previously thought, warrants a reconsideration of its role in visual perception, and indicates that sensory memory could be a unique behavioral correlate of the earlier perceptual inference that can be studied post hoc.

How previous experience shapes perception in different sensory modalities

What has transpired immediately before has a strong influence on how sensory stimuli are processed and perceived. In particular, temporal context can have contrastive effects, repelling perception away from the interpretation of the context stimulus, and attractive effects (TCEs), whereby perception repeats upon successive presentations of the same stimulus. For decades, scientists have documented contrastive and attractive temporal context effects mostly with simple visual stimuli. But both types of effects also occur in other modalities, e.g., audition and touch, and for stimuli of varying complexity, raising the possibility that context effects reflect general computational principles of sensory systems. Neuroimaging shows that contrastive and attractive context effects arise from neural processes in different areas of the cerebral cortex, suggesting two separate operations with distinct functional roles. Bayesian models can provide a functional account of both context effects, whereby prior experience adjusts sensory systems to optimize perception of future stimuli.

Transformation priming helps to disambiguate sudden changes of sensory inputs

Retinal input is riddled with abrupt transients due to self-motion, changes in illumination, object-motion, etc. Our visual system must correctly interpret each of these changes to keep visual perception consistent and sensitive. This poses an enormous challenge, as many transients are highly ambiguous in that they are consistent with many alternative physical transformations. Here we investigated inter-trial effects in three situations with sudden and ambiguous transients, each presenting two alternative appearances (rotation-reversing structure-from-motion, polarity-reversing shape-from-shading, and streaming-bouncing object collisions). In every situation, we observed priming of transformations as the outcome perceived in earlier trials tended to repeat in subsequent trials and this repetition was contingent on perceptual experience. The observed priming was specific to transformations and did not originate in priming of perceptual states preceding a transient. Moreover, transformation priming was independent of attention and specific to low level stimulus attributes. In summary, we show how "transformation priors" and experience-driven updating of such priors helps to disambiguate sudden changes of sensory inputs. We discuss how dynamic transformation priors can be instantiated as "transition energies" in an "energy landscape" model of the visual perception.

Perceptual adaptation to structure-from-motion depends on the size of adaptor and probe objects, but not on the similarity of their shapes

Perceptual adaptation destabilizes the phenomenal appearance of multistable visual displays. Prolonged dominance of a perceptual state fatigues the associated neural population, lowering the likelihood of renewed perception of the same appearance (Nawrot & Blake in Perception & Psychophysics, 49, 230-44, 1991). Here, we used a selective adaptation paradigm to investigate perceptual adaptation for the illusory rotation of ambiguous structure-from-motion (SFM) displays. Specifically, we generated SFM objects with different three-dimensional shapes and presented them in random order, separating successive objects by brief blank periods, which included a mask. To assess the specificity of perceptual adaptation to the shape of SFM objects, we established the probability that a perceived direction of rotation persisted between successive objects of similar or dissimilar shape. We found that the strength of negative aftereffects depended on the volume, but not the shape, of adaptor and probe objects. More voluminous objects were both more effective as adaptor objects and more sensitive as probe objects. Surprisingly, we found these volume effects to be completely independent, since any relationship between two shapes (such as overlap between volumes, similarity of shape, or similarity of velocity profiles) failed to modulate the negative aftereffect. This pattern of results was the opposite of that observed for sensory memory of SFM objects, which reflects similarity between objects, but not volume of individual objects (Pastukhov et al. in Attention, Perception & Psychophysics, 75, 1215-1229, 2013). The disparate specificities of perceptual adaptation and sensory memory for identical SFM objects suggest that the two aftereffects engage distinct neural representations, consistent with recent brain imaging results (Schwiedrzik et al. in Cerebral Cortex, 2012).

Sensory memory of illusory depth in structure-from-motion

When multistable displays (stimuli consistent with two or more equally plausible perceptual interpretations) are presented intermittently, their perceptions are stabilized by sensory memory. Independent memory traces are generated not only for different types of multistable displays (Maier, Wilke, Logothetis, & Leopold, Current Biology 13:1076-1085, 2003), but also for different ambiguous features of binocular rivalry (Pearson & Clifford, Journal of Vision 4:196-202, 2004). In the present study, we examined whether a similar independence of sensory memories is observed in structure-from-motion (SFM), a multistable display with two ambiguous properties. In SFM, a 2-D planar motion creates a vivid impression of a rotating 3-D volume. Both the illusory rotation and illusory depth (i.e., how close parts of an object appear to the observer) of an SFM object are ambiguous. We dissociated the sensory memories of these two ambiguous properties by using an intermittent presentation in combination with a forced-ambiguous-switch paradigm (Pastukhov, Vonau, & Braun, PLoS ONE 7:e37734, 2012). We demonstrated that the illusory depth of SFM generates a sensory memory trace that is independent from that of illusory rotation. Despite this independence, the specificities levels of the sensory memories were identical for illusory depth and illusory rotation. The history effect was weakened by a change in the volumetric property of a shape (whether it was a hollow band or a filled drum volume), but not by changes in color or size. We discuss how these new results constrain models of sensory memory and SFM processing.