Implicit expectation modulates multisensory perception

Stimulus statistics can induce expectations that in turn can influence multisensory perception. In three experiments, we manipulate perceptual history by biasing stimulus statistics and examined the effect of implicit expectations on the perceptual resolution of a bistable visual stimulus that is modulated by sound. First, we found a general effect of expectation such that responses were biased in line with the biased statistics and interpret this as a bias towards an implicitly expected outcome. Second, expectation did not influence the perception of all types of stimuli. In both Experiment 1 and Experiment 2, integrated audio-visual stimuli were affected by expectation but visual-only and unintegrated audio-visual stimuli were not. In Experiment 3 we examined the sensory versus interpretational effects of expectation and found that contrary to our predictions, an expectation of audio-visually integrated stimuli was associated with impaired multisensory integration compared to visual-only or unintegrated audio-visual stimuli. Our findings suggest that perceptual experience implicitly creates expectations that influence multisensory perception, which appear to be about perceptual outcomes rather than sensory stimuli. Finally, in the case of resolving perceptual ambiguity, the expectation effect is an effect on cognitive rather than sensory processes.

The effect of movement speed on audiovisual temporal integration in streaming-bouncing illusion

Motion perception in real situations is often stimulated by multisensory information. Speed is an essential characteristic of moving objects; however, at present, it is not clear whether speed affects the process of audiovisual temporal integration in motion perception. Therefore, this study used a streaming-bouncing task (a bistable motion perception; SB task) combined with a simultaneous judgment task (SJ task) to explore the effect of speed on audiovisual temporal integration from implicit and explicit perspectives. The experiment had a within-subjects design, two speed conditions (fast/slow), eleven audiovisual conditions [stimulus onset asynchrony (SOA): 0 ms/ ± 60 ms/ ± 120 ms/ ± 180 ms/ ± 240 ms/ ± 300 ms], and a visual-only condition. A total of 30 subjects were recruited for the study. These participants completed the SB task and the SJ task successively. The results showed the following outcomes: (1) the optimal times needed to induce the "bouncing" illusion and maximum audiovisual bounce-inducing effect (ABE) magnitude were much earlier than that for the optimal time of audiovisual synchrony, (2) speed as a bottom-up factor could affect the proportion of "bouncing" perception in SB illusions but did not affect the ABE magnitude, (3) speed could also affect the ability of audiovisual temporal integration in motion perception, and the main manifestation was that the point of subjective simultaneity (PSS) in fast speed conditions was earlier than that of slow speed conditions in the SJ task and (4) the SB task and SJ task were not related. In conclusion, the time to complete the maximum audiovisual integration was different from the optimal time for synchrony perception; moreover, speed could affect audiovisual temporal integration in motion perception but only in explicit temporal tasks.

Temporal dynamics of a perceptual decision

Previous research suggests that cognitive factors acting in a top-down manner influence the perceptual interpretation of ambiguous stimuli. To examine the temporal unfolding of these influences as a perceptual decision evolves, we have implemented a modified version of the stream-bounce display. Our novel approach allows us to track responses to stream-bounce stimuli dynamically over the entire course of the motion sequence rather than collecting a subjective report after the fact. Using a trackpad, we had participants control a cursor to track a stream-bounce target actively from start to end and measured tracking speed throughout as the dependent variable. Our paradigm replicated the typical effect of visual-only displays being associated with a streaming bias and audiovisual displays with a bouncing bias. Our main finding is a significant behavioral change preceding a perceptual decision that then predicts that decision. Specifically, for trials in which the sound was presented, tracking speeds were significantly slower starting 500 ms before the point of coincidence and presentation of the sound for bounce compared to stream responses. We suggest that behavioral response may reflect a cognitive expectation of a perceptual outcome that then biases action and the interpretation of sensory input to favor that forthcoming percept in a manner consistent with both the predictive-coding and common-coding theoretical frameworks. Our approach provides a novel behavioral corroboration of recent imaging studies that are suggestive of early brain activity in perception and action.

The Effect of Transient Location on the Resolution of Bistable Visual and Audiovisual Motion Sequences

We examined the attention and inference accounts of audiovisual perception using the stream/bounce display, a visual stimulus wherein two identical objects move toward each other, completely superimpose, then move apart. This display has two candidate percepts: stream past each other or bounce off each other. Presented without additional visual or auditory transients, the motion sequence tends to yield the streaming percept, but when coupled with a tone or flash at the point of coincidence, the response bias flips toward bouncing. We explored two competing accounts of this effect: the attentional hypothesis and the inference hypothesis. Participants watched a series of motion sequences where a transient, when present, occurred at the moment of coincidence either colocalised with the motion sequence (congruent presentation) or on the opposite side of the display (incongruent presentation). Assuming the spotlight or zoom lens metaphor, an attentional account predicts that incongruent presentations should be associated with a higher percentage of bouncing responses than congruent presentations, while the inferential account predicts the opposite effect. No effect was found for tone-only trials. However, in trials containing a visual transient, results showed higher proportions of bounce responses within congruent over incongruent presentations, favouring the inference hypothesis over a spotlight or zoom lens attentional account.

Streaming, Bouncing, and Rotation: The Polka Dance Stimulus

When the objects in a typical stream-bounce stimulus are made to rotate on a circular trajectory, not two but four percepts can be observed: streaming, bouncing, clockwise rotation, and counterclockwise rotation, often with spontaneous reversals between them. When streaming or bouncing is perceived, the objects seem to move on individual, opposite trajectories. When rotation is perceived, however, the objects seem to move in unison on the same circular trajectory, as if constituting the edges of a virtual pane that pivots around its axis. We called this stimulus the Polka Dance stimulus. Experiments showed that with some viewing experience, the viewer can "hold" the rotation percepts. Yet even when doing so, a short sound at the objects' point of coincidence can induce a bouncing percept. Besides this fast percept switching from rotation to bouncing, an external stimulus might also induce slower rotation direction switches, from clockwise to counterclockwise, or vice versa.

Arousing emoticons edit stream/bounce perception of objects moving past each other

When two identical objects move toward each other, overlap completely, and continue toward opposite ends of a space, observers might perceive them as streaming through or bouncing off each other. This phenomenon is known as ‘stream/bounce perception’. In this study, we investigated the effect of the presentation of emoticons on stream/bounce perception in five experiments. In Experiment 1, we used emoticons representing anger (‘(‘∧’)’), a smile (‘(^_^)’), and a sober face (‘(°_°)’, as a control), and observers were asked to judge whether two objects unrelated to the emoticon had streamed through or bounced off each other. The anger emoticon biased perception toward bouncing when compared with the smile or sober face emoticon. In Experiments 2 and 3, we controlled for the valence and arousal of emoticons, and found that arousal influenced stream/bounce perception but valence did not. Experiments 4 and 5 ruled out the possibility of attentional capture and response bias for the emoticon with higher arousal. Taken together, the findings indicate that emoticons with higher arousal evoke a mental image of a ‘collision’ in observers, thereby eliciting the bounce perception.

Auditory-induced bouncing is a perceptual (rather than a cognitive) phenomenon: Evidence from illusory crescents

A central task for vision is to identify objects as the same persisting individuals over time and motion. The need for such processing is made especially clear in ambiguous situations such as the bouncing/streaming display: two discs move toward each other, superimpose, and then continue along their trajectories. Did the discs stream past each other, or bounce off each other? When people are likely to perceive streaming, playing a brief tone at the moment of overlap can readily cause them to see bouncing instead. Recent research has attributed this effect to decisional (rather than perceptual) processes by showing that auditory tones alter response biases but not the underlying sensitivity for detecting objective bounces. Here we explore the nature of this phenomenon using 'illusory causal crescents': when people perceive bouncing (or causal 'launching'), they also perceive the second disc to begin moving before being fully overlapped with the first disc (i.e. leaving an uncovered crescent). Here we demonstrate that merely playing a sound coincident to the moment of overlap can also reliably induce the perception of such illusory crescents. Moreover, this effect is due to the coincidence of the tone, per se, since the effect disappears when the tone is embedded in a larger regular tone sequence. Because observers never have to explicitly categorize their percept (e.g. as streaming)-and because the effect involves a subtle quantitative influence on another clearly visual property (i.e. the crescent's width)-we conclude that this audiovisual influence on the perception of identity over time reflects perceptual processing rather than higher-level decisions.