Effects of the orientation of moving objects on the perception of streaming/bouncing motion displays

Age-related changes to the attentional modulation of temporal binding

During multisensory integration, the time range within which visual and auditory information can be perceived as synchronous and bound together is known as the temporal binding window (TBW). With increasing age, the TBW becomes wider, such that older adults erroneously, and often dangerously, integrate sensory inputs that are asynchronous. Recent research suggests that attentional cues can narrow the width of the TBW in younger adults, sharpening temporal perception and increasing the accuracy of integration. However, due to their age-related declines in attentional control, it is not yet known whether older adults can deploy attentional resources to narrow the TBW in the same way as younger adults. This study investigated the age-related changes to the attentional modulation of the TBW. Thirty younger and 30 older adults completed a cued-spatial-attention version of the stream-bounce illusion, assessing the extent to which the visual and auditory stimuli were integrated when presented at three different stimulus-onset asynchronies, and when attending to a validly cued or invalidly cued location. A 2 × 2 × 3 mixed ANOVA revealed that when participants attended to the validly cued location (i.e., when attention was present), susceptibility to the stream-bounce illusion decreased. However, crucially, this attentional manipulation significantly affected audiovisual integration in younger adults, but not in older adults. These findings suggest that older adults have multisensory integration-related attentional deficits. Directions for future research and practical applications surrounding treatments to improve the safety of older adults' perception and navigation through the environment are discussed.

Sound-modulations of visual motion perception implicate the cortico-vestibular brain

A widely used example of the intricate (yet poorly understood) intertwining of multisensory signals in the brain is the audiovisual bounce inducing effect (ABE). This effect presents two identical objects moving along the azimuth with uniform motion and towards opposite directions. The perceptual interpretation of the motion is ambiguous and is modulated if a transient (sound) is presented in coincidence with the point of overlap of the two objects' motion trajectories. This phenomenon has long been written-off to simple attentional or decision-making mechanisms, although the neurological underpinnings for the effect are not well understood. Using behavioural metrics concurrently with event-related fMRI, we show that sound-induced modulations of motion perception can be further modulated by changing motion dynamics of the visual targets. The phenomenon engages the posterior parietal cortex and the parieto-insular-vestibular cortical complex, with a close correspondence of activity in these regions with behaviour. These findings suggest that the insular cortex is engaged in deriving a probabilistic perceptual solution through the integration of multisensory data.

Exploring the effectiveness of auditory, visual, and audio-visual sensory cues in a multiple object tracking environment

Maintaining object correspondence among multiple moving objects is an essential task of the perceptual system in many everyday life activities. A substantial body of research has confirmed that observers are able to track multiple target objects amongst identical distractors based only on their spatiotemporal information. However, naturalistic tasks typically involve the integration of information from more than one modality, and there is limited research investigating whether auditory and audio-visual cues improve tracking. In two experiments, we asked participants to track either five target objects or three versus five target objects amongst similarly indistinguishable distractor objects for 14 s. During the tracking interval, the target objects bounced occasionally against the boundary of a centralised orange circle. A visual cue, an auditory cue, neither or both coincided with these collisions. Following the motion interval, the participants were asked to indicate all target objects. Across both experiments and both set sizes, our results indicated that visual and auditory cues increased tracking accuracy although visual cues were more effective than auditory cues. Audio-visual cues, however, did not increase tracking performance beyond the level of purely visual cues for both high and low load conditions. We discuss the theoretical implications of our findings for multiple object tracking as well as for the principles of multisensory integration.

The Relationship Between Illusory Crescents and the Stream/Bounce Effect

We conducted two experiments to evaluate Meyerhoff and Scholl's (2018, Cognition 170, 88-94) hypothesis that illusory crescents contribute to resolutions in audiovisual stream/bounce displays. In Experiment 1, we measured illusory crescent size in the launching effect as a function of speed, overlap, and sound. In Experiment 2, we tabulated stream and bounce responses to similar stimuli with the same speed, sound, and overlap conditions as Experiment 1. Our critical manipulation of target speed spanned the range of values from typical stream/bounce investigations of ∼5 degrees/s up to the target speeds employed by Meyerhoff and Scholl ∼38 degrees/s. We replicated Meyerhoff and Scholl's findings at higher speeds, but not at slower speeds. Critically, we found that speed influenced crescent size judgements and bouncing responses in opposite directions. As target speed increased, illusory crescent size increased (Experiment 1), but the overall percentage of bounce responses decreased (Experiment 2). Additionally, we found that sound failed to enhance illusory crescent size at slower speeds but promotes bouncing responses at all speeds. The disassociation of the effects of speed and sound on illusory crescents with those effects on reported streaming/bouncing in similar displays provides compelling evidence against Meyerhoff and Scholl's hypothesis. Therefore, we conclude that illusory crescents do not account for the pattern of responses attributed to the stream/bounce effect.

Temporal dynamics of the flash-induced bouncing effect

Two identical visual disks moving toward each other on a two‐dimensional (2D) display are more likely to be perceived as “streaming through” than “bouncing off” each other after their coincidence. However, either a brief auditory tone or visual flash presented at the coincident moment of the disks can strikingly increase the incidence of the bouncing percept. Despite the neural substrates underlying the sound‐induced bouncing effect have been widely investigated, little is known about the neural mechanisms underlying the flash‐induced bouncing effect. The present study used event‐related potential recordings to explore the temporal dynamics of the flash‐induced bouncing effect. The results showed that the amplitude of the postcoincidence parietooccipital P2 component (190–230 ms after coincidence) elicited by the visual motion was significantly smaller on bouncing relative to streaming trials only when the flash was presented but not when absent. In addition, the parietal P3 component (330–430 ms) was found to be larger on bouncing than streaming trials when the flash was presented, but the opposite was true when no flash was presented. These electrophysiological findings suggest that the flash‐induced bouncing effect may occur at both perceptual and postperceptual stages of processing.

Multiple phases of cross-sensory interactions associated with the audiovisual bounce-inducing effect

Using event-related potential (ERP) recordings, the present study investigated the cross-modal neural activities underlying the audiovisual bounce-inducing effect (ABE) via a novel experimental design wherein the audiovisual bouncing trials were induced solely by the ABE. The within-subject (percept-based) analysis showed that early cross-modal interactions within 100-200 ms after sound onset over fronto-central and occipital regions were associated with the occurrence of the ABE, but the cross-modal interaction at a later latency (ND250, 220-280 ms) over fronto-central region did not differ between ABE trials and non-ABE trials. The between-subject analysis indicated that the cross-modal interaction revealed by ND250 was larger for subjects who perceived the ABE more frequently. These findings suggest that the ABE is generated as a consequence of the rapid interplay between the variations of early cross-modal interactions and the general multisensory binding predisposition at an individual level.

The Effect of Transient Location on the Resolution of Bistable Visual and Audiovisual Motion Sequences

We examined the attention and inference accounts of audiovisual perception using the stream/bounce display, a visual stimulus wherein two identical objects move toward each other, completely superimpose, then move apart. This display has two candidate percepts: stream past each other or bounce off each other. Presented without additional visual or auditory transients, the motion sequence tends to yield the streaming percept, but when coupled with a tone or flash at the point of coincidence, the response bias flips toward bouncing. We explored two competing accounts of this effect: the attentional hypothesis and the inference hypothesis. Participants watched a series of motion sequences where a transient, when present, occurred at the moment of coincidence either colocalised with the motion sequence (congruent presentation) or on the opposite side of the display (incongruent presentation). Assuming the spotlight or zoom lens metaphor, an attentional account predicts that incongruent presentations should be associated with a higher percentage of bouncing responses than congruent presentations, while the inferential account predicts the opposite effect. No effect was found for tone-only trials. However, in trials containing a visual transient, results showed higher proportions of bounce responses within congruent over incongruent presentations, favouring the inference hypothesis over a spotlight or zoom lens attentional account.

Early cross-modal interactions underlie the audiovisual bounce-inducing effect

Two identical visual disks moving towards one another on a two-dimensional display can be perceived as either "streaming through" or "bouncing off" each other after their coincidence/overlapping. A brief sound presented at the moment of the coincidence of the disks could strikingly bias the percept towards bouncing, which was termed the audiovisual bounce-inducing effect (ABE). Although the ABE has been studied intensively since its discovery, the debate about its origin is still unresolved so far. The present study used event-related potential (ERP) recordings to investigate whether or not early neural activities associated with cross-modal interactions play a role on the ABE. The results showed that the fronto-central P2 component ∼200 ms before the coincidence of the disks was predictive of subsequent streaming or bouncing percept in the unimodal visual display but not in auditory-visual display. More importantly, the cross-modal interactions revealed by the fronto-central positivity PD170 (125-175 ms after sound onset), as well as the occipital positivity PD190 (180-200 ms), were substantially enhanced on bouncing trials compared to streaming trials in the auditory-visual display. These findings provide direct electrophysiological evidence that early cross-modal interactions contribute to the origin of ABE phenomenon at the perceptual stage of processing.

Tactile stimulation disambiguates the perception of visual motion paths

Although visual perception traditionally has been considered to be impenetrable by non-visual information, there are a rising number of reports discussing cross-modal influences on visual perception. In two experiments, we investigated how coinciding vibrotactile stimulation affects the perception of two discs that move toward each other, superimpose in the center of the screen, and then move apart. Whereas two discs streaming past each other was the dominant impression when the visual event was presented in isolation, a brief coinciding vibrotactile stimulation at the moment of overlap biased the visual impression toward two discs bouncing off each other (Experiment 1). Further, the vibrotactile stimulation actually changed perceptual processing by reducing the amount of perceived overlap between the discs (Experiment 2), which has been demonstrated to be associated with a higher proportion of bouncing impressions. We propose that tactile-induced quantitative changes in the visual percept might alter the quality of the visual percept (from streaming to bouncing), thereby adding to the understanding of how cross-modal information interacts with early visual perception and how this interaction influences subsequent visual impressions.

Disambiguating the Stream/Bounce Illusion With Inference

The ‘stream/bounce’ illusion refers to the perception of an ambiguous visual display in which two discs approach each other on a collision course. The display can be seen as two discs streaming through each other or bouncing off each other. Which perception dominates, may be influenced by a brief transient, usually a sound, presented around the time of simulated contact. Several theories have been proposed to account for the switching in dominance based on sensory processing, attention and cognitive inference, but a universally applicable, parsimonious explanation has not emerged. We hypothesized that only cognitive inference would be influenced by the perceptual history of the display. We rendered the display technically unambiguous by vertically offsetting the targets’ trajectories and manipulated their history by allowing the objects to switch from one trajectory to the other up to four times before the potential collision point. As the number of switches increased, the number of ‘bounce’ responses also increased. These observations show that expectancy is a critical factor in determining whether a bounce or streaming is perceived and may form the basis for a universal explanation of instances of the stream/bounce illusion.

The Posture of Putting One's Palms Together Modulates Visual Motion Event Perception

We investigated the effect of an observer's hand postures on visual motion perception using the stream/bounce display. When two identical visual objects move across collinear horizontal trajectories toward each other in a two-dimensional display, observers perceive them as either streaming or bouncing. In our previous study, we found that when observers put their palms together just below the coincidence point of the two objects, the percentage of bouncing responses increased, mainly depending on the proprioceptive information from their own hands. However, it remains unclear if the tactile or haptic (force) information produced by the postures mostly influences the stream/bounce perception. We solved this problem by changing the tactile and haptic information on the palms of the hands. Experiment 1 showed that the promotion of bouncing perception was observed only when the posture of directly putting one's palms together was used, while there was no effect when a brick was sandwiched between the participant's palms. Experiment 2 demonstrated that the strength of force used when putting the palms together had no effect on increasing bounce perception. Our findings indicate that the hands-induced bounce effect derives from the tactile information produced by the direct contact between both palms.

Auditory-induced bouncing is a perceptual (rather than a cognitive) phenomenon: Evidence from illusory crescents

A central task for vision is to identify objects as the same persisting individuals over time and motion. The need for such processing is made especially clear in ambiguous situations such as the bouncing/streaming display: two discs move toward each other, superimpose, and then continue along their trajectories. Did the discs stream past each other, or bounce off each other? When people are likely to perceive streaming, playing a brief tone at the moment of overlap can readily cause them to see bouncing instead. Recent research has attributed this effect to decisional (rather than perceptual) processes by showing that auditory tones alter response biases but not the underlying sensitivity for detecting objective bounces. Here we explore the nature of this phenomenon using 'illusory causal crescents': when people perceive bouncing (or causal 'launching'), they also perceive the second disc to begin moving before being fully overlapped with the first disc (i.e. leaving an uncovered crescent). Here we demonstrate that merely playing a sound coincident to the moment of overlap can also reliably induce the perception of such illusory crescents. Moreover, this effect is due to the coincidence of the tone, per se, since the effect disappears when the tone is embedded in a larger regular tone sequence. Because observers never have to explicitly categorize their percept (e.g. as streaming)-and because the effect involves a subtle quantitative influence on another clearly visual property (i.e. the crescent's width)-we conclude that this audiovisual influence on the perception of identity over time reflects perceptual processing rather than higher-level decisions.

Pre-coincidence brain activity predicts the perceptual outcome of streaming/bouncing motion display

When two identical visual discs move toward each other on a two-dimensional visual display, they can be perceived as either "streaming through" or "bouncing off" each other after their coincidence. Previous studies have observed a strong bias toward the streaming percept. Additionally, the incidence of the bouncing percept in this ambiguous display could be increased by various factors, such as a brief sound at the moment of coincidence and a momentary pause of the two discs. The streaming/bouncing bistable motion phenomenon has been studied intensively since its discovery. However, little is known regarding the neural basis underling the perceptual ambiguity in the classic version of the streaming/bouncing motion display. The present study investigated the neural basis of the perception disambiguating underling the processing of the streaming/bouncing bistable motion display using event-related potential (ERP) recordings. Surprisingly, the amplitude of frontal central P2 (220-260 ms) that was elicited by the moving discs ~200 ms before the coincidence of the two discs was observed to be predictive of subsequent streaming or bouncing percept. A larger P2 amplitude was observed for streaming percept than the bouncing percept. These findings suggest that the streaming/bouncing bistable perception may have been disambiguated unconsciously ~200 ms before the coincidence of the two discs.

Low-Level Motion Characteristics Do Not Account for Perceptions of Stream-Bounce Stimuli

The stream-bounce effect refers to a bistable motion stimulus that is interpreted as two targets either “streaming” past or “bouncing” off one another, and the manipulations that bias responses. Directional bias, according to Bertenthal et al., is an account of the effect proposing that low-level motion integration promotes streaming, and its disruption leads to bouncing, and it is sometimes cited either directly in a bottom-up fashion or indirectly under top-down control despite Sekuler and Sekuler finding evidence inconsistent with it. We tested two key aspects of the hypothesis: (a) comparable changes in speed should produce comparable disruptions and lead to similar effects; and (b) speed changes alone should disrupt integration without the need for additional more complex changes of motion. We found that target motion influences stream-bounce perception, but not as directional bias predicts. Our results support Sekuler and Sekuler and argue against the low-level motion signals driving perceptual outcomes in stream-bounce displays (directly or indirectly) and point to higher level inferential processes involving perceptual history and expectation. Directional bias as a mechanism should be abandoned and either another specific bottom-up process must be proposed and tested or consideration should be given to top-down factors alone driving the effect.

Hand Positions Alter Bistable Visual Motion Perception

We found that a hand posture with the palms together located just below the stream/bounce display could increase the proportion of bouncing perception. This effect, called the hands-induced bounce (HIB) effect, did not occur in the hands-cross condition or in the one-hand condition. By using rubber hands or covering the participants’ hands with a cloth, we demonstrated that the visual information of the hand shapes was not a critical factor in producing the HIB effect, whereas proprioceptive information seemed to be important. We also found that the HIB effect did not occur when the participants’ hands were far from the coincidence point, suggesting that the HIB effect might be produced within a limited spatial area around the hands.

Transformation priming helps to disambiguate sudden changes of sensory inputs

Retinal input is riddled with abrupt transients due to self-motion, changes in illumination, object-motion, etc. Our visual system must correctly interpret each of these changes to keep visual perception consistent and sensitive. This poses an enormous challenge, as many transients are highly ambiguous in that they are consistent with many alternative physical transformations. Here we investigated inter-trial effects in three situations with sudden and ambiguous transients, each presenting two alternative appearances (rotation-reversing structure-from-motion, polarity-reversing shape-from-shading, and streaming-bouncing object collisions). In every situation, we observed priming of transformations as the outcome perceived in earlier trials tended to repeat in subsequent trials and this repetition was contingent on perceptual experience. The observed priming was specific to transformations and did not originate in priming of perceptual states preceding a transient. Moreover, transformation priming was independent of attention and specific to low level stimulus attributes. In summary, we show how "transformation priors" and experience-driven updating of such priors helps to disambiguate sudden changes of sensory inputs. We discuss how dynamic transformation priors can be instantiated as "transition energies" in an "energy landscape" model of the visual perception.

The effects of attention on the temporal integration of multisensory stimuli

In unisensory contexts, spatially-focused attention tends to enhance perceptual processing. How attention influences the processing of multisensory stimuli, however, has been of much debate. In some cases, attention has been shown to be important for processes related to the integration of audio-visual stimuli, but in other cases such processes have been reported to occur independently of attention. To address these conflicting results, we performed three experiments to examine how attention interacts with a key facet of multisensory processing: the temporal window of integration (TWI). The first two experiments used a novel cued-spatial-attention version of the bounce/stream illusion, wherein two moving visual stimuli with intersecting paths tend to be perceived as bouncing off rather than streaming through each other when a brief sound occurs near in time. When the task was to report whether the visual stimuli appeared to bounce or stream, attention served to narrow this measure of the TWI and bias perception toward “streaming”. When the participants’ task was to explicitly judge the simultaneity of the sound with the intersection of the moving visual stimuli, however, the results were quite different. Specifically, attention served to mainly widen the TWI, increasing the likelihood of simultaneity perception, while also substantially increasing the simultaneity judgment accuracy when the stimuli were actually physically simultaneous. Finally, in Experiment 3, where the task was to judge the simultaneity of a simple, temporally discrete, flashed visual stimulus and the same brief tone pip, attention had no effect on the measured TWI. These results highlight the flexibility of attention in enhancing multisensory perception and show that the effects of attention on multisensory processing are highly dependent on the task demands and observer goals.

Aging and audio-visual and multi-cue integration in motion

The perception of naturalistic events relies on the ability to integrate information from multiple sensory systems, an ability that may change with healthy aging. When two objects move toward and then past one another, their trajectories are perceptually ambiguous: the objects may seem to stream past one another or bounce off one another. Previous research showed that auditory or visual events that occur at the time of disks' coincidence could bias the percept toward bouncing or streaming. We exploited this malleable percept to assay age-related changes in the integration of multiple inter- and intra-modal cues. The disks' relative luminances were manipulated to produce stimuli strongly favoring either bouncing or streaming, or to produce ambiguous motion (equal luminances). A sharp sound coincident with the disks' overlap increased both groups' perception of bouncing, but did so significantly less for older subjects. An occluder's impact on motion perception varied with its duration: a long duration occluder promoted streaming in both groups; a brief occluder promoted bouncing in younger subjects, but not older ones. Control experiments demonstrated that the observed differences between younger and older subjects resulted from neither age-related changes in retinal illuminance nor age-related changes in hearing, pointing to weakened inter- and intra-modal integration with aging. These changes could contribute to previously demonstrated age-related perceptual and memory deficits.

Effect of form cues on 1D and 2D motion pooling

Local-motion information can provide either 1-dimensional (1D) or 2-dimensional (2D) solutions. 1D signals occur when the aperture problem has not been solved, so each signal is an estimate of the local-orthogonal component of the object's motion. 2D signals occur when the aperture problem has been solved, so each signal is an estimate of the object's motion. Previous research (JoV, 2009, 9, 1-25) has shown that 1D and 2D signals are pooled differently, via intersection-of-constraints (IOC) and vector-average processes, respectively. Previous research (e.g. Vis. Res., 2003, 2290-2301) has also indicated that form cues can influence how motion signals are perceived. We investigated whether forms cues can affect the pooling of motion signals and whether they differentially affect the pooling of 1D and 2D signals. Global-Gabor (GG) and global-plaid (GP) stimuli were used. These stimuli consist of multiple apertures that contain either Gabors or plaids, respectively. In the GG stimulus the global solution is defined by having the Gabor carriers move (1D signals) such that they are consistent with a single IOC-defined solution. In the GP stimuli the plaid motion (2D signals) are consistent with a vector-average solution defined by a Gaussian distribution. Form cues can be introduced by adding orientation information to the apertures that is either consistent (aligned with) or inconsistent (orthogonal to) with the global-solution. With the 1D stimuli, form cues affect how the motion signals are pooled, with motion being perceived in the direction defined by the orientation cue. Orientation cues had no direct effect on the pooling of the 2D signals.

The origin of the audiovisual bounce inducing effect: a TMS study

The audiovisual bounce inducing effect (ABE) is a bouncing percept induced by the presence of a sound at the moment of two moving objects intercepting in a motion display otherwise perceived as streaming. The origin of the ABE is still debated: the effect could arise from the subtraction of attentional resources caused by the sound (needed to favor the perception of streaming), and/or from the cross-modal integration (binding) of visual and auditory information: indeed bouncing-like sounds are best in inducing the ABE. The neural mechanism responsible for the ABE is still unknown. Here, by using offline TMS, we investigated the role of the posterior parietal cortex (PPC), thought to be involved in both attentional and binding processes, in the generation of the ABE. Results show that disrupting the functional integrity of the right (but not the left) PPC has the effect of weakening the binding of cross-modal information, which reduces the magnitude of the ABE. Indeed, if the effect of parietal stimulation was merely to disrupt attention, we would expect an increase (not a decrease) of bouncing percepts. The present study not only shows the involvement of the right PPC in the ABE, but also support the notion that cross-modal binding (and not attention) is at the origin of the ABE.

The Stream/Bounce Effect Occurs for Luminance- and Disparity-Defined Motion Targets

We generalised the stream/bounce effect to dynamic random element displays containing luminance- or disparity-defined targets. Previous studies investigating audio-visual interactions in this context have exclusively employed motion sequences with luminance-defined disks or squares and have focused on properties of the accompanying brief stimuli rather than the visual properties of the motion targets. We found that the presence of a brief sound temporally close to coincidence, or a visual flash at coincidence significantly promote bounce perception for motion targets defined by either luminance contrast or disparity contrast. A brief tone significantly promoted bouncing of luminance-defined targets above a no-sound baseline when it was presented at least 250 ms before coincidence and 100 ms after coincidence. A similar pattern was observed for disparity-defined targets, though the tone promoted bouncing above the no-sound baseline when presented at least 350 ms before and 300 ms after coincidence. We further explored the temporal properties of audio-visual interactions for these two display types and found that bounce perception saturated at similar durations after coincidence. The stream/bounce illusion manifests itself in dynamic random-element displays and is similar for luminance- and disparity-defined motion targets.

Cognitive binding in schizophrenia: weakened integration of temporal intersensory information

Cognitive functioning is based on binding processes, by which different features and elements of neurocognition are integrated and coordinated. Binding is an essential ingredient of, for instance, Gestalt perception. We have implemented a paradigm of causality perception based on the work of Albert Michotte, in which 2 identical discs move from opposite sides of a monitor, steadily toward, and then past one another. Their coincidence generates an ambiguous percept of either "streaming" or "bouncing," which the subjects (34 schizophrenia spectrum patients and 34 controls with mean age 27.9 y) were instructed to report. The latter perception is a marker of the binding processes underlying perceived causality (type I binding). In addition to this visual task, acoustic stimuli were presented at different times during the task (150 ms before and after visual coincidence), which can modulate perceived causality. This modulation by intersensory and temporally delayed stimuli is viewed as a different type of binding (type II). We show here, using a mixed-effects hierarchical analysis, that type II binding distinguishes schizophrenia spectrum patients from healthy controls, whereas type I binding does not. Type I binding may even be excessive in some patients, especially those with positive symptoms; Type II binding, however, was generally attenuated in patients. The present findings point to ways in which the disconnection (or Gestalt) hypothesis of schizophrenia can be refined, suggesting more specific markers of neurocognitive functioning and potential targets of treatment.

Stream/bounce event perception reveals a temporal limit of motion correspondence based on surface feature over space and time

We examined how stream/bounce event perception is affected by motion correspondence based on the surface features of moving objects passing behind an occlusion. In the stream/bounce display two identical objects moving across each other in a two-dimensional display can be perceived as either streaming through or bouncing off each other at coincidence. Here, surface features such as colour (Experiments 1 and 2) or luminance (Experiment 3) were switched between the two objects at coincidence. The moment of coincidence was invisible to observers due to an occluder. Additionally, the presentation of the moving objects was manipulated in duration after the feature switch at coincidence. The results revealed that a postcoincidence duration of approximately 200 ms was required for the visual system to stabilize judgments of stream/bounce events by determining motion correspondence between the objects across the occlusion on the basis of the surface feature. The critical duration was similar across motion speeds of objects and types of surface features. Moreover, controls (Experiments 4a–4c) showed that cognitive bias based on feature (colour/luminance) congruency across the occlusion could not fully account for the effects of surface features on the stream/bounce judgments. We discuss the roles of motion correspondence, visual feature processing, and attentive tracking in the stream/bounce judgments.

Auditory induced bounce perception persists as the probability of a motion reversal is reduced

When two identical targets move toward one another from opposite sides of a display and continue past one another along collinear trajectories, they can be perceived to either stream past or bounce off of one another. Streaming is the dominant perception in motion displays free of additional transients, while bouncing predominates when a transient (eg auditory or visual) is presented at the point of coincidence. We investigated whether the auditory induced bias towards bouncing would persist as the probability of a motion reversal was reduced by introducing a spatial offset either vertically in a 2-D display or in depth in a 3-D display. Offset conditions were combined with two auditory conditions (tone or no-tone at the point of coincidence) in the presence or absence of a central occluder. In conditions with no sound, streaming was reported on a clear majority of trials, regardless of spatial offset. When a transient tone was presented, reported motion reversals dominated and persisted for increasing verbal offsets up to 17.9 min of arc and for 3-D trajectory offsets up to 25.6 min of arc. The bounce-promoting effect of an auditory tone at the point of coincidence in stream/bounce displays persists in spite of rendering the visual motion sequence unambiguous and more consistent with streaming.

The audiovisual tau effect in infancy

BACKGROUND

Perceived spatial intervals between successive flashes can be distorted by varying the temporal intervals between them (the "tau effect"). A previous study showed that a tau effect for visual flashes could be induced when they were accompanied by auditory beeps with varied temporal intervals (an audiovisual tau effect).

METHODOLOGY/PRINCIPAL FINDINGS

We conducted two experiments to investigate whether the audiovisual tau effect occurs in infancy. Forty-eight infants aged 5-8 months took part in this study. In Experiment 1, infants were familiarized with audiovisual stimuli consisting of three pairs of two flashes and three beeps. The onsets of the first and third pairs of flashes were respectively matched to those of the first and third beeps. The onset of the second pair of flashes was separated from that of the second beep by 150 ms. Following the familiarization phase, infants were exposed to a test stimulus composed of two vertical arrays of three static flashes with different spatial intervals. We hypothesized that if the audiovisual tau effect occurred in infancy then infants would preferentially look at the flash array with spatial intervals that would be expected to be different from the perceived spatial intervals between flashes they were exposed to in the familiarization phase. The results of Experiment 1 supported this hypothesis. In Experiment 2, the first and third beeps were removed from the familiarization stimuli, resulting in the disappearance of the audiovisual tau effect. This indicates that the modulation of temporal intervals among flashes by beeps was essential for the audiovisual tau effect to occur (Experiment 2).

CONCLUSIONS/SIGNIFICANCE

These results suggest that the cross-modal processing that underlies the audiovisual tau effect occurs even in early infancy. In particular, the results indicate that audiovisual modulation of temporal intervals emerges by 5-8 months of age.

Audiovisual temporal capture underlies flash fusion

When sequential visual flashes are accompanied by a lower number of sequential auditory pulses, the perceived number of visual flashes is lower than the actual number, an illusion termed 'flash fusion'. We examined whether temporal capture of flashes by pulses underlay flash fusion. One of the visual flashes was given a luminance increment, and observers reported which flash had the luminance increment. Results showed that the pulse strongly captured the flashes in its temporal vicinity, resulting in flash fusion. Moreover, when one of the successive pulses was given a higher frequency than others, the luminance increment was perceptually paired with the pulse with the higher frequency. The pairing of audiovisual features disappeared when the temporal pattern of the pulse frequency was difficult for the observer to anticipate. These data indicate that flash fusion is caused by temporal capture of flashes by the pulse, and that feature matching between auditory and visual signals also contributes to the modulation of perceived temporal structure of flashes during flash fusion.