The effect of looming and receding sounds on the perceived in-depth orientation of depth-ambiguous biological motion figures

Audiovisual correspondence facilitates the visual search for biological motion

Hearing synchronous sounds may facilitate the visual search for the concurrently changed visual targets. Evidence for this audiovisual attentional facilitation effect mainly comes from studies using artificial stimuli with relatively simple temporal dynamics, indicating a stimulus-driven mechanism whereby synchronous audiovisual cues create a salient object to capture attention. Here, we investigated the crossmodal attentional facilitation effect on biological motion (BM), a natural, biologically significant stimulus with complex and unique dynamic profiles. We found that listening to temporally congruent sounds, compared with incongruent sounds, enhanced the visual search for BM targets. More intriguingly, such a facilitation effect requires the presence of distinctive local motion cues (especially the accelerations in feet movement) independent of the global BM configuration, suggesting a crossmodal mechanism triggered by specific biological features to enhance the salience of BM signals. These findings provide novel insights into how audiovisual integration boosts attention to biologically relevant motion stimuli and extend the function of a proposed life detection system driven by local kinematics of BM to multisensory life motion perception.

Cross-modal social attention triggered by biological motion cues

Previous research has demonstrated that biological motion (BM) cues can induce reflexive attentional orienting. This BM-triggered social attention has hitherto only been investigated within visual modality. It remains unknown whether and to what extent social attention induced by BM cues can occur across different sensory modalities. By introducing auditory stimuli to a modified central cueing paradigm, we showed that observers responded significantly faster to auditory targets presented in the walking direction of BM than in the opposite direction, reflecting the notion that BM cues can trigger cross-modal social attention. This effect was not due to the viewpoint effect of the global configuration and could be extended to local BM cues without any global configuration. Critically, such cross-modal social attention was sensitive to the orientation of BM cues and completely disappeared when critical biological characteristics were removed. Our findings, taken together, support the existence of a special multimodal attention mechanism tuned to life motion signals and shed new light on the unique and cross-modal nature of social attention.

Biased estimations of interpersonal distance in non-clinical social anxiety

Previous studies have indicated that socially-anxious individuals prefer to maintain a greater interpersonal distance from others, specifically from strangers. Notwithstanding, it has yet to be examined whether this preference for distance is associated with estimating the physical interpersonal distance in a distorted manner. In the current study, 100 participants performed a computerized task that measured estimated distance (Study 1). An additional sample of 75 participants performed the same task for the purpose of replication, and further took part in a new task that measured estimated distance from a stranger in a real-life setting (Study 2). In both studies social anxiety correlated with estimating the interpersonal distance from strangers as shorter. Furthermore, ones' preferred distance from a stranger was predicted by this distance estimation bias. Taken together, our findings are the first to reveal distance estimation bias in social anxiety, suggesting a role for distorted distance estimation in avoidance behavior.

Aging Impairs Audiovisual Facilitation of Object Motion Within Self-Motion

The presence of a moving sound has been shown to facilitate the detection of an independently moving visual target embedded among an array of identical moving objects simulating forward self-motion (Calabro et al., Proc. R. Soc. B, 2011). Given that the perception of object motion within self-motion declines with aging, we investigated whether older adults can also benefit from the presence of a congruent dynamic sound when detecting object motion within self-motion. Visual stimuli consisted of nine identical spheres randomly distributed inside a virtual rectangular prism. For 1 s, all the spheres expanded outward simulating forward observer translation at a constant speed. One of the spheres (the target) had independent motion either approaching or moving away from the observer at three different speeds. In the visual condition, stimuli contained no sound. In the audiovisual condition, the visual stimulus was accompanied by a broadband noise sound co-localized with the target, whose loudness increased or decreased congruent with the target’s direction. Participants reported which of the spheres had independent motion. Younger participants showed higher target detection accuracy in the audiovisual compared to the visual condition at the slowest speed level. Older participants showed overall poorer target detection accuracy than the younger participants, but the presence of the sound had no effect on older participants’ target detection accuracy at either speed level. These results indicate that aging may impair cross-modal integration in some contexts. Potential reasons for the absence of auditory facilitation in older adults are discussed.

Auditory motion in the sighted and blind: Early visual deprivation triggers a large-scale imbalance between auditory and "visual" brain regions

How early blindness reorganizes the brain circuitry that supports auditory motion processing remains controversial. We used fMRI to characterize brain responses to in-depth, laterally moving, and static sounds in early blind and sighted individuals. Whole-brain univariate analyses revealed that the right posterior middle temporal gyrus and superior occipital gyrus selectively responded to both in-depth and laterally moving sounds only in the blind. These regions overlapped with regions selective for visual motion (hMT+/V5 and V3A) that were independently localized in the sighted. In the early blind, the right planum temporale showed enhanced functional connectivity with right occipito-temporal regions during auditory motion processing and a concomitant reduced functional connectivity with parietal and frontal regions. Whole-brain searchlight multivariate analyses demonstrated higher auditory motion decoding in the right posterior middle temporal gyrus in the blind compared to the sighted, while decoding accuracy was enhanced in the auditory cortex bilaterally in the sighted compared to the blind. Analyses targeting individually defined visual area hMT+/V5 however indicated that auditory motion information could be reliably decoded within this area even in the sighted group. Taken together, the present findings demonstrate that early visual deprivation triggers a large-scale imbalance between auditory and "visual" brain regions that typically support the processing of motion information.

Content congruency and its interplay with temporal synchrony modulate integration between rhythmic audiovisual streams

Both lower-level stimulus factors (e.g., temporal proximity) and higher-level cognitive factors (e.g., content congruency) are known to influence multisensory integration. The former can direct attention in a converging manner, and the latter can indicate whether information from the two modalities belongs together. The present research investigated whether and how these two factors interacted in the perception of rhythmic, audiovisual (AV) streams derived from a human movement scenario. Congruency here was based on sensorimotor correspondence pertaining to rhythm perception. Participants attended to bimodal stimuli consisting of a humanlike figure moving regularly to a sequence of auditory beat, and detected a possible auditory temporal deviant. The figure moved either downwards (congruently) or upwards (incongruently) to the downbeat, while in both situations the movement was either synchronous with the beat, or lagging behind it. Greater cross-modal binding was expected to hinder deviant detection. Results revealed poorer detection for congruent than for incongruent streams, suggesting stronger integration in the former. False alarms increased in asynchronous stimuli only for congruent streams, indicating greater tendency for deviant report due to visual capture of asynchronous auditory events. In addition, a greater increase in perceived synchrony was associated with a greater reduction in false alarms for congruent streams, while the pattern was reversed for incongruent ones. These results demonstrate that content congruency as a top-down factor not only promotes integration, but also modulates bottom-up effects of synchrony. Results are also discussed regarding how theories of integration and attentional entrainment may be combined in the context of rhythmic multisensory stimuli.

Bodily movement of approach is detected faster than that of receding

The human visual system is efficient at detecting an approaching object. In detecting approaching human beings, bodily movement serves as a cue for the visual system to compute moving direction. On the basis of this knowledge, we hypothesized that bodily movement implying approach is detected faster than receding bodily movement even when only bodily movement is available as a clue to discerning motion direction. To examine this hypothesis, we conducted a visual search experiment in which participants searched for a point-light figure with approaching or receding walking movement. Results showed that an approaching point-light figure was detected faster than a receding one. This search asymmetry was eliminated when the figures were presented upside-down. These findings indicate the potency of bodily movement that implies approach in effectively capturing visuospatial attention.

Further explorations of the facing bias in biological motion perception: perspective cues, observer sex, and response times

The human visual system has evolved to be highly sensitive to visual information about other persons and their movements as is illustrated by the effortless perception of point-light figures or ‘biological motion’. When presented orthographically, a point-light walker is interpreted in two anatomically plausible ways: As ‘facing the viewer’ or as ‘facing away’ from the viewer. However, human observers show a ‘facing bias’: They perceive such a point-light walker as facing towards them in about 70-80% of the cases. In studies exploring the role of social and biological relevance as a possible account for the facing bias, we found a ‘figure gender effect’: Male point-light figures elicit a stronger facing bias than female point-light figures. Moreover, we also found an ‘observer gender effect’: The ‘figure gender effect’ was stronger for male than for female observers. In the present study we presented to 11 males and 11 females point-light walkers of which, very subtly, the perspective information was manipulated by modifying the earlier reported ‘perspective technique’. Proportions of ‘facing the viewer’ responses and reaction times were recorded. Results show that human observers, even in the absence of local shape or size cues, easily pick up on perspective cues, confirming recent demonstrations of high visual sensitivity to cues on whether another person is potentially approaching. We also found a consistent difference in how male and female observers respond to stimulus variations (figure gender or perspective cues) that cause variations in the perceived in-depth orientation of a point-light walker. Thus, the ‘figure gender effect’ is possibly caused by changes in the relative locations and motions of the dots that the perceptual system tends to interpret as perspective cues. Third, reaction time measures confirmed the existence of the facing bias and recent research showing faster detection of approaching than receding biological motion.

Reference repulsion in the categorical perception of biological motion

Perceiving biological motion is important for understanding the intentions and future actions of others. Perceiving an approaching person's behavior may be particularly important, because such behavior often precedes social interaction. To this end, the visual system may devote extra resources for perceiving an oncoming person's heading. If this were true, humans should show increased sensitivity for perceiving approaching headings, and as a result, a repulsive perceptual effect around the categorical boundary of leftward/rightward motion. We tested these predictions and found evidence for both. First, observers were especially sensitive to the heading of an approaching person; variability in estimates of a person's heading decreased near the category boundary of leftward/rightward motion. Second, we found a repulsion effect around the category boundary; a person walking approximately toward the observer was perceived as being repelled away from straight ahead. This repulsive effect was greatly exaggerated for perception of a very briefly presented person or perception of a chaotic crowd, suggesting that repulsion may protect against categorical errors when sensory noise is high. The repulsion effect with a crowd required integration of local motion and human form, suggesting an origin in high-level stages of visual processing. Similar repulsive effects may underlie categorical perception with other social features. Overall, our results show that a person's direction of walking is categorically perceived, with improved sensitivity at the category boundary and a concomitant repulsion effect.

Multidimensional processing of dynamic sounds: more than meets the ear

Strong cross-modal interactions exist between visual and auditory processing. The relative contributions of perceptual versus decision-related processes to such interactions are only beginning to be understood. We used methodological and statistical approaches to control for potential decision-related contributions such as response interference, decisional criterion shift, and strategy selection. Participants were presented with rising-, falling-, and constant-amplitude sounds and were asked to detect change (increase or decrease) in sound amplitude while ignoring an irrelevant visual cue of a disk that grew, shrank, or stayed constant in size. Across two experiments, testing context was manipulated by varying the grouping of visual cues during testing, and cross-modal congruency showed independent perceptual and decision-related effects. Whereas a change in testing context greatly affected criterion shifts, cross-modal effects on perceptual sensitivity remained relatively consistent. In general, participants were more sensitive to increases in sound amplitude and less sensitive to sounds paired with dynamic visual cues. As compared with incongruent visual cues, congruent cues enhanced detection of amplitude decreases, but not increases. These findings suggest that the relative contributions of perceptual and decisional processing and the impacts of these processes on cross-modal interactions can vary significantly depending on asymmetries in within-modal processing, as well as consistencies in cross-modal dynamics.

Premotor Cortex Is Sensitive to Auditory–Visual Congruence for Biological Motion

The auditory and visual perception systems have developed special processing strategies for ecologically valid motion stimuli, utilizing some of the statistical properties of the real world. A well-known example is the perception of biological motion, for example, the perception of a human walker. The aim of the current study was to identify the cortical network involved in the integration of auditory and visual biological motion signals. We first determined the cortical regions of auditory and visual coactivation (Experiment 1); a conjunction analysis based on unimodal brain activations identified four regions: middle temporal area, inferior parietal lobule, ventral premotor cortex, and cerebellum. The brain activations arising from bimodal motion stimuli (Experiment 2) were then analyzed within these regions of coactivation. Auditory footsteps were presented concurrently with either an intact visual point-light walker (biological motion) or a scrambled point-light walker; auditory and visual motion in depth (walking direction) could either be congruent or incongruent. Our main finding is that motion incongruency (across modalities) increases the activity in the ventral premotor cortex, but only if the visual point-light walker is intact. Our results extend our current knowledge by providing new evidence consistent with the idea that the premotor area assimilates information across the auditory and visual modalities by comparing the incoming sensory input with an internal representation.