Auditory-visual integration during nonconscious perception

Unconscious multisensory integration: behavioral and neural evidence from subliminal stimuli

Introduction

The prevailing theories of consciousness consider the integration of different sensory stimuli as a key component for this phenomenon to rise on the brain level. Despite many theories and models have been proposed for multisensory integration between supraliminal stimuli (e.g., the optimal integration model), we do not know if multisensory integration occurs also for subliminal stimuli and what psychophysical mechanisms it follows.

Methods

To investigate this, subjects were exposed to visual (Virtual Reality) and/or haptic stimuli (Electro-Cutaneous Stimulation) above or below their perceptual threshold. They had to discriminate, in a two-Alternative Forced Choice Task, the intensity of unimodal and/or bimodal stimuli. They were then asked to discriminate the sensory modality while recording their EEG responses.

Results

We found evidence of multisensory integration for supraliminal condition, following the classical optimal model. Importantly, even for subliminal trials participant's performances in the bimodal condition were significantly more accurate when discriminating the intensity of the stimulation. Moreover, significant differences emerged between unimodal and bimodal activity templates in parieto-temporal areas known for their integrative role.

Discussion

These converging evidences - even if preliminary and needing confirmation from the collection of further data - suggest that subliminal multimodal stimuli can be integrated, thus filling a meaningful gap in the debate about the relationship between consciousness and multisensory integration.

Influences of Music Reading on Auditory Chord Discrimination: A Novel Test Bed for Nonconscious Processing of Irrelevant Prime Meaning

The question whether nonconscious processing could involve higher-level, semantic representations is of broad interest. Here, we demonstrate semantic processing of task-relevant and task-irrelevant features of nonconscious primes within a novel, empirical test bed. In two experiments, musicians were visually primed with musical note triads varying in mode (i.e., major vs minor) and position (i.e., the arrangement of notes within a triad). The task required to discriminate only the mode in the following auditory target chord. In two experimental blocks, primes were either consciously visible or masked, respectively. Response times for auditory discrimination of the modes (relevant dimension) of heard triads were measured. Crucially, the targets also varied with respect to mode and position, creating different grades of congruency with the visual primes. Based on the Theory of Event Coding, we expected and found interactions between relevant and irrelevant semantic characteristics of masked primes, illustrating that even irrelevant prime meaning was processed. Moreover, our results indicated that both task-relevant and task-irrelevant prime characteristics are processed in nonconscious conditions only, and that practice in ignoring uninformative conscious primes can be transferred to a subsequent block. In conclusion, this study demonstrates cross-modal, automatic semantic processing using a novel approach to study such effects.

Unconscious integration: Current evidence for integrative processing under subliminal conditions

Integrative processing is traditionally believed to be dependent on consciousness. While earlier studies within the last decade reported many types of integration under subliminal conditions (i.e. without perceptual awareness), these findings are widely challenged recently. This review evaluates the current evidence for 10 types of subliminal integration that are widely studied: arithmetic processing, object-context integration, multi-word processing, same-different processing, multisensory integration and 5 different types of associative learning. Potential methodological issues concerning awareness measures are also taken into account. It is concluded that while there is currently no reliable evidence for subliminal integration, this does not necessarily refute 'unconscious' integration defined through non-subliminal (e.g. implicit) approaches.

Remembrance of things perceived: Adding thalamocortical function to artificial neural networks

Recent research has illuminated the complexity and importance of the thalamocortical system but it has been difficult to identify what computational functions it performs. Meanwhile, deep-learning artificial neural networks (ANNs) based on bio-inspired models of purely cortical circuits have achieved surprising success solving sophisticated cognitive problems associated historically with human intelligence. Nevertheless, the limitations and shortcomings of artificial intelligence (AI) based on such ANNs are becoming increasingly clear. This review considers how the addition of thalamocortical connectivity and its putative functions related to cortical attention might address some of those shortcomings. Such bio-inspired models are now providing both testable theories of biological cognition and improved AI technology, much of which is happening outside the usual academic venues.

Semantic and spatial congruency mould audiovisual integration depending on perceptual awareness

Information integration is considered a hallmark of human consciousness. Recent research has challenged this tenet by showing multisensory interactions in the absence of awareness. This psychophysics study assessed the impact of spatial and semantic correspondences on audiovisual binding in the presence and absence of visual awareness by combining forward-backward masking with spatial ventriloquism. Observers were presented with object pictures and synchronous sounds that were spatially and/or semantically congruent or incongruent. On each trial observers located the sound, identified the picture and rated the picture's visibility. We observed a robust ventriloquist effect for subjectively visible and invisible pictures indicating that pictures that evade our perceptual awareness influence where we perceive sounds. Critically, semantic congruency enhanced these visual biases on perceived sound location only when the picture entered observers' awareness. Our results demonstrate that crossmodal influences operating from vision to audition and vice versa are interactively controlled by spatial and semantic congruency in the presence of awareness. However, when visual processing is disrupted by masking procedures audiovisual interactions no longer depend on semantic correspondences.