Auditory-visual multisensory interactions attenuate subsequent visual responses in humans

Differences in eccentricity for sound-induced flash illusion in four visual fields

A sound-induced flash illusion (SiFI) is a multisensory illusion dominated by auditory stimuli, in which the individual perceives that the number of visual flashes is equal to the number of auditory stimuli when visual flashes are presented along with an unequal number of auditory stimuli. Although the mechanisms underlying fission and fusion illusions have been documented, there is not yet a consensus on how they vary according to the different eccentricities. In the present study, by incorporating the classic SiFI paradigm into four different eccentricities, we aimed to investigate whether the SiFI varies under the different eccentricities. The results showed that the fission illusion varied significantly across the four eccentricities, with the perifovea (7°) and peripheral (11°) illusions being greater than the fovea and parafovea (3°) illusions. In contrast, the fusion illusion did not vary significantly across the four eccentricities. Our findings revealed that SiFI was affected by different visual fields and that there were differences between the fission and the fusion illusions. Furthermore, by examining the SiFI of eccentricity across visual fields, this study also suggests that bottom-up factors affect the SiFI.

Effects of audiovisual interactions on working memory: Use of the combined N-back + Go/NoGo paradigm

Background

Approximately 94% of sensory information acquired by humans originates from the visual and auditory channels. Such information can be temporarily stored and processed in working memory, but this system has limited capacity. Working memory plays an important role in higher cognitive functions and is controlled by central executive function. Therefore, elucidating the influence of the central executive function on information processing in working memory, such as in audiovisual integration, is of great scientific and practical importance.

Purpose

This study used a paradigm that combined N-back and Go/NoGo tasks, using simple Arabic numerals as stimuli, to investigate the effects of cognitive load (modulated by varying the magnitude of N) and audiovisual integration on the central executive function of working memory as well as their interaction.

Methods

Sixty college students aged 17-21 years were enrolled and performed both unimodal and bimodal tasks to evaluate the central executive function of working memory. The order of the three cognitive tasks was pseudorandomized, and a Latin square design was used to account for order effects. Finally, working memory performance, i.e., reaction time and accuracy, was compared between unimodal and bimodal tasks with repeated-measures analysis of variance (ANOVA).

Results

As cognitive load increased, the presence of auditory stimuli interfered with visual working memory by a moderate to large extent; similarly, as cognitive load increased, the presence of visual stimuli interfered with auditory working memory by a moderate to large effect size.

Conclusion

Our study supports the theory of competing resources, i.e., that visual and auditory information interfere with each other and that the magnitude of this interference is primarily related to cognitive load.

Effects of Audiovisual Interactions on Working Memory Task Performance—Interference or Facilitation

(1) Background: The combined n-back + Go/NoGo paradigm was used to investigate whether audiovisual interactions interfere with or facilitate WM. (2) Methods: College students were randomly assigned to perform the working memory task based on either a single (visual or auditory) or dual (audiovisual) stimulus. Reaction times, accuracy, and WM performance were compared across the two groups to investigate effects of audiovisual interactions. (3) Results: With low cognitive load (2-back), auditory stimuli had no effect on visual working memory, whereas visual stimuli had a small effect on auditory working memory. With high cognitive load (3-back), auditory stimuli interfered (large effect size) with visual WM, and visual stimuli interfered (medium effect size) with auditory WM. (4) Conclusions: Audiovisual effects on WM follow the resource competition theory, and the cognitive load of a visual stimulus is dominated by competition; vision always interferes with audition, and audition conditionally interferes with vision. With increased visual cognitive load, competitive effects of audiovisual interactions were more obvious than those with auditory stimuli. Compared with visual stimuli, audiovisual stimuli showed significant interference only when visual cognitive load was high. With low visual cognitive load, the two stimulus components neither facilitated nor interfered with the other in accordance with a speed–accuracy trade-off.

Reward reduces the fission illusion in the sound-induced flash illusion

Pairing a single visual stimulus with multiple auditory stimuli will lead to the illusory perception of multiple visual stimuli, which is known as sound-induced flash illusion (SIFI). The present study adopted the classic SIFI paradigm to investigate whether value-associated tasks could affect the SIFI. By adjusting the sequence of reward and nonreward conditions, we also examined the effect of reward history on SIFI. The results showed that the fission illusion was reduced when associated with momentary reward, demonstrating significantly higher accuracy and discriminability than the nonreward condition. However, the fusion illusion was not affected by the momentary reward, and the explanation was that the fusion illusion was not as stable as the fission illusion and disappeared across different trials and conditions. Moreover, the robustness of reward history in the present study was not as strong as previous studies have suggested, indicating that the effect of sound on the perceptual representation of visual stimuli is strong and robust to reward history. These findings demonstrated that the reward could reduce the SIFI and broaden the existing dichotomy of SIFI. New evidence for the operation of value-driven attention mechanisms is also provided, suggesting that the underlying value-driven attention operates across multiple sensory systems.

The Role of the Interaction between the Inferior Parietal Lobule and Superior Temporal Gyrus in the Multisensory Go/No-go Task

Information from multiple sensory modalities interacts. Using functional magnetic resonance imaging (fMRI), we aimed to identify the neural structures correlated with how cooccurring sound modulates the visual motor response execution. The reaction time (RT) to audiovisual stimuli was significantly faster than the RT to visual stimuli. Signal detection analyses showed no significant difference in the perceptual sensitivity (d') between audiovisual and visual stimuli, while the response criteria (β or c) of the audiovisual stimuli was decreased compared to the visual stimuli. The functional connectivity between the left inferior parietal lobule (IPL) and bilateral superior temporal gyrus (STG) was enhanced in Go processing compared with No-go processing of audiovisual stimuli. Furthermore, the left precentral gyrus (PreCG) showed enhanced functional connectivity with the bilateral STG and other areas of the ventral stream in Go processing compared with No-go processing of audiovisual stimuli. These results revealed that the neuronal network correlated with modulations of the motor response execution after the presentation of both visual stimuli along with cooccurring sound in a multisensory Go/Nogo task, including the left IPL, left PreCG, bilateral STG and some areas of the ventral stream. The role of the interaction between the IPL and STG in transforming audiovisual information into motor behavior is discussed. The current study provides a new perspective for exploring potential brain mechanisms underlying how humans execute appropriate behaviors on the basis of multisensory information.

Neural Correlates of Enhanced Audiovisual Processing in the Bilingual Brain

Bilingualism is associated with enhancements in perceptual and cognitive processing necessary for juggling multiple languages. Recent psychophysical studies demonstrate bilinguals also show enhanced multisensory processing and more restricted temporal binding windows for integrating audiovisual information. Here, we probed the neural mechanisms of bilinguals' audiovisual benefits. We recorded neuroelectric responses in mono- and bi-lingual listeners to the double-flash paradigm in which auditory beeps concurrent with a single visual flash induces the perceptual illusion of multiple flashes. Relative to monolinguals, bilinguals showed less susceptibility to the illusion (fewer false perceptual reports) coupled with stronger and faster event-related potentials to audiovisual information. Source analyses of EEG data revealed monolinguals' increased propensity for erroneously perceiving audiovisual stimuli was attributed to increased activity in primary visual (V1) and auditory cortex (PAC), increases in multisensory association areas (BA 37), but reduced frontal activity (BA 10). Regional activations were associated with an opposite pattern of behaviors: whereas stronger V1 and PAC activity predicted slower behavioral responses, stronger frontal BA10 responses elicited faster judgments. Our results suggest bilinguals' higher precision in audiovisual perception reflects more veridical sensory coding of physical cues coupled with superior top-down gating of sensory information to suppress the generation of false percepts. Findings underscore that the plasticity afforded by speaking multiple languages shapes extra-linguistic brain regions and can enhance audiovisual brain processing in a domain-general manner.

The origins of cortical multisensory dynamics: Evidence from human infants

Classic views of multisensory processing suggest that cortical sensory regions are specialized. More recent views argue that cortical sensory regions are inherently multisensory. To date, there are no published neuroimaging data that directly test these claims in infancy. Here we used fNIRS to show that temporal and occipital cortex are functionally coupled in 3.5-5-month-old infants (N = 65), and that the extent of this coupling during a synchronous, but not an asynchronous, audiovisual event predicted whether occipital cortex would subsequently respond to sound-only information. These data suggest that multisensory experience may shape cortical dynamics to adapt to the ubiquity of synchronous multisensory information in the environment, and invoke the possibility that adaptation to the environment can also reflect broadening of the computational range of sensory systems.

Auditory-visual integration modulates location-specific repetition suppression of auditory responses

Space is a dimension shared by different modalities, but at what stage spatial encoding is affected by multisensory processes is unclear. Early studies observed attenuation of N1/P2 auditory evoked responses following repetition of sounds from the same location. Here, we asked whether this effect is modulated by audiovisual interactions. In two experiments, using a repetition-suppression paradigm, we presented pairs of tones in free field, where the test stimulus was a tone presented at a fixed lateral location. Experiment 1 established a neural index of auditory spatial sensitivity, by comparing the degree of attenuation of the response to test stimuli when they were preceded by an adapter sound at the same location versus 30° or 60° away. We found that the degree of attenuation at the P2 latency was inversely related to the spatial distance between the test stimulus and the adapter stimulus. In Experiment 2, the adapter stimulus was a tone presented from the same location or a more medial location than the test stimulus. The adapter stimulus was accompanied by a simultaneous flash displayed orthogonally from one of the two locations. Sound-flash incongruence reduced accuracy in a same-different location discrimination task (i.e., the ventriloquism effect) and reduced the location-specific repetition-suppression at the P2 latency. Importantly, this multisensory effect included topographic modulations, indicative of changes in the relative contribution of underlying sources across conditions. Our findings suggest that the auditory response at the P2 latency is affected by spatially selective brain activity, which is affected crossmodally by visual information.

The context-contingent nature of cross-modal activations of the visual cortex

Real-world environments are nearly always multisensory in nature. Processing in such situations confers perceptual advantages, but its automaticity remains poorly understood. Automaticity has been invoked to explain the activation of visual cortices by laterally-presented sounds. This has been observed even when the sounds were task-irrelevant and spatially uninformative about subsequent targets. An auditory-evoked contralateral occipital positivity (ACOP) at ~250ms post-sound onset has been postulated as the event-related potential (ERP) correlate of this cross-modal effect. However, the spatial dimension of the stimuli was nevertheless relevant in virtually all prior studies where the ACOP was observed. By manipulating the implicit predictability of the location of lateralised sounds in a passive auditory paradigm, we tested the automaticity of cross-modal activations of visual cortices. 128-channel ERP data from healthy participants were analysed within an electrical neuroimaging framework. The timing, topography, and localisation resembled previous characterisations of the ACOP. However, the cross-modal activations of visual cortices by sounds were critically dependent on whether the sound location was (un)predictable. Our results are the first direct evidence that this particular cross-modal process is not (fully) automatic; instead, it is context-contingent. More generally, the present findings provide novel insights into the importance of context-related factors in controlling information processing across the senses, and call for a revision of current models of automaticity in cognitive sciences.

The multisensory function of the human primary visual cortex

It has been nearly 10 years since Ghazanfar and Schroeder (2006) proposed that the neocortex is essentially multisensory in nature. However, it is only recently that sufficient and hard evidence that supports this proposal has accrued. We review evidence that activity within the human primary visual cortex plays an active role in multisensory processes and directly impacts behavioural outcome. This evidence emerges from a full pallet of human brain imaging and brain mapping methods with which multisensory processes are quantitatively assessed by taking advantage of particular strengths of each technique as well as advances in signal analyses. Several general conclusions about multisensory processes in primary visual cortex of humans are supported relatively solidly. First, haemodynamic methods (fMRI/PET) show that there is both convergence and integration occurring within primary visual cortex. Second, primary visual cortex is involved in multisensory processes during early post-stimulus stages (as revealed by EEG/ERP/ERFs as well as TMS). Third, multisensory effects in primary visual cortex directly impact behaviour and perception, as revealed by correlational (EEG/ERPs/ERFs) as well as more causal measures (TMS/tACS). While the provocative claim of Ghazanfar and Schroeder (2006) that the whole of neocortex is multisensory in function has yet to be demonstrated, this can now be considered established in the case of the human primary visual cortex.

Multisensory integration: flexible use of general operations

Research into the anatomical substrates and "principles" for integrating inputs from separate sensory surfaces has yielded divergent findings. This suggests that multisensory integration is flexible and context dependent and underlines the need for dynamically adaptive neuronal integration mechanisms. We propose that flexible multisensory integration can be explained by a combination of canonical, population-level integrative operations, such as oscillatory phase resetting and divisive normalization. These canonical operations subsume multisensory integration into a fundamental set of principles as to how the brain integrates all sorts of information, and they are being used proactively and adaptively. We illustrate this proposition by unifying recent findings from different research themes such as timing, behavioral goal, and experience-related differences in integration.

Neural responses in parietal and occipital areas in response to visual events are modulated by prior multisensory stimuli

The effect of multi-modal vs uni-modal prior stimuli on the subsequent processing of a simple flash stimulus was studied in the context of the audio-visual 'flash-beep' illusion, in which the number of flashes a person sees is influenced by accompanying beep stimuli. EEG recordings were made while combinations of simple visual and audio-visual stimuli were presented. The experiments found that the electric field strength related to a flash stimulus was stronger when it was preceded by a multi-modal flash/beep stimulus, compared to when it was preceded by another uni-modal flash stimulus. This difference was found to be significant in two distinct timeframes--an early timeframe, from 130-160 ms, and a late timeframe, from 300-320 ms. Source localisation analysis found that the increased activity in the early interval was localised to an area centred on the inferior and superior parietal lobes, whereas the later increase was associated with stronger activity in an area centred on primary and secondary visual cortex, in the occipital lobe. The results suggest that processing of a visual stimulus can be affected by the presence of an immediately prior multisensory event. Relatively long-lasting interactions generated by the initial auditory and visual stimuli altered the processing of a subsequent visual stimulus.

Why do we need to use a zero reference? Reference influences on the ERPs of audiovisual effects

Using ERPs in the audiovisual stimulus, the current study is the first to investigate the influence of the reference on experimental effects (between two conditions). Three references, the average reference (AR), the mean mastoid (MM), and a new infinity zero reference (IR), were comparatively investigated via ERPs, statistical parametric scalp mappings (SPSM), and LORETA. Specifically, for the N1 (170-190 ms), the SPSM results showed an anterior distribution for MM, a posterior distribution for IR, and both anterior and posterior distributions for AR. However, the circumstantial evidence provided by LORETA is consistent with SPSM of IR. These results indicated that the newly developed IR could provide increased accuracy; thus, we recommend IR for future ERP studies.

The Efficacy of Single-Trial Multisensory Memories

This review article summarizes evidence that multisensory experiences at one point in time have long-lasting effects on subsequent unisensory visual and auditory object recognition. The efficacy of single-trial exposure to task-irrelevant multisensory events is its ability to modulate memory performance and brain activity to unisensory components of these events presented later in time. Object recognition (either visual or auditory) is enhanced if the initial multisensory experience had been semantically congruent and can be impaired if this multisensory pairing was either semantically incongruent or entailed meaningless information in the task-irrelevant modality, when compared to objects encountered exclusively in a unisensory context. Processes active during encoding cannot straightforwardly explain these effects; performance on all initial presentations was indistinguishable despite leading to opposing effects with stimulus repetitions. Brain responses to unisensory stimulus repetitions differ during early processing stages (∼100 ms post-stimulus onset) according to whether or not they had been initially paired in a multisensory context. Plus, the network exhibiting differential responses varies according to whether or not memory performance is enhanced or impaired. The collective findings we review indicate that multisensory associations formedviasingle-trial learning exert influences on later unisensory processing to promote distinct object representations that manifest as differentiable brain networks whose activity is correlated with memory performance. These influences occur incidentally, despite many intervening stimuli, and are distinguishable from the encoding/learning processes during the formation of the multisensory associations. The consequences of multisensory interactions thus persist over time to impact memory retrieval and object discrimination.

Visual anticipatory information modulates multisensory interactions of artificial audiovisual stimuli

The neural activity of speech sound processing (the N1 component of the auditory ERP) can be suppressed if a speech sound is accompanied by concordant lip movements. Here we demonstrate that this audiovisual interaction is neither speech specific nor linked to humanlike actions but can be observed with artificial stimuli if their timing is made predictable. In Experiment 1, a pure tone synchronized with a deformation of a rectangle induced a smaller auditory N1 than auditory-only presentations if the temporal occurrence of this audiovisual event was made predictable by two moving disks that touched the rectangle. Local autoregressive average source estimation indicated that this audiovisual interaction may be related to integrative processing in auditory areas. When the moving disks did not precede the audiovisual stimulus--making the onset unpredictable--there was no N1 reduction. In Experiment 2, the predictability of the leading visual signal was manipulated by introducing a temporal asynchrony between the audiovisual event and the collision of moving disks. Audiovisual events occurred either at the moment, before (too "early"), or after (too "late") the disks collided on the rectangle. When asynchronies varied from trial to trial--rendering the moving disks unreliable temporal predictors of the audiovisual event--the N1 reduction was abolished. These results demonstrate that the N1 suppression is induced by visual information that both precedes and reliably predicts audiovisual onset, without a necessary link to human action-related neural mechanisms.

Crossmodal recruitment of primary visual cortex following brief exposure to bimodal audiovisual stimuli

Several lines of evidence suggest that exposure to only one component of typically audiovisual events can lead to crossmodal cortical activation. These effects are likely explained by long-term associations formed between the auditory and visual components of such events. It is not certain whether such crossmodal recruitment can occur in the absence of explicit conditioning, semantic factors, or long-term association; nor is it clear whether primary sensory cortices can be recruited in such paradigms. In the present study we tested the hypothesis that crossmodal cortical recruitment would occur even after a brief exposure to bimodal stimuli without semantic association. We used positron emission tomography, and an apparatus allowing presentation of spatially and temporally congruous audiovisual stimuli (noise bursts and light flashes). When presented with only the auditory or visual components of the bimodal stimuli, naïve subjects showed only modality-specific cortical activation, as expected. However, subjects who had previously been exposed to the audiovisual stimuli showed increased cerebral blood flow in the primary visual cortex when presented with sounds alone. Functional connectivity analysis suggested that the auditory cortex was the source of visual cortex activity. This crossmodal activation appears to be the result of implicit associations of the two stimuli, likely driven by their spatiotemporal characteristics; it was observed after a relatively short period of exposure (approximately 45 min), and lasted for a relatively long period after the initial exposure (approximately 1 day). The findings indicate that auditory and visual cortices interact with one another to a larger degree than typically assumed.

The impact of spatial incongruence on an auditory-visual illusion

Background

The sound-induced flash illusion is an auditory-visual illusion – when a single flash is presented along with two or more beeps, observers report seeing two or more flashes. Previous research has shown that the illusion gradually disappears as the temporal delay between auditory and visual stimuli increases, suggesting that the illusion is consistent with existing temporal rules of neural activation in the superior colliculus to multisensory stimuli. However little is known about the effect of spatial incongruence, and whether the illusion follows the corresponding spatial rule. If the illusion occurs less strongly when auditory and visual stimuli are separated, then integrative processes supporting the illusion must be strongly dependant on spatial congruence. In this case, the illusion would be consistent with both the spatial and temporal rules describing response properties of multisensory neurons in the superior colliculus.

Methodology/Principal Findings

The main aim of this study was to investigate the importance of spatial congruence in the flash-beep illusion. Selected combinations of one to four short flashes and zero to four short 3.5 KHz tones were presented. Observers were asked to count the number of flashes they saw. After replication of the basic illusion using centrally-presented stimuli, the auditory and visual components of the illusion stimuli were presented either both 10 degrees to the left or right of fixation (spatially congruent) or on opposite (spatially incongruent) sides, for a total separation of 20 degrees.

Conclusions/Significance

The sound-induced flash fission illusion was successfully replicated. However, when the sources of the auditory and visual stimuli were spatially separated, perception of the illusion was unaffected, suggesting that the “spatial rule” does not extend to describing behavioural responses in this illusion. We also find no evidence for an associated “fusion” illusion reportedly occurring when multiple flashes are accompanied by a single beep.

Cortical processes underlying sound-induced flash fusion

When two brief flashes presented in rapid succession (<100 ms apart) are paired with a single auditory stimulus, subjects often report perceiving only a single flash [Andersen, T.S., Tiippana, K., Sams, M., 2004. Factors influencing audiovisual fission and fusion illusions. Brain Res. Cogn. Brain Res. 21, 301-308; Shams, L., Iwaki, S., Chawla, A., Bhattacharya, J., 2005a. Early modulation of visual cortex by sound: an MEG study. Neurosci. Lett. 378, 76-81, Shams, L., Ma, W.J., Beierholm, U., 2005b. Sound-induced flash illusion as an optimal percept. Neuroreport 16, 1923-1927]. We used event-related potentials (ERPs) to investigate the timing and localization of the cortical processes that underlie this sound induced flash fusion, which is complementary to the sound-induced extra flash illusion that we analyzed previously [Mishra, J., Martinez, A., Sejnowski, T.J. and Hillyard, S.A., Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion. J. Neurosci. 27 (2007) 4120-4131]. The difference ERP that represented the cross-modal interaction between the visual (two flashes) and auditory (one sound) constituents of the bimodal stimulus revealed a positive component elicited 160-190 ms after stimulus onset, which was markedly attenuated in subjects who did not perceive the second flash. This component, previously designated as PD180 [Mishra, J., Martinez, A., Sejnowski, T.J. and Hillyard, S.A., Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion. J. Neurosci. 27 (2007) 4120-4131], was localized by dipole modeling to polysensory superior temporal cortex. PD180 was found to covary in amplitude across subjects with the visual evoked N1 component (148-184 ms), suggesting that inter-individual differences in perceiving the illusion are based at least in part on differences in visual processing. A trial-by-trial analysis found that the PD180 as well as a subsequent modulation in visual cortex at 228-248 ms was diminished on trials when the two flashes were perceived as one relative to trials when two flashes were correctly reported. These results suggest that the sound induced flash fusion is based on an interaction between polysensory and visual cortical areas.

Occipital transcranial magnetic stimulation has opposing effects on visual and auditory stimulus detection: implications for multisensory interactions

Multisensory interactions occur early in time and in low-level cortical areas, including primary cortices. To test current models of early auditory-visual (AV) convergence in unisensory visual brain areas, we studied the effect of transcranial magnetic stimulation (TMS) of visual cortex on behavioral responses to unisensory (auditory or visual) or multisensory (simultaneous auditory-visual) stimulus presentation. Single-pulse TMS was applied over the occipital pole at short delays (30-150 ms) after external stimulus onset. Relative to TMS over a control site, reactions times (RTs) to unisensory visual stimuli were prolonged by TMS at 60-75 ms poststimulus onset (visual suppression effect), confirming stimulation of functional visual cortex. Conversely, RTs to unisensory auditory stimuli were significantly shortened when visual cortex was stimulated by TMS at the same delays (beneficial interaction effect of auditory stimulation and occipital TMS). No TMS-effect on RTs was observed for AV stimulation. The beneficial interaction effect of combined unisensory auditory and TMS-induced visual cortex stimulation matched and was correlated with the RT-facilitation after external multisensory AV stimulation without TMS, suggestive of multisensory interactions between the stimulus-evoked auditory and TMS-induced visual cortex activities. A follow-up experiment showed that auditory input enhances excitability within visual cortex itself (using phosphene-induction via TMS as a measure) over a similarly early time-window (75-120 ms). The collective data support a mechanism of early auditory-visual interactions that is mediated by auditory-driven sensitivity changes in visual neurons that coincide in time with the initial volleys of visual input.

Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion

When a single flash of light is presented interposed between two brief auditory stimuli separated by 60-100 ms, subjects typically report perceiving two flashes (Shams et al., 2000, 2002). We investigated the timing and localization of the cortical processes that underlie this illusory flash effect in 34 subjects by means of 64-channel recordings of event-related potentials (ERPs). A difference ERP calculated to isolate neural activity associated with the illusory second flash revealed an early modulation of visual cortex activity at 30-60 ms after the second sound, which was larger in amplitude in subjects who saw the illusory flash more frequently. These subjects also showed this early modulation in response to other combinations of auditory and visual stimuli, thus pointing to consistent individual differences in the neural connectivity that underlies cross-modal integration. The overall pattern of cortical activity associated with the cross-modally induced illusory flash, however, differed markedly from that evoked by a real second flash. A trial-by-trial analysis showed that short-latency ERP activity localized to auditory cortex and polymodal cortex of the temporal lobe, concurrent with gamma bursts in visual cortex, were associated with perception of the double-flash illusion. These results provide evidence that perception of the illusory second flash is based on a very rapid dynamic interplay between auditory and visual cortical areas that is triggered by the second sound.