The auditory redundant signals effect: An influence of number of stimuli or number of percepts?

Multidimensional regularity processing in music: an examination using redundant signals effect

Music is based on various regularities, ranging from the repetition of physical sounds to theoretically organized harmony and counterpoint. How are multidimensional regularities processed when we listen to music? The present study focuses on the redundant signals effect (RSE) as a novel approach to untangling the relationship between these regularities in music. The RSE refers to the occurrence of a shorter reaction time (RT) when two or three signals are presented simultaneously than when only one of these signals is presented, and provides evidence that these signals are processed concurrently. In two experiments, chords that deviated from tonal (harmonic) and acoustic (intensity and timbre) regularities were presented occasionally in the final position of short chord sequences. The participants were asked to detect all deviant chords while withholding their responses to non-deviant chords (i.e., the Go/NoGo task). RSEs were observed in all double- and triple-deviant combinations, reflecting processing of multidimensional regularities. Further analyses suggested evidence of coactivation by separate perceptual modules in the combination of tonal and acoustic deviants, but not in the combination of two acoustic deviants. These results imply that tonal and acoustic regularities are different enough to be processed as two discrete pieces of information. Examining the underlying process of RSE may elucidate the relationship between multidimensional regularity processing in music.

Vision rivals audition in alerting humans for fast action

Successful behaviour requires that humans act promptly upon the ubiquitous rapid changes in the environment. Prompt actions are supported by phasic alertness, the increased readiness for perception and action elicited by warning stimuli (alerting cues). Audition is assumed to induce phasic alertness for action faster and more strongly than other senses. Here, we show that vision can be equally effective as audition. We investigated the temporal evolution and the effectiveness of visual and auditory alerting for action in a speeded choice task, while controlling for basic sensitivity differences between the modalities that are unrelated to action control (by matching auditory and visual stimuli according to reaction times in a prior simple detection task). Results revealed that alerting sped up responses, but this happened equally fast and equally strong for visual and auditory alerting cues. Thus, these findings argue that vision rivals audition in phasic alerting for prompt actions, and suggest that the underlying mechanisms work across both modalities.

Merging the Psychophysical Function With Response Times for Auditory Detection of One vs. Two Tones

The purpose of this study is to take preliminary steps to unify psychoacoustic techniques with reaction-time methodologies to address the perceptual mechanisms responsible for the detection of one vs. multiple sounds. We measured auditory redundancy gains for auditory detection of pure tones widely spaced in frequency using the tools of Systems Factorial Technology to evince the system architecture and workload capacity in two different scenarios (SOFT and LOUD). We adopted an experimental design in which the presence or absence of a target at each of two frequencies was combined factorially with two stimulus levels. Replicating previous work, results did not allow an assessment of system architecture due to a failure to observe factor influence at the level of distribution ordering for dual-target stimuli for both SOFT and LOUD scenarios. All subjects demonstrated very modest redundancy gains for the dual-target compared to the single-target stimuli, and results were similar for both LOUD and SOFT. We propose that these results can be predicted by a mental architecture that falls into the class of integrated subadditive parallel systems, using a well-supported assumption that reaction time is driven by loudness. We demonstrate that modeled loudness of the experimental sounds (which ranged between about 0.2 and 14 sones) is highly correlated with mean reaction time (r = −0.87), and we provide a proof-of-concept model based on Steven’s Power law that predicts both a failure of distributional ordering for dual-target stimuli and very modest redundancy gains.

Effects of unintentional coordination on attentional load

The goal of this study was to evaluate the impact of unintentional (spontaneous) coordination on high attentional visual load. More precisely, we wondered whether such coordination could free up some attentional resources and help improve performance in other more demanding attentional tasks. An experiment was performed in which participant attentional allocation was challenged by performing three tasks simultaneously while simultaneously being induced to unintentional entrain to an environmental rhythm. The first task was an interception task associated with a Stroop test to increase their attentional load. The second task was a reaction time test to alarms in different modalities (auditory, visual and bimodal) which was used to assess participant attentional load. The third task was a motor task in which participants were asked to swing their legs at a preferred frequency. The interface background brightness intensity was either synchronized in real time using a bidirectional coupling to participant leg movement or the background brightness was not changing at all. Our results on the reaction time task demonstrated that participants exhibited better reaction times for alarms in the bimodal condition than in the auditory condition and lastly for the visual condition. Also, participants exhibited a lower reaction time to alarms when the background brightness was synchronizing with their leg regardless the alarm modality. Overall, our study suggests a beneficial effect of unintentional environmental coordination on attentional resource allocation and highlights the importance of bidirectionality in interaction.

The Redundant Signals Effect and the Full Body Illusion: not Multisensory, but Unisensory Tactile Stimuli Are Affected by the Illusion

During a full body illusion (FBI), participants experience a change in self-location towards a body that they see in front of them from a third-person perspective and experience touch to originate from this body. Multisensory integration is thought to underlie this illusion. In the present study we tested the redundant signals effect (RSE) as a new objective measure of the illusion that was designed to directly tap into the multisensory integration underlying the illusion. The illusion was induced by an experimenter who stroked and tapped the participant's shoulder and underarm, while participants perceived the touch on the virtual body in front of them via a head-mounted display. Participants performed a speeded detection task, responding to visual stimuli on the virtual body, to tactile stimuli on the real body and to combined (multisensory) visual and tactile stimuli. Analysis of the RSE with a race model inequality test indicated that multisensory integration took place in both the synchronous and the asynchronous condition. This surprising finding suggests that simultaneous bodily stimuli from different (visual and tactile) modalities will be transiently integrated into a multisensory representation even when no illusion is induced. Furthermore, this finding suggests that the RSE is not a suitable objective measure of body illusions. Interestingly however, responses to the unisensory tactile stimuli in the speeded detection task were found to be slower and had a larger variance in the asynchronous condition than in the synchronous condition. The implications of this finding for the literature on body representations are discussed.

Age-related changes in the functional integrity of the phasic alerting system: a pupillometric investigation

Enhanced processing following a warning cue is thought to be mediated by a phasic alerting response involving the locus coeruleus-noradrenergic (LC-NA) system. We examined the effect of aging on phasic alerting using pupil dilation as a marker of LC-NA activity in conjunction with a novel assessment of task-evoked pupil dilation. While both young and older adults displayed behavioral and pupillary alerting effects, reflected in decreased RT and increased pupillary response under high (tone) versus low (no tone) alerting conditions, older adults displayed a weaker pupillary response that benefited more from the alerting tone. The strong association between dilation and speed displayed by older adults in both alerting conditions was reduced in young adults in the high alerting condition, suggesting that in young (but not older) adults the tone conferred relatively little behavioral benefit beyond that provided by the alerting effect elicited by the target. These findings suggest a functioning but deficient LC-NA alerting system in older adults, and help reconcile previous results concerning the effects of aging on phasic alerting.

Multisensory Integration and Exogenous Spatial Attention: A Time-window-of-integration Analysis

Although it is well documented that occurrence of an irrelevant and nonpredictive sound facilitates motor responses to a subsequent target light appearing nearby, the cause of this "exogenous spatial cuing effect" has been under discussion. On the one hand, it has been postulated to be the result of a shift of visual spatial attention possibly triggered by parietal and/or cortical supramodal "attention" structures. On the other hand, the effect has been considered to be due to multisensory integration based on the activation of multisensory convergence structures in the brain. Recent RT experiments have suggested that multisensory integration and exogenous spatial cuing differ in their temporal profiles of facilitation: When the nontarget occurs 100-200 msec before the target, facilitation is likely driven by crossmodal exogenous spatial attention, whereas multisensory integration effects are still seen when target and nontarget are presented nearly simultaneously. Here, we develop an extension of the time-window-of-integration model that combines both mechanisms within the same formal framework. The model is illustrated by fitting it to data from a focused attention task with a visual target and an auditory nontarget presented at horizontally or vertically varying positions. Results show that both spatial cuing and multisensory integration may coexist in a single trial in bringing about the crossmodal facilitation of RT effects. Moreover, the formal analysis via time window of integration allows to predict and quantify the contribution of either mechanism as they occur across different spatiotemporal conditions.

Differential coactivation in a redundant signals task with weak and strong go/no-go stimuli

When participants respond to stimuli of two sources, response times (RTs) are often faster when both stimuli are presented together relative to the RTs obtained when presented separately (redundant signals effect [RSE]). Race models and coactivation models can explain the RSE. In race models, separate channels process the two stimulus components, and the faster processing time determines the overall RT. In audiovisual experiments, the RSE is often higher than predicted by race models, and coactivation models have been proposed that assume integrated processing of the two stimuli. Where does coactivation occur? We implemented a go/no-go task with randomly intermixed weak and strong auditory, visual, and audiovisual stimuli. In one experimental session, participants had to respond to strong stimuli and withhold their response to weak stimuli. In the other session, these roles were reversed. Interestingly, coactivation was only observed in the experimental session in which participants had to respond to strong stimuli. If weak stimuli served as targets, results were widely consistent with the race model prediction. The pattern of results contradicts the inverse effectiveness law. We present two models that explain the result in terms of absolute and relative thresholds.

Multisensory Decisions: the Test of a Race Model, Its Logic, and Power

The use of separate multisensory signals is often beneficial. A prominent example is the speed-up of responses to two redundant signals relative to the components, which is known as the redundant signals effect (RSE). A convenient explanation for the effect is statistical facilitation, which is inherent in the basic architecture of race models (Raab, 1962,Trans. N. Y. Acad. Sci.24, 574–590). However, this class of models has been largely rejected in multisensory research, which we think results from an ambiguity in definitions and misinterpretations of the influential race model test (Miller, 1982,Cogn. Psychol.14, 247–279). To resolve these issues, we here discuss four main items. First, we clarify definitions and ask how successful models of perceptual decision making can be extended from uni- to multisensory decisions. Second, we review the race model test and emphasize elements leading to confusion with its interpretation. Third, we introduce a new approach to study the RSE. As a major change of direction, our working hypothesis is that the basic race model architecture is correct even if the race model test seems to suggest otherwise. Based on this approach, we argue that understanding the variability of responses is the key to understand the RSE. Finally, we highlight the critical role of model testability to advance research on multisensory decisions. Despite being largely rejected, it should be recognized that race models, as part of a broader class of parallel decision models, demonstrate, in fact, a convincing explanatory power in a range of experimental paradigms. To improve research consistency in the future, we conclude with a short checklist for RSE studies.

Does semantic redundancy gain result from multiple semantic priming?

Fiedler, Schröter, and Ulrich (2013) reported faster responses to a single written word when the semantic content of this word (e.g., "elephant") matched both targets (e.g., "animal", "gray") as compared to a single target (e.g., "animal", "brown"). This semantic redundancy gain was explained by statistical facilitation due to a race of independent memory retrieval processes. The present experiment addresses one alternative explanation, namely that semantic redundancy gain results from multiple pre-activation of words that match both targets. In different blocks of trials, participants performed a redundant-targets task and a lexical decision task. The targets of the redundant-targets task served as primes in the lexical decision task. Replicating the findings of Fiedler et al., a semantic redundancy gain was observed in the redundant-targets task. Crucially, however, there was no evidence of a multiple semantic priming effect in the lexical decision task. This result suggests that semantic redundancy gain cannot be explained by multiple pre-activation of words that match both targets.

A new perspective on binaural integration using response time methodology: super capacity revealed in conditions of binaural masking release

This study applied reaction-time based methods to assess the workload capacity of binaural integration by comparing reaction time (RT) distributions for monaural and binaural tone-in-noise detection tasks. In the diotic contexts, an identical tone + noise stimulus was presented to each ear. In the dichotic contexts, an identical noise was presented to each ear, but the tone was presented to one of the ears 180° out of phase with respect to the other ear. Accuracy-based measurements have demonstrated a much lower signal detection threshold for the dichotic vs. the diotic conditions, but accuracy-based techniques do not allow for assessment of system dynamics or resource allocation across time. Further, RTs allow comparisons between these conditions at the same signal-to-noise ratio. Here, we apply a reaction-time based capacity coefficient, which provides an index of workload efficiency and quantifies the resource allocations for single ear vs. two ear presentations. We demonstrate that the release from masking generated by the addition of an identical stimulus to one ear is limited-to-unlimited capacity (efficiency typically less than 1), consistent with less gain than would be expected by probability summation. However, the dichotic presentation leads to a significant increase in workload capacity (increased efficiency)-most specifically at lower signal-to-noise ratios. These experimental results provide further evidence that configural processing plays a critical role in binaural masking release, and that these mechanisms may operate more strongly when the signal stimulus is difficult to detect, albeit still with nearly 100% accuracy.

Redundancy gain for semantic features

In a go/no-go experiment, semantic redundancy gain was assessed for responses to single written words. Specifically, we asked participants to respond only to words whose meaning matched at least one semantic target feature-that is, the target category (e.g., animal), the target color (e.g., gray), or both. On redundant-target trials, the word (e.g., elephant) matched both semantic target features (i.e., gray and animal). On single-target trials, the word (e.g., beaver) matched one target feature (i.e., animal) and a nontarget feature (i.e., brown). We observed shorter reaction times in the redundant-target condition than in the faster single-target condition. Hence, the present study provides the first evidence that redundancy gain is not limited to responses to redundant proximal stimulus features but can also be observed for responses to semantic feature information.

Showing that the race model inequality is not violated

When participants are asked to respond in the same way to stimuli from different sources (e.g., auditory and visual), responses are often observed to be substantially faster when both stimuli are presented simultaneously (redundancy gain). Different models account for this effect, the two most important being race models and coactivation models. Redundancy gains consistent with the race model have an upper limit, however, which is given by the well-known race model inequality (Miller, 1982). A number of statistical tests have been proposed for testing the race model inequality in single participants and groups of participants. All of these tests use the race model as the null hypothesis, and rejection of the null hypothesis is considered evidence in favor of coactivation. We introduce a statistical test in which the race model prediction is the alternative hypothesis. This test controls the Type I error if a theory predicts that the race model prediction holds in a given experimental condition.

Illusory double flashes can speed up responses like physical ones: evidence from the sound-induced flash illusion

When a single brief flash is accompanied by two auditory beeps, participants often report perceiving two flashes. The present experiment examined whether the perception of illusory redundant flashes can result in faster responses as compared to the perception of a single flash, because previous research has shown such a redundancy gain for physical stimuli. To this end, participants were asked to respond as rapidly as possible to the onset of any flash. Following their response, they additionally indicated whether they perceived a single flash or a double flash. Most importantly, we observed significant shorter reaction times in response to redundant flashes, irrespective of whether they were physically presented or illusorily perceived. Taken together, our results suggest that an illusory percept can affect simple reaction time in much the same manner as the corresponding physical stimulation.

The influence of dichotical fusion on the redundant signals effect, localization performance, and the mismatch negativity

In two experiments, each including a simple reaction time (RT) task, a localization task, and a passive oddball paradigm, the physical similarity between two dichotically presented auditory stimuli was manipulated. In both experiments, a redundant signals effect (RSE), high localization performance, and a reliable mismatch negativity (MMN) was observed for largely differing stimuli, suggesting that these are coded separately in auditory memory. In contrast, no RSE and a localization rate close to chance level (experiment 1) or at chance (experiment 2) were observed for stimuli differing to a lesser degree. Crucially, for such stimuli a small (experiment 1) or no (experiment 2) MMN were observed. These MMN results indicate that such stimuli tend to fuse into a single percept and that this fusion occurs rather early within information processing.

Coactive processing of dimensionally redundant targets within the auditory modality?

Previous reaction time studies have demonstrated coactivation processes within the visual modality for redundant stimuli that differ in two dimensions (e.g., shape and color). The present study provides novel results of analogous processes within the auditory modality. A redundant-target effect (RTE) was obtained in a Go/NoGo experiment using tones that differed in location and/or frequency. Participants were asked to respond to a specific tone location (e.g., left) and/or tone frequency (e.g., 200 Hz) of auditory stimuli. For redundant targets (e.g., a 200 Hz tone presented to the left), an RTE was observed which was too large to be explained by mere statistical facilitation. Therefore, responses to redundant targets were triggered by a combined activation of the target dimensions. The results are consistent with the modular hybrid account of Mordkoff and Yantis (1993).

The effect of a cross-trial shift of auditory warning signals on the sequential foreperiod effect

When a warning signal (WS) precedes an imperative signal (IS) by a certain amount of time (the foreperiod, FP), responses are speeded. Moreover, this effect is modulated by the FP length in the previous trial. This sequential FP effect has lately been attributed to a trace-conditioning mechanism according to which individuals learn (and re-learn) temporal relationships between the WS and the IS. Recent evidence suggests that sensory WS attributes are critical to trigger time-related response activation. Specifically, when WS modality is shifted in subsequent trials (e.g., from auditory to visual modality), the sequential FP effect becomes attenuated. This study examined whether the sequential FP effect is reduced only by between-modality shifts or whether this attenuation generalizes to cross-trial shifts of WS attributes within modalities. We compared dimensional (low vs. high tone frequency) and qualitative shifts (pure tone vs. noise) of equal-intense auditory WS events. The results of four experiments revealed that shifts of tone frequency did not, whereas shifts of qualitative tone characteristics did attenuate the sequential FP effect. These results support the view that the WS acts as a trigger cue that unintentionally activates responses at previously reinforced critical moments.