On the relationship between the redundant signals effect and temporal order judgments: Parametric data and a new model

Healthful choices depend on the latency and rate of information accumulation

The drift diffusion model provides a parsimonious explanation of decisions across neurobiological, psychological and behavioural levels of analysis. Although most drift diffusion model implementations assume that only a single value guides decisions, choices often involve multiple attributes that could make separable contributions to choice. Here we fit incentive-compatible dietary choices to a multi-attribute, time-dependent drift diffusion model, in which taste and health could differentially influence the evidence accumulation process. We find that these attributes shaped both the relative value signal and the latency of evidence accumulation in a manner consistent with participants' idiosyncratic preferences. Moreover, by using a dietary prime, we showed how a healthy choice intervention alters multi-attribute, time-dependent drift diffusion model parameters that in turn predict prime-dependent choices. Our results reveal that different decision attributes make separable contributions to the strength and timing of evidence accumulation, providing new insights into the construction of interventions to alter the processes of choice.

Audiovisual Integration Varies With Target and Environment Richness in Immersive Virtual Reality

We are continually bombarded by information arriving to each of our senses; however, the brain seems to effortlessly integrate this separate information into a unified percept. Although multisensory integration has been researched extensively using simple computer tasks and stimuli, much less is known about how multisensory integration functions in real-world contexts. Additionally, several recent studies have demonstrated that multisensory integration varies tremendously across naturalistic stimuli. Virtual reality can be used to study multisensory integration in realistic settings because it combines realism with precise control over the environment and stimulus presentation. In the current study, we investigated whether multisensory integration as measured by the redundant signals effects (RSE) is observable in naturalistic environments using virtual reality and whether it differs as a function of target and/or environment cue-richness. Participants detected auditory, visual, and audiovisual targets which varied in cue-richness within three distinct virtual worlds that also varied in cue-richness. We demonstrated integrative effects in each environment-by-target pairing and further showed a modest effect on multisensory integration as a function of target cue-richness but only in the cue-rich environment. Our study is the first to definitively show that minimal and more naturalistic tasks elicit comparable redundant signals effects. Our results also suggest that multisensory integration may function differently depending on the features of the environment. The results of this study have important implications in the design of virtual multisensory environments that are currently being used for training, educational, and entertainment purposes.

Recalibration of the multisensory temporal window of integration results from changing task demands

The notion of the temporal window of integration, when applied in a multisensory context, refers to the breadth of the interval across which the brain perceives two stimuli from different sensory modalities as synchronous. It maintains a unitary perception of multisensory events despite physical and biophysical timing differences between the senses. The boundaries of the window can be influenced by attention and past sensory experience. Here we examined whether task demands could also influence the multisensory temporal window of integration. We varied the stimulus onset asynchrony between simple, short-lasting auditory and visual stimuli while participants performed two tasks in separate blocks: a temporal order judgment task that required the discrimination of subtle auditory-visual asynchronies, and a reaction time task to the first incoming stimulus irrespective of its sensory modality. We defined the temporal window of integration as the range of stimulus onset asynchronies where performance was below 75% in the temporal order judgment task, as well as the range of stimulus onset asynchronies where responses showed multisensory facilitation (race model violation) in the reaction time task. In 5 of 11 participants, we observed audio-visual stimulus onset asynchronies where reaction time was significantly accelerated (indicating successful integration in this task) while performance was accurate in the temporal order judgment task (indicating successful segregation in that task). This dissociation suggests that in some participants, the boundaries of the temporal window of integration can adaptively recalibrate in order to optimize performance according to specific task demands.

Coactive processing of dimensionally redundant targets within the auditory modality?

Previous reaction time studies have demonstrated coactivation processes within the visual modality for redundant stimuli that differ in two dimensions (e.g., shape and color). The present study provides novel results of analogous processes within the auditory modality. A redundant-target effect (RTE) was obtained in a Go/NoGo experiment using tones that differed in location and/or frequency. Participants were asked to respond to a specific tone location (e.g., left) and/or tone frequency (e.g., 200 Hz) of auditory stimuli. For redundant targets (e.g., a 200 Hz tone presented to the left), an RTE was observed which was too large to be explained by mere statistical facilitation. Therefore, responses to redundant targets were triggered by a combined activation of the target dimensions. The results are consistent with the modular hybrid account of Mordkoff and Yantis (1993).

Redundancy gains in simple responses and go/no-go tasks

In divided-attention tasks with two classes of target stimuli, participants typically respond more quickly if both targets are presented simultaneously, as compared with single-target presentation (redundant-signals effect). Different explanations exist for this effect, including serial, parallel, and coactivation models of information processing. In two experiments, we investigated redundancy gains in simple and go/no-go responses to auditory-visual stimuli presented with an onset asynchrony. In Experiment 1, go/no-go discrimination was performed for near-threshold and suprathreshold stimuli. Response times in both the simple and go/no-go responses were well explained by a common coactivation model assuming linear superposition of modality-specific activation. In Experiment 2, the go/no-go task was made more difficult. Participants had to respond to high-frequency tones or right-tilted Gabor patches and to withhold their response for low tones and left-tilted Gabors. Redundancy gains were consistent with coactivation models; however, channel-specific buildup of evidence seems to occur at different speeds in the two tasks. Response times of 1 participant support a serial self-terminating model of modality-specific information processing. Supplemental materials for this article may be downloaded from http://app.psychonomic-journals.org/content/supplemental.

The influence of a sensitive period for auditory-visual integration in children with cochlear implants

PURPOSE

Children who experience long periods of auditory deprivation are susceptible to large-scale reorganization of auditory cortical areas responsible for the perception of speech and language. One consequence of this reorganization is that integration of combined auditory and visual information may be altered after hearing is restored with a cochlear implant. Our goal was to investigate the effects of reorganization in a task that examines performance during multisensory integration.

METHODS

Reaction times to the detection of basic auditory (A), visual (V), and combined auditory-visual (AV) stimuli were examined in a group of normally hearing children, and in two groups of cochlear implanted children: (1) early implanted children in whom cortical auditory evoked potentials (CAEPs) fell within normal developmental limits, and (2) late implanted children in whom CAEPs were outside of normal developmental limits. Miller's test of the race model inequality was performed for each group in order to examine the effects of auditory deprivation on multisensory integration abilities after implantation.

RESULTS

Results revealed a significant violation of the race model inequality in the normally hearing and early implanted children, but not in the group of late implanted children.

CONCLUSION

These results suggest that coactivation to multi-modal sensory input cannot explain the decreased reaction times to multi-modal input in late implanted children. These results are discussed in regards to current models for coactivation to redundant sensory information.