Components of visual prior entry

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments

This protocol describes how to conduct temporal-order experiments to measure visual processing speed and the attentional resource distribution. The proposed method is based on a new and synergistic combination of three components: the temporal-order judgments (TOJ) paradigm, Bundesen's Theory of Visual Attention (TVA), and a hierarchical Bayesian estimation framework. The method provides readily interpretable parameters, which are supported by the theoretical and neurophysiological underpinnings of TVA. Using TOJs, TVA-based estimates can be obtained for a broad range of stimuli, whereas traditional paradigms used with TVA are mainly limited to letters and digits. Finally, the meaningful parameters of the proposed model allow for the establishment of a hierarchical Bayesian model. Such a statistical model allows assessing results in one coherent analysis both on the subject and the group level. To demonstrate the feasibility and versatility of this new approach, three experiments are reported with attention manipulations in synthetic pop-out displays, natural images, and a cued letter-report paradigm.

Audiovisual Temporal Processing and Synchrony Perception in the Rat

Extensive research on humans has improved our understanding of how the brain integrates information from our different senses, and has begun to uncover the brain regions and large-scale neural activity that contributes to an observer’s ability to perceive the relative timing of auditory and visual stimuli. In the present study, we developed the first behavioral tasks to assess the perception of audiovisual temporal synchrony in rats. Modeled after the parameters used in human studies, separate groups of rats were trained to perform: (1) a simultaneity judgment task in which they reported whether audiovisual stimuli at various stimulus onset asynchronies (SOAs) were presented simultaneously or not; and (2) a temporal order judgment task in which they reported whether they perceived the auditory or visual stimulus to have been presented first. Furthermore, using in vivo electrophysiological recordings in the lateral extrastriate visual (V2L) cortex of anesthetized rats, we performed the first investigation of how neurons in the rat multisensory cortex integrate audiovisual stimuli presented at different SOAs. As predicted, rats (n = 7) trained to perform the simultaneity judgment task could accurately (~80%) identify synchronous vs. asynchronous (200 ms SOA) trials. Moreover, the rats judged trials at 10 ms SOA to be synchronous, whereas the majority (~70%) of trials at 100 ms SOA were perceived to be asynchronous. During the temporal order judgment task, rats (n = 7) perceived the synchronous audiovisual stimuli to be “visual first” for ~52% of the trials, and calculation of the smallest timing interval between the auditory and visual stimuli that could be detected in each rat (i.e., the just noticeable difference (JND)) ranged from 77 ms to 122 ms. Neurons in the rat V2L cortex were sensitive to the timing of audiovisual stimuli, such that spiking activity was greatest during trials when the visual stimulus preceded the auditory by 20–40 ms. Ultimately, given that our behavioral and electrophysiological results were consistent with studies conducted on human participants and previous recordings made in multisensory brain regions of different species, we suggest that the rat represents an effective model for studying audiovisual temporal synchrony at both the neuronal and perceptual level.

Peripheral Visual Cues: Their Fate in Processing and Effects on Attention and Temporal-Order Perception

Peripheral visual cues lead to large shifts in psychometric distributions of temporal-order judgments. In one view, such shifts are attributed to attention speeding up processing of the cued stimulus, so-called prior entry. However, sometimes these shifts are so large that it is unlikely that they are caused by attention alone. Here we tested the prevalent alternative explanation that the cue is sometimes confused with the target on a perceptual level, bolstering the shift of the psychometric function. We applied a novel model of cued temporal-order judgments, derived from Bundesen's Theory of Visual Attention. We found that cue-target confusions indeed contribute to shifting psychometric functions. However, cue-induced changes in the processing rates of the target stimuli play an important role, too. At smaller cueing intervals, the cue increased the processing speed of the target. At larger intervals, inhibition of return was predominant. Earlier studies of cued TOJs were insensitive to these effects because in psychometric distributions they are concealed by the conjoint effects of cue-target confusions and processing rate changes.

No Evidence for Threat-Induced Spatial Prioritization of Somatosensory Stimulation during Pain Control Using a Synchrony Judgment Paradigm

Topical research efforts on attention to pain often take a critical look at the modulatory role of top-down factors. For instance, it has been shown that the fearful expectation of pain at a location of the body directs attention towards that body part. In addition, motivated attempts to control this pain were found to modulate this prioritization effect. Such studies have often used a temporal order judgment task, requiring participants to judge the order in which two stimuli are presented by indicating which one they perceived first. As this constitutes a forced-choice response format, such studies may be subject to response bias. The aim of the current study was to address this concern. We used a ternary synchrony judgment paradigm, in which participants judged the order in which two somatosensory stimuli occurred. Critically, participants now also had the option to give a 'simultaneous' response when they did not perceive a difference. This way we eliminated the need for guessing, and thus reduced the risk of response bias. One location was threatened with the possibility of pain in half of the trials, as predicted by an auditory cue. Additionally, half of the participants (pain control group) were encouraged to avoid pain stimuli by executing a quick button press. The other half (comparison group) performed a similar action, albeit unrelated to the occurrence of pain. Our data did not support threat-induced spatial prioritization, nor did we find evidence that pain control attempts influenced attention in any way.

Dissociating the neural correlates of tactile temporal order and simultaneity judgements

Perceiving temporal relationships between sensory events is a key process for recognising dynamic environments. Temporal order judgement (TOJ) and simultaneity judgement (SJ) are used for probing this perceptual process. TOJ and SJ exhibit identical psychometric parameters. However, there is accumulating psychophysical evidence that distinguishes TOJ from SJ. Some studies have proposed that the perceptual processes for SJ (e.g., detecting successive/simultaneity) are also included in TOJ, whereas TOJ requires more processes (e.g., determination of the temporal order). Other studies have proposed two independent processes for TOJ and SJ. To identify differences in the neural activity associated with TOJ versus SJ, we performed functional magnetic resonance imaging of participants during TOJ and SJ with identical tactile stimuli. TOJ-specific activity was observed in multiple regions (e.g., left ventral and bilateral dorsal premotor cortices and left posterior parietal cortex) that overlap the general temporal prediction network for perception and motor systems. SJ-specific activation was observed only in the posterior insular cortex. Our results suggest that TOJ requires more processes than SJ and that both TOJ and SJ implement specific process components. The neural differences between TOJ and SJ thus combine features described in previous psychophysical hypotheses that proposed different mechanisms.

A Roving Dual-Presentation Simultaneity-Judgment Task to Estimate the Point of Subjective Simultaneity

The most popular tasks with which to investigate the perception of subjective synchrony are the temporal order judgment (TOJ) and the simultaneity judgment (SJ). Here, we discuss a complementary approach—a dual-presentation (2x) SJ task—and focus on appropriate analysis methods for a theoretically desirable “roving” design. Two stimulus pairs are presented on each trial and the observer must select the most synchronous. To demonstrate this approach, in Experiment 1 we tested the 2xSJ task alongside TOJ, SJ, and simple reaction-time (RT) tasks using audiovisual stimuli. We interpret responses from each task using detection-theoretic models, which assume variable arrival times for sensory signals at critical brain structures for timing perception. All tasks provide similar estimates of the point of subjective simultaneity (PSS) on average, and PSS estimates from some tasks were correlated on an individual basis. The 2xSJ task produced lower and more stable estimates of model-based (and thus comparable) sensory/decision noise than the TOJ. In Experiment 2 we obtained similar results using RT, TOJ, ternary, and 2xSJ tasks for all combinations of auditory, visual, and tactile stimuli. In Experiment 3 we investigated attentional prior entry, using both TOJs and 2xSJs. We found that estimates of prior-entry magnitude correlated across these tasks. Overall, our study establishes the practicality of the roving dual-presentation SJ task, but also illustrates the additional complexity of the procedure. We consider ways in which this task might complement more traditional procedures, particularly when it is important to estimate both PSS and sensory/decisional noise.

Awareness of Temporal Lag is Necessary for Motor-Visual Temporal Recalibration

Consistent exposure to a temporal lag between observers' voluntary action and its visual feedback induced recalibration of temporal order perception between a motor action and a visual stimulus. It remains unclear what kinds of processing underlie this motor–visual temporal recalibration. This study examined the necessity of awareness of a temporal lag between a motor action and its visual feedback for motor–visual temporal recalibration. In Experiment 1, we allocated observers to either the multiple-step or single-step lag conditions. In the multiple-step lag condition, we first inserted a small temporal lag and subsequently increased it with progress of the adaptation period, to make observers unaware of the temporal lag during the adaptation period. In the single-step lag condition, we instructed observers about the temporal lag before adaptation, and inserted a substantial temporal lag from the beginning of the adaptation period to ensure that they were aware of the temporal lag. We found significant recalibration only in the single-step lag condition. In Experiment 2, we exposed all observers to a substantial temporal lag from the beginning of adaptation period with no instruction about insertion of the temporal lag. We asked observers at the end of the experiment whether they were aware of the temporal lag. We found significant recalibration for only observers who were aware of the lag. These results suggest that awareness of the temporal lag plays a crucial role in motor–visual temporal recalibration.

Involuntary attentional orienting in the absence of awareness speeds up early sensory processing

A long-standing controversy in the field of human neuroscience has revolved around the question whether attended stimuli are processed more rapidly compared to unattended stimuli. We conducted two event-related potential (ERP) experiments employing a temporal order judgment procedure in order to assess whether involuntary attention accelerates sensory processing, as indicated by latency modulations of early visual ERP components. A non-reportable exogenous cue could precede the first target with equal probability at the same (compatible) or opposite (incompatible) location. The use of non-reportable cues promoted automatic, bottom-up attentional capture, and ensured the elimination of any confounds related to the use of stimulus features that are common to both cue and target. Behavioral results confirmed involuntary exogenous orienting towards the unaware cue. ERP results showed that the N1pc, an electrophysiological measure of attentional orienting, was smaller and peaked earlier in compatible as opposed to incompatible trials, indicating cue-dependent changes in magnitude and speed of first target processing in extrastriate visual areas. Complementary Bayesian analysis confirmed the presence of this effect regardless of whether participants were actively looking for the cue (Experiment 1) or were not informed of it (Experiment 2), indicating purely automatic, stimulus-driven orienting mechanisms.

The simultaneous perception of auditory-tactile stimuli in voluntary movement

The simultaneous perception of multimodal information in the environment during voluntary movement is very important for effective reactions to the environment. Previous studies have found that voluntary movement affects the simultaneous perception of auditory and tactile stimuli. However, the results of these experiments are not completely consistent, and the differences may be attributable to methodological differences in the previous studies. In this study, we investigated the effect of voluntary movement on the simultaneous perception of auditory and tactile stimuli using a temporal order judgment task with voluntary movement, involuntary movement, and no movement. To eliminate the potential effect of stimulus predictability and the effect of spatial information associated with large-scale movement in the previous studies, we randomized the interval between the start of movement and the first stimulus, and used small-scale movement. As a result, the point of subjective simultaneity (PSS) during voluntary movement shifted from the tactile stimulus being first during involuntary movement or no movement to the auditory stimulus being first. The just noticeable difference (JND), an indicator of temporal resolution, did not differ across the three conditions. These results indicate that voluntary movement itself affects the PSS in auditory–tactile simultaneous perception, but it does not influence the JND. In the discussion of these results, we suggest that simultaneous perception may be affected by the efference copy.

Tracking Visual Events in Time in the Absence of Time Perception: Implicit Processing at the ms Level

Previous studies have suggested that even if subjects deem two visual stimuli less than 20 ms apart to be simultaneous, implicitly they are nonetheless distinguished in time. It is unclear, however, how information is encoded within this short timescale. We used a priming paradigm to demonstrate how successive visual stimuli are processed over time intervals of less than 20 ms. The primers were two empty square frames displayed either simultaneously or with a 17ms asynchrony. The primers were followed by the target information after a delay of 25 ms to 100 ms. The two square frames were filled in one after another with a delay of 100 ms between them, and subjects had to decide on the location of the first of the frames to be filled in. In a second version of the paradigm, only one square frame was filled in, and subjects had to decide where it was positioned. The influence of the primers is revealed through faster response times depending on the location of the first and second primers. Experiment 1 replicates earlier results, with a bias towards the side of the second primer, but only when there is a delay of 75 to 100 ms between primers and targets. The following experiments suggest this effect to be relatively independent of the task context, except for a slight effect on the time course of the biases. For the temporal order judgment task, identical results were observed when subjects have to answer to the side of the second rather than the first target, showing the effect to be independent of the hand response, and suggesting it might be related to a displacement of attention. All in all the results suggest the flow of events is followed more efficiently than suggested by explicit asynchrony judgment studies. We discuss the possible impact of these results on our understanding of the sense of time continuity.

Differences in perceptual latency estimated from judgments of temporal order, simultaneity and duration are inconsistent

Differences in perceptual latency (ΔL) for two stimuli, such as an auditory and a visual stimulus, can be estimated from temporal order judgments (TOJ) and simultaneity judgments (SJ), but previous research has found evidence that ΔL estimated from these tasks do not coincide. Here, using an auditory and a visual stimulus we confirmed this and further show that ΔL as estimated from duration judgments also does not coincide with ΔL estimated from TOJ or SJ. These inconsistencies suggest that each judgment is subject to different processes that bias ΔL in different ways: TOJ might be affected by sensory interactions, a bias associated with the method of single stimuli and an order difficulty bias; SJ by sensory interactions and an asymmetrical criterion bias; duration judgments by an order duration bias.

The effect of visual apparent motion on audiovisual simultaneity

Visual motion information from dynamic environments is important in multisensory temporal perception. However, it is unclear how visual motion information influences the integration of multisensory temporal perceptions. We investigated whether visual apparent motion affects audiovisual temporal perception. Visual apparent motion is a phenomenon in which two flashes presented in sequence in different positions are perceived as continuous motion. Across three experiments, participants performed temporal order judgment (TOJ) tasks. Experiment 1 was a TOJ task conducted in order to assess audiovisual simultaneity during perception of apparent motion. The results showed that the point of subjective simultaneity (PSS) was shifted toward a sound-lead stimulus, and the just noticeable difference (JND) was reduced compared with a normal TOJ task with a single flash. This indicates that visual apparent motion affects audiovisual simultaneity and improves temporal discrimination in audiovisual processing. Experiment 2 was a TOJ task conducted in order to remove the influence of the amount of flash stimulation from Experiment 1. The PSS and JND during perception of apparent motion were almost identical to those in Experiment 1, but differed from those for successive perception when long temporal intervals were included between two flashes without motion. This showed that the result obtained under the apparent motion condition was unaffected by the amount of flash stimulation. Because apparent motion was produced by a constant interval between two flashes, the results may be accounted for by specific prediction. In Experiment 3, we eliminated the influence of prediction by randomizing the intervals between the two flashes. However, the PSS and JND did not differ from those in Experiment 1. It became clear that the results obtained for the perception of visual apparent motion were not attributable to prediction. Our findings suggest that visual apparent motion changes temporal simultaneity perception and improves temporal discrimination in audiovisual processing.

Bias-free double judgment accuracy during spatial attention cueing: performance enhancement from voluntary and involuntary attention

Recent research has demonstrated that involuntary attention improves target identification accuracy for letters using non-predictive peripheral cues, helping to resolve some of the controversy over performance enhancement from involuntary attention. While various cueing studies have demonstrated that their reported cueing effects were not due to response bias to the cue, very few investigations have quantified the extent of any response bias or developed methods of removing bias from observed results in a double judgment accuracy task. We have devised a method to quantify and remove response bias to cued locations in a double judgment accuracy cueing task, revealing the true, unbiased performance enhancement from involuntary and voluntary attention. In a 7-alternative forced choice cueing task using backward masked stimuli to temporally constrain stimulus processing, non-predictive cueing increased target detection and discrimination at cued locations relative to uncued locations even after cue location bias had been corrected.

The bisection point across variants of the task

Bisection tasks are used in research on normal space and time perception and to assess the perceptual distortions accompanying neurological disorders. Several variants of the bisection task are used, which often yield inconsistent results, prompting the question of which variant is most dependable and which results are to be trusted. We addressed this question using theoretical and experimental approaches. Theoretical performance in bisection tasks is derived from a general model of psychophysical performance that includes sensory components and decisional processes. The model predicts how performance should differ across variants of the task, even when the sensory component is fixed. To test these predictions, data were collected in a within-subjects study with several variants of a spatial bisection task, including a two-response variant in which observers indicated whether a line was transected to the right or left of the midpoint, a three-response variant (which included the additional option to respond "midpoint"), and a paired-comparison variant of the three-response format. The data supported the model predictions, revealing that estimated bisection points were least dependable with the two-response variant, because this format confounds perceptual and decisional influences. Only the three-response paired-comparison format can separate out these influences. Implications for research in basic and clinical fields are discussed.

Spatial distribution of attentional bias in visuo-spatial working memory following multiple cues

When attention is focused on one location, its spatial distribution depends on many factors, such as the distance between the attended location and the target location, the presence of visual meridians in between them, and the way, endogenous or exogenous, by which attention is oriented. However, it is not well known how attention distributes when more than one location is endogenously or exogenously cued, which was the focus of the current study. Furthermore, the distribution of attention has been manly investigated in perception. In the present study we faced this issue from a different perspective, by examining the spatial distribution of the attentional bias in visuo-spatial working memory (VSWM), when attention is oriented either exogenously or endogenously, i.e., after two peripheral vs. central symbolic cues (also manipulating cue-target predictability). Results indicated a systematic difference between endogenous and exogenous attention regarding the distribution of the attentional bias over VSWM. In fact, attentional bias following endogenous cues was affected by the presence of visual meridians and by the split of the attentional focus, converging in a unipolar attentional distribution, independently of cue-target predictability. On the other hand, when pulled by exogenous cues, attention distributed uni-modally or multi-modally depending on the distance between the cued locations, with larger effects for highly predictive cues. Results are discussed in terms of space-based, object-based and perceptual grouping mechanisms.

Illusory line motion is not caused by object-differentiating mechanisms or endogenous attention

Christie and Klein [2005. Does attention cause illusory line motion? Perception & Psychophysics, 67(6), 1032-1043] published line motion ratings consistent with illusory line motion (ILM) after peripheral endogenous cues but not central arrow cues. When attention was directed endogenously on the basis of the shape of one of two peripherally presented objects, participants reported small, but significant motion away from the attended object, and this was attributed to participant bias, or to a peripherally directed object-based attention system endogenously recruited to differentiate the peripheral shapes. By using a unique cueing method with identical peripheral markers, but still allowing them to act as cues, the findings of Christie and Klein Experiment 4 were replicated. This reduces the likelihood that object discrimination or object attention mechanisms are responsible for the reported ILM-like effects.

Relative timing: from behaviour to neurons

Processing of temporal information is critical to behaviour. Here, we review the phenomenology and mechanism of relative timing, ordinal comparisons between the timing of occurrence of events. Relative timing can be an implicit component of particular brain computations or can be an explicit, conscious judgement. Psychophysical measurements of explicit relative timing have revealed clues about the interaction of sensory signals in the brain as well as in the influence of internal states, such as attention, on those interactions. Evidence from human neurophysiological and functional imaging studies, neuropsychological examination in brain-lesioned patients, and temporary disruptive interventions such as transcranial magnetic stimulation (TMS), point to a role of the parietal cortex in relative timing. Relative timing has traditionally been modelled as a 'race' between competing neural signals. We propose an updated race process based on the integration of sensory evidence towards a decision threshold rather than simple signal propagation. The model suggests a general approach for identifying brain regions involved in relative timing, based on looking for trial-by-trial correlations between neural activity and temporal order judgements (TOJs). Finally, we show how the paradigm can be used to reveal signals related to TOJs in parietal cortex of monkeys trained in a TOJ task.

Prism adaptation does not alter object-based attention in healthy participants

Hemispatial neglect (‘neglect’) is a disabling condition that can follow damage to the right side of the brain, in which patients show difficulty in responding to or orienting towards objects and events that occur on the left side of space. Symptoms of neglect can manifest in both space- and object-based frames of reference. Although patients can show a combination of these two forms of neglect, they are considered separable and have distinct neurological bases. In recent years considerable evidence has emerged to demonstrate that spatial symptoms of neglect can be reduced by an intervention called prism adaptation. Patients point towards objects viewed through prismatic lenses that shift the visual image to the right. Approximately five minutes of repeated pointing results in a leftward recalibration of pointing and improved performance on standard clinical tests for neglect. The understanding of prism adaptation has also been advanced through studies of healthy participants, in whom adaptation to leftward prismatic shifts results in temporary neglect-like performance. Here we examined the effect of prism adaptation on the performance of healthy participants who completed a computerised test of space- and object-based attention. Participants underwent adaptation to leftward- or rightward-shifting prisms, or performed neutral pointing according to a between-groups design. Significant pointing after-effects were found for both prism groups, indicating successful adaptation. In addition, the results of the computerised test revealed larger reaction-time costs associated with shifts of attention between two objects compared to shifts of attention within the same object, replicating previous work. However there were no differences in the performance of the three groups, indicating that prism adaptation did not influence space- or object-based attention for this task. When combined with existing literature, the results are consistent with the proposal that prism adaptation may only perturb cognitive functions for which normal baseline performance is already biased.

Sight and sound out of synch: fragmentation and renormalisation of audiovisual integration and subjective timing

The sight and sound of a person speaking or a ball bouncing may seem simultaneous, but their corresponding neural signals are spread out over time as they arrive at different multisensory brain sites. How subjective timing relates to such neural timing remains a fundamental neuroscientific and philosophical puzzle. A dominant assumption is that temporal coherence is achieved by sensory resynchronisation or recalibration across asynchronous brain events. This assumption is easily confirmed by estimating subjective audiovisual timing for groups of subjects, which is on average similar across different measures and stimuli, and approximately veridical. But few studies have examined normal and pathological individual differences in such measures. Case PH, with lesions in pons and basal ganglia, hears people speak before seeing their lips move. Temporal order judgements (TOJs) confirmed this: voices had to lag lip-movements (by ∼200 msec) to seem synchronous to PH. Curiously, voices had to lead lips (also by ∼200 msec) to maximise the McGurk illusion (a measure of audiovisual speech integration). On average across these measures, PH's timing was therefore still veridical. Age-matched control participants showed similar discrepancies. Indeed, normal individual differences in TOJ and McGurk timing correlated negatively: subjects needing an auditory lag for subjective simultaneity needed an auditory lead for maximal McGurk, and vice versa. This generalised to the Stream-Bounce illusion. Such surprising antagonism seems opposed to good sensory resynchronisation, yet average timing across tasks was still near-veridical. Our findings reveal remarkable disunity of audiovisual timing within and between subjects. To explain this we propose that the timing of audiovisual signals within different brain mechanisms is perceived relative to the average timing across mechanisms. Such renormalisation fully explains the curious antagonistic relationship between disparate timing estimates in PH and healthy participants, and how they can still perceive the timing of external events correctly, on average.

Brain deactivation in the outperformance in bimodal tasks: an FMRI study

While it is known that some individuals can effectively perform two tasks simultaneously, other individuals cannot. How the brain deals with performing simultaneous tasks remains unclear. In the present study, we aimed to assess which brain areas corresponded to various phenomena in task performance. Nineteen subjects were requested to sequentially perform three blocks of tasks, including two unimodal tasks and one bimodal task. The unimodal tasks measured either visual feature binding or auditory pitch comparison, while the bimodal task required performance of the two tasks simultaneously. The functional magnetic resonance imaging (fMRI) results are compatible with previous studies showing that distinct brain areas, such as the visual cortices, frontal eye field (FEF), lateral parietal lobe (BA7), and medial and inferior frontal lobe, are involved in processing of visual unimodal tasks. In addition, the temporal lobes and Brodmann area 43 (BA43) were involved in processing of auditory unimodal tasks. These results lend support to concepts of modality-specific attention. Compared to the unimodal tasks, bimodal tasks required activation of additional brain areas. Furthermore, while deactivated brain areas were related to good performance in the bimodal task, these areas were not deactivated where the subject performed well in only one of the two simultaneous tasks. These results indicate that efficient information processing does not require some brain areas to be overly active; rather, the specific brain areas need to be relatively deactivated to remain alert and perform well on two tasks simultaneously. Meanwhile, it can also offer a neural basis for biofeedback in training courses, such as courses in how to perform multiple tasks simultaneously.

No prior entry for threat-related faces: evidence from temporal order judgments

Previous research showed that threat-related faces, due to their intrinsic motivational relevance, capture attention more readily than neutral faces. Here we used a standard temporal order judgment (TOJ) task to assess whether negative (either angry or fearful) emotional faces, when competing with neutral faces for attention selection, may lead to a prior entry effect and hence be perceived as appearing first, especially when uncertainty is high regarding the order of the two onsets. We did not find evidence for this conjecture across five different experiments, despite the fact that participants were invariably influenced by asynchronies in the respective onsets of the two competing faces in the pair, and could reliably identify the emotion in the faces. Importantly, by systematically varying task demands across experiments, we could rule out confounds related to suboptimal stimulus presentation or inappropriate task demands. These findings challenge the notion of an early automatic capture of attention by (negative) emotion. Future studies are needed to investigate whether the lack of systematic bias of attention by emotion is imputed to the primacy of a non-emotional cue to resolve the TOJ task, which in turn prevents negative emotion to exert an early bottom-up influence on the guidance of spatial and temporal attention.

Cross-modal stimulus conflict: the behavioral effects of stimulus input timing in a visual-auditory Stroop task

Cross-modal processing depends strongly on the compatibility between different sensory inputs, the relative timing of their arrival to brain processing components, and on how attention is allocated. In this behavioral study, we employed a cross-modal audio-visual Stroop task in which we manipulated the within-trial stimulus-onset-asynchronies (SOAs) of the stimulus-component inputs, the grouping of the SOAs (blocked vs. random), the attended modality (auditory or visual), and the congruency of the Stroop color-word stimuli (congruent, incongruent, neutral) to assess how these factors interact within a multisensory context. One main result was that visual distractors produced larger incongruency effects on auditory targets than vice versa. Moreover, as revealed by both overall shorter response times (RTs) and relative shifts in the psychometric incongruency-effect functions, visual-information processing was faster and produced stronger and longer-lasting incongruency effects than did auditory. When attending to either modality, stimulus incongruency from the other modality interacted with SOA, yielding larger effects when the irrelevant distractor occurred prior to the attended target, but no interaction with SOA grouping. Finally, relative to neutral-stimuli, and across the wide range of the SOAs employed, congruency led to substantially more behavioral facilitation than did incongruency to interference, in contrast to findings that within-modality stimulus-compatibility effects tend to be more evenly split between facilitation and interference. In sum, the present findings reveal several key characteristics of how we process the stimulus compatibility of cross-modal sensory inputs, reflecting stimulus processing patterns that are critical for successfully navigating our complex multisensory world.

Attention and the speed of information processing: posterior entry for unattended stimuli instead of prior entry for attended stimuli

Why are nearly simultaneous stimuli frequently perceived in reversed order? The origin of errors in temporal judgments is a question older than experimental psychology itself. One of the earliest suspects is attention. According to the concept of prior entry, attention accelerates attended stimuli; thus they have "prior entry" to perceptive processing stages, including consciousness. Although latency advantages for attended stimuli have been revealed in psychophysical studies many times, these measures (e.g. temporal order judgments, simultaneity judgments) cannot test the prior-entry hypothesis completely. Since they assess latency differences between an attended and an unattended stimulus, they cannot distinguish between faster processing of attended stimuli and slower processing of unattended stimuli. Therefore, we present a novel paradigm providing separate estimates for processing advantages respectively disadvantages of attended and unattended stimuli. We found that deceleration of unattended stimuli contributes more strongly to the prior-entry illusion than acceleration of attended stimuli. Thus, in the temporal domain, attention fulfills its selective function primarily by deceleration of unattended stimuli. That means it is actually posterior entry, not prior entry which accounts for the largest part of the effect.

The timing of change detection and change perception in complex acoustic scenes

We investigated how listeners perceive the temporal relationship of a light flash and a complex acoustic signal. The stimulus mimics ubiquitous events in busy scenes which are manifested as a change in the pattern of on-going fluctuation. Detecting pattern emergence inherently requires integration over time; resulting in such events being detected later than when they occurred. How does delayed detection time affect the perception of such events relative to other events in the scene? To model these situations, we use rapid sequences of tone pips with a time-frequency pattern that changes from random to regular ("REG-RAND") or vice versa ("RAND-REG"). REG-RAND transitions are detected rapidly, but RAND-REG take longer to detect (∼880 ms post nominal transition). Using a Temporal Order Judgment task, we instructed subjects to indicate whether the flash appeared before or after the acoustic transition. The point of subjective simultaneity between the flash and RAND-REG does not occur at the point of detection (∼880 ms post nominal transition) but ∼470 ms closer to the nominal acoustic transition. In a second experiment we halved the tone pip duration. The resulting pattern of performance was qualitatively similar to that in Experiment 1, but scaled by half. Our results indicates that the brain possesses mechanisms that survey the proximal history of an on-going stimulus and automatically adjust perception so as to compensate for prolonged detection time, thus producing more accurate representations of scene dynamics. However, this readjustment is not complete.

Shifts of the psychometric function: distinguishing bias from perceptual effects

Morgan, Dillenburger, Raphael, and Solomon have shown that observers can use different response strategies when unsure of their answer, and, thus, they can voluntarily shift the location of the psychometric function estimated with the method of single stimuli (MSS; sometimes also referred to as the single-interval, two-alternative method). They wondered whether MSS could distinguish response bias from a true perceptual effect that would also shift the location of the psychometric function. We demonstrate theoretically that the inability to distinguish response bias from perceptual effects is an inherent shortcoming of MSS, although a three-response format including also an "undecided" response option may solve the problem under restrictive assumptions whose validity cannot be tested with MSS data. We also show that a proper two-alternative forced-choice (2AFC) task with the three-response format is free of all these problems so that bias and perceptual effects can easily be separated out. The use of a three-response 2AFC format is essential to eliminate a confound (response bias) in studies of perceptual effects and, hence, to eliminate a threat to the internal validity of research in this area.

Evidence for multisensory integration in the elicitation of prior entry by bimodal cues

This study reports an experiment investigating the relative effects of intramodal, crossmodal and bimodal cues on visual and auditory temporal order judgements. Pairs of visual or auditory targets, separated by varying stimulus onset asynchronies, were presented to either side of a central fixation (±45°), and participants were asked to identify the target that had occurred first. In some of the trials, one of the targets was preceded by a short, non-predictive visual, auditory or audiovisual cue stimulus. The cue and target stimuli were presented at the exact same locations in space. The point of subjective simultaneity revealed a consistent spatiotemporal bias towards targets at the cued location. For the visual targets, the intramodal cue elicited the largest, and the crossmodal cue the smallest, bias. The bias elicited by the bimodal cue fell between the intramodal and crossmodal cue biases, with significant differences between all cue types. The pattern for the auditory targets was similar apart from a scaling factor and greater variance, so the differences between the cue conditions did not reach significance. These results provide evidence for multisensory integration in exogenous attentional cueing. The magnitude of the bimodal cueing effect was equivalent to the average of the facilitation elicited by the intramodal and crossmodal cues. Under the assumption that the visual and auditory cues were equally informative, this is consistent with the notion that exogenous attention, like perception, integrates multimodal information in an optimal way.

The Build-Up and Transfer of Sensorimotor Temporal Recalibration Measured via a Synchronization Task

The timing relation between a motor action and the sensory consequences of that action can be adapted by exposing participants to artificially delayed feedback (temporal recalibration). Here, we demonstrate that a sensorimotor synchronization task (i.e., tapping the index finger in synchrony with a pacing signal) can be used as a measure of temporal recalibration. Participants were first exposed to a constant delay (~150 ms) between a voluntary action (a finger tap) and an external feedback stimulus of that action (a visual flash or auditory tone). A subjective "no-delay" condition (~50 ms) served as baseline. After a short exposure phase to delayed feedback participants performed the tapping task in which they tapped their finger in synchrony with a flash or tone. Temporal recalibration manifested itself in that taps were given ~20 ms earlier after exposure to 150 ms delays than in the case of 50 ms delays. This effect quickly built up (within 60 taps) and was bigger for auditory than visual adapters. In Experiment 2, we tested whether temporal recalibration would transfer across modalities by switching the modality of the adapter and pacing signal. Temporal recalibration transferred from visual adapter to auditory test, but not from auditory adapter to visual test. This asymmetric transfer suggests that sensory-specific effects are at play.

Changes in search rate but not in the dynamics of exogenous attention in action videogame players

Many previous studies have shown that the speed of processing in attentionally demanding tasks seems enhanced following habitual action videogame play. However, using one of the diagnostic tasks for efficiency of attentional processing, a visual search task, Castel and collaborators (Castel, Pratt, & Drummond, Acta Psychologica 119:217-230, 2005) reported no difference in visual search rates, instead proposing that action gaming may change response execution time rather than the efficiency of visual selective attention per se. Here we used two hard visual search tasks, one measuring reaction time and the other accuracy, to test whether visual search rate may be changed by action videogame play. We found greater search rates in the gamer group than in the nongamer controls, consistent with increased efficiency in visual selective attention. We then asked how general the change in attentional throughput noted so far in gamers might be by testing whether exogenous attentional cues would lead to a disproportional enhancement in throughput in gamers as compared to nongamers. Interestingly, exogenous cues were found to enhance throughput equivalently between gamers and nongamers, suggesting that not all mechanisms known to enhance throughput are similarly enhanced in action videogamers.

Response errors explain the failure of independent-channels models of perception of temporal order

Independent-channels models of perception of temporal order (also referred to as threshold models or perceptual latency models) have been ruled out because two formal properties of these models (monotonicity and parallelism) are not borne out by data from ternary tasks in which observers must judge whether stimulus A was presented before, after, or simultaneously with stimulus B. These models generally assume that observed responses are authentic indicators of unobservable judgments, but blinks, lapses of attention, or errors in pressing the response keys (maybe, but not only, motivated by time pressure when reaction times are being recorded) may make observers misreport their judgments or simply guess a response. We present an extension of independent-channels models that considers response errors and we show that the model produces psychometric functions that do not satisfy monotonicity and parallelism. The model is illustrated by fitting it to data from a published study in which the ternary task was used. The fitted functions describe very accurately the absence of monotonicity and parallelism shown by the data. These characteristics of empirical data are thus consistent with independent-channels models when response errors are taken into consideration. The implications of these results for the analysis and interpretation of temporal order judgment data are discussed.

Intentional binding is driven by the mere presence of an action and not by motor prediction

Intentional binding refers to the fact that when a voluntary action produces a sensory outcome, action and outcome are perceived as being closer together in time. This phenomenon is often attributed, at least partially, to predictive motor mechanisms. However, previous studies failed to unequivocally attribute intentional binding to these mechanisms, since the contrasts that have been used to demonstrate intentional binding covered not only one but two processes: temporal control and motor identity prediction. In the present study we aimed to isolate the respective role of each of these processes in the emergence of intentional binding of action-effects. The results show that motor identity prediction does not modulate intentional binding of action-effects. Our findings cast doubts on the assumption that intentional binding of action effects is linked to internal forward predictive process.

At the mercy of prior entry: Prior entry induced by invisible primes is not susceptible to current intentions

If one of two events is attended to, it will be perceived earlier than a simultaneously occurring unattended event. Since 150 years, this effect has been ascribed to the facilitating influence of attention, also known as prior entry. Yet, the attentional origin of prior-entry effects(1) has been repeatedly doubted. One criticism is that prior-entry effects might be due to biased decision processes that would mimic a temporal advantage for attended stimuli. Although most obvious biases have already been excluded experimentally (e.g. judgment criteria, response compatibility) and prior-entry effects have shown to persist (Shore, Spence, & Klein, 2001), many other biases are conceivable, which makes it difficult to put the debate to an end. Thus, we approach this problem the other way around by asking whether prior-entry effects can be biased voluntarily. Observers were informed about prior entry and instructed to reduce it as far as possible. For this aim they received continuous feedback about the correctness of their temporal judgments. If elicited by invisible primes the effect could not be reduced at all and only by 12 ms if elicited by visible cues. This challenges decision biases as primary source of prior-entry effects - at least if attention is oriented exogenously.

The cross-modal spread of attention reveals differential constraints for the temporal and spatial linking of visual and auditory stimulus events

The integration of multisensory information has been shown to be guided by spatial and temporal proximity, as well as to be influenced by attention. Here we used neural measures of the multisensory spread of attention to investigate the spatial and temporal linking of synchronous versus near-synchronous auditory and visual events. Human participants attended selectively to one of two lateralized visual-stimulus streams while task-irrelevant tones were presented centrally. Electrophysiological measures of brain activity showed that tones occurring simultaneously or delayed by 100 ms were temporally linked to an attended visual stimulus, as reflected by robust cross-modal spreading-of-attention activity, but not when delayed by 300 ms. The neural data also indicated a ventriloquist-like spatial linking of the auditory to the attended visual stimuli, but only when occurring simultaneously. These neurophysiological results thus provide unique insight into the temporal and spatial principles of multisensory feature integration and the fundamental role attention plays in such integration.

Shifts of criteria or neural timing? The assumptions underlying timing perception studies

In timing perception studies, the timing of one event is usually manipulated relative to another, and participants are asked to judge if the two events were synchronous, or to judge which of the two events occurred first. Responses are analyzed to determine a measure of central tendency, which is taken as an estimate of the timing at which the two events are perceptually synchronous. When these estimates do not coincide with physical synchrony, it is often assumed that the sensory signals are asynchronous, as though the transfer of information concerning one input has been accelerated or decelerated relative to the other. Here we show that, while this is a viable interpretation, it is equally plausible that such effects are driven by shifts in the criteria used to differentiate simultaneous from asynchronous inputs. Our analyses expose important ambiguities concerning the interpretation of simultaneity judgement data, which have hitherto been underappreciated.

Temporal binding of auditory and rotational stimuli

Integration of cues from multiple sensory channels improves our ability to sense and respond to stimuli. Cues arising from a single event may arrive at the brain asynchronously, requiring them to be "bound" in time. The perceptual asynchrony between vestibular and auditory stimuli has been reported to be several times greater than other stimulus pairs. However, these data were collected using electrically evoked vestibular stimuli, which may not provide similar results to those obtained using actual head rotations. Here, we tested whether auditory stimuli and vestibular stimuli consisting of physiologically relevant mechanical rotations are perceived with asynchronies consistent with other sensory systems. We rotated 14 normal subjects about the earth-vertical axis over a raised-cosine trajectory (0.5 Hz, peak velocity 10 deg/s) while isolated from external noise and light. This trajectory minimized any input from extravestibular sources such as proprioception. An 800-Hz, 10-ms auditory tone was presented at stimulus onset asynchronies ranging from 200 ms before to 700 ms after the onset of motion. After each trial, subjects reported whether the stimuli were "simultaneous" or "not simultaneous." The experiment was repeated, with subjects reporting whether the tone or rotation came first. After correction for the time the rotational stimulus took to reach vestibular perceptual threshold, asynchronies spanned from -41 ms (auditory stimulus leading vestibular) to 91 ms (vestibular stimulus leading auditory). These values are significantly lower than those previously reported for stimulus pairs involving electrically evoked vestibular stimuli and are more consistent with timing relationships between pairs of non-vestibular stimuli.

Simultaneity and temporal order perception: Different sides of the same coin? Evidence from a visual prior-entry study

Attended stimuli are perceived as occurring earlier than unattended stimuli. This phenomenon of prior entry is usually identified by a shift in the point of subjective simultaneity (PSS) in temporal order judgements (TOJs). According to its traditional psychophysical interpretation, the PSS coincides with the perception of simultaneity. This assumption is, however, questionable. Technically, the PSS represents the temporal interval between two stimuli at which the two alternative TOJs are equally likely. Thus it also seems possible that observers perceive not simultaneity, but uncertainty of temporal order. This possibility is supported by prior-entry studies, which find that perception of simultaneity is not very likely at the PSS. The present study tested the percept at the PSS in prior entry, using peripheral cues to orient attention. We found that manipulating attention caused varying temporal perceptions around the PSS. On some occasions observers perceived the two stimuli as simultaneous, but on others they were simply uncertain about the order in which they had been presented. This finding contradicts the implicit assumption of most models of temporal order perception, that perception of simultaneity inevitably results if temporal order cannot be discriminated.