Components of visual prior entry

Endogenous attention enhances contrast appearance regardless of stimulus contrast

There has been enduring debate on how attention alters contrast appearance. Recent research indicates that exogenous attention enhances contrast appearance for low-contrast stimuli but attenuates it for high-contrast stimuli. Similarly, one study has demonstrated that endogenous attention heightens perceived contrast for low-contrast stimuli, yet none have explored its impact on high-contrast stimuli. In this study, we investigated how endogenous attention alters contrast appearance, with a specific focus on high-contrast stimuli. In Experiment 1, we utilized the rapid serial visual presentation (RSVP) paradigm to direct endogenous attention, revealing that contrast appearance was enhanced for both low- and high-contrast stimuli. To eliminate potential influences from the confined attention field in the RSVP paradigm, Experiment 2 adopted the letter identification paradigm, deploying attention across a broader visual field. Results consistently indicated that endogenous attention increased perceived contrast for high-contrast stimuli. Experiment 3 employed equiluminant chromatic letters as stimuli in the letter identification task to eliminate potential interference from contrast adaption, which might have occurred in Experiment 2. Remarkably, the boosting effect of endogenous attention persisted. Combining the results from these experiments, we propose that endogenous attention consistently enhances contrast appearance, irrespective of stimulus contrast levels. This stands in contrast to the effects of exogenous attention, suggesting that mechanisms through which endogenous attention alters contrast appearance may differ from those of exogenous attention.

The role of attention in the effect of facial attractiveness on time perception

Recent research has indicated that attractive faces often cause a dilation of our time perception thus affecting physical and mental health, and speculates that this could be relevant to the fact that attractive faces capture people's attention. Nevertheless, there was no direct experimental data to support this speculation. The present work was designed to illustrate how attention affects time perception of facial attractiveness. It utilized two experiments to investigate this phenomenon. In Experiment 1, perception of timing and attention bias were assessed using a temporal reproduction task and a dot-probe task. Increased attention bias was found to mediate the time dilation effect of facial attractiveness. Experiment 2 adopted dual-task paradigm, combining a temporal reproduction task and attractiveness rating task, to manipulate attention allocation. The findings suggested that allocating more attention to the task requiring timing enhanced the time dilation effect caused by the faces. Results of Experiments 1 and 2 converge to show that attention plays an essential role in the effects of facial attractiveness on time perception.

Audiovisual simultaneity windows reflect temporal sensory uncertainty

The ability to judge the temporal alignment of visual and auditory information is a prerequisite for multisensory integration and segregation. However, each temporal measurement is subject to error. Thus, when judging whether a visual and auditory stimulus were presented simultaneously, observers must rely on a subjective decision boundary to distinguish between measurement error and truly misaligned audiovisual signals. Here, we tested whether these decision boundaries are relaxed with increasing temporal sensory uncertainty, i.e., whether participants make the same type of adjustment an ideal observer would make. Participants judged the simultaneity of audiovisual stimulus pairs with varying temporal offset, while being immersed in different virtual environments. To obtain estimates of participants' temporal sensory uncertainty and simultaneity criteria in each environment, an independent-channels model was fitted to their simultaneity judgments. In two experiments, participants' simultaneity decision boundaries were predicted by their temporal uncertainty, which varied unsystematically with the environment. Hence, observers used a flexibly updated estimate of their own audiovisual temporal uncertainty to establish subjective criteria of simultaneity. This finding implies that, under typical circumstances, audiovisual simultaneity windows reflect an observer's cross-modal temporal uncertainty.

Does cross-modal correspondence modulate modality-specific perceptual processing? Study using timing judgment tasks

Cross-modal correspondences refer to associations between stimulus features across sensory modalities. Previous studies have shown that cross-modal correspondences modulate reaction times for detecting and identifying stimuli in one modality when uninformative stimuli from another modality are present. However, it is unclear whether such modulation reflects changes in modality-specific perceptual processing. We used two psychophysical timing judgment tasks to examine the effects of audiovisual correspondences on visual perceptual processing. In Experiment 1, we conducted a temporal order judgment (TOJ) task that asked participants to judge which of two visual stimuli presented with various stimulus onset asynchronies (SOAs) appeared first. In Experiment 2, we conducted a simultaneous judgment (SJ) task that asked participants to report whether the two visual stimuli were simultaneous or successive. We also presented an unrelated auditory stimulus, simultaneously or preceding the first visual stimulus, and manipulated the congruency between audiovisual stimuli. Experiment 1 indicated that the points of subjective simultaneity (PSSs) between the two visual stimuli estimated in the TOJ task shifted according to the audiovisual correspondence between the auditory pitch and visual features of vertical location and size. However, these audiovisual correspondences did not affect PSS estimated using the SJ task in Experiment 2. The different results of the two tasks can be explained through the response bias triggered by audiovisual correspondence that only the TOJ task included. We concluded that audiovisual correspondence would not modulate visual perceptual timing and that changes in modality-specific perceptual processing might not trigger the congruency effects reported in previous studies.

The moment of awareness influences the content of awareness in orientation repulsion

Through the neurally evolving process of dynamic contextual modulation of perceptual contents, it remains unclear how the content of awareness is determined. Here we quantified the visual illusion of orientation repulsion, wherein the target appears tilted against the surrounding's orientation, and examined whether its extent changed when the target awareness was quickened by a preceding flanker. Independently of spatial cueing, repulsion was reduced when the flanker preceded the target by 100 ms compared with when they appeared simultaneously. We confirmed that the preceding flanker quickened the awareness of a nearby target relative to distant ones by 40 ms. Furthermore, the preceding flanker that was greater than 7 degrees away from the target still evoked such reduction of repulsion. These findings imply that the content of awareness is determined by the temporal interaction of two distinct processes: one controls the moment of awareness, and the other represents the perceptual content.

Audiovisual illusion training improves multisensory temporal integration

When we perceive external physical stimuli from the environment, the brain must remain somewhat flexible to unaligned stimuli within a specific range, as multisensory signals are subject to different transmission and processing delays. Recent studies have shown that the width of the 'temporal binding window (TBW)' can be reduced by perceptual learning. However, to date, the vast majority of studies examining the mechanisms of perceptual learning have focused on experience-dependent effects, failing to reach a consensus on its relationship with the underlying perception influenced by audiovisual illusion. The sound-induced flash illusion (SiFI) training is a reliable function for improving perceptual sensitivity. The present study utilized the classic auditory-dominated SiFI paradigm with feedback training to investigate the effect of a 5-day SiFI training on multisensory temporal integration, as evaluated by a simultaneity judgment (SJ) task and temporal order judgment (TOJ) task. We demonstrate that audiovisual illusion training enhances multisensory temporal integration precision in the form of (i) the point of subjective simultaneity (PSS) shifts to reality (0 ms) and (ii) a narrowing TBW. The results are consistent with a Bayesian model of causal inference, suggesting that perception learning reduce the susceptibility to SiFI, whilst improving the precision of audiovisual temporal estimation.

Older Adults' Response to Color Visibility in Indoor Residential Environment Using Eye-Tracking Technology

Older adults spend relatively more time in indoor residential environments than young people. As their visual and cognitive abilities decline, they experience a high risk of accidents in indoor environments; thus, understanding their gaze behavior while considering cognitive capacity is essential in preventing potential accidents and planning for aging-friendly environments. The purpose of this study was to examine whether the use of color in living environments affects the visual attention of older adults. The following two experiments were conducted using eye-tracking technology: Experiment 1: Older adults' gaze behaviors were measured while viewing four images consisting of nine sets of door-door frames with different color combinations of safety colors, black and white; and Experiment 2: Based on results from Experiment 1, images of simulated living environments were created, and older adults' gaze behaviors were measured while viewing eight images in which two door colors were applied to verify spatial color visibility. Their cognitive state represented by the Mini-Mental State Examination (MMSE) was then compared with their gaze behavior. The results show that: (a) participants paid more attention to doors with color combinations including red (red-black and white-red) and remained longer on the red-black combination; (b) older adults looked at red-black doors faster than white doors in the same position; (c) the dwell time on a red-black door was longer than on that of the corresponding position of a white door; and (d) based on participants' MMSE values, the gaze behaviors of the group with low cognition were less focused and observable than those of the group with normal cognition. The results of this study are significant in that they reveal that indoor color can improve the visibility of elements that can impact safety in the living environments of older adults and enhance their quality of life.

Temporal order judgments and presaccadic shifts of attention: What can prior entry teach us about the premotor theory?

A temporal order judgment (TOJ) 2-alternative forced choice design was used to examine presaccadic shifts of attention. Prior work on the premotor theory of attention (PTA) has predominantly focused on single-target discrimination tasks as a tool to measure accuracy and shifts of attention. It is important to demonstrate that the PTA is effective across attentional tasks that have been shown to be reliable in other contexts. Therefore, it was decided to use a perceptual task that probes multiple locations simultaneously and can equally be used to examine spatial spread of attention in more detail. In typical TOJ studies, prior entry is the metric used to measure an attentional effect. Prior entry is the biasing of temporal perception toward an attentionally cued location. This generally manifests as observers processing events at the cued location more rapidly, altering their perspective of temporal order. Participants were required to prepare saccades toward one of four targets, two of which would light up either synchronously or sequentially after a GO signal but before saccadic execution. Results demonstrated that in conditions with critical stimulus onset asynchronies, saccade preparation had a significant effect on performance. Prior entry effects were observed at saccade congruent locations with probes at these locations being typically perceived earlier than probes presented at a neutral location. These effects were not observed in control trials without a saccade. A further spatial effect was demonstrated for the attentional modulation, suggesting that this effect is restricted predominantly to horizontal configurations. Overall, results demonstrated that presaccadic attention is effective at eliciting a prior entry effect in TOJ designs and that such effects are more pronounced when the probes are distributed across the two lateral hemifields.

Pitch-elevation and pitch-size cross-modal correspondences do not affect temporal ventriloquism

Temporal ventriloquism refers to the shift in the perceived timing of a visual stimulus towards a transient auditory stimulus presented close in time. This effect is demonstrated by greater sensitivity of temporal order judgments of two visual stimuli when a sound is presented before the first visual stimulus and after the second visual stimulus. Recent studies suggest that temporal ventriloquism is affected by cross-modal correspondence between auditory pitch and visual elevation but not by correspondence between pitch and visual size. Here we examined the possibility that these results do not reflect a difference in the effects of different types of cross-modal correspondences on temporal ventriloquism but are rather mediated by shifts in visual-spatial attention triggered by preceding auditory stimuli. In Experiment 1, we replicated the results of previous studies that varied with the type of correspondence. Experiment 2 investigated the effects of the second audiovisual stimuli's asynchrony while the first audiovisual stimuli were synchronized. The results, unlike in Experiment 1, revealed that the magnitude of the temporal ventriloquism effect did not change with the congruency of pitch-elevation correspondence. Experiment 3 also indicated that the asynchrony of the first audiovisual stimuli modulated visual discrimination sensitivity irrespective of temporal ventriloquism. These results suggest that cross-modal correspondences do not affect temporal ventriloquism. Greater visual sensitivity when audiovisual stimuli are congruent with pitch-elevation correspondence may be attributable to shifts in visual attention caused by pitches of the preceding auditory stimulus, which speeds up detection of the first visual stimulus.

Attention to a Moment in Time Impairs Episodic Distinctiveness during Rapid Serial Visual Presentation

Human attention is limited in the ability to select and segregate relevant distinct events from the continuous flow of external information while concurrently encoding their temporal succession. While it is well-known that orienting attention to one external target stimulus impairs the encoding of ensuing relevant external events, it is still unknown whether orienting attention to internally generated events can interfere with concurrent processing of external input. We addressed this issue by asking participants to identify a single target embedded among distractors in a non-spatial rapid serial visual presentation (RSVP) stream and to indicate whether that target appeared before or after an internally estimated midpoint of the stream. The results indicate that (a) such an internally generated temporal benchmark does not interfere with the identification of a subsequent physical target stimulus but (b) the two events cannot be accurately segregated when the physical target immediately follows the internally generated temporal event. These findings indicate that the asymmetrical distribution around the midpoint of order reversals reflects an impaired temporal discrimination ability. Orienting attention to a moment in time reduces episodic distinctiveness as much as orienting attention to external events.

Does feature intertrial priming guide attention? The jury is still out

Our search performance is strongly influenced by our past experience. In the lab, this influence has been demonstrated by investigating a variety of phenomena, including intertrial priming, statistical learning, and reward history, and collectively referred to as selection history. The resulting findings have led researchers to claim that selection history guides attention, thereby challenging the prevailing dichotomy, according to which top-down and bottom-up factors alone determine attentional priority. Here, we re-examine this claim with regard to one selection-history phenomenon, feature intertrial priming (aka priming of pop-out). We evaluate the evidence that specifically pertains to the role of feature intertrial priming in attentional guidance, rather than in later selective processes occurring after the target is found. We distinguish between the main experimental rationales, while considering the extent to which feature intertrial priming, as studied through different protocols, shares characteristics of top-down attention. We show that there is strong evidence that feature intertrial priming guides attention when the experimental protocol departs from the canonical paradigm and encourages observers to maintain the critical feature in visual working memory or to form expectations about the upcoming target. By contrast, the current evidence regarding the standard feature intertrial priming phenomenon is inconclusive. We propose directions for future research and suggest that applying the methodology used here in order to re-evaluate of the role of other selection history phenomena in attentional guidance should clarify the mechanisms underlying the strong impact of past experience on visual search performance.

Cortical activation by a salient sound modulates visual temporal order judgments: An electrophysiological study of multisensory attentional processes

Previous event-related potential (ERP) studies show that a salient lateral sound activates the visual cortex more strongly contralateral to the sound, observed as an auditory-evoked contralateral occipital positivity (ACOP). Studies showed that this activation enhances the early cortical processing of co-localized visual stimuli presented after, reflected by better detection rates, better discrimination, and sharper perceived contrast. We replicated the ACOP, using earphones, and tested whether auditory cuing can influence temporal order judgments (TOJ) for two visual stimuli (horizontal arrangement) as well as if the ACOP would predict the amplitude of this influence. A lateral salient sound was followed, after 150 or 630 ms, by the visual presentation of a pair of disks, one in left and one in right hemifield, with variable SOA. The TOJ task was to indicate which disk appeared first or which disk appeared second (controlling for response bias). We observed an ACOP at posterior electrode sites and confirmed our hypothesis that the lateral sound influenced TOJ by accelerating the perception of the disk presented on the cued side, even though the sound was irrelevant to the task. Furthermore, the ACOP amplitude was correlated to this visual perceptual change, indicating that a larger change in brain activity was associated with a faster processing of co-localized visual stimuli.

Early Capture of Attention by Self-Face: Investigation Using a Temporal Order Judgment Task

Earlier work on self-face processing has reported a bias in the processing of self-face result in faster response to self-face in comparison to other familiar and unfamiliar faces (termed as self-face advantage or SFA). Even though most studies agree that the SFA occurs due to an attentional bias, there is little agreement regarding the stage at which it occurs. While a large number of studies show self-face influencing processing later at disengagement stage, early event-related potential components show differential activity for the self-face suggesting that SFA occurs early. We address this contradiction using a cueless temporal order judgment task that allows us to investigate early perceptual processing, while bias due to top-down expectation is controlled. A greater shift in point of subjective simultaneity for self-face would indicate a greater processing advantage at early perceptual stage. With help of two experiments, we show an early perceptual advantage for self-face, compared to both a friend's face and an unfamiliar face (Experiment 1). This advantage is present even when the effect of criterion shift is minimized (Experiment 2). Interestingly, the magnitude of advantage is similar for self-friend and self-unfamiliar pair. The evidence from the two experiments suggests early capture of attention as a likely reason for the SFA, which is present for the self-face but not for other familiar faces.

Binocular Enhancement of Multisensory Temporal Perception

Purpose

The goal of this study was to examine the behavioral effects and to suggest possible underlying mechanisms of binocularity on audiovisual temporal perception in normally-sighted individuals.

Methods

Participants performed two audiovisual simultaneity judgment tasks-one using simple flashes and beeps and the other using audiovisual speech stimuli-with the left eye, right eye, and both eyes. Two measures, the point of subjective simultaneity (PSS) and the temporal binding window (TBW), an index for audiovisual temporal acuity, were derived for each viewing condition, stimulus type, and participant. The data were then modeled using causal inference, allowing us to determine whether binocularity affected low-level unisensory mechanisms (i.e., sensory noise level) or high-level multisensory mechanisms (i.e., prior probability of interring a common cause, pC=1).

Results

Whereas for the PSS there was no significant effect of viewing condition, for the TBW, a significant interaction between stimulus type and viewing condition was found. Post hoc analyses revealed a significantly narrower TBW during binocular than monocular viewing (average of left and right eyes) for the flash-beep condition but no difference between the viewing conditions for the speech stimuli. Modeling results showed no significant difference in pC=1 but a significant reduction in sensory noise during binocular performance on flash-beep trials.

Conclusions

Binocular viewing was found to enhance audiovisual temporal acuity as indexed by the TBW for simple low-level audiovisual stimuli. Furthermore, modeling results suggest that this effect may stem from enhanced sensory representations evidenced as a reduction in sensory noise affecting the measurement of physical asynchrony during audiovisual temporal perception.

The Effect of Rhythmic Tactile Stimuli Under the Voluntary Movement on Audio-Tactile Temporal Order Judgement

The simultaneous perception of multimodal sensory information is important for effective reactions to the external environment. In relation to the effect on time perception, voluntary movement and rhythmic stimuli have already been identified in previous studies to be associated with improved accuracy of temporal order judgments (TOJs). Here, we examined whether the combination of voluntary movement and rhythmic stimuli improves the just noticeable difference (JND) in audio-tactile TOJ Tasks. Four different experimental conditions were studied, involving two types of movements (voluntary movement, involuntary movement) and two types of stimulus presentation (rhythmic, one-time only). In the voluntary movement condition (VM), after the auditory stimulus (cue sound) participants moved their right index finger voluntarily and naturally, while in the involuntary movement condition (IM), their right index finger was moved by the tactile device. The stimuli were provided in a rhythmic or one-time only manner by hitting inside the first joint of the participants' right index finger using a tactile device. Furthermore, in the rhythmical tactile (RT) conditions, tactile stimuli were presented rhythmically to the right index finger 5 times consecutively. On the other hand, in the one-time tactile (1T) conditions, tactile stimuli was presented one-time only to the right index finger. Participants made an order judgment for the fifth tactile stimuli and the first and only auditory stimuli. In our TOJ tasks, auditory-tactile stimulus pairs were presented to participants with varying stimulus-onset asynchronies (SOAs; intervals between the within-pair onsets of the auditory and tactile stimuli). For the two stimuli presented at a time that were shifted by the SOA, the participants were asked to judge which one was presented first, and they were given a two-choice answer. Using a non-parametric test, our results showed that voluntary movement and rhythmic tactile stimuli were both effective in improving the JNDs in TOJ Tasks. However, in the combination of voluntary movement and rhythmic tactile stimuli, we found that there was no significant difference in JNDs in our experiments.

Body-environment integration: Temporal processing of tactile and auditory inputs along the schizophrenia continuum

According to the dimensional approach to psychosis, there is a continuum from low schizotypy to schizophrenia patients. The temporal aspect of sensory processing seems to be compromised across such continuum, as suggested by different studies separately investigating unisensory or multisensory domains. Most of these studies have so far focused primarily on the temporal processing of visual and auditory stimuli, either in schizotypy or schizophrenia, while leaving the tactile domain and the integration of touch with other senses mostly unexplored. Given the relevance of body-related perceptual abnormalities for psychosis proneness, we aimed at filling this gap in the literature across two studies. We asked participants with increasing levels of schizotypy (study 1) and schizophrenia patients (study 2) to perform three simultaneity judgement tasks: a unimodal tactile task, a unimodal auditory task and a bimodal audio-tactile task. Each task allowed estimating a simultaneity range (SR), as a proxy of the individual tolerance to asynchronies in the tactile, auditory and audio-tactile domains, respectively. Results showed larger SRs as the level of schizotypy increases. Specifically, the linear effect of schizotypy levels on the audio-tactile task was stronger than on the auditory task, which in turn was greater than the effect on the tactile task (study 1). Differently, schizophrenia patients showed larger SRs than controls in all the three tasks (study 2). The current study is the first empirical investigation across the continuum from low schizotypy to schizophrenia of the tolerance to asynchronies in the processing of external (auditory) and body-related (tactile) inputs.

Brief period of monocular deprivation drives changes in audiovisual temporal perception

The human brain retains a striking degree of plasticity into adulthood. Recent studies have demonstrated that a short period of altered visual experience (via monocular deprivation) can change the dynamics of binocular rivalry in favor of the deprived eye, a compensatory action thought to be mediated by an upregulation of cortical gain control mechanisms. Here, we sought to better understand the impact of monocular deprivation on multisensory abilities, specifically examining audiovisual temporal perception. Using an audiovisual simultaneity judgment task, we discovered that 90 minutes of monocular deprivation produced opposing effects on the temporal binding window depending on the eye used in the task. Thus, in those who performed the task with their deprived eye there was a narrowing of the temporal binding window, whereas in those performing the task with their nondeprived eye there was a widening of the temporal binding window. The effect was short lived, being observed only in the first 10 minutes of postdeprivation testing. These findings indicate that changes in visual experience in the adult can rapidly impact multisensory perceptual processes, a finding that has important clinical implications for those patients with adult-onset visual deprivation and for therapies founded on monocular deprivation.

Exposure to first-person shooter videogames is associated with multisensory temporal precision and migraine incidence

Adaptive interactions with the environment require optimal integration and segregation of sensory information. Yet, temporal misalignments in the presentation of visual and auditory stimuli may generate illusory phenomena such as the sound-induced flash illusion, in which a single flash paired with multiple auditory stimuli induces the perception of multiple illusory flashes. This phenomenon has been shown to be robust and resistant to feedback training. According to a Bayesian account, this is due to a statistically optimal combination of the signals operated by the nervous system. From this perspective, individual susceptibility to the illusion might be moulded through prolonged experience. For example, repeated exposure to the illusion and prolonged training sessions partially impact on the reported illusion. Therefore, extensive and immersive audio-visual experience, such as first-person shooter videogames, should sharpen individual capacity to correctly integrate multisensory information over time, leading to more veridical perception. We tested this hypothesis by comparing the temporal profile of the sound-induced illusion in a group of expert first-person shooter gamers and a non-players group. In line with the hypotheses, gamers experience significantly narrower windows of illusion (~87 ms) relative to non-players (~105 ms), leading to higher veridical reports in gamers (~68%) relative to non-players (~59%). Moreover, according to recent literature, we tested whether audio-visual intensive training in gamers could be related to the incidence of migraine, and found that its severity may be directly proportioned to the time spent on videogames. Overall, these results suggest that continued training within audio-visual environments such as first-person shooter videogames improves temporal discrimination and sensory integration. This finding may pave the way for future therapeutic strategies based on self-administered multisensory training. On the other hand, the impact of intensive training on visual-related stress disorders, such as migraine incidence, should be taken into account as a risk factor during therapeutic planning.

Mindfulness Meditation Biases Visual Temporal Order Discrimination but Not Under Conditions of Temporal Ventriloquism

This study examined how cognitive plasticity acquired from a long (8 weeks) course of mindfulness training can modulate the perceptual processing of temporal order judgment (TOJ) on a sub-second scale. Observers carried out a TOJ on two visual disks, with or without concurrent paired beeps. A temporal ventriloquism paradigm was used in which the sound beeps either were synchronized with the two disks or bracketed the visual stimuli by leading the first disk by 50 ms and lagging the other by 50 ms. A left-to-right bias in TOJ was found under the visual-only condition after mindfulness training. This bias was positively correlated with “acting with awareness,” a factor in the Five Facet Mindfulness Questionnaire, showing that awareness of every moment and enhanced attention focus magnify the left-to-right bias. However, the effect of mindfulness training may be short-lived and was not present when attention was diverted by auditory events in the cross-modal temporal ventriloquism illusion.

Event timing in human vision: Modulating factors and independent functions

Essential for successful interaction with the environment is the human capacity to resolve events in time. Typical event timing paradigms are judgements of simultaneity (SJ) and of temporal order (TOJ). It remains unclear whether SJ and TOJ are based on the same underlying mechanism and whether there are fixed thresholds for resolution. The current study employed four visual event timing task versions: horizontal and vertical SJ and TOJ. Binary responses were analysed using multilevel binary regression modelling. Modulatory effects of potential explanatory variables on event timing perception were investigated: (1) Individual factors (sex and age), (2) temporal factors (SOA, trial number, order of experiment, order of stimuli orientation, time of day) and (3) spatial factors (left or right stimulus first, top or bottom stimulus first, horizontal vs. vertical orientation). The current study directly compares for the first time, performance on SJ and TOJ tasks using the same paradigm and presents evidence that a variety of factors and their interactions selectively modulate event timing functions in humans, explaining the variance found in previous studies. We conclude that SJ and TOJ are partially independent functions, because they are modulated differently by individual and contextual variables.

Optimality and Limitations of Audio-Visual Integration for Cognitive Systems

Multimodal integration is an important process in perceptual decision-making. In humans, this process has often been shown to be statistically optimal, or near optimal: sensory information is combined in a fashion that minimizes the average error in perceptual representation of stimuli. However, sometimes there are costs that come with the optimization, manifesting as illusory percepts. We review audio-visual facilitations and illusions that are products of multisensory integration, and the computational models that account for these phenomena. In particular, the same optimal computational model can lead to illusory percepts, and we suggest that more studies should be needed to detect and mitigate these illusions, as artifacts in artificial cognitive systems. We provide cautionary considerations when designing artificial cognitive systems with the view of avoiding such artifacts. Finally, we suggest avenues of research toward solutions to potential pitfalls in system design. We conclude that detailed understanding of multisensory integration and the mechanisms behind audio-visual illusions can benefit the design of artificial cognitive systems.

The modulation of background color on perceiving audiovisual simultaneity

Perceiving simultaneity is critical in integrating visual and auditory signals that give rise to a unified perception. We examined whether background color modulates people's perception of audiovisual simultaneity. Two hypotheses were proposed and examined: (1) the red-impairment hypothesis: visual processing speed deteriorates when viewing a red background because the magnocellular system is inhibited by red light; and (2) the blue-enhancement hypothesis: the detection of both visual and auditory signals is enhanced when viewing a blue background because it stimulates the blue-light sensitive intrinsically photosensitive retinal ganglion cells (ipRGCs), which trigger a higher alert state. Participants were exposed to different backgrounds while performing an audiovisual simultaneity judgment (SJ) task: a flash and a beep were presented at pre-designated stimulus onset asynchronies (SOAs) and participants judged whether or not the two stimuli were presented simultaneously. Experiment 1 demonstrated a shift of the point of subjective simultaneity (PSS) toward the visual-leading condition in the red compared to the blue background when the flash was presented in the periphery. In Experiment 2, the stimulation of ipRGCs was specifically manipulated to test the blue-enhancement hypothesis. The results showed no support for this hypothesis, perhaps due to top-down cortical modulations. Taken together, the shift of PSS toward the visual-leading condition in the red background was attributed to impaired visual processing speed with respect to auditory processing speed, caused by the inhibition of the magnocellular system under red light.

Judging Relative Onsets and Offsets of Audiovisual Events

This study assesses the fidelity with which people can make temporal order judgments (TOJ) between auditory and visual onsets and offsets. Using an adaptive staircase task administered to a large sample of young adults, we find that the ability to judge temporal order varies widely among people, with notable difficulty created when auditory events closely follow visual events. Those findings are interpretable within the context of an independent channels model. Visual onsets and offsets can be difficult to localize in time when they occur within the temporal neighborhood of sound onsets or offsets.

The Do's and Don'ts of Psychophysical Methods for Interpretability of Psychometric Functions and Their Descriptors

Many areas of research require measuring psychometric functions or their descriptors (thresholds, slopes, etc.). Data for this purpose are collected with psychophysical methods of various types and justification for the interpretation of results arises from a model of performance grounded in signal detection theory. Decades of research have shown that psychophysical data display features that are incompatible with such framework, questioning the validity of interpretations obtained under it and revealing that psychophysical performance is more complex than this framework entertains. This paper describes the assumptions and formulation of the conventional framework for the two major classes of psychophysical methods (single- and dual-presentation methods) and presents various lines of empirical evidence that the framework is inconsistent with. An alternative framework is then described and shown to account for all the characteristics that the conventional framework regards as anomalies. This alternative process model explicitly separates the sensory, decisional, and response components of performance and represents them via parameters whose estimation characterizes the corresponding processes. Retrospective and prospective evidence of the validity of the alternative framework is also presented. A formal analysis also reveals that some psychophysical methods and response formats are unsuitable for separation of the three components of observed performance. Recommendations are thus given regarding practices that should be avoided and those that should be followed to ensure interpretability of the psychometric function, or descriptors (detection threshold, difference limen, point of subjective equality, etc.) obtained with shortcut methods that do not require estimation of psychometric functions.

Neural correlates of tactile simultaneity judgement: a functional magnetic resonance imaging study

Simultaneity judgement (SJ) is a temporal discrimination task in which the targets span an ultimately short time range (zero or not). Psychophysical studies suggest that SJ is adequate to probe the perceptual components of human time processing in pure form. Thus far, time-relevant neural correlates for tactile SJ are unclear. We performed functional magnetic resonance imaging (fMRI) to investigate the neural correlates of tactile SJ using tactile number judgement as a time-irrelevant control task. As our main result, we demonstrated that the right inferior parietal lobule (IPL) is an SJ-specific region. The right IPL was detected by both parametric and non-parametric statistical analyses, and its activation intensity fulfilled a strict statistical criterion. In addition, we observed that some left-dominant regions (e.g., the striatum) were specifically activated by successive stimuli during SJ. Meanwhile, no region was specifically activated by simultaneous stimuli during SJ. Accordingly, we infer that the neural process for tactile SJ is as follows: the striatum estimates the time interval between tactile stimuli; based on this interval, the right IPL discriminates the successiveness or simultaneity of the stimuli. Moreover, taking detailed behavioural results into account, we further discuss possible concurrent or alternative mechanisms that can explain the fMRI results.

Influence of Expected Reward on Temporal Order Judgment

Perceptual decision-making has been shown to be influenced by reward expected from alternative options or actions, but the underlying neural mechanisms are currently unknown. More specifically, it is debated whether reward effects are mediated through changes in sensory processing, later stages of decision-making, or both. To address this question, we conducted two experiments in which human participants made saccades to what they perceived to be either the first or second of two visually identical but asynchronously presented targets while we manipulated expected reward from correct and incorrect responses on each trial. By comparing reward-induced bias in target selection (i.e., reward bias) during the two experiments, we determined whether reward caused changes in sensory or decision-making processes. We found similar reward biases in the two experiments indicating that reward information mainly influenced later stages of decision-making. Moreover, the observed reward biases were independent of the individual's sensitivity to sensory signals. This suggests that reward effects were determined heuristically via modulation of decision-making processes instead of sensory processing. To further explain our findings and uncover plausible neural mechanisms, we simulated our experiments with a cortical network model and tested alternative mechanisms for how reward could exert its influence. We found that our experimental observations are more compatible with reward-dependent input to the output layer of the decision circuit. Together, our results suggest that, during a temporal judgment task, reward exerts its influence via changing later stages of decision-making (i.e., response bias) rather than early sensory processing (i.e., perceptual bias).

Brief Sensory Training Narrows the Temporal Binding Window and Enhances Long-Term Multimodal Speech Perception

Our ability to integrate multiple sensory-based representations of our surrounding supplies us with a more holistic view of our world. There are many complex algorithms our nervous system uses to construct a coherent perception. An indicator to solve this 'binding problem' are the temporal characteristics with the specificity that environmental information has different propagation speeds (e.g., sound and electromagnetic waves) and sensory processing time and thus the temporal relationship of a stimulus pair derived from the same event must be flexibly adjusted by our brain. This tolerance can be conceptualized in the form of the cross-modal temporal binding window (TBW). Several studies showed the plasticity of the TBW and its importance concerning audio-visual illusions, synesthesia, as well as psychiatric disturbances. Using three audio-visual paradigms, we investigated the importance of length (short vs. long) as well as modality (uni- vs. multimodal) of a perceptual training aiming at reducing the TBW in a healthy population. We also investigated the influence of the TBW on speech intelligibility, where participants had to integrate auditory and visual speech information from a videotaped speaker. We showed that simple sensory trainings can change the TBW and are capable of optimizing speech perception at a very naturalistic level. While the training-length had no different effect on the malleability of the TBW, the multisensory trainings induced a significantly stronger narrowing of the TBW than their unisensory counterparts. Furthermore, a narrowing of the TBW was associated with a better performance in speech perception, meaning that participants showed a greater capacity for integrating informations from different sensory modalities in situations with one modality impaired. All effects persisted at least seven days. Our findings show the significance of multisensory temporal processing regarding ecologically valid measures and have important clinical implications for interventions that may be used to alleviate debilitating conditions (e.g., autism, schizophrenia), in which multisensory temporal function is shown to be impaired.

Orienting spatial attention to sounds enhances visual processing

Attention, the mechanism by which information is selected for further processing, has mostly been studied within the visual system. While this research has been exceptionally successful, it is important to understand how attention operates across the sensory modalities. This review focuses on recent studies showing that orienting to a peripheral, salient sound affects visual processing: it enhances visual perception, boosts visual-cortical responses, and modulates visual cortex activity before the appearance of a visual object. Critically, all of these effects are spatially selective, indicating that spatial attention facilitates perceptual processing at an attended location across sensory modalities. The neural changes in visual cortex triggered by the sounds not only resemble some of the neural modulations reported in uni-modal visual attention studies, but also reveal some important differences.

Visuotactile synchrony of stimulation-induced sensation and natural somatosensation

OBJECTIVE

Previous studies suggest that somatosensory feedback has the potential to improve the functional performance of prostheses, reduce phantom pain, and enhance embodiment of sensory-enabled prosthetic devices. To maximize such benefits for amputees, the temporal properties of the sensory feedback must resemble those of natural somatosensation in an intact limb.

APPROACH

To better understand temporal perception of artificial sensation, we characterized the perception of visuotactile synchrony for tactile perception restored via peripheral nerve stimulation. We electrically activated nerves in the residual limbs of two trans-tibial amputees and two trans-radial amputees via non-penetrating nerve cuff electrodes, which elicited sensations referred to the missing limbs.

MAIN RESULTS

Our findings suggest that with respect to vision, stimulation-induced sensation has a point of subjective simultaneity (PSS; processing time) and just noticeable difference (JND; temporal sensitivity) that are similar to natural touch. The JND was not significantly different between the participants with upper- and lower-limb amputations. However, the PSS indicated that sensations evoked in the missing leg must occur significantly earlier than those in the hand to be perceived as maximally synchronous with vision. Furthermore, we examined visuotactile synchrony in the context of a functional task during which stimulation was triggered by pressure applied to the prosthesis. Stimulation-induced sensation could be delayed up to 111  ±  62 ms without the delay being reliably detected.

SIGNIFICANCE

The quantitative temporal properties of stimulation-induced perception were previously unknown and will contribute to design specifications for future sensory neuroprostheses.

Decoupling sensory from decisional choice biases in perceptual decision making

The contribution of sensory and decisional processes to perceptual decision making is still unclear, even in simple perceptual tasks. When decision makers need to select an action from a set of balanced alternatives, any tendency to choose one alternative more often-choice bias-is consistent with a bias in the sensory evidence, but also with a preference to select that alternative independently of the sensory evidence. To decouple sensory from decisional biases, here we asked humans to perform a simple perceptual discrimination task with two symmetric alternatives under two different task instructions. The instructions varied the response mapping between perception and the category of the alternatives. We found that from 32 participants, 30 exhibited sensory biases and 15 decisional biases. The decisional biases were consistent with a criterion change in a simple signal detection theory model. Perceptual decision making, thus, even in simple scenarios, is affected by sensory and decisional choice biases.

Cancelling Flash Illusory Line Motion by Cancelling the Attentional Gradient and a Consideration of Consciousness

Illusory line motion (ILM) refers to the perception of motion in a line that is, in fact, presented in full at one time. One form of this illusion (flashILM) occurs when the line is presented between two objects following a brief luminance change in one of them and flashILM is thought to result from exogenous attention being captured by the flash. Exogenous attention fades with increasing delays, which predicts that flashILM should show a similar temporal pattern. Exogenous attention appears to follow flashILM to become more or less equally distributed along the line.The current study examines flashILM in order to test these predictions derived from the attentional explanation for flashILM and the results were consistent with them. The discussion then concludes with an exploratory analysis approach concerning states of consciousness and decision making and suggests a possible role for attention.

Violation of the Unity Assumption Disrupts Temporal Ventriloquism Effect in Starlings

When stimuli from different sensory modalities are received, they may be combined by the brain to form a multisensory percept. One key mechanism for multisensory binding is the unity assumption under which multisensory stimuli that share certain physical properties like temporal and/or spatial correspondence are grouped together as deriving from one object. In humans, evidence for a role of the unity assumption has been found in spatial tasks and also in temporal tasks using stimuli that share physical properties (speech-related stimuli, musical and synesthetically congruent stimuli). In our study, we investigate the role of the unity assumption in an animal model in a temporal order judgment task. When subjects are asked to indicate which of two spatially separated visual stimuli appeared first in time, performance improves when the visual stimuli are paired (in time) with spatially non-informative acoustic cues, a phenomenon known as the temporal ventriloquism effect. Here, we show that European starlings perform better when one singleton acoustic cue is paired with the first visual stimulus as compared to pairing with the second visual stimulus. This shows, in combination with our previous study, that a non-informative singleton acoustic cue, when temporally paired with the first visual stimulus, triggers alerting while, when temporally pairing with the second visual stimulus, it prevents a temporal ventriloquism effect because the unity assumption is violated. Thus, the unity assumption influences sensory perception not only in humans but also in an animal model. The importance of the unity assumption in this task supports the idea that the temporal ventriloquism effect, similar to the spatial ventriloquism effect, is based on multisensory binding and integration but not on alerting effects.

The Flash-Lag, Fröhlich and Related Motion Illusions Are Natural Consequences of Discrete Sampling in the Visual System

The Fröhlich effect and flash-lag effect, in which moving objects appear advanced along their trajectories compared to their actual positions, have defied a simple and consistent explanation. Here, I show that these illusions can be understood as a natural consequence of temporal compression in the human visual system. Discrete sampling at some stage of sensory perception has long been considered, and if it were true, it would necessarily lead to these illusions of motion. I show that the discrete perception hypothesis, with a single free parameter, the perceptual moment or sampling rate, can quantitatively explain all of the scenarios of the Fröhlich and flash-lag effect. I interpret discrete perception as the implementation of data compression in the brain, and our conscious perception as the reconstruction of the compressed input.

Audiovisual Temporal Processing in Postlingually Deafened Adults with Cochlear Implants

For many cochlear implant (CI) users, visual cues are vitally important for interpreting the impoverished auditory speech information that an implant conveys. Although the temporal relationship between auditory and visual stimuli is crucial for how this information is integrated, audiovisual temporal processing in CI users is poorly understood. In this study, we tested unisensory (auditory alone, visual alone) and multisensory (audiovisual) temporal processing in postlingually deafened CI users (n = 48) and normal-hearing controls (n = 54) using simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks. We varied the timing onsets between the auditory and visual components of either a syllable/viseme or a simple flash/beep pairing, and participants indicated either which stimulus appeared first (TOJ) or if the pair occurred simultaneously (SJ). Results indicate that temporal binding windows-the interval within which stimuli are likely to be perceptually 'bound'-are not significantly different between groups for either speech or non-speech stimuli. However, the point of subjective simultaneity for speech was less visually leading in CI users, who interestingly, also had improved visual-only TOJ thresholds. Further signal detection analysis suggests that this SJ shift may be due to greater visual bias within the CI group, perhaps reflecting heightened attentional allocation to visual cues.

Stuck on a Plateau? A Model-Based Approach to Fundamental Issues in Visual Temporal-Order Judgments

Humans are incapable of judging the temporal order of visual events at brief temporal separations with perfect accuracy. Their performance-which is of much interest in visual cognition and attention research-can be measured with the temporal-order judgment (TOJ) task, which typically produces S-shaped psychometric functions. Occasionally, researchers reported plateaus within these functions, and some theories predict such deviation from the basic S shape. However, the centers of the psychometric functions result from the weakest performance at the most difficult presentations and therefore fluctuate strongly, leaving the existence and exact shapes of plateaus unclear. This study set out to investigate whether plateaus disappear if the data accuracy is enhanced, or if we are "stuck on a plateau", or rather with it. For this purpose, highly accurate data were assessed by model-based analysis. The existence of plateaus is confidently confirmed and two plausible mechanisms derived from very different models are presented. Neither model, however, performs well in the presence of a strong attention manipulation, and model comparison remains unclear on the question of which of the models describes the data best. Nevertheless, the present study includes the highest accuracy in visual TOJ data and the most explicit models of plateaus in TOJ studied so far.

Temporal trimming: Evidence that common-onset masking shortens perceptual sampling of conscious object representations

Common-onset masking (COM) refers to a methodology where a mask can impair awareness of an object if the mask's offset is delayed relative to the offset of the object. This method has classically been used to understand how discontinuities in visual input lead to the discrete removal of object representations before they reach conscious awareness. However, COM has recently been shown to reduce the precision of conscious object representations (Harrison, Rajsic, & Wilson, Psychonomic Bulletin & Review, 23(1), 180-186, 2016). As a result, Harrison et al. proposed that COM shortens the temporal window for perceptual sampling of an object's representation, an account consistent with interruption-based theories of masking. In the present study we modified the standard COM methodology to assess the impact of a delayed mask offset on the temporal perception of an object's representation. Across two experiments we provide novel evidence that a delayed mask offset can impair temporal perception of a conscious percept, such that it reduces the percept's perceived duration (Experiment 1), and prematurely terminates updating of the percept's dynamic orientation (Experiment 2). We refer to these results as temporal trimming, and suggest that the mechanism responsible for COM operates during the sustained perception of an object.

Audiovisual integration in depth: multisensory binding and gain as a function of distance

The integration of information across sensory modalities is dependent on the spatiotemporal characteristics of the stimuli that are paired. Despite large variation in the distance over which events occur in our environment, relatively little is known regarding how stimulus-observer distance affects multisensory integration. Prior work has suggested that exteroceptive stimuli are integrated over larger temporal intervals in near relative to far space, and that larger multisensory facilitations are evident in far relative to near space. Here, we sought to examine the interrelationship between these previously established distance-related features of multisensory processing. Participants performed an audiovisual simultaneity judgment and redundant target task in near and far space, while audiovisual stimuli were presented at a range of temporal delays (i.e., stimulus onset asynchronies). In line with the previous findings, temporal acuity was poorer in near relative to far space. Furthermore, reaction time to asynchronously presented audiovisual targets suggested a temporal window for fast detection-a range of stimuli asynchronies that was also larger in near as compared to far space. However, the range of reaction times over which multisensory response enhancement was observed was limited to a restricted range of relatively small (i.e., 150 ms) asynchronies, and did not differ significantly between near and far space. Furthermore, for synchronous presentations, these distance-related (i.e., near vs. far) modulations in temporal acuity and multisensory gain correlated negatively at an individual subject level. Thus, the findings support the conclusion that multisensory temporal binding and gain are asymmetrically modulated as a function of distance from the observer, and specifies that this relationship is specific for temporally synchronous audiovisual stimulus presentations.

I before U: Temporal order judgements reveal bias for self-owned objects

A multitude of studies demonstrate that self-relevant stimuli influence attention. Self-owned objects are a special class of self-relevant stimuli. If a self-owned object can indeed be characterised as a self-relevant stimulus then, consistent with theoretical predictions, a behavioural effect of ownership on attention should be present. To test this prediction, a task was selected that is known to be particularly sensitive measure of the prioritisation of visual information: the temporal order judgement. Participants completed temporal order judgements with pictures of "own" and "experimenter" owned objects (mugs) presented on either side of a central fixation cross. There was a variable onset delay between each picture, ranging between 0 ms and 105 ms, and participants were asked to indicate which mug appeared first. The results indicated a reliable change in the point of subjective simultaneity (PSS) in favour of their own mug. Such a change in the PSS was not observed for two groups of participants who were exposed to a mug but did not keep the mug. A further experiment indicated that the source of the bias in PSS was more consistent with a criterion shift or top-down attentional prioritisation rather than a perceptual bias. These findings suggest that ownership, beyond mere-touch, mere-choice, or familiarity, leads to prioritised processing and responses, but the mechanism underlying the effect is not likely to be perceptual in nature.

We prefer what we fear: A response preference bias mimics attentional capture in spider fear

The extent to which emotionally significant stimuli capture visual attention remains elusive because a preference for reporting or choosing emotionally significant stimuli could mimic attentional capture by these stimuli. We conducted two prior-entry experiments to disentangle whether phobic and fear-relevant stimuli capture attention or merely produce a response bias in spider-fearful participants. Prior entry denotes the effect that attended stimuli are perceived earlier than unattended stimuli as indicated by temporal order judgments. We presented phobic (spiders), fear-relevant (snakes) and neutral stimuli in pairs with varying temporal onset. The participants' task was to indicate which stimulus was presented first (Experiment 1) or second (Experiment 2). In the first experiment, spider-fearful but not control participants indicated that they had perceived spiders as occurring earlier in time, suggesting a prior-entry effect for spiders in this group. But surprisingly, in the second experiment, spider-fearful participants indicated more frequently that they had seen spiders as being presented second. This finding rules out a genuine prior-entry effect and instead suggests a strong preference for the response option associated with the feared animal. This response bias may result from a hypervigilance toward the feared stimulus and contribute to maintaining avoidance behavior in individuals with specific phobias.

Further evidence in favor of prior entry from endogenous attention to a location in space

Titchener's (1908) law of prior entry states that "the object of attention comes to consciousness more quickly than the objects which we are not attending to," or otherwise, that attended stimuli are perceived earlier than unattended stimuli. Shore, Spence, and Klein (Psychological Science, 12, 205-212. doi: 10.1111/1467-9280.00337 , 2001) showed that endogenous visuospatial orienting does in fact elicit prior-entry effects, albeit to a smaller degree than does exogenous visuospatial orienting. In disagreement with this finding, Schneider and Bavelier (Cognitive Psychology, 47, 333-366. doi: 10.1016/S0010-0285(03)00035-5 , 2003) found no effect of their instruction to attend. They concluded that nonattentional effects could masquerade as prior entry, which could account for findings such as those in Shore et al.'s endogenous condition. We investigated this empirical and theoretical discord by replicating the temporal-order judgment task used by Shore, Spence, and Klein, while manipulating and measuring endogenous orienting by way of an orthogonal color probe task. We showed evidence of prior entry as a consequence of endogenous orienting, supporting the conclusions of Shore, Spence, and Klein.

Multisensory integration and ADHD-like traits: Evidence for an abnormal temporal integration window in ADHD

Abnormalities in multimodal processing have been found in many developmental disorders such as autism and dyslexia. However, surprisingly little empirical work has been conducted to test the integrity of multisensory integration in Attention Deficit Hyperactivity Disorder (ADHD). The main aim of the present study was to examine links between symptoms of ADHD (as measured using a self-report scale in a healthy adult population) and the temporal aspects of multisensory processing. More specifically, a Simultaneity Judgement (SJ) and a Temporal Order Judgement (TOJ) task were used in participants with low and high levels of ADHD-like traits to measure the temporal integration window and Just-Noticeable Difference (JND) (respectively) between the timing of an auditory beep and a visual pattern presented over a broad range of stimulus onset asynchronies. The Point of Subjective Similarity (PSS) was also measured in both cases. In the SJ task, participants with high levels of ADHD-like traits considered significantly fewer stimuli to be simultaneous than participants with high levels of ADHD-like traits, and the former were found to have significantly smaller temporal windows of integration (although no difference was found in the PSS in the SJ or TOJ tasks, or the JND in the latter). This is the first study to identify an abnormal temporal integration window in individuals with ADHD-like traits. Perceived temporal misalignment of two or more modalities can lead to distractibility (e.g., when the stimulus components from different modalities occur separated by too large of a temporal gap). Hence, an abnormality in the perception of simultaneity could lead to the increased distractibility seen in ADHD.

Learned value and object perception: Accelerated perception or biased decisions?

Learned value is known to bias visual search toward valued stimuli. However, some uncertainty exists regarding the stage of visual processing that is modulated by learned value. Here, we directly tested the effect of learned value on preattentive processing using temporal order judgments. Across four experiments, we imbued some stimuli with high value and some with low value, using a nonmonetary reward task. In Experiment 1, we replicated the value-driven distraction effect, validating our nonmonetary reward task. Experiment 2 showed that high-value stimuli, but not low-value stimuli, exhibit a prior-entry effect. Experiment 3, which reversed the temporal order judgment task (i.e., reporting which stimulus came second), showed no prior-entry effect, indicating that although a response bias may be present for high-value stimuli, they are still reported as appearing earlier. However, Experiment 4, using a simultaneity judgment task, showed no shift in temporal perception. Overall, our results support the conclusion that learned value biases perceptual decisions about valued stimuli without speeding preattentive stimulus processing.

Temporal Order Judgements of Dynamic Gaze Stimuli Reveal a Postdictive Prioritisation of Averted Over Direct Shifts

We studied temporal order judgements (TOJs) of gaze shift behaviours and evaluated the impact of gaze direction (direct and averted gaze) and face context information (both eyes set within a single face or each eye within two adjacent hemifaces) on TOJ performance measures. Avatar faces initially gazed leftwards or rightwards (Starting Gaze Direction). This was followed by sequential and independent left and right eye gaze shifts with various amounts of stimulus onset asynchrony. Gaze shifts could be either Matching (both eyes end up pointing direct or averted) or Mismatching (one eye ends up pointing direct, the other averted). Matching shifts revealed an attentional cueing mechanism, where TOJs were biased in favour of the eye lying in the hemispace cued by the avatar’s Starting Gaze Direction. For example, the left eye was more likely to be judged as shifting first when the avatar initially gazed toward the left side of the screen. Mismatching shifts showed biased TOJs in favour of the eye performing the averted shift, but only in the context of two separate hemifaces that does not violate expectations of directional gaze shift congruency. This suggests a postdictive inferential strategy that prioritises eye movements based on the type of gaze shift, independently of where attention is initially allocated. Averted shifts are prioritised over direct, as these might signal the presence of behaviourally relevant information in the environment.

Differences between endogenous attention to spatial locations and sensory modalities

Vibell et al. (J Cogn Neurosci 19:109-120, 2007) reported that endogenously attending to a sensory modality (vision or touch) modulated perceptual processing, in part, by the relative speeding-up of neural activation (i.e., as a result of prior entry). However, it was unclear whether it was the fine temporal discrimination required by the temporal-order judgment task that was used, or rather, the type of attentional modulation (spatial locations or sensory modalities) that was responsible for the shift in latencies that they observed. The present study used a similar experimental design to evaluate whether spatial attention would also yield similar latency effects suggestive of prior entry in the early visual P1 potentials. Intriguingly, while the results demonstrate similar neural latency shifts attributable to spatial attention, they started at a somewhat later stage than seen in Vibell et al.'s study. These differences are consistent with different neural mechanisms underlying attention to a specific sensory modality versus to a spatial location.

SOAP Opera: Self as Object and Agent in Prioritizing Attention

A growing body of evidence has demonstrated that multiple sources of salience tune attentional sets toward aspects of the environment, including affectively and motivationally significant categories of stimuli such as angry faces and reward-associated target locations. Recent evidence further indicates that objects that have gained personal significance through ownership can elicit similar attentional prioritization. Here we discuss current research on sources of attentional prioritization that shape our awareness of the visual world from moment to moment and the underlying neural systems and contextualize what is known about attentional prioritization of our possessions within that research. We review behavioral and neuroimaging research on the influence of self-relevance and ownership on cognition and discuss challenges to this literature stemming from different modes of conceptualizing and operationalizing the self. We argue that ownership taps into both “self-as-object,” which characterizes the self as an object with a constellation of traits and attributes, and “self-as-subject,” which characterizes the self as an agentic perceiver and knower. Despite an abundance of research probing neural and behavioral indices of self-as-object and its effects on attention, there exists a paucity of research on the influence of self-relevance of attention when self is operationalized from the perspective of a first-person subject. To begin to address this gap, we propose the Self as Ownership in Attentional Prioritization (SOAP) framework to explain how ownership increases salience through attention to external representations of self-identity (i.e., self as object) and attention to contextually mediated permission to act (i.e., self as subject).

Sight and sound persistently out of synch: stable individual differences in audiovisual synchronisation revealed by implicit measures of lip-voice integration

Are sight and sound out of synch? Signs that they are have been dismissed for over two centuries as an artefact of attentional and response bias, to which traditional subjective methods are prone. To avoid such biases, we measured performance on objective tasks that depend implicitly on achieving good lip-synch. We measured the McGurk effect (in which incongruent lip-voice pairs evoke illusory phonemes), and also identification of degraded speech, while manipulating audiovisual asynchrony. Peak performance was found at an average auditory lag of ~100 ms, but this varied widely between individuals. Participants’ individual optimal asynchronies showed trait-like stability when the same task was re-tested one week later, but measures based on different tasks did not correlate. This discounts the possible influence of common biasing factors, suggesting instead that our different tasks probe different brain networks, each subject to their own intrinsic auditory and visual processing latencies. Our findings call for renewed interest in the biological causes and cognitive consequences of individual sensory asynchronies, leading potentially to fresh insights into the neural representation of sensory timing. A concrete implication is that speech comprehension might be enhanced, by first measuring each individual’s optimal asynchrony and then applying a compensatory auditory delay.