What you saw is what you will hear: Two new illusions with audiovisual postdictive effects

Neuroscience investigations are most often focused on the prediction of future perception or decisions based on prior brain states or stimulus presentations. However, the brain can also process information retroactively, such that later stimuli impact conscious percepts of the stimuli that have already occurred (called “postdiction”). Postdictive effects have thus far been mostly unimodal (such as apparent motion), and the models for postdiction have accordingly been limited to early sensory regions of one modality. We have discovered two related multimodal illusions in which audition instigates postdictive changes in visual perception. In the first illusion (called the “Illusory Audiovisual Rabbit”), the location of an illusory flash is influenced by an auditory beep-flash pair that follows the perceived illusory flash. In the second illusion (called the “Invisible Audiovisual Rabbit”), a beep-flash pair following a real flash suppresses the perception of the earlier flash. Thus, we showed experimentally that these two effects are influenced significantly by postdiction. The audiovisual rabbit illusions indicate that postdiction can bridge the senses, uncovering a relatively-neglected yet critical type of neural processing underlying perceptual awareness. Furthermore, these two new illusions broaden the Double Flash Illusion, in which a single real flash is doubled by two sounds. Whereas the double flash indicated that audition can create an illusory flash, these rabbit illusions expand audition’s influence on vision to the suppression of a real flash and the relocation of an illusory flash. These new additions to auditory-visual interactions indicate a spatio-temporally fine-tuned coupling of the senses to generate perception.

A Framework Supporting the Development of a Grid Portal for Analysis Based on ROI

Objectives: In our research on brain function analysis, users require two different simultaneous types of processing: interactive processing to a specific part of data and high-performance batch processing to an entire dataset. The difference between these two types of processing is in whether or not the analysis is for data in the region of interest (ROI). In this study, we propose a Grid portal that has a mechanism to freely assign computing resources to the users on a Grid environment according to the users’ two different types of processing requirements.

Methods: We constructed a Grid portal which integrates interactive processing and batch processing by the following two mechanisms. First, a job steering mechanism controls job execution based on user-tagged priority among organizations with heterogeneous computing resources. Interactive jobs are processed in preference to batch jobs by this mechanism. Second, a priority-based result delivery mechanism that administrates a rank of data significance.

Results: The portal ensures a turn-around time of interactive processing by the priority-based job controlling mechanism, and provides the users with quality of services (QoS) for interactive processing. The users can access the analysis results of interactive jobs in preference to the analysis results of batch jobs. The Grid portal has also achieved high-performance computation of MEG analysis with batch processing on the Grid environment.

Conclusion: The priority-based job controlling mechanism has been realized to freely assign computing resources to the users’ requirements. Furthermore the achievement of high-performance computation contributes greatly to the overall progress of brain science. The portal has thus made it possible for the users to flexibly include the large computational power in what they want to analyze.

Triple dissociation of duration perception regulating mechanisms: Top-down attention is inherent

The brain constantly adjusts perceived duration based on the recent event history. One such lab phenomenon is subjective time expansion induced in an oddball paradigm ("oddball chronostasis"), where the duration of a distinct item (oddball) appears subjectively longer when embedded in a series of other repeated items (standards). Three hypotheses have been separately proposed but it remains unresolved which or all of them are true: 1) attention prolongs oddball duration, 2) repetition suppression reduces standards duration, and 3) accumulative temporal preparation (anticipation) expedites the perceived item onset so as to lengthen its duration. We thus conducted critical systematic experiments to dissociate the relative contribution of all hypotheses, by orthogonally manipulating sequences types (repeated, ordered, or random) and target serial positions. Participants' task was to judge whether a target lasts shorter or longer than its reference. The main finding was that a random item sequence still elicited significant chronostasis even though each item was odd. That is, simply being a target draws top-down attention and induces chronostasis. In Experiments 1 (digits) and 2 (orientations), top-down attention explained about half of the effect while saliency/adaptation explained the other half. Additionally, for non-repeated (ordered and random) sequence types, a target with later serial position still elicited stronger chronostasis, favoring a temporal preparation over a repetition suppression account. By contrast, in Experiment 3 (colors), top-down attention was likely the sole factor. Consequently, top-down attention is necessary and sometimes sufficient to explain oddball chronostasis; saliency/adaptation and temporal preparation are contingent factors. These critical boundary conditions revealed in our study serve as quantitative constraints for neural models of duration perception.

Perceptual Alternation Induced by Visual Transients

When our visual system is confronted with ambiguous stimuli, the perceptual interpretation spontaneously alternates between the competing incompatible interpretations. The timing of such perceptual alternations is highly stochastic and the underlying neural mechanisms are poorly understood. We show that perceptual alternations can be triggered by a transient stimulus presented nearby. The induction was tested for four types of bistable stimuli: structure-from-motion, binocular rivalry, Necker cube, and ambiguous apparent motion. While underlying mechanisms may vary among them, a transient flash induced time-locked perceptual alternations in all cases. The effect showed a dependence on the adaptation to the dominant percept prior to the presentation of a flash. These perceptual alternations show many similarities to perceptual disappearances induced by transient stimuli (Kanai and Kamitani, 2003 Journal of Cognitive Neuroscience15 664–672; Moradi and Shimojo, 2004 Vision Research44 449–460). Mechanisms linking these two transient-induced phenomena are discussed.

Temporal and Spatial Characteristics of Attention to Facilitate Manual and Eye-Movement Responses

Previous investigation found that the speed of saccadic eye movements is enhanced when a temporal interval (gap) is introduced between the disappearance of a foveal fixation mark and the appearance of a peripheral target (the gap paradigm). Attention was shown to be involved in the gap paradigm. Here, we investigated relevant temporal and spatial characteristics of attention, manipulating central fixation marks and peripheral targets. Results from three experiments indicate that (i) the speed of manual and eye-movement detection is accelerated when a fixation mark changes abruptly (in less than 100 ms) before its termination in the gap paradigm; (ii) the speed is further accelerated when a peripheral target location is pre-cued; (iii) sufficient time for fixation (1000 ms) is necessary for the facilitation. These results suggest that fast and transient attention at the fixation spot facilitates attentional disengagement process that urges a spatial-orienting mechanism. Sustained attention is required in the engagement process during the fixation.

Length and orientation constancy learning in 2-dimensions with auditory sensory substitution: the importance of self-initiated movement

A subset of sensory substitution (SS) devices translate images into sounds in real time using a portable computer, camera, and headphones. Perceptual constancy is the key to understanding both functional and phenomenological aspects of perception with SS. In particular, constancies enable object externalization, which is critical to the performance of daily tasks such as obstacle avoidance and locating dropped objects. In order to improve daily task performance by the blind, and determine if constancies can be learned with SS, we trained blind (N = 4) and sighted (N = 10) individuals on length and orientation constancy tasks for 8 days at about 1 h per day with an auditory SS device. We found that blind and sighted performance at the constancy tasks significantly improved, and attained constancy performance that was above chance. Furthermore, dynamic interactions with stimuli were critical to constancy learning with the SS device. In particular, improved task learning significantly correlated with the number of spontaneous left-right head-tilting movements while learning length constancy. The improvement from previous head-tilting trials even transferred to a no-head-tilt condition. Therefore, not only can SS learning be improved by encouraging head movement while learning, but head movement may also play an important role in learning constancies in the sighted. In addition, the learning of constancies by the blind and sighted with SS provides evidence that SS may be able to restore vision-like functionality to the blind in daily tasks.

Functional connectivity of the striatum in experts of stenography

INTRODUCTION

Stenography, or shorthand, is a unique set of skills that involves intensive training which is nearly life-long and orchestrating various brain functional modules, including auditory, linguistic, cognitive, mnemonic, and motor. Stenography provides cognitive neuroscientists with a unique opportunity to investigate the neural mechanisms underlying the neural plasticity that enables such a high degree of expertise. However, shorthand is quickly being replaced with voice recognition technology. We took this nearly final opportunity to scan the brains of the last alive shorthand experts of the Japanese language.

METHODS

Thirteen right-handed stenographers and fourteen right-handed controls participated in the functional magnetic resonance imaging (fMRI) study.

RESULTS

The fMRI data revealed plastic reorganization of the neural circuits around the putamen. The acquisition of expert skills was accompanied by structural and functional changes in the area. The posterior putamen is known as the execution center of acquired sensorimotor skills. Compared to nonexperts, the posterior putamen in stenographers had high covariation with the cerebellum and midbrain.The stenographers' brain developed different neural circuits from those of the nonexpert brain.

CONCLUSIONS

The current data illustrate the vigorous plasticity in the putamen and in its connectivity to other relevant areas in the expert brain. This is a case of vigorous neural plastic reorganization in response to massive overtraining, which is rare especially considering that it occurred in adulthood.

Neural computations mediating one-shot learning in the human brain

Incremental learning, in which new knowledge is acquired gradually through trial and error, can be distinguished from one-shot learning, in which the brain learns rapidly from only a single pairing of a stimulus and a consequence. Very little is known about how the brain transitions between these two fundamentally different forms of learning. Here we test a computational hypothesis that uncertainty about the causal relationship between a stimulus and an outcome induces rapid changes in the rate of learning, which in turn mediates the transition between incremental and one-shot learning. By using a novel behavioral task in combination with functional magnetic resonance imaging (fMRI) data from human volunteers, we found evidence implicating the ventrolateral prefrontal cortex and hippocampus in this process. The hippocampus was selectively “switched” on when one-shot learning was predicted to occur, while the ventrolateral prefrontal cortex was found to encode uncertainty about the causal association, exhibiting increased coupling with the hippocampus for high-learning rates, suggesting this region may act as a “switch,” turning on and off one-shot learning as required.

A combination of neuroimaging and computational modeling suggests that a part of ventrolateral prefrontal cortex, in cooperation with the hippocampus, is responsible for switching between incremental and one-shot strategies for learning about causal relationships. Read the Synopsis.

There are at least two distinct learning strategies for identifying the relationship between a cause and its consequence: (1) incremental learning, in which we gradually acquire knowledge through trial and error, and (2) one-shot learning, in which we rapidly learn from only a single pairing of a potential cause and a consequence. Little is known about how the brain switches between these two forms of learning. In this study, we provide evidence that the amount of uncertainty about the relationship between cause and consequence mediates the transition between incremental and one-shot learning. Specifically, the more uncertainty there is about the causal relationship, the higher the learning rate that is assigned to that stimulus. By imaging the brain while participants were performing the learning task, we also found that uncertainty about the causal association is encoded in the ventrolateral prefrontal cortex and that the degree of coupling between this region and the hippocampus increases during one-shot learning. We speculate that this prefrontal region may act as a “switch,” turning on and off one-shot learning as required.

Eyetracking of social preference choices reveals normal but faster processing in autism

People with Autism Spectrum Disorder (ASD) have been reported to show atypical attention and evaluative processing, in particular for social stimuli such as faces. The usual measure in these studies is an explicit, subjective judgment, which is the culmination of complex-temporally extended processes that are not typically dissected in detail. Here we addressed a neglected aspect of social decision-making in order to gain further insight into the underlying mechanisms: the temporal evolution of the choice. We investigated this issue by quantifying the alternating patterns of gaze onto faces, as well as nonsocial stimuli, while subjects had to decide which of the two stimuli they preferred. Surprisingly, the temporal profile of fixations relating to choice (the so-called "gaze cascade") was entirely normal in ASD, as were the eventual preference choices. Despite these similarities, we found two key abnormalities: people with ASD made choices more rapidly than did control subjects across the board, and their reaction times for social preference judgments were insensitive to choice difficulty. We suggest that ASD features an altered decision-making process when basing choice on social preferences. One hypothesis motivated by these data is that a choice criterion is reached in ASD regardless of the discriminability of the options.

Visual attractiveness is leaky: the asymmetrical relationship between face and hair

Predicting personality is crucial when communicating with people. It has been revealed that the perceived attractiveness or beauty of the face is a cue. As shown in the well-known "what is beautiful is good" stereotype, perceived attractiveness is often associated with desirable personality. Although such research on attractiveness used mainly the face isolated from other body parts, the face is not always seen in isolation in the real world. Rather, it is surrounded by one's hairstyle, and is perceived as a part of total presence. In human vision, perceptual organization/integration occurs mostly in a bottom up, task-irrelevant fashion. This raises an intriguing possibility that task-irrelevant stimulus that is perceptually integrated with a target may influence our affective evaluation. In such a case, there should be a mutual influence between attractiveness perception of the face and surrounding hair, since they are assumed to share strong and unique perceptual organization. In the current study, we examined the influence of a task-irrelevant stimulus on our attractiveness evaluation, using face and hair as stimuli. The results revealed asymmetrical influences in the evaluation of one while ignoring the other. When hair was task-irrelevant, it still affected attractiveness of the face, but only if the hair itself had never been evaluated by the same evaluator. On the other hand, the face affected the hair regardless of whether the face itself was evaluated before. This has intriguing implications on the asymmetry between face and hair, and perceptual integration between them in general. Together with data from a post hoc questionnaire, it is suggested that both implicit non-selective and explicit selective processes contribute to attractiveness evaluation. The findings provide an understanding of attractiveness perception in real-life situations, as well as a new paradigm to reveal unknown implicit aspects of information integration for emotional judgment.

Rate perception adapts across the senses: evidence for a unified timing mechanism

The brain constructs a representation of temporal properties of events, such as duration and frequency, but the underlying neural mechanisms are under debate. One open question is whether these mechanisms are unisensory or multisensory. Duration perception studies provide some evidence for a dissociation between auditory and visual timing mechanisms; however, we found active crossmodal interaction between audition and vision for rate perception, even when vision and audition were never stimulated together. After exposure to 5 Hz adaptors, people perceived subsequent test stimuli centered around 4 Hz to be slower, and the reverse after exposure to 3 Hz adaptors. This aftereffect occurred even when the adaptor and test were different modalities that were never presented together. When the discrepancy in rate between adaptor and test increased, the aftereffect was attenuated, indicating that the brain uses narrowly-tuned channels to process rate information. Our results indicate that human timing mechanisms for rate perception are not entirely segregated between modalities and have substantial implications for models of how the brain encodes temporal features. We propose a model of multisensory channels for rate perception, and consider the broader implications of such a model for how the brain encodes timing.