Slower and Less Variable Binocular Rivalry Rates in Patients With Bipolar Disorder, OCD, Major Depression, and Schizophrenia

When two different images are presented to the two eyes dichoptically, observers usually experience a perceptual alternation between the two images. This phenomenon, known as binocular rivalry, has been used as a powerful tool to investigate mechanisms of visual awareness. It was also found that the rates of perceptual alternation are slower in patients with bipolar disorder than in healthy controls (Pettigrew and Miller, 1998; Miller et al., 2003). To investigate the broader clinical relevance of binocular rivalry in psychiatric disorders, we measured the perceptual alternation rates during rivalry in healthy controls (n = 39) and in patients with different types of psychiatric disorders, including bipolar disorder type I (BD, n = 28), obsessive-compulsive disorder (OCD, n = 22), major depression (MD, n = 50), schizophrenia (SCZ, n = 44), and first-degree relatives (FDRs) of SCZ patients (n = 32). Participants viewed competing red-green images on a computer monitor through red-green anaglyph glasses and pressed buttons to record their perceptual alternations. The distributions of the rivalry rates were well described by a lognormal function in all groups. Critically, the median rate of perceptual alternation was 0.27 Hz for BD patients, 0.26 Hz for the OCD patients, 0.25 Hz for the MD patients, and 0.23 Hz and 0.27 Hz for the SCZ patients and their FDRs, respectively. All of which were significantly slower than the rate of 0.41 Hz obtained for the healthy controls, suggesting there may be shared genotypes between these different disorders. While rivalry alternations were generally slower in different types of psychiatric disorders compared to healthy controls, adding variance of rivalry rates in the analysis helped to partially separate among the different patient groups. Our results suggest that the slowing of binocular rivalry is likely due to certain common factors among the patient groups, but more subtle differences between different patient groups could be revealed when additional properties of rivalry dynamics are considered.

Transcutaneous Vagus Nerve Stimulation (tVNS) and the Dynamics of Visual Bistable Perception

Transcutaneous vagus nerve stimulation (tVNS) is widely used for clinical applications, but its mechanism of action is poorly understood. One candidate pathway that might mediate the effects of tVNS is an increase in GABAergic neurotransmission. In this study, we investigated the effect of tVNS on visual bistable perception, which is highly coupled to GABA. Participants were 34 healthy young subjects. We used a static (Necker cube) and a dynamic (structure from motion) bistable perception task. Each subject underwent tVNS as well as sham (placebo) stimulation for ∼45 min. We analyze effects of tVNS on percept durations by means of Bayesian multilevel regression. We find no evidence for a modulation of bistable perception dynamics through tVNS in either task, but the analyses do not ultimately confirm the null hypothesis either. We discuss different possible implications of our finding and propose that GABAergic effects of tVNS should be further investigated using more direct measures of GABA concentration, and, more generally, that a better understanding of the mechanisms of action of vagus nerve stimulation is needed. Finally, we discuss limitations of our study design, data analysis, and conclusions.

Frequency of alpha oscillation predicts individual differences in perceptual stability during binocular rivalry

When ambiguous visual stimuli have multiple interpretations, human perception can alternate between them, producing perceptual multistability. There is a large variation between individuals in how long stable percepts endure, on average, between switches, but the underlying neural basis of this individual difference in perceptual dynamics remains obscure. Here, we show that in one widely studied multistable paradigm-binocular rivalry-perceptual stability in individuals is predicted by the frequency of their neural oscillations within the alpha range (7-13 Hz). Our results suggest revising models of rivalry to incorporate effects of neural oscillations on perceptual alternations, and raise the possibility that a common factor may influence dynamics in many neural processes.

Neuronal correlates of auditory streaming in the auditory cortex of behaving monkeys

This study tested the hypothesis that spiking activity in the primary auditory cortex of monkeys is related to auditory stream formation. Evidence for this hypothesis was previously obtained in animals that were passively exposed to stimuli and in which differences in the streaming percept were confounded with differences between the stimuli. In this study, monkeys performed an operant task on sequences that were composed of light flashes and tones. The tones alternated between a high and a low frequency and could be perceived either as one auditory stream or two auditory streams. The flashes promoted either a one-stream percept or a two-stream percept. Comparison of different types of sequences revealed that the neuronal responses to the alternating tones were more similar when the flashes promoted auditory stream integration, and were more dissimilar when the flashes promoted auditory stream segregation. Thus our findings show that the spiking activity in the monkey primary auditory cortex is related to auditory stream formation.

A Generic Mechanism for Perceptual Organization in the Parietal Cortex

Our visual system's ability to group visual elements into meaningful entities and to separate them from others is referred to as scene segmentation. Visual motion often provides a powerful cue for this process as parallax or coherence can inform the visual system about scene or object structure. Here we tested the hypothesis that scene segmentation by motion cues relies on a common neural substrate in the parietal cortex. We used fMRI and a set of three entirely distinct motion stimuli to examine scene segmentation in the human brain. The stimuli covered a wide range of high-level processes, including perceptual grouping, transparent motion, and depth perception. All stimuli were perceptually bistable such that percepts alternated every few seconds while the physical stimulation remained constant. The perceptual states were asymmetric, in that one reflected the default (nonsegmented) interpretation, and the other the non-default (segmented) interpretation. We confirmed behaviorally that upon stimulus presentation, the default percept was always perceived first, before perceptual alternations ensued. Imaging results showed that across all stimulus classes perceptual scene-segmentation was associated with an increase of activity in the posterior parietal cortex together with a decrease of neural signal in the early visual cortex. This pattern of activation is compatible with predictive coding models of visual perception, and suggests that parietal cortex hosts a generic mechanism for scene segmentation.SIGNIFICANCE STATEMENT Making sense of cluttered visual scenes is crucial for everyday perception. An important cue to scene segmentation is visual motion: slight movements of scene elements give away which elements belong to the foreground or background or to the same object. We used three distinct stimuli that engage visual scene segmentation mechanisms based on motion. They involved perceptual grouping, transparent motion, and depth perception. Brain activity associated with all three mechanisms converged in the same parietal region with concurrent deactivation of early visual areas. The results suggest that posterior parietal cortex is a hub involved in structuring visual scenes based on different motion cues, and that feedback modulates early cortical processing in accord with predictive coding theory.

Long-Range Temporal Correlations in Alpha Oscillations Stabilize Perception of Ambiguous Visual Stimuli

Ongoing brain dynamics have been proposed as a type of “neuronal noise” that can trigger perceptual switches when viewing an ambiguous, bistable stimulus. However, no prior study has directly quantified how such neuronal noise relates to the rate of percept reversals. Specifically, it has remained unknown whether individual differences in complexity of resting-state oscillations—as reflected in long-range temporal correlations (LRTC)—are associated with perceptual stability. We hypothesized that participants with stronger resting-state LRTC in the alpha band experience more stable percepts, and thereby fewer perceptual switches. Furthermore, we expected that participants who report less discontinuous thoughts during rest, experience less switches. To test this, we recorded electroencephalography (EEG) in 65 healthy volunteers during 5 min Eyes-Closed Rest (ECR), after which they filled in the Amsterdam Resting-State Questionnaire (ARSQ). This was followed by three conditions where participants attended an ambiguous structure-from-motion stimulus—Neutral (passively observe the stimulus), Hold (the percept for as long as possible), and Switch (as often as possible). LRTC of resting-state alpha oscillations predicted the number of switches only in the Hold condition, with stronger LRTC associated with less switches. Contrary to our expectations, there was no association between resting-state Discontinuity of Mind and percept stability. Participants were capable of controlling switching according to task goals, and this was accompanied by increased alpha power during Hold and decreased power during Switch. Fewer switches were associated with stronger task-related alpha LRTC in all conditions. Together, our data suggest that bistable visual perception is to some extent under voluntary control and influenced by LRTC of alpha oscillations.

The Independent and Shared Mechanisms of Intrinsic Brain Dynamics: Insights From Bistable Perception

In bistable perception, constant input leads to alternating perception. The dynamics of the changing perception reflects the intrinsic dynamic properties of the “unconscious inferential” process in the brain. Under the same condition, individuals differ in how fast they experience the perceptual alternation. In this study, testing many forms of bistable perception in a large number of observers, we investigated the key question of whether there is a general and common mechanism or multiple and independent mechanisms that control the dynamics of the inferential brain. Bistable phenomena tested include binocular rivalry, vase-face, Necker cube, moving plaid, motion induced blindness, biological motion, spinning dancer, rotating cylinder, Lissajous-figure, rolling wheel, and translating diamond. Switching dynamics for each bistable percept was measured in 100 observers. Results show that the switching rates of subsets of bistable percept are highly correlated. The clustering of dynamic properties of some bistable phenomena but not an overall general control of switching dynamics implies that the brain’s inferential processes are both shared and independent – faster in constructing 3D structure from motion does not mean faster in integrating components into an objects.

Can Rotational Grouping Be Determined by the Initial Conditions?

Objects rotating in depth with an ambiguous rotation direction frequently appear to rotate together. Corotation is especially strong when the objects are interpretable as having a shared axis. We manipulated the initial conditions of the experiment by having pairs of objects initially appear to be unambiguous, and then make either a sudden or gradual transition to ambiguous spin. We find that in neither case do coaxial counter-rotating objects persist in being perceived as counter-rotating. This implies that the perceptual constraint that favors coaxial corotation overrides the initial perceptual state of the objects.

Spectral fingerprints of large-scale cortical dynamics during ambiguous motion perception

Ambiguous stimuli have been widely used to study the neuronal correlates of consciousness. Recently, it has been suggested that conscious perception might arise from the dynamic interplay of functionally specialized but widely distributed cortical areas. While previous research mainly focused on phase coupling as a correlate of cortical communication, more recent findings indicated that additional coupling modes might coexist and possibly subserve distinct cortical functions. Here, we studied two coupling modes, namely phase and envelope coupling, which might differ in their origins, putative functions and dynamics. Therefore, we recorded 128-channel EEG while participants performed a bistable motion task and utilized state-of-the-art source-space connectivity analysis techniques to study the functional relevance of different coupling modes for cortical communication. Our results indicate that gamma-band phase coupling in extrastriate visual cortex might mediate the integration of visual tokens into a moving stimulus during ambiguous visual stimulation. Furthermore, our results suggest that long-range fronto-occipital gamma-band envelope coupling sustains the horizontal percept during ambiguous motion perception. Additionally, our results support the idea that local parieto-occipital alpha-band phase coupling controls the inter-hemispheric information transfer. These findings provide correlative evidence for the notion that synchronized oscillatory brain activity reflects the processing of sensory input as well as the information integration across several spatiotemporal scales. The results indicate that distinct coupling modes are involved in different cortical computations and that the rich spatiotemporal correlation structure of the brain might constitute the functional architecture for cortical processing and specific multi-site communication. Hum Brain Mapp 37:4099-4111, 2016. © 2016 Wiley Periodicals, Inc.

Category Selectivity of Human Visual Cortex in Perception of Rubin Face-Vase Illusion

When viewing the Rubin face-vase illusion, our conscious perception spontaneously alternates between the face and the vase; this illusion has been widely used to explore bistable perception. Previous functional magnetic resonance imaging (fMRI) studies have studied the neural mechanisms underlying bistable perception through univariate and multivariate pattern analyses; however, no studies have investigated the issue of category selectivity. Here, we used fMRI to investigate the neural mechanisms underlying the Rubin face-vase illusion by introducing univariate amplitude and multivariate pattern analyses. The results from the amplitude analysis suggested that the activity in the fusiform face area was likely related to the subjective face perception. Furthermore, the pattern analysis results showed that the early visual cortex (EVC) and the face-selective cortex could discriminate the activity patterns of the face and vase perceptions. However, further analysis of the activity patterns showed that only the face-selective cortex contains the face information. These findings indicated that although the EVC and face-selective cortex activities could discriminate the visual information, only the activity and activity pattern in the face-selective areas contained the category information of face perception in the Rubin face-vase illusion.

Attention model of binocular rivalry

When the corresponding retinal locations in the two eyes are presented with incompatible images, a stable percept gives way to perceptual alternations in which the two images compete for perceptual dominance. As perceptual experience evolves dynamically under constant external inputs, binocular rivalry has been used for studying intrinsic cortical computations and for understanding how the brain regulates competing inputs. Converging behavioral and EEG results have shown that binocular rivalry and attention are intertwined: binocular rivalry ceases when attention is diverted away from the rivalry stimuli. In addition, the competing image in one eye suppresses the target in the other eye through a pattern of gain changes similar to those induced by attention. These results require a revision of the current computational theories of binocular rivalry, in which the role of attention is ignored. Here, we provide a computational model of binocular rivalry. In the model, competition between two images in rivalry is driven by both attentional modulation and mutual inhibition, which have distinct selectivity (feature vs. eye of origin) and dynamics (relatively slow vs. relatively fast). The proposed model explains a wide range of phenomena reported in rivalry, including the three hallmarks: (i) binocular rivalry requires attention; (ii) various perceptual states emerge when the two images are swapped between the eyes multiple times per second; (iii) the dominance duration as a function of input strength follows Levelt's propositions. With a bifurcation analysis, we identified the parameter space in which the model's behavior was consistent with experimental results.

Changing What You See by Changing What You Know: The Role of Attention

Attending is a cognitive process that incorporates a person's knowledge, goals, and expectations. What we perceive when we attend to one thing is different from what we perceive when we attend to something else. Yet, it is often argued that attentional effects do not count as evidence that perception is influenced by cognition. I investigate two arguments often given to justify excluding attention. The first is arguing that attention is a post-perceptual process reflecting selection between fully constructed perceptual representations. The second is arguing that attention as a pre-perceptual process that simply changes the input to encapsulated perceptual systems. Both of these arguments are highly problematic. Although some attentional effects can indeed be construed as post-perceptual, others operate by changing perceptual content across the entire visual hierarchy. Although there is a natural analogy between spatial attention and a change of input, the analogy falls apart when we consider other forms of attention. After dispelling these arguments, I make a case for thinking of attention not as a confound, but as one of the mechanisms by which cognitive states affect perception by going through cases in which the same or similar visual inputs are perceived differently depending on the observer's cognitive state, and instances where cuing an observer using language affects what one sees. Lastly, I provide two compelling counter-examples to the critique that although cognitive influences on perception can be demonstrated in the laboratory, it is impossible to really experience them for oneself in a phenomenologically compelling way. Taken together, the current evidence strongly supports the thesis that what we know routinely influences what we see, that the same sensory input can be perceived differently depending on the current cognitive state of the viewer, and that phenomenologically salient demonstrations are possible if certain conditions are met.

Binocular rivalry in children on the autism spectrum

When different images are presented to the eyes, the brain is faced with ambiguity, causing perceptual bistability: visual perception continuously alternates between the monocular images, a phenomenon called binocular rivalry. Many models of rivalry suggest that its temporal dynamics depend on mutual inhibition among neurons representing competing images. These models predict that rivalry should be different in autism, which has been proposed to present an atypical ratio of excitation and inhibition [the E/I imbalance hypothesis; Rubenstein & Merzenich, 2003]. In line with this prediction, some recent studies have provided evidence for atypical binocular rivalry dynamics in autistic adults. In this study, we examined if these findings generalize to autistic children. We developed a child‐friendly binocular rivalry paradigm, which included two types of stimuli, low‐ and high‐complexity, and compared rivalry dynamics in groups of autistic and age‐ and intellectual ability‐matched typical children. Unexpectedly, the two groups of children presented the same number of perceptual transitions and the same mean phase durations (times perceiving one of the two stimuli). Yet autistic children reported mixed percepts for a shorter proportion of time (a difference which was in the opposite direction to previous adult studies), while elevated autistic symptomatology was associated with shorter mixed perception periods. Rivalry in the two groups was affected similarly by stimulus type, and consistent with previous findings. Our results suggest that rivalry dynamics are differentially affected in adults and developing autistic children and could be accounted for by hierarchical models of binocular rivalry, including both inhibition and top‐down influences. Autism Res2017. ©2017 The Authors Autism Research published by Wiley Periodicals, Inc. on behalf of International Society for Autism Research Autism Res 2017, 10: 1096–1106. © 2017 International Society for Autism Research, Wiley Periodicals, Inc.

Learning What to See in a Changing World

Visual perception is strongly shaped by expectations, but it is poorly understood how such perceptual expectations are learned in our dynamic sensory environment. Here, we applied a Bayesian framework to investigate whether perceptual expectations are continuously updated from different aspects of ongoing experience. In two experiments, human observers performed an associative learning task in which rapidly changing expectations about the appearance of ambiguous stimuli were induced. We found that perception of ambiguous stimuli was biased by both learned associations and previous perceptual outcomes. Computational modeling revealed that perception was best explained by a model that continuously updated priors from associative learning and perceptual history and combined these priors with the current sensory information in a probabilistic manner. Our findings suggest that the construction of visual perception is a highly dynamic process that incorporates rapidly changing expectations from different sources in a manner consistent with Bayesian learning and inference.

Quantum-like model of unconscious-conscious dynamics

We present a quantum-like model of sensation–perception dynamics (originated in Helmholtz theory of unconscious inference) based on the theory of quantum apparatuses and instruments. We illustrate our approach with the model of bistable perception of a particular ambiguous figure, the Schröder stair. This is a concrete model for unconscious and conscious processing of information and their interaction. The starting point of our quantum-like journey was the observation that perception dynamics is essentially contextual which implies impossibility of (straightforward) embedding of experimental statistical data in the classical (Kolmogorov, 1933) framework of probability theory. This motivates application of nonclassical probabilistic schemes. And the quantum formalism provides a variety of the well-approved and mathematically elegant probabilistic schemes to handle results of measurements. The theory of quantum apparatuses and instruments is the most general quantum scheme describing measurements and it is natural to explore it to model the sensation–perception dynamics. In particular, this theory provides the scheme of indirect quantum measurements which we apply to model unconscious inference leading to transition from sensations to perceptions.

Imagine that: elevated sensory strength of mental imagery in individuals with Parkinson's disease and visual hallucinations

Visual hallucinations occur when our conscious experience does not accurately reflect external reality. However, these dissociations also regularly occur when we imagine the world around us in the absence of visual stimulation. We used two novel behavioural paradigms to objectively measure visual hallucinations and voluntary mental imagery in 19 individuals with Parkinson's disease (ten with visual hallucinations; nine without) and ten healthy, age-matched controls. We then used this behavioural overlap to interrogate the connectivity both within and between the major attentional control networks using resting-state functional magnetic resonance imaging. Patients with visual hallucinations had elevated mental imagery strength compared with patients without hallucinations and controls. Specifically, the sensory strength of imagery predicted the frequency of visual hallucinations. Together, hallucinations and mental imagery predicted multiple abnormalities in functional connectivity both within and between the attentional control networks, as measured with resting-state functional magnetic resonance imaging. However, the two phenomena were also dissociable at the neural level, with both mental imagery and visual misperceptions associated with specific abnormalities in attentional network connectivity. Our results provide the first evidence of both the shared and unique neural correlates of these two similar, yet distinct phenomena.

An alternating renewal process describes the buildup of perceptual segregation

For some ambiguous scenes perceptual conflict arises between integration and segregation. Initially, all stimulus features seem integrated. Then abruptly, perhaps after a few seconds, a segregated percept emerges. For example, segregation of acoustic features into streams may require several seconds. In behavioral experiments, when a subject's reports of stream segregation are averaged over repeated trials, one obtains a buildup function, a smooth time course for segregation probability. The buildup function has been said to reflect an underlying mechanism of evidence accumulation or adaptation. During long duration stimuli perception may alternate between integration and segregation. We present a statistical model based on an alternating renewal process (ARP) that generates buildup functions without an accumulative process. In our model, perception alternates during a trial between different groupings, as in perceptual bistability, with random and independent dominance durations sampled from different percept-specific probability distributions. Using this theory, we describe the short-term dynamics of buildup observed on short trials in terms of the long-term statistics of percept durations for the two alternating perceptual organizations. Our statistical-dynamics model describes well the buildup functions and alternations in simulations of pseudo-mechanistic neuronal network models with percept-selective populations competing through mutual inhibition. Even though the competition model can show history dependence through slow adaptation, our statistical switching model, that neglects history, predicts well the buildup function. We propose that accumulation is not a necessary feature to produce buildup. Generally, if alternations between two states exhibit independent durations with stationary statistics then the associated buildup function can be described by the statistical dynamics of an ARP.

Top-down modulation in human visual cortex predicts the stability of a perceptual illusion

Conscious perception sometimes fluctuates strongly, even when the sensory input is constant. For example, in motion-induced blindness (MIB), a salient visual target surrounded by a moving pattern suddenly disappears from perception, only to reappear after some variable time. Whereas such changes of perception result from fluctuations of neural activity, mounting evidence suggests that the perceptual changes, in turn, may also cause modulations of activity in several brain areas, including visual cortex. In this study, we asked whether these latter modulations might affect the subsequent dynamics of perception. We used magnetoencephalography (MEG) to measure modulations in cortical population activity during MIB. We observed a transient, retinotopically widespread modulation of beta (12-30 Hz)-frequency power over visual cortex that was closely linked to the time of subjects' behavioral report of the target disappearance. This beta modulation was a top-down signal, decoupled from both the physical stimulus properties and the motor response but contingent on the behavioral relevance of the perceptual change. Critically, the modulation amplitude predicted the duration of the subsequent target disappearance. We propose that the transformation of the perceptual change into a report triggers a top-down mechanism that stabilizes the newly selected perceptual interpretation.

Auditory event-related potentials associated with perceptual reversals of bistable pitch motion

Previous event-related potential (ERP) experiments have consistently identified two components associated with perceptual transitions of bistable visual stimuli, the "reversal negativity" (RN) and the "late positive complex" (LPC). The RN (~200 ms post-stimulus, bilateral occipital-parietal distribution) is thought to reflect transitions between neural representations that form the moment-to-moment contents of conscious perception, while the LPC (~400 ms, central-parietal) is considered an index of post-perceptual processing related to accessing and reporting one's percept. To explore the generality of these components across sensory modalities, the present experiment utilized a novel bistable auditory stimulus. Pairs of complex tones with ambiguous pitch relationships were presented sequentially while subjects reported whether they perceived the tone pairs as ascending or descending in pitch. ERPs elicited by the tones were compared according to whether perceived pitch motion changed direction or remained the same across successive trials. An auditory reversal negativity (aRN) component was evident at ~170 ms post-stimulus over bilateral fronto-central scalp locations. An auditory LPC component (aLPC) was evident at subsequent latencies (~350 ms, fronto-central distribution). These two components may be auditory analogs of the visual RN and LPC, suggesting functionally equivalent but anatomically distinct processes in auditory vs. visual bistable perception.

Probing the time course of head-motion cues integration during auditory scene analysis

The perceptual organization of auditory scenes is a hard but important problem to solve for human listeners. It is thus likely that cues from several modalities are pooled for auditory scene analysis, including sensory-motor cues related to the active exploration of the scene. We previously reported a strong effect of head motion on auditory streaming. Streaming refers to an experimental paradigm where listeners hear sequences of pure tones, and rate their perception of one or more subjective sources called streams. To disentangle the effects of head motion (changes in acoustic cues at the ear, subjective location cues, and motor cues), we used a robotic telepresence system, Telehead. We found that head motion induced perceptual reorganization even when the acoustic scene had not changed. Here we reanalyzed the same data to probe the time course of sensory-motor integration. We show that motor cues had a different time course compared to acoustic or subjective location cues: motor cues impacted perceptual organization earlier and for a shorter time than other cues, with successive positive and negative contributions to streaming. An additional experiment controlled for the effects of volitional anticipatory components, and found that arm or leg movements did not have any impact on scene analysis. These data provide a first investigation of the time course of the complex integration of sensory-motor cues in an auditory scene analysis task, and they suggest a loose temporal coupling between the different mechanisms involved.

Online tracking of the contents of conscious perception using real-time fMRI

Perception is an active process that interprets and structures the stimulus input based on assumptions about its possible causes. We use real-time functional magnetic resonance imaging (rtfMRI) to investigate a particularly powerful demonstration of dynamic object integration in which the same physical stimulus intermittently elicits categorically different conscious object percepts. In this study, we simulated an outline object that is moving behind a narrow slit. With such displays, the physically identical stimulus can elicit categorically different percepts that either correspond closely to the physical stimulus (vertically moving line segments) or represent a hypothesis about the underlying cause of the physical stimulus (a horizontally moving object that is partly occluded). In the latter case, the brain must construct an object from the input sequence. Combining rtfMRI with machine learning techniques we show that it is possible to determine online the momentary state of a subject's conscious percept from time resolved BOLD-activity. In addition, we found that feedback about the currently decoded percept increased the decoding rates compared to prior fMRI recordings of the same stimulus without feedback presentation. The analysis of the trained classifier revealed a brain network that discriminates contents of conscious perception with antagonistic interactions between early sensory areas that represent physical stimulus properties and higher-tier brain areas. During integrated object percepts, brain activity decreases in early sensory areas and increases in higher-tier areas. We conclude that it is possible to use BOLD responses to reliably track the contents of conscious visual perception with a relatively high temporal resolution. We suggest that our approach can also be used to investigate the neural basis of auditory object formation and discuss the results in the context of predictive coding theory.

Do audio-visual motion cues promote segregation of auditory streams?

An audio-visual experiment using moving sound sources was designed to investigate whether the analysis of auditory scenes is modulated by synchronous presentation of visual information. Listeners were presented with an alternating sequence of two pure tones delivered by two separate sound sources. In different conditions, the two sound sources were either stationary or moving on random trajectories around the listener. Both the sounds and the movement trajectories were derived from recordings in which two humans were moving with loudspeakers attached to their heads. Visualized movement trajectories modeled by a computer animation were presented together with the sounds. In the main experiment, behavioral reports on sound organization were collected from young healthy volunteers. The proportion and stability of the different sound organizations were compared between the conditions in which the visualized trajectories matched the movement of the sound sources and when the two were independent of each other. The results corroborate earlier findings that separation of sound sources in space promotes segregation. However, no additional effect of auditory movement per se on the perceptual organization of sounds was obtained. Surprisingly, the presentation of movement-congruent visual cues did not strengthen the effects of spatial separation on segregating auditory streams. Our findings are consistent with the view that bistability in the auditory modality can occur independently from other modalities.

Predictability effects in auditory scene analysis: a review

Many sound sources emit signals in a predictable manner. The idea that predictability can be exploited to support the segregation of one source's signal emissions from the overlapping signals of other sources has been expressed for a long time. Yet experimental evidence for a strong role of predictability within auditory scene analysis (ASA) has been scarce. Recently, there has been an upsurge in experimental and theoretical work on this topic resulting from fundamental changes in our perspective on how the brain extracts predictability from series of sensory events. Based on effortless predictive processing in the auditory system, it becomes more plausible that predictability would be available as a cue for sound source decomposition. In the present contribution, empirical evidence for such a role of predictability in ASA will be reviewed. It will be shown that predictability affects ASA both when it is present in the sound source of interest (perceptual foreground) and when it is present in other sound sources that the listener wishes to ignore (perceptual background). First evidence pointing toward age-related impairments in the latter capacity will be addressed. Moreover, it will be illustrated how effects of predictability can be shown by means of objective listening tests as well as by subjective report procedures, with the latter approach typically exploiting the multi-stable nature of auditory perception. Critical aspects of study design will be delineated to ensure that predictability effects can be unambiguously interpreted. Possible mechanisms for a functional role of predictability within ASA will be discussed, and an analogy with the old-plus-new heuristic for grouping simultaneous acoustic signals will be suggested.

Neural substrates of perceptual integration during bistable object perception

The way we perceive an object depends both on feedforward, bottom-up processing of its physical stimulus properties and on top-down factors such as attention, context, expectation, and task relevance. Here we compared neural activity elicited by varying perceptions of the same physical image--a bistable moving image in which perception spontaneously alternates between dissociated fragments and a single, unified object. A time-frequency analysis of EEG changes associated with the perceptual switch from object to fragment and vice versa revealed a greater decrease in alpha (8-12 Hz) accompanying the switch to object percept than to fragment percept. Recordings of event-related potentials elicited by irrelevant probes superimposed on the moving image revealed an enhanced positivity between 184 and 212 ms when the probes were contained within the boundaries of the perceived unitary object. The topography of the positivity (P2) in this latency range elicited by probes during object perception was distinct from the topography elicited by probes during fragment perception, suggesting that the neural processing of probes differed as a function of perceptual state. Two source localization algorithms estimated the neural generator of this object-related difference to lie in the lateral occipital cortex, a region long associated with object perception. These data suggest that perceived objects attract attention, incorporate visual elements occurring within their boundaries into unified object representations, and enhance the visual processing of elements occurring within their boundaries. Importantly, the perceived object in this case emerged as a function of the fluctuating perceptual state of the viewer.