Temporal windows of unconscious processing cannot easily be disrupted

Conscious perception is preceded by long periods of unconscious processing. These periods are crucial for analyzing temporal information and for solving the many ill-posed problems of vision. An important question is what starts and ends these windows and how they may be interrupted. Most experimental paradigms do not offer the methodology required for such investigation. Here, we used the sequential metacontrast paradigm, in which two streams of lines, expanding from the center to the periphery, are presented, and participants are asked to attend to one of the motion streams. If several lines in the attended motion stream are offset, the offsets are known to integrate mandatorily and unconsciously, even if separated by up to 450 ms. Using this paradigm, we here found that external visual objects, such as an annulus, presented during the motion stream, do not disrupt mandatory temporal integration. Thus, if a window is started once, it appears to remain open even in the presence of disruptions that are known to interrupt visual processes normally. Further, we found that interrupting the motion stream with a gap disrupts temporal integration but does not terminate the overall unconscious processing window. Thus, while temporal integration is key to unconscious processing, not all stimuli in the same processing window are integrated together. These results strengthen the case for unconscious processing taking place in windows of sensemaking, during which temporal integration occurs in a flexible and perceptually meaningful manner.

Weak correlations between visual abilities in healthy older adults, despite long-term performance stability

Using batteries of visual tests, most studies have found that there are only weak correlations between the performance levels of the tests. Factor analysis has confirmed these results. This means that a participant excelling in one test may rank low in another test. Hence, there is very little evidence for a common factor in vision. In visual aging research, cross-sectional studies have repeatedly found that healthy older adults' performance is strongly deteriorated in most visual tests compared to young adults. However, also within the healthy older population, there is no evidence for a visual common factor. To investigate whether the weak between-tests correlations are due to fluctuations in individual performance throughout time, we conducted a longitudinal study. Healthy older adults performed a battery of eight visual tests, with two re-tests after approximately four and seven years. Pearson's, Spearman's and intraclass correlations of most visual tests were significant across the three testing, indicating that the tests are reliable and individual differences are stable across years. Yet, we found low between-tests correlations at each visit, which is consistent with previous studies finding no evidence for a visual common factor. Our results exclude the possibility that the weak correlations between tests are due to high within-individual variance across time.

Pre-stimulus alpha activity modulates long-lasting unconscious feature integration

Pre-stimulus alpha (α) activity can influence perception of shortly presented, low-contrast stimuli. The underlying mechanisms are often thought to affect perception exactly at the time of presentation. In addition, it is suggested that α cycles determine temporal windows of integration. However, in everyday situations, stimuli are usually presented for periods longer than ∼100 ms and perception is often an integration of information across space and time. Moving objects are just one example. Hence, the question is whether α activity plays a role also in temporal integration, especially when stimuli are integrated over several α cycles. Using electroencephalography (EEG), we investigated the relationship between pre-stimulus brain activity and long-lasting integration in the sequential metacontrast paradigm (SQM), where two opposite vernier offsets, embedded in a stream of lines, are unconsciously integrated into a single percept. We show that increases in α power, even 300 ms before the stimulus, affected the probability of reporting the first offset, shown at the very beginning of the SQM. This effect was mediated by the systematic slowing of the α rhythm that followed the peak in α power. No phase effects were found. Together, our results demonstrate a cascade of neural changes, following spontaneous bursts of α activity and extending beyond a single moment, which influences the sensory representation of visual features for hundreds of milliseconds. Crucially, as feature integration in the SQM occurs before a conscious percept is elicited, this also provides evidence that α activity is linked to mechanisms regulating unconscious processing.

Processing load, and not stimulus evidence, determines the duration of unconscious visual feature integration

Integration across space and time is essential for the analysis of motion, low contrast, and many more stimuli. A crucial question is what determines the duration of integration. Based on classical models of decision-making, one might expect that integration terminates as soon as sufficient evidence about a stimulus is accumulated and a threshold is crossed. However, there is very little research on this question as most experimental paradigms cannot monitor processing following stimulus presentation. In particular, it is difficult to determine when processing terminates. Here, using the sequential metacontrast paradigm (SQM), in which information is mandatorily integrated along motion trajectories, we show that the processing load determines the extent of integration but that evidence accumulation does not. Further, the extent of integration is determined by absolute time instead of the number of elements presented. These results have important implications for understanding the time course and mechanisms of temporal integration.

Do we really measure what we think we are measuring?

Tests used in the empirical sciences are often (implicitly) assumed to be representative of a given research question in the sense that similar tests should lead to similar results. Here, we show that this assumption is not always valid. We illustrate our argument with the example of resting-state electroencephalogram (EEG). We used multiple analysis methods, contrary to typical EEG studies where one analysis method is used. We found, first, that many EEG features correlated significantly with cognitive tasks. However, these EEG features correlated weakly with each other. Similarly, in a second analysis, we found that many EEG features were significantly different in older compared to younger participants. When we compared these EEG features pairwise, we did not find strong correlations. In addition, EEG features predicted cognitive tasks poorly as shown by cross-validated regression analysis. We discuss several explanations of these results.

Intact and deficient contextual processing in schizophrenia patients

Schizophrenia patients are known to have deficits in contextual vision. However, results are often very mixed. In some paradigms, patients do not take the context into account and, hence, perform more veridically than healthy controls. In other paradigms, context deteriorates performance much more strongly in patients compared to healthy controls. These mixed results may be explained by differences in the paradigms as well as by small or biased samples, given the large heterogeneity of patients' deficits. Here, we show that mixed results may also come from idiosyncrasies of the stimuli used because in variants of the same visual paradigm, tested with the same participants, we found intact and deficient processing.

The EEG multiverse of schizophrenia

Research on schizophrenia typically focuses on one paradigm for which clear-cut differences between patients and controls are established. Great efforts are made to understand the underlying genetical, neurophysiological, and cognitive mechanisms, which eventually may explain the clinical outcome. One tacit assumption of these "deep rooting" approaches is that paradigms tap into common and representative aspects of the disorder. Here, we analyzed the resting-state electroencephalogram (EEG) of 121 schizophrenia patients and 75 controls. Using multiple signal processing methods, we extracted 194 EEG features. Sixty-nine out of the 194 EEG features showed a significant difference between patients and controls, indicating that these features detect an important aspect of schizophrenia. Surprisingly, the correlations between these features were very low. We discuss several explanations to our results and propose that complementing "deep" with "shallow" rooting approaches might help in understanding the underlying mechanisms of the disorder.

No Common Factor Underlying Decline of Visual Abilities in Mild Cognitive Impairment

INTRODUCTION

Recent work has shown an association between cognitive and visual impairments and two main theories were advanced, namely the sensory deprivation and the common cause theories. Most studies considered only basic visual functions such as visual acuity or visual field size and evaluated the association with dementia.

OBJECTIVES

To reconcile between these theories and to test the link between visual and cognitive decline in mildly cognitive impaired people.

METHODS

We employed a battery of 19 visual tasks on 39 older adults with mild cognitive impairment and 91 without any evidence of cognitive decline, as measured by the Montreal Cognitive Assessment.

RESULTS

Our results show a strong association between visual impairment and mild cognitive impairment. In agreement with previous results with younger and healthy older adults, we found also only weak correlations between most tests in older adults with mild cognitive impairment.

CONCLUSION

Our results suggest that visual and cognitive abilities decline simultaneously, but they do so independently across visual and cognitive functions and across participants.

The Irreducibility of Vision: Gestalt, Crowding and the Fundamentals of Vision

What is fundamental in vision has been discussed for millennia. For philosophical realists and the physiological approach to vision, the objects of the outer world are truly given, and failures to perceive objects properly, such as in illusions, are just sporadic misperceptions. The goal is to replace the subjectivity of the mind by careful physiological analyses. Continental philosophy and the Gestaltists are rather skeptical or ignorant about external objects. The percepts themselves are their starting point, because it is hard to deny the truth of one own′s percepts. I will show that, whereas both approaches can well explain many visual phenomena with classic visual stimuli, they both have trouble when stimuli become slightly more complex. I suggest that these failures have a deeper conceptual reason, namely that their foundations (objects, percepts) do not hold true. I propose that only physical states exist in a mind independent manner and that everyday objects, such as bottles and trees, are perceived in a mind-dependent way. The fundamental processing units to process objects are extended windows of unconscious processing, followed by short, discrete conscious percepts.

Embedded figures in schizophrenia: A main deficit but no specificity

Visual deficits are core deficits of schizophrenia. Classically, deficits are determined with demanding psychophysical tasks requiring fine-grained spatial or temporal resolution. Less is known about holistic processing. Here, we employed the Leuven Embedded Figures Test (L-EFT) measuring classic aspects of Gestalt processing. A target shape is embedded in a context and observers have to detect as quickly as possible in which display the target is embedded. Targets vary in closure, symmetry, complexity, and good continuation. In all conditions, schizophrenia patients had longer RTs compared to controls and depressive patients and to a lesser extent compared to their siblings. There was no interaction suggesting that, once the main deficit of schizophrenia patients is discarded, there are no further deficits in Gestalt perception between the groups. This result is in line with a growing line of research showing that when schizophrenia patients are given sufficient time to accomplish the task, they perform as well as controls.

First-person experience cannot rescue causal structure theories from the unfolding argument

We recently put forward an argument, the Unfolding Argument (UA), that integrated information theory (IIT) and other causal structure theories are either already falsified or unfalsifiable, which provoked significant criticism. It seems that we and the critics agree that the main question in this debate is whether first-person experience, independent of third-person data, is a sufficient foundation for theories of consciousness. Here, we argue that pure first-person experience cannot be a scientific foundation for IIT because science relies on taking measurements, and pure first-person experience is not measurable except through reports, brain activity, and the relationship between them. We also argue that pure first-person experience cannot be taken as ground truth because science is about backing up theories with data, not about asserting that we have ground truth independent of data. Lastly, we explain why no experiment based on third-person data can test IIT as a theory of consciousness. IIT may be a good theory of something, but not of consciousness. We conclude by exposing a deeper reason for the above conclusions: IIT's consciousness is by construction fully dissociated from any measurable thing and, for this reason, IIT implies that both the level and content of consciousness are epiphenomenal, with no causal power. IIT and other causal structure theories end up in a form of dissociative epiphenomenalism, in which we cannot even trust reports about first-person experiences. But reports about first-person experiences are taken as ground truth and the foundation for IIT's axioms. Therefore, accepting IIT leads to rejecting its own axioms. We also respond to several other criticisms against the UA.

Information Integration and Information Storage in Retinotopic and Non-Retinotopic Sensory Memory

The first stage of the Atkinson–Shiffrin model of human memory is a sensory memory (SM). The visual component of the SM was shown to operate within a retinotopic reference frame. However, a retinotopic SM (rSM) is unable to account for vision under natural viewing conditions because, for example, motion information needs to be analyzed across space and time. For this reason, the SM store of the Atkinson–Shiffrin model has been extended to include a non-retinotopic component (nrSM). In this paper, we analyze findings from two experimental paradigms and show drastically different properties of rSM and nrSM. We show that nrSM involves complex processes such as motion-based reference frames and Gestalt grouping, which establish object identities across space and time. We also describe a quantitative model for nrSM and show drastic differences between the spatio-temporal properties of rSM and nrSM. Since the reference-frame of the latter is non-retinotopic and motion-stream based, we suggest that the spatiotemporal properties of the nrSM are in accordance with the spatiotemporal properties of the motion system. Overall, these findings indicate that, unlike the traditional rSM, which is a relatively passive store, nrSM exhibits sophisticated processing properties to manage the complexities of ecological perception.

Features integrate along a motion trajectory when object integrity is preserved

Information about a moving object is usually poor at each retinotopic location because photoreceptor activation is short, noisy, and affected by shadows, reflections of other objects, and so on. Integration across the motion trajectory may yield a much better estimate about the objects’ features. Using the sequential metacontrast paradigm, we have shown previously that features, indeed, integrate along a motion trajectory in a long-lasting window of unconscious processing. In the sequential metacontrast paradigm, a percept of two diverging streams is elicited by the presentation of a central line followed by a sequence of flanking pairs of lines. When several lines are spatially offset, the offsets integrate mandatorily for several hundreds of milliseconds along the motion trajectory of the streams. We propose that, within these long-lasting windows, stimuli are first grouped based on Gestalt principles of grouping. These processes establish reference frames that are used to attribute features. Features are then integrated following their respective reference frame. Here using occlusion and bouncing effects, we show that indeed such grouping operations are in place. We found that features integrate only when the spatiotemporal integrity of the object is preserved. Moreover, when several moving objects are present, only features belonging to the same object integrate. Overall, our results show that feature integration is a deliberate strategy of the brain and long-lasting windows of processing can be seen as periods of sense making.

Global and high-level effects in crowding cannot be predicted by either high-dimensional pooling or target cueing

In visual crowding, the perception of a target deteriorates in the presence of nearby flankers. Traditionally, target-flanker interactions have been considered as local, mostly deleterious, low-level, and feature specific, occurring when information is pooled along the visual processing hierarchy. Recently, a vast literature of high-level effects in crowding (grouping effects and face-holistic crowding in particular) led to a different understanding of crowding, as a global, complex, and multilevel phenomenon that cannot be captured or explained by simple pooling models. It was recently argued that these high-level effects may still be captured by more sophisticated pooling models, such as the Texture Tiling model (TTM). Unlike simple pooling models, the high-dimensional pooling stage of the TTM preserves rich information about a crowded stimulus and, in principle, this information may be sufficient to drive high-level and global aspects of crowding. In addition, it was proposed that grouping effects in crowding may be explained by post-perceptual target cueing. Here, we extensively tested the predictions of the TTM on the results of six different studies that highlighted high-level effects in crowding. Our results show that the TTM cannot explain any of these high-level effects, and that the behavior of the model is equivalent to a simple pooling model. In addition, we show that grouping effects in crowding cannot be predicted by post-perceptual factors, such as target cueing. Taken together, these results reinforce once more the idea that complex target-flanker interactions determine crowding and that crowding occurs at multiple levels of the visual hierarchy.

Dissecting (un)crowding

In crowding, perception of a target deteriorates in the presence of nearby flankers. Surprisingly, perception can be rescued from crowding if additional flankers are added (uncrowding). Uncrowding is a major challenge for all classic models of crowding and vision in general, because the global configuration of the entire stimulus is crucial. However, it is unclear which characteristics of the configuration impact (un)crowding. Here, we systematically dissected flanker configurations and showed that (un)crowding cannot be easily explained by the effects of the sub-parts or low-level features of the stimulus configuration. Our modeling results suggest that (un)crowding requires global processing. These results are well in line with previous studies showing the importance of global aspects in crowding.

A comparative biology approach to DNN modeling of vision: A focus on differences, not similarities

Deep neural networks (DNNs) have revolutionized computer science and are now widely used for neuroscientific research. A hot debate has ensued about the usefulness of DNNs as neuroscientific models of the human visual system; the debate centers on to what extent certain shortcomings of DNNs are real failures and to what extent they are redeemable. Here, we argue that the main problem is that we often do not understand which human functions need to be modeled and, thus, what counts as a falsification. Hence, not only is there a problem on the DNN side, but there is also one on the brain side (i.e., with the explanandum-the thing to be explained). For example, should DNNs reproduce illusions? We posit that we can make better use of DNNs by adopting an approach of comparative biology by focusing on the differences, rather than the similarities, between DNNs and humans to improve our understanding of visual information processing in general.

Adaptive mechanisms of visual motion discrimination, integration, and segregation

Under ecological conditions, the luminance impinging on the retina varies within a dynamic range of 220 dB. Stimulus contrast can also vary drastically within a scene and eye movements leave little time for sampling luminance. Given these fundamental problems, the human brain allocates a significant amount of resources and deploys both structural and functional solutions that work in tandem to compress this range. Here we propose a new dynamic neural model built upon well-established canonical neural mechanisms. The model consists of two feed-forward stages. The first stage encodes the stimulus spatially and normalizes its activity by extracting contrast and discounting the background luminance. These normalized activities allow a second stage to implement a contrast-dependent spatial-integration strategy. We show how the properties of this model can account for adaptive properties of motion discrimination, integration, and segregation.

How do visual skills relate to action video game performance?

It has been claimed that video gamers possess increased perceptual and cognitive skills compared to non-video gamers. Here, we examined to which extent gaming performance in CS:GO (Counter-Strike: Global Offensive) correlates with visual performance. We tested 94 players ranging from beginners to experts with a battery of visual paradigms, such as visual acuity and contrast detection. In addition, we assessed performance in specific gaming skills, such as shooting and tracking, and administered personality traits. All measures together explained about 70% of the variance of the players’ rank. In particular, regression models showed that a few visual abilities, such as visual acuity in the periphery and the susceptibility to the Honeycomb illusion, were strongly associated with the players’ rank. Although the causality of the effect remains unknown, our results show that high-rank players perform better in certain visual skills compared to low-rank players.

Shrinking Bouma's window: How to model crowding in dense displays

In crowding, perception of a target deteriorates in the presence of nearby flankers. Traditionally, it is thought that visual crowding obeys Bouma's law, i.e., all elements within a certain distance interfere with the target, and that adding more elements always leads to stronger crowding. Crowding is predominantly studied using sparse displays (a target surrounded by a few flankers). However, many studies have shown that this approach leads to wrong conclusions about human vision. Van der Burg and colleagues proposed a paradigm to measure crowding in dense displays using genetic algorithms. Displays were selected and combined over several generations to maximize human performance. In contrast to Bouma's law, only the target's nearest neighbours affected performance. Here, we tested various models to explain these results. We used the same genetic algorithm, but instead of selecting displays based on human performance we selected displays based on the model's outputs. We found that all models based on the traditional feedforward pooling framework of vision were unable to reproduce human behaviour. In contrast, all models involving a dedicated grouping stage explained the results successfully. We show how traditional models can be improved by adding a grouping stage.

A guideline for linking brain wave findings to the various aspects of discrete perception

Brain waves, determined by electrical and magnetic brain recordings (e.g., EEG and MEG), and fluctuating behavioral responses, determined by response time or accuracy measures, are frequently taken to support discrete perception. For example, it has been proposed that humans experience only one conscious percept per brain wave (e.g., during one alpha cycle). However, the proposed link between brain waves and discrete perception is typically rather vague. More importantly, there are many models and aspects of discrete perception and it is often not apparent in what theoretical framework brain wave findings are interpreted and to what specific aspects of discrete perception they relate. Here, we review different approaches to discrete perception and highlight issues with particular interpretations. We then discuss how certain findings on brain waves may relate to certain aspects of discrete perception. The main purpose of this meta‐contribution is to give a short overview of discrete models of perception and to illustrate the need to make explicit what aspects of discrete theories are addressed by what aspects of brain wave findings.

Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making

Classic reinforcement learning (RL) theories cannot explain human behavior in the absence of external reward or when the environment changes. Here, we employ a deep sequential decision-making paradigm with sparse reward and abrupt environmental changes. To explain the behavior of human participants in these environments, we show that RL theories need to include surprise and novelty, each with a distinct role. While novelty drives exploration before the first encounter of a reward, surprise increases the rate of learning of a world-model as well as of model-free action-values. Even though the world-model is available for model-based RL, we find that human decisions are dominated by model-free action choices. The world-model is only marginally used for planning, but it is important to detect surprising events. Our theory predicts human action choices with high probability and allows us to dissociate surprise, novelty, and reward in EEG signals.

Individual differences in the perception of visual illusions are stable across eyes, time, and measurement methods

Vision scientists have tried to classify illusions for more than a century. For example, some studies suggested that there is a unique common factor for all visual illusions. Other studies proposed that there are several subclasses of illusions, such as illusions of linear extent or distortions. We previously observed strong within-illusion correlations but only weak between-illusion correlations, arguing in favor of an even higher multifactorial space with-more or less-each illusion making up its own factor. These mixed results are surprising. Here, we examined to what extent individual differences in the perception of visual illusions are stable across eyes, time, and measurement methods. First, we did not find any significant differences in the magnitudes of the seven illusions tested with monocular or binocular viewing conditions. In addition, illusion magnitudes were not significantly predicted by visual acuity. Second, we observed stable individual differences over time. Last, we compared two illusion measurements, namely an adjustment procedure and a method of constant stimuli, which both led to similar individual differences. Hence, it is unlikely that the individual differences in the perception of visual illusions arise from instability across eyes, time, and measurement methods.

Electrophysiological correlates of visual backward masking in patients with bipolar disorder

In visual backward masking (VBM), a target is followed by a mask that decreases target discriminability. Schizophrenia patients (SZ) show strong and reproducible masking impairments, which are associated with reduced EEG amplitudes. Patients with bipolar disorder (BP) show masking deficits, too. Here, we investigated the neural EEG correlates of VBM in BP. 122 SZ, 94 unaffected controls, and 38 BP joined a standard VBM experiment. 123 SZ, 94 unaffected controls and 16 BP joined a corresponding EEG experiment, analyzed in terms of global field power. As in previous studies, SZ and BP show strong masking deficits. Importantly and similarly to SZ, BP show decreased global field power amplitudes at approximately 200 ms after the target onset, compared to controls. These results suggest that VBM deficits are not specific for schizophrenia but for a broader range of functional psychoses. Potentially, both SZ and BP show deficient target enhancement.

Response to commentaries on 'hard criteria for empirical theories of consciousness'

In consciousness research, we have a very large number of theories, which exceeds by far the number of theories in other fields. We recently presented a set of criteria for evaluating and comparing theories of consciousness, and then applied the criteria to a number of different theories. Our publication sparked strong responses as evident by the many comments published in Cognitive Neuroscience (this issue). Overall, there seems to be consensus that a theory of consciousness (ToC) needs to have an unconscious alternative, but other criteria sparked controversy. The hottest debate is to what extent consciousness needs to work with purely 1^st person data, containing information not available in 3^rd person reports. We would like to thank all the commentators for their lively input and we look forward to continued dialog as theories evolve and compete.

All in Good Time: Long-Lasting Postdictive Effects Reveal Discrete Perception

Is consciousness a continuous stream of percepts or is it discrete, occurring only at certain moments in time? This question has puzzled philosophers, psychologists, and neuroscientists for centuries. Both hypotheses have fallen repeatedly in and out of favor. Here, we review recent studies exploring long-lasting postdictive effects and show that the results favor a two-stage discrete model, in which substantial periods of continuous unconscious processing precede discrete conscious percepts. We propose that such a model marries the advantages of both continuous and discrete models and resolves centuries old debates about perception and consciousness.

When illusions merge

We recently found only weak correlations between the susceptibility to various visual illusions. However, we observed strong correlations among different variants of an illusion, suggesting that the visual space of illusions includes several illusion-specific factors. Here, we specifically examined how factors for the vertical-horizontal, Müller-Lyer, and Ponzo illusions relate to each other. We measured the susceptibility to each illusion separately and to combinations of two illusions, which we refer to as a merged illusion; for example, we tested the Müller-Lyer illusion and the vertical-horizontal illusion, as well as a merged version of both illusions. We used an adjustment procedure in two experiments with 306 and 98 participants, respectively. Using path analyses, correlations, and exploratory factor analyses, we found that the susceptibility to a merged illusion is well predicted from the susceptibilities to the individual illusions. We suggest that there are illusion-specific factors that, by independent combinations, represent the whole visual structure underlying illusions.

Hard criteria for empirical theories of consciousness

Consciousness is now a well-established field of empirical research. A large body of experimental results has been accumulated and is steadily growing. In parallel, many Theories of Consciousness (ToCs) have been proposed. These theories are diverse in nature, ranging from computational to neurophysiological and quantum theoretical approaches. This contrasts with other fields of natural science, which host a smaller number of competing theories. We suggest that one reason for this abundance of extremely different theories may be the lack of stringent criteria specifying how empirical data constrains ToCs. First, we argue that consciousness is a well-defined topic from an empirical point of view and motivate a purely empirical stance on the quest for consciousness. Second, we present a checklist of criteria that, we propose, empirical ToCs need to cope with. Third, we review 13 of the most influential ToCs and subject them to the criteria. Our analysis helps to situate these different ToCs in the theoretical landscapeand sheds light on their strengths and weaknesses from a strictly empirical point of view.

Neural Compensation Mechanisms of Siblings of Schizophrenia Patients as Revealed by High-Density EEG

Visual backward masking (VBM) deficits are candidate endophenotypes of schizophrenia indexing genetic liability of the disorder. In VBM, a target is followed by a mask that deteriorates target perception. Schizophrenia patients and, to a lesser extent, their unaffected relatives show strong and reproducible VBM deficits. In patients, VBM deficits are associated with strongly decreased amplitudes in the evoked-related potentials (ERPs). Here, to unveil the neural mechanisms of VBM in schizophrenia, circumventing illness-specific confounds, we investigated the electroencephalogram correlates of VBM in unaffected siblings of schizophrenia patients. We tested 110 schizophrenia patients, 60 siblings, and 83 healthy controls. As in previous studies, patients showed strong behavioral deficits and decreased ERP amplitudes compared to controls. Surprisingly, the ERP amplitudes of siblings were even higher than the ones of controls, while their performances were similar. ERP amplitudes in siblings were found to correlate with performance. These results suggest that VBM is deteriorated in patients and siblings. However, siblings, unlike patients, can partially compensate for the deficits by over-activating a network of brain regions.

Object identity determines trans-saccadic integration

Humans make two to four rapid eye movements (saccades) per second, which, surprisingly, does not lead to abrupt changes in vision. To the contrary, we perceive a stable world. Hence, an important question is how information is integrated across saccades. To investigate this question, we used the sequential metacontrast paradigm (SQM), where two expanding streams of lines are presented. When one line is spatially offset, the other lines are perceived as being offset, too. When more lines are offset, all offsets integrate mandatorily; that is, observers cannot report the individual offsets but perceive one integrated offset. Here, we asked observers to make a saccade during the SQM. Even though the saccades caused a highly disrupted motion trajectory on the retina, offsets presented before and after the saccade integrated mandatorily. When observers made no saccade and the streams were displaced on the screen so that a similarly disrupted retinal image occurred as in the previous condition, no integration occurred. We suggest that trans-saccadic integration and perception are determined by object identity in spatiotopic coordinates and not by the retinal image.

Capsule networks as recurrent models of grouping and segmentation

Classically, visual processing is described as a cascade of local feedforward computations. Feedforward Convolutional Neural Networks (ffCNNs) have shown how powerful such models can be. However, using visual crowding as a well-controlled challenge, we previously showed that no classic model of vision, including ffCNNs, can explain human global shape processing. Here, we show that Capsule Neural Networks (CapsNets), combining ffCNNs with recurrent grouping and segmentation, solve this challenge. We also show that ffCNNs and standard recurrent CNNs do not, suggesting that the grouping and segmentation capabilities of CapsNets are crucial. Furthermore, we provide psychophysical evidence that grouping and segmentation are implemented recurrently in humans, and show that CapsNets reproduce these results well. We discuss why recurrence seems needed to implement grouping and segmentation efficiently. Together, we provide mutually reinforcing psychophysical and computational evidence that a recurrent grouping and segmentation process is essential to understand the visual system and create better models that harness global shape computations.

EEG microstates are a candidate endophenotype for schizophrenia

Electroencephalogram microstates are recurrent scalp potential configurations that remain stable for around 90 ms. The dynamics of two of the four canonical classes of microstates, commonly labeled as C and D, have been suggested as a potential endophenotype for schizophrenia. For endophenotypes, unaffected relatives of patients must show abnormalities compared to controls. Here, we examined microstate dynamics in resting-state recordings of unaffected siblings of patients with schizophrenia, patients with schizophrenia, healthy controls, and patients with first episodes of psychosis (FEP). Patients with schizophrenia and their siblings showed increased presence of microstate class C and decreased presence of microstate class D compared to controls. No difference was found between FEP and chronic patients. Our findings suggest that the dynamics of microstate classes C and D are a candidate endophenotype for schizophrenia.

Individual differences in the Müller-Lyer and Ponzo illusions are stable across different contexts

Vision scientists have attempted to classify visual illusions according to certain aspects, such as brightness or spatial features. For example, Piaget proposed that visual illusion magnitudes either decrease or increase with age. Subsequently, it was suggested that illusions are segregated according to their context: real-world contexts enhance and abstract contexts inhibit illusion magnitudes with age. We tested the effects of context on the Müller-Lyer and Ponzo illusions with a standard condition (no additional context), a line-drawing perspective condition, and a real-world perspective condition. A mixed-effects model analysis, based on data from 76 observers with ages ranging from 6 to 66 years, did not reveal any significant interaction between context and age. Although we found strong intra-illusion correlations for both illusions, we found only weak inter-illusion correlations, suggesting that the structure underlying these two spatial illusions includes several specific factors.

How stable is perception in #TheDress and #TheShoe?

#TheDress is perceived by some people as black and blue while others perceive it as white and gold. We have previously shown that the first encounter with #TheDress strongly biases its perception. This percept remained stable during the experiment, suggesting a role of one-shot learning. #TheShoe is another image that elicits similar bimodal color percepts. Here, we investigated how percepts change over time in both #TheShoe and #TheDress. First, we show that the important role of one-shot learning, which we found for #TheDress extends to #TheShoe. Similarly to our previous results with the dress, hiding large parts of the image with occluders biased the percept of the shoe. The percept did not change for the majority of observers when the occluders were removed. Second, we investigated if and how percepts switch over a time course of 14 days. We found that although some observers experienced percept switches, the percept was largely stable for most observers.

Crowding reveals fundamental differences in local vs. global processing in humans and machines

Feedforward Convolutional Neural Networks (ffCNNs) have become state-of-the-art models both in computer vision and neuroscience. However, human-like performance of ffCNNs does not necessarily imply human-like computations. Previous studies have suggested that current ffCNNs do not make use of global shape information. However, it is currently unclear whether this reflects fundamental differences between ffCNN and human processing or is merely an artefact of how ffCNNs are trained. Here, we use visual crowding as a well-controlled, specific probe to test global shape computations. Our results provide evidence that ffCNNs cannot produce human-like global shape computations for principled architectural reasons. We lay out approaches that may address shortcomings of ffCNNs to provide better models of the human visual system.

Electrophysiological correlates of visual backward masking in patients with major depressive disorder

Depression and schizophrenia are two psychiatric diseases with high co-morbidity. For this reason, it is important to find sensitive endophenotypes, which may disentangle the two disorders. The Shine-Through paradigm, a visual backward masking task, is a potential endophenotype for schizophrenia. Masking is strongly deteriorated in schizophrenia patients, which is reflected in reduced EEG amplitudes. Here, we tested whether masking deficits and associated EEG changes are also found in patients with major depressive disorder. First, we replicated previous findings showing that depressive patients exhibit, at most, only weak masking deficits. Second, we found that the EEG amplitudes of depressive patients were reduced compared to controls and slightly increased compared to schizophrenia patients. As a secondary analysis, we compared the performance in the masking paradigm with three cognitive tasks, namely: the Wisconsin card sorting test, a verbal fluency test and a degraded continuous performance test. Performance in all but the verbal fluency test could discriminate schizophrenia from depression.

One-shot learning and behavioral eligibility traces in sequential decision making

In many daily tasks, we make multiple decisions before reaching a goal. In order to learn such sequences of decisions, a mechanism to link earlier actions to later reward is necessary. Reinforcement learning (RL) theory suggests two classes of algorithms solving this credit assignment problem: In classic temporal-difference learning, earlier actions receive reward information only after multiple repetitions of the task, whereas models with eligibility traces reinforce entire sequences of actions from a single experience (one-shot). Here, we show one-shot learning of sequences. We developed a novel paradigm to directly observe which actions and states along a multi-step sequence are reinforced after a single reward. By focusing our analysis on those states for which RL with and without eligibility trace make qualitatively distinct predictions, we find direct behavioral (choice probability) and physiological (pupil dilation) signatures of reinforcement learning with eligibility trace across multiple sensory modalities.

Dominant men are faster in decision-making situations and exhibit a distinct neural signal for promptness

Social dominance, the main organizing principle of social hierarchies, facilitates priority access to resources by dominant individuals. Throughout taxa, individuals are more likely to become dominant if they act first in social situations and acting fast may provide evolutionary advantage; yet whether fast decision-making is a behavioral predisposition of dominant persons outside of social contexts is not known. Following characterization of participants for social dominance motivation, we found that, indeed, men high in social dominance respond faster–without loss of accuracy–than those low in dominance across a variety of decision-making tasks. Both groups did not differ in a simple reaction task. Then, we selected a decision-making task and applied high-density electroencephalography (EEG) to assess temporal dynamics of brain activation through event related potentials. We found that promptness to respond in the choice task in dominant individuals is related to a strikingly amplified brain signal at approximately 240 ms post-stimulus presentation. Source imaging analyses identified higher activity in the left insula and in the cingulate, right inferior temporal and right angular gyri in high than in low dominance participants. Our findings suggest that promptness to respond in choice situations, regardless of social context, is a biomarker for social disposition.