One-shot learning and behavioral eligibility traces in sequential decision making

In many daily tasks, we make multiple decisions before reaching a goal. In order to learn such sequences of decisions, a mechanism to link earlier actions to later reward is necessary. Reinforcement learning (RL) theory suggests two classes of algorithms solving this credit assignment problem: In classic temporal-difference learning, earlier actions receive reward information only after multiple repetitions of the task, whereas models with eligibility traces reinforce entire sequences of actions from a single experience (one-shot). Here, we show one-shot learning of sequences. We developed a novel paradigm to directly observe which actions and states along a multi-step sequence are reinforced after a single reward. By focusing our analysis on those states for which RL with and without eligibility trace make qualitatively distinct predictions, we find direct behavioral (choice probability) and physiological (pupil dilation) signatures of reinforcement learning with eligibility trace across multiple sensory modalities.

Dominant men are faster in decision-making situations and exhibit a distinct neural signal for promptness

Social dominance, the main organizing principle of social hierarchies, facilitates priority access to resources by dominant individuals. Throughout taxa, individuals are more likely to become dominant if they act first in social situations and acting fast may provide evolutionary advantage; yet whether fast decision-making is a behavioral predisposition of dominant persons outside of social contexts is not known. Following characterization of participants for social dominance motivation, we found that, indeed, men high in social dominance respond faster–without loss of accuracy–than those low in dominance across a variety of decision-making tasks. Both groups did not differ in a simple reaction task. Then, we selected a decision-making task and applied high-density electroencephalography (EEG) to assess temporal dynamics of brain activation through event related potentials. We found that promptness to respond in the choice task in dominant individuals is related to a strikingly amplified brain signal at approximately 240 ms post-stimulus presentation. Source imaging analyses identified higher activity in the left insula and in the cingulate, right inferior temporal and right angular gyri in high than in low dominance participants. Our findings suggest that promptness to respond in choice situations, regardless of social context, is a biomarker for social disposition.

Investigating prospective memory via eye tracking: No evidence for a monitoring deficit in older adults

Prospective memory (PM) refers to remembering to perform intended actions in the future. Older adults in particular have been shown to be negatively affected by PM tasks that require a high amount of attentional resources (i.e., nonfocal tasks). This age-related PM deficit has been attributed to reduced target monitoring in this age group older adults. However, this conclusion was based on indirect measures of monitoring, such as costs of the ongoing task. The present study set out to 1) investigate older adults' PM target monitoring by, for the first time, employing a direct measure (i.e., eye tracking), 2) assess differences in monitoring between PM tasks that differ in their focality, and 3) examine whether differences in PM monitoring can indeed explain older adults' reduced PM performance in nonfocal tasks. Results demonstrate that while older, but not younger adults, showed reduced performance in a nonfocal PM task, overt monitoring (eye movements) of these groups did not differ between focality conditions. Further analyses showed that older adults' performance was still reduced on the strategically more demanding task after controlling for overt target monitoring (i.e., including only trials in which the participant monitored). In contrast to indirect measures of cue monitoring, our findings illustrate that older adults' deficits on nonfocal PM tasks cannot (exclusively) be explained by reduced monitoring. Instead, processing that takes place after target monitoring are discussed as possible mechanisms underlying older adults' reduced PM performance in nonfocal tasks.

Feature integration within discrete time windows

Sensory information must be integrated over time to perceive, for example, motion and melodies. Here, to study temporal integration, we used the sequential metacontrast paradigm in which two expanding streams of lines are presented. When a line in one stream is offset observers perceive all other lines to be offset too, even though they are straight. When more lines are offset the offsets integrate mandatorily, i.e., observers cannot report the individual offsets. We show that mandatory integration lasts for up to 450 ms, depending on the observer. Importantly, integration occurs only when offsets are presented within a discrete window of time. Even stimuli that are in close spatio-temporal proximity do not integrate if they are in different windows. A window of integration starts with stimulus onset and integration in the next window has similar characteristics. We present a two-stage computational model based on discrete time windows that captures these effects.

In order to perceive moving or changing objects, sensory information must be integrated over time. Here, using a visual sequential metacontrast paradigm, the authors show that integration occurs only when subsequent stimuli are presented within a discrete window of time after the initial stimulus.

Dopaminergic modulation of motor network compensatory mechanisms in Parkinson's disease

The dopaminergic system has a unique gating function in the initiation and execution of movements. When the interhemispheric imbalance of dopamine inherent to the healthy brain is disrupted, as in Parkinson's disease (PD), compensatory mechanisms act to stave off behavioral changes. It has been proposed that two such compensatory mechanisms may be (a) a decrease in motor lateralization, observed in drug-naïve PD patients and (b) reduced inhibition - increased facilitation. Seeking to investigate the differential effect of dopamine depletion and subsequent substitution on compensatory mechanisms in non-drug-naïve PD, we studied 10 PD patients and 16 healthy controls, with patients undergoing two test sessions - "ON" and "OFF" medication. Using a simple visually-cued motor response task and fMRI, we investigated cortical motor activation - in terms of laterality, contra- and ipsilateral percent BOLD signal change and effective connectivity in the parametric empirical Bayes framework. We found that decreased motor lateralization persists in non-drug-naïve PD and is concurrent with decreased contralateral activation in the cortical motor network. Normal lateralization is not reinstated by dopamine substitution. In terms of effective connectivity, disease-related changes primarily affect ipsilaterally-lateralized homotopic cortical motor connections, while medication-related changes affect contralaterally-lateralized homotopic connections. Our findings suggest that, in non-drug-naïve PD, decreased lateralization is no longer an adaptive cortical mechanism, but rather the result of maladaptive changes, related to disease progression and long-term dopamine replacement. These findings highlight the need for the development of noninvasive therapies, which would promote the adaptive mechanisms of the PD brain.

Running Large-Scale Simulations on the Neurorobotics Platform to Understand Vision - The Case of Visual Crowding

Traditionally, human vision research has focused on specific paradigms and proposed models to explain very specific properties of visual perception. However, the complexity and scope of modern psychophysical paradigms undermine the success of this approach. For example, perception of an element strongly deteriorates when neighboring elements are presented in addition (visual crowding). As it was shown recently, the magnitude of deterioration depends not only on the directly neighboring elements but on almost all elements and their specific configuration. Hence, to fully explain human visual perception, one needs to take large parts of the visual field into account and combine all the aspects of vision that become relevant at such scale. These efforts require sophisticated and collaborative modeling. The Neurorobotics Platform (NRP) of the Human Brain Project offers a unique opportunity to connect models of all sorts of visual functions, even those developed by different research groups, into a coherently functioning system. Here, we describe how we used the NRP to connect and simulate a segmentation model, a retina model, and a saliency model to explain complex results about visual perception. The combination of models highlights the versatility of the NRP and provides novel explanations for inward-outward anisotropy in visual crowding.

The unfolding argument: Why IIT and other causal structure theories cannot explain consciousness

How can we explain consciousness? This question has become a vibrant topic of neuroscience research in recent decades. A large body of empirical results has been accumulated, and many theories have been proposed. Certain theories suggest that consciousness should be explained in terms of brain functions, such as accessing information in a global workspace, applying higher order to lower order representations, or predictive coding. These functions could be realized by a variety of patterns of brain connectivity. Other theories, such as Information Integration Theory (IIT) and Recurrent Processing Theory (RPT), identify causal structure with consciousness. For example, according to these theories, feedforward systems are never conscious, and feedback systems always are. Here, using theorems from the theory of computation, we show that causal structure theories are either false or outside the realm of science.

Schizophrenia patients using atypical medication perform better in visual tasks than patients using typical medication

Schizophrenia (SCZ) patients show deficits in many domains, including cognition and perception. However, results are often mixed. One reason for mixed results may be differences in medication. Very little is known about the role of medication in visual processing. Here, we investigated the effects of typical vs. atypical medication on contrast sensitivity (spatial frequencies ranging from 0.2 to 20 cycles per degree), vernier acuity, and visual backward masking. From a large pool of patients, we selected 50 patients (Study 1, conducted in Brazil) and 97 patients (Study 2, conducted in Georgia) taking either only typical or atypical medication. Patients with atypical medication performed significantly better than patients with typical medication for contrast sensitivity, vernier duration, and backward masking. As a secondary result, we found similar, but not significant, trends for the cognitive tasks (Stroop, Flanker, Trail-Making Test-B, Wisconsin Card Sorting Test and Continuous Performance Test) in the same patients. No correlations were found between demographics, psychopathology, chlorpromazine equivalents and visual processing. A conclusion of our study is that one needs to be careful comparing studies when medication is not comparable.

Beyond Bouma's window: How to explain global aspects of crowding?

In crowding, perception of an object deteriorates in the presence of nearby elements. Although crowding is a ubiquitous phenomenon, since elements are rarely seen in isolation, to date there exists no consensus on how to model it. Previous experiments showed that the global configuration of the entire stimulus must be taken into account. These findings rule out simple pooling or substitution models and favor models sensitive to global spatial aspects. In order to investigate how to incorporate global aspects into models, we tested a large number of models with a database of forty stimuli tailored for the global aspects of crowding. Our results show that incorporating grouping like components strongly improves model performance.

Building perception block by block: a response to Fekete et al

Is consciousness a continuous stream, or do percepts occur only at certain moments of time? This age-old question is still under debate. Both positions face difficult problems, which we proposed to overcome with a 2-stage model, where unconscious processing continuously integrates information before a discrete, conscious percept occurs. Recently, Fekete et al. criticized our model. Here, we show that, contrary to their proposal, simple sliding windows cannot explain apparent motion and related phenomena within a continuous framework, and that their supervenience argument only holds true for qualia realists, a philosophical position we do not adopt.

Electrophysiological correlates of visual backward masking in patients with first episode psychosis

Visual backward masking is strongly impaired in patients with schizophrenia. Masking deficits have been proposed as potential endophenotypes of schizophrenia. Masking performance deficits manifest as strongly reduced amplitudes in the electroencephalogram (EEG). In order to fulfill the criteria of an endophenotype, masking deficits should not vary substantially across time and should be present at the first psychotic event. To verify whether these conditions are met for visual backward masking, we tested patients with first episode psychosis (n = 21) in a longitudinal study. Patients were tested with visual backward masking and EEG three times every six months over a period of one year. We found that the EEG amplitudes of patients with first episode psychosis were higher as compared to those of patients with schizophrenia but lower as compared to those of unaffected controls. More interestingly, we found that the EEG amplitudes of patients with first episode psychosis remained stable over the course of one year. Since chronic schizophrenia patients have strongly reduced amplitudes, we speculate that the neural correlates of masking deficits (EEG amplitudes) continue to decrease as the disease progresses.

Is the perception of illusions abnormal in schizophrenia?

There seems to be no common factor for visual perception, i.e., performance in visual tasks correlates only weakly with each other. Similar results were found with visual illusions. One may expect common visual factors for individuals suffering from pathologies that alter brain functioning, such as schizophrenia. For example, patients who are more severely affected by the disease, e.g., stronger positive symptoms, may show increased illusion magnitudes. Here, in the first experiment, we used a battery of seven visual illusions and a mental imagery questionnaire. Illusion magnitudes for the seven illusions did not differ significantly between the patients and controls. In addition, correlations between the different illusions and mental imagery were low. In the second experiment, we tested 59 patients (mostly outpatients) with ten visual illusions. As for the first experiment, patients and controls showed similar susceptibility to all but one visual illusion. Moreover, there were no significant correlations between different illusions, symptoms, or medication type. Thus, it seems that perception of visual illusions is mostly intact in schizophrenia.

Perceptual grouping

A Quick guide to perceptual grouping, the way that we group individual elements that we perceive, such as parts of a visual scene, together to make sense of the world.

Sustained spatial attention can affect feature fusion

When two verniers are presented in rapid succession at the same location, feature fusion occurs. Instead of perceiving two separate verniers, participants typically report perceiving one fused vernier, whose offset is a combination of the two previous verniers, with the later one slightly dominating. Here, we examined the effects of sustained attention-the voluntary component of spatial attention-on feature fusion. One way to manipulate sustained attention is via the degree of certainty regarding the stimulus location. In the attended condition, the stimulus appeared always in the same location, and in the unattended condition it could appear in one of two possible locations. Participants had to report the offset of the fused vernier. Experiments 1 and 2 measured attentional effects on feature fusion with and without eye-tracking. In both experiments, we found a higher rate of reports corresponding to the offset of the second vernier with focused attention than without focused attention, suggesting that attention strengthened the final percept emerging from the fusion operation. In Experiment 3, we manipulated the stimulus duration to encourage a final fused percept that is dominated by either the first or second vernier. We found that attention strengthened the already dominant percept, regardless of whether it corresponded to the offset of the first or second vernier. These results are consistent with an attentional mechanism of signal enhancement at the encoding stage.