Reinforcement learning modulates the stability of cognitive control settings for object selection

Learning Cognitive Flexibility: Neural Substrates of Adapting Switch-Readiness to Time-varying Demands

An individual's readiness to switch tasks (cognitive flexibility) varies over time, in part, as the result of reinforcement learning based on the statistical structure of the world around them. Consequently, the behavioral cost associated with task-switching is smaller in contexts where switching is frequent than where it is rare, but the underlying brain mechanisms of this adaptation in cognitive flexibility are not well understood. Here, we manipulated the likelihood of switches across blocks of trials in a classic cued task-switching paradigm while participants underwent fMRI. As anticipated, behavioral switch costs decreased as the probability of switching increased, and neural switch costs were observed in lateral and medial frontoparietal cortex. To study moment-by-moment adjustments in cognitive flexibility at the neural level, we first fitted the behavioral RT data with reinforcement learning algorithms and then used the resulting trial-wise prediction error estimate as a regressor in a model-based fMRI analysis. The results revealed that lateral frontal and parietal cortex activity scaled positively with unsigned switch prediction error and that there were no brain regions encoding signed (i.e., switch- or repeat-specific) prediction error. Taken together, this study documents that adjustments in cognitive flexibility to time-varying switch demands are mediated by frontoparietal cortex tracking the likelihood of forthcoming task switches.

The location independence of learned attentional flexibility

Individuals can adjust their shift readiness, known as attentional flexibility, according to the statistical structure of the environment. However, the extent to which these modulations in attentional flexibility are associated with a global readiness to shift attention to any location versus an anticipated shift to a single location remains unknown. Across two experiments, participants shifted attention among three rapid serial visual presentation (RSVP) streams of alphanumeric characters in response to embedded visual cues and made button presses in response to targets at the cued location. We manipulated the likelihood that participants would receive a cue that signaled a shift between two of the streams across blocks of trials. The likelihood of a cued shift of attention to the third location was held constant across all blocks. Participants demonstrated smaller target detection shift costs (Experiments 1 and 2) and shorter saccade latencies (Experiment 1) when the overall likelihood of shifting was high than when the overall shift likelihood was low. Critically, we observed evidence of both global shift readiness and location-specific shift readiness in both experiments such that participants shifted attention to the most-likely-to-be cued location the fastest, but still demonstrated a difference in the time to shift attention to the unlikely location according to the overall shift likelihood. Our findings provide evidence that moment-by-moment changes in attentional flexibility are not limited to an expectation to shift to a single location, but rather reflect, in part, a location-independent state of control.

Impulsivity Relates to Multi-Trial Choice Strategy in Probabilistic Reversal Learning

Impulsivity is defined as a trait-like tendency to engage in rash actions that are poorly thought out or expressed in an untimely manner. Previous research has found that impulsivity relates to deficits in decision making, in particular when it necessitates executive control or reward outcomes. Reinforcement learning (RL) relies on the ability to integrate reward or punishment outcomes to make good decisions, and has recently been shown to often recruit executive function; as such, it is unsurprising that impulsivity has been studied in the context of RL. However, how impulsivity relates to the mechanisms of RL remains unclear. We aimed to investigate the relationship between impulsivity and learning in a reward-driven learning task with probabilistic feedback and reversal known to recruit executive function. Based on prior literature in clinical populations, we predicted that higher impulsivity would be associated with poorer performance on the task, driven by more frequent switching following unrewarded outcomes. Our results did not support this prediction, but more advanced, trial-history dependent analyses revealed specific effects of impulsivity on switching behavior following consecutive unrewarded trials. Computational modeling captured group-level behavior, but not impulsivity results. Our results support previous findings highlighting the importance of sensitivity to negative outcomes in understanding how impulsivity relates to learning, but indicate that this may stem from more complex strategies than usually considered in computational models of learning. This should be an important target for future research.

Learned states of preparatory attentional control

Individuals regularly experience fluctuations in the ability to perform cognitive operations. Although previous research has focused on predicting cognitive flexibility from persistent individual traits, as well as from spontaneous fluctuations in neural activity, the role of learning in shaping preparatory attentional control remains poorly understood. Across 3 experiments, we manipulated the statistical regularities of an attentional orienting paradigm to examine whether individuals modulated attentional flexibility, the readiness to perform a spatial shift of attention, across learned contexts. We found evidence of learning-based modulations in preparatory attentional control settings when the probability of shifting the focus of attention differed based on temporally or color-defined contexts. Furthermore, in the case of color-defined contexts, these modulations in preparatory control persisted even after a change in the underlying statistical properties. Our results indicate that dynamic adjustments in preparatory attentional control are sensitive to the underlying statistical regularities of an environment. This finding has implications for understanding disordered patterns of attentional control and how these patterns might be modified with training.