Deterministic response strategies in a trial-and-error learning task

The dynamics of functional brain network segregation in feedback-driven learning

Prior evidence suggests that increasingly efficient task performance in human learning is associated with large scale brain network dynamics. However, the specific nature of this general relationship has remained unclear. Here, we characterize performance improvement during feedback-driven stimulus-response (S-R) learning by learning rate as well as S-R habit strength and test whether and how these two behavioral measures are associated with a functional brain state transition from a more integrated to a more segregated brain state across learning. Capitalizing on two separate fMRI studies using similar but not identical experimental designs, we demonstrate for both studies that a higher learning rate is associated with a more rapid brain network segregation. By contrast, S-R habit strength is not reliably related to changes in brain network segregation. Overall, our current study results highlight the utility of dynamic functional brain state analysis. From a broader perspective taking into account previous study results, our findings align with a framework that conceptualizes brain network segregation as a general feature of processing efficiency not only in feedback-driven learning as in the present study but also in other types of learning and in other task domains.

The modulation of acute stress on model-free and model-based reinforcement learning in gambling disorder

BACKGROUND AND AIMS

Experiencing acute stress is common in behavioral addictions such as gambling disorder. Additionally, like most substance-induced addictions, aberrant decision-making wherein a reactive habit-induced response (conceptualized as a Model-free [MF] in reinforcement learning) suppresses a flexible goal-directed response (conceptualized as a Model-based [MB]) is also common in gambling disorder. In the current study we investigated the influence of acute stress on the balance between habitual response and the goal-directed system.

METHODS

A sample of N = 116 problem gamblers (PG) and healthy controls (HC) performed an acute stress task - the Socially Evaluated Cold pressure task (SECPT) - or a control task. Self-reported stress and salivary cortisol were collected as measures of acute stress. Following the SECPT, participants performed the Two-Step Markov Task to account for the relative contribution of MB and MF strategies. Additionally, verbal working memory and IQ measures were collected to account for their mediating effects on the orchestration between MB/MF and the impact of stress.

RESULTS

Both groups had comparable baseline and stress-induced cortisol response to the SECPT. Non-stressed PG displayed lower MB learning than HC. MANOVA and regression analyses showed a deleterious effect of stress-induced cortisol response on the orchestration between MB and MF learning in HC but not in PG. These effects remained when controlling for working memory and IQ.

DISCUSSION AND CONCLUSIONS

We found an abnormal pattern of modulation of stress on the orchestration between MB and MF learning among PG. Several interpretations and future research directions are discussed.

Effects of a Pair Programming Educational Robot-Based Approach on Students' Interdisciplinary Learning of Computational Thinking and Language Learning

Using educational robots (ERs) to integrate computational thinking (CT) with cross-disciplinary content has gone beyond Science, Technology, Engineering, and Mathematics (STEM), to include foreign-language learning (FL) and further cross-context target-language (TL) acquisition. Such integration must not solely emphasise CT problem-solving skills. Rather, it must provide students with interactive learning to support their target-language (TL) interaction while reducing potential TL anxiety. This study aimed to validate the effects of the proposed method of pair programming (PP) along with question-and-response interaction in a board-game activity on young learners’ CT skills and TL learning across contexts. Two Grade 6 classes, one with 15 students who were studying Chinese as a Second Language (CSL) and the other with 15 students who were studying English as a Foreign Language (EFL), participated in the activity. A series of instruments on achievement assessment, questionnaires on CT skills and TL anxiety, and sequential learning behaviour analysis were used to critically examine the results. The main conclusion is that the EFL group showed better social skills of cooperation on CT and lower TL learning anxiety, while the CSL group demonstrated better problem-solving skills in CT, but presented more behaviours of trial-and-error loops. Results not only contribute suggestions for cross-disciplinary learning but also provide support for cross-context instruction beyond educational coursework.

Advances in modeling learning and decision-making in neuroscience

An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.

Goal-Based Binding of Irrelevant Stimulus Features for Action Slips

Abstract. Binding between representations of stimuli and actions and later retrieval of these compounds provide efficient shortcuts in action control. Recent observations indicate that these mechanisms are not only effective when action episodes go as planned, but they also seem to be at play when actions go awry. Moreover, the human cognitive system even corrects traces of error commission on the fly because it binds the intended but not actually executed response to concurrent task-relevant stimuli, thus enabling retrieval of a correct, but not actually executed response when encountering the stimulus again. However, a plausible alternative interpretation of this finding is that error commission triggers selective strengthening of the instructed stimulus–response mapping instead, thus promoting its efficient application in the future. The experiment presented here makes an unequivocal case for episodic binding and retrieval in erroneous action episodes by showing binding between task-irrelevant stimuli and correct responses.

Modeling the influence of working memory, reinforcement, and action uncertainty on reaction time and choice during instrumental learning

What determines the speed of our decisions? Various models of decision-making have focused on perceptual evidence, past experience, and task complexity as important factors determining the degree of deliberation needed for a decision. Here, we build on a sequential sampling decision-making framework to develop a new model that captures a range of reaction time (RT) effects by accounting for both working memory and instrumental learning processes. The model captures choices and RTs at various stages of learning, and in learning environments with varying complexity. Moreover, the model generalizes from tasks with deterministic reward contingencies to probabilistic ones. The model succeeds in part by incorporating prior uncertainty over actions when modeling RT. This straightforward process model provides a parsimonious account of decision dynamics during instrumental learning and makes unique predictions about internal representations of action values.

Beyond dichotomies in reinforcement learning

Reinforcement learning (RL) is a framework of particular importance to psychology, neuroscience and machine learning. Interactions between these fields, as promoted through the common hub of RL, has facilitated paradigm shifts that relate multiple levels of analysis in a singular framework (for example, relating dopamine function to a computationally defined RL signal). Recently, more sophisticated RL algorithms have been proposed to better account for human learning, and in particular its oft-documented reliance on two separable systems: a model-based (MB) system and a model-free (MF) system. However, along with many benefits, this dichotomous lens can distort questions, and may contribute to an unnecessarily narrow perspective on learning and decision-making. Here, we outline some of the consequences that come from overconfidently mapping algorithms, such as MB versus MF RL, with putative cognitive processes. We argue that the field is well positioned to move beyond simplistic dichotomies, and we propose a means of refocusing research questions towards the rich and complex components that comprise learning and decision-making.