1
|
Wang X, Zwosta K, Hennig J, Böhm I, Ehrlich S, Wolfensteller U, Ruge H. The dynamics of functional brain network segregation in feedback-driven learning. Commun Biol 2024; 7:531. [PMID: 38710773 DOI: 10.1038/s42003-024-06210-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Accepted: 04/17/2024] [Indexed: 05/08/2024] Open
Abstract
Prior evidence suggests that increasingly efficient task performance in human learning is associated with large scale brain network dynamics. However, the specific nature of this general relationship has remained unclear. Here, we characterize performance improvement during feedback-driven stimulus-response (S-R) learning by learning rate as well as S-R habit strength and test whether and how these two behavioral measures are associated with a functional brain state transition from a more integrated to a more segregated brain state across learning. Capitalizing on two separate fMRI studies using similar but not identical experimental designs, we demonstrate for both studies that a higher learning rate is associated with a more rapid brain network segregation. By contrast, S-R habit strength is not reliably related to changes in brain network segregation. Overall, our current study results highlight the utility of dynamic functional brain state analysis. From a broader perspective taking into account previous study results, our findings align with a framework that conceptualizes brain network segregation as a general feature of processing efficiency not only in feedback-driven learning as in the present study but also in other types of learning and in other task domains.
Collapse
Affiliation(s)
- Xiaoyu Wang
- Faculty of Psychology, Technische Universität Dresden, Dresden, Germany.
| | - Katharina Zwosta
- Faculty of Psychology, Technische Universität Dresden, Dresden, Germany
| | - Julius Hennig
- Translational Developmental Neuroscience Section, Division of Psychological and Social Medicine and Developmental Neurosciences, Faculty of Medicine, TU Dresden, Dresden, Germany
| | - Ilka Böhm
- Translational Developmental Neuroscience Section, Division of Psychological and Social Medicine and Developmental Neurosciences, Faculty of Medicine, TU Dresden, Dresden, Germany
| | - Stefan Ehrlich
- Translational Developmental Neuroscience Section, Division of Psychological and Social Medicine and Developmental Neurosciences, Faculty of Medicine, TU Dresden, Dresden, Germany
- Eating Disorder Treatment and Research Center, Department of Child and Adolescent Psychiatry, Faculty of Medicine, Technische Universität Dresden, Dresden, Germany
| | - Uta Wolfensteller
- Faculty of Psychology, Technische Universität Dresden, Dresden, Germany
| | - Hannes Ruge
- Faculty of Psychology, Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
2
|
Wyckmans F, Banerjee N, Saeremans M, Otto R, Kornreich C, Vanderijst L, Gruson D, Carbone V, Bechara A, Buchanan T, Noël X. The modulation of acute stress on model-free and model-based reinforcement learning in gambling disorder. J Behav Addict 2022; 11:831-844. [PMID: 36112488 PMCID: PMC9872530 DOI: 10.1556/2006.2022.00059] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Revised: 07/19/2022] [Accepted: 07/25/2022] [Indexed: 02/03/2023] Open
Abstract
BACKGROUND AND AIMS Experiencing acute stress is common in behavioral addictions such as gambling disorder. Additionally, like most substance-induced addictions, aberrant decision-making wherein a reactive habit-induced response (conceptualized as a Model-free [MF] in reinforcement learning) suppresses a flexible goal-directed response (conceptualized as a Model-based [MB]) is also common in gambling disorder. In the current study we investigated the influence of acute stress on the balance between habitual response and the goal-directed system. METHODS A sample of N = 116 problem gamblers (PG) and healthy controls (HC) performed an acute stress task - the Socially Evaluated Cold pressure task (SECPT) - or a control task. Self-reported stress and salivary cortisol were collected as measures of acute stress. Following the SECPT, participants performed the Two-Step Markov Task to account for the relative contribution of MB and MF strategies. Additionally, verbal working memory and IQ measures were collected to account for their mediating effects on the orchestration between MB/MF and the impact of stress. RESULTS Both groups had comparable baseline and stress-induced cortisol response to the SECPT. Non-stressed PG displayed lower MB learning than HC. MANOVA and regression analyses showed a deleterious effect of stress-induced cortisol response on the orchestration between MB and MF learning in HC but not in PG. These effects remained when controlling for working memory and IQ. DISCUSSION AND CONCLUSIONS We found an abnormal pattern of modulation of stress on the orchestration between MB and MF learning among PG. Several interpretations and future research directions are discussed.
Collapse
Affiliation(s)
- Florent Wyckmans
- Psychological Medicine Laboratory, Université Libre de Bruxelles, Brussels, Belgium
| | - Nilosmita Banerjee
- Psychological Medicine Laboratory, Université Libre de Bruxelles, Brussels, Belgium
| | - Mélanie Saeremans
- Psychiatric Institute, Universitary Hospital Brugmann, Brussels, Belgium
| | - Ross Otto
- Department of Psychology, McGill University, Montréal, Canada
| | - Charles Kornreich
- Psychological Medicine Laboratory, Université Libre de Bruxelles, Brussels, Belgium,Psychiatric Institute, Universitary Hospital Brugmann, Brussels, Belgium
| | - Laetitia Vanderijst
- Psychological Medicine Laboratory, Université Libre de Bruxelles, Brussels, Belgium
| | - Damien Gruson
- Department of Laboratory Medicine, Cliniques Universitaires St-Luc, Brussels, Belgium
| | - Vincenzo Carbone
- Department of Laboratory Medicine, Cliniques Universitaires St-Luc, Brussels, Belgium
| | - Antoine Bechara
- Faculty of Psychology, University of Southern California, Los Angeles, USA
| | - Tony Buchanan
- Department of Psychology, Saint Louis University, USA
| | - Xavier Noël
- Psychological Medicine Laboratory, Université Libre de Bruxelles, Brussels, Belgium,Corresponding author. E-mail:
| |
Collapse
|
3
|
Hsu TC, Chang C, Wu LK, Looi CK. Effects of a Pair Programming Educational Robot-Based Approach on Students' Interdisciplinary Learning of Computational Thinking and Language Learning. Front Psychol 2022; 13:888215. [PMID: 35686069 PMCID: PMC9172847 DOI: 10.3389/fpsyg.2022.888215] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Accepted: 04/08/2022] [Indexed: 11/18/2022] Open
Abstract
Using educational robots (ERs) to integrate computational thinking (CT) with cross-disciplinary content has gone beyond Science, Technology, Engineering, and Mathematics (STEM), to include foreign-language learning (FL) and further cross-context target-language (TL) acquisition. Such integration must not solely emphasise CT problem-solving skills. Rather, it must provide students with interactive learning to support their target-language (TL) interaction while reducing potential TL anxiety. This study aimed to validate the effects of the proposed method of pair programming (PP) along with question-and-response interaction in a board-game activity on young learners’ CT skills and TL learning across contexts. Two Grade 6 classes, one with 15 students who were studying Chinese as a Second Language (CSL) and the other with 15 students who were studying English as a Foreign Language (EFL), participated in the activity. A series of instruments on achievement assessment, questionnaires on CT skills and TL anxiety, and sequential learning behaviour analysis were used to critically examine the results. The main conclusion is that the EFL group showed better social skills of cooperation on CT and lower TL learning anxiety, while the CSL group demonstrated better problem-solving skills in CT, but presented more behaviours of trial-and-error loops. Results not only contribute suggestions for cross-disciplinary learning but also provide support for cross-context instruction beyond educational coursework.
Collapse
Affiliation(s)
- Ting-Chia Hsu
- Department of Technology Application and Human Resource Development, National Taiwan Normal University, Taipei City, Taiwan
| | - Ching Chang
- Department of Technology Application and Human Resource Development, National Taiwan Normal University, Taipei City, Taiwan
| | - Long-Kai Wu
- Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan, China
| | - Chee-Kit Looi
- National Institute of Education, Nanyang Technological University, Singapore, Singapore
| |
Collapse
|
4
|
Collins AGE, Shenhav A. Advances in modeling learning and decision-making in neuroscience. Neuropsychopharmacology 2022; 47:104-118. [PMID: 34453117 PMCID: PMC8617262 DOI: 10.1038/s41386-021-01126-y] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/03/2021] [Revised: 07/14/2021] [Accepted: 07/22/2021] [Indexed: 02/07/2023]
Abstract
An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.
Collapse
Affiliation(s)
- Anne G E Collins
- Department of Psychology and Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.
| | - Amitai Shenhav
- Department of Cognitive, Linguistic, & Psychological Sciences and Carney Institute for Brain Science, Brown University, Providence, RI, USA.
| |
Collapse
|
5
|
Foerster A, Rothermund K, Parmar JJ, Moeller B, Frings C, Pfister R. Goal-Based Binding of Irrelevant Stimulus Features for Action Slips. Exp Psychol 2021; 68:206-213. [PMID: 34918539 PMCID: PMC8691204 DOI: 10.1027/1618-3169/a000525] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
Abstract. Binding between representations of stimuli and actions and
later retrieval of these compounds provide efficient shortcuts in action
control. Recent observations indicate that these mechanisms are not only
effective when action episodes go as planned, but they also seem to be at play
when actions go awry. Moreover, the human cognitive system even corrects traces
of error commission on the fly because it binds the intended but not actually
executed response to concurrent task-relevant stimuli, thus enabling retrieval
of a correct, but not actually executed response when encountering the stimulus
again. However, a plausible alternative interpretation of this finding is that
error commission triggers selective strengthening of the instructed
stimulus–response mapping instead, thus promoting its efficient
application in the future. The experiment presented here makes an unequivocal
case for episodic binding and retrieval in erroneous action episodes by showing
binding between task-irrelevant stimuli and correct responses.
Collapse
Affiliation(s)
- Anna Foerster
- Department of Psychology III, Julius-Maximilians-Universität of Würzburg, Germany
| | | | | | - Birte Moeller
- Department of Cognitive Psychology, University of Trier, Germany
| | - Christian Frings
- Department of Cognitive Psychology, University of Trier, Germany
| | - Roland Pfister
- Department of Psychology III, Julius-Maximilians-Universität of Würzburg, Germany
| |
Collapse
|
6
|
Modeling the influence of working memory, reinforcement, and action uncertainty on reaction time and choice during instrumental learning. Psychon Bull Rev 2021; 28:20-39. [PMID: 32710256 DOI: 10.3758/s13423-020-01774-z] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
What determines the speed of our decisions? Various models of decision-making have focused on perceptual evidence, past experience, and task complexity as important factors determining the degree of deliberation needed for a decision. Here, we build on a sequential sampling decision-making framework to develop a new model that captures a range of reaction time (RT) effects by accounting for both working memory and instrumental learning processes. The model captures choices and RTs at various stages of learning, and in learning environments with varying complexity. Moreover, the model generalizes from tasks with deterministic reward contingencies to probabilistic ones. The model succeeds in part by incorporating prior uncertainty over actions when modeling RT. This straightforward process model provides a parsimonious account of decision dynamics during instrumental learning and makes unique predictions about internal representations of action values.
Collapse
|
7
|
Fischer T, Demiris Y. Computational Modeling of Embodied Visual Perspective Taking. IEEE Trans Cogn Dev Syst 2020. [DOI: 10.1109/tcds.2019.2949861] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
8
|
Collins AGE, Cockburn J. Beyond dichotomies in reinforcement learning. Nat Rev Neurosci 2020; 21:576-586. [PMID: 32873936 DOI: 10.1038/s41583-020-0355-6] [Citation(s) in RCA: 46] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/20/2020] [Indexed: 11/09/2022]
Abstract
Reinforcement learning (RL) is a framework of particular importance to psychology, neuroscience and machine learning. Interactions between these fields, as promoted through the common hub of RL, has facilitated paradigm shifts that relate multiple levels of analysis in a singular framework (for example, relating dopamine function to a computationally defined RL signal). Recently, more sophisticated RL algorithms have been proposed to better account for human learning, and in particular its oft-documented reliance on two separable systems: a model-based (MB) system and a model-free (MF) system. However, along with many benefits, this dichotomous lens can distort questions, and may contribute to an unnecessarily narrow perspective on learning and decision-making. Here, we outline some of the consequences that come from overconfidently mapping algorithms, such as MB versus MF RL, with putative cognitive processes. We argue that the field is well positioned to move beyond simplistic dichotomies, and we propose a means of refocusing research questions towards the rich and complex components that comprise learning and decision-making.
Collapse
Affiliation(s)
- Anne G E Collins
- Department of Psychology and the Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.
| | - Jeffrey Cockburn
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| |
Collapse
|