1
|
Loosen AM, Seow TXF, Hauser TU. Consistency within change: Evaluating the psychometric properties of a widely used predictive-inference task. Behav Res Methods 2024; 56:7410-7426. [PMID: 38844601 PMCID: PMC11362202 DOI: 10.3758/s13428-024-02427-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/12/2024] [Indexed: 08/30/2024]
Abstract
Rapid adaptation to sudden changes in the environment is a hallmark of flexible human behaviour. Many computational, neuroimaging, and even clinical investigations studying this cognitive process have relied on a behavioural paradigm known as the predictive-inference task. However, the psychometric quality of this task has never been examined, leaving unanswered whether it is indeed suited to capture behavioural variation on a within- and between-subject level. Using a large-scale test-retest design (T1: N = 330; T2: N = 219), we assessed the internal (internal consistency) and temporal (test-retest reliability) stability of the task's most used measures. We show that the main measures capturing flexible belief and behavioural adaptation yield good internal consistency and overall satisfying test-retest reliability. However, some more complex markers of flexible behaviour show lower psychometric quality. Our findings have implications for the large corpus of previous studies using this task and provide clear guidance as to which measures should and should not be used in future studies.
Collapse
Affiliation(s)
- Alisa M Loosen
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK.
- Wellcome Centre for Human Neuroimaging, University College London, University College London, London, UK.
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| | - Tricia X F Seow
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, University College London, London, UK
| | - Tobias U Hauser
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, University College London, London, UK
- Department of Psychiatry and Psychotherapy, Medical School and University Hospital, Eberhard Karls University of Tübingen, Tübingen, Germany
- German Center for Mental Health (DZPG), Tübingen, Germany
| |
Collapse
|
2
|
Loosen AM, Kato A, Gu X. Revisiting the role of computational neuroimaging in the era of integrative neuroscience. Neuropsychopharmacology 2024:10.1038/s41386-024-01946-8. [PMID: 39242921 DOI: 10.1038/s41386-024-01946-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/05/2024] [Revised: 07/12/2024] [Accepted: 07/17/2024] [Indexed: 09/09/2024]
Abstract
Computational models have become integral to human neuroimaging research, providing both mechanistic insights and predictive tools for human cognition and behavior. However, concerns persist regarding the ecological validity of lab-based neuroimaging studies and whether their spatiotemporal resolution is not sufficient for capturing neural dynamics. This review aims to re-examine the utility of computational neuroimaging, particularly in light of the growing prominence of alternative neuroscientific methods and the growing emphasis on more naturalistic behaviors and paradigms. Specifically, we will explore how computational modeling can both enhance the analysis of high-dimensional imaging datasets and, conversely, how neuroimaging, in conjunction with other data modalities, can inform computational models through the lens of neurobiological plausibility. Collectively, this evidence suggests that neuroimaging remains critical for human neuroscience research, and when enhanced by computational models, imaging can serve an important role in bridging levels of analysis and understanding. We conclude by proposing key directions for future research, emphasizing the development of standardized paradigms and the integrative use of computational modeling across neuroimaging techniques.
Collapse
Affiliation(s)
- Alisa M Loosen
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| | - Ayaka Kato
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| | - Xiaosi Gu
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|
3
|
Goodwin I, Hester R, Garrido MI. Temporal stability of Bayesian belief updating in perceptual decision-making. Behav Res Methods 2024; 56:6349-6362. [PMID: 38129733 PMCID: PMC11335944 DOI: 10.3758/s13428-023-02306-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/24/2023] [Indexed: 12/23/2023]
Abstract
Bayesian inference suggests that perception is inferred from a weighted integration of prior contextual beliefs with current sensory evidence (likelihood) about the world around us. The perceived precision or uncertainty associated with prior and likelihood information is used to guide perceptual decision-making, such that more weight is placed on the source of information with greater precision. This provides a framework for understanding a spectrum of clinical transdiagnostic symptoms associated with aberrant perception, as well as individual differences in the general population. While behavioral paradigms are commonly used to characterize individual differences in perception as a stable characteristic, measurement reliability in these behavioral tasks is rarely assessed. To remedy this gap, we empirically evaluate the reliability of a perceptual decision-making task that quantifies individual differences in Bayesian belief updating in terms of the relative precision weighting afforded to prior and likelihood information (i.e., sensory weight). We analyzed data from participants (n = 37) who performed this task twice. We found that the precision afforded to prior and likelihood information showed high internal consistency and good test-retest reliability (ICC = 0.73, 95% CI [0.53, 0.85]) when averaged across participants, as well as at the individual level using hierarchical modeling. Our results provide support for the assumption that Bayesian belief updating operates as a stable characteristic in perceptual decision-making. We discuss the utility and applicability of reliable perceptual decision-making paradigms as a measure of individual differences in the general population, as well as a diagnostic tool in psychiatric research.
Collapse
Affiliation(s)
- Isabella Goodwin
- Melbourne School of Psychological Sciences, The University of Melbourne, Parkville Campus, Melbourne, Victoria, 3010, Australia.
| | - Robert Hester
- Melbourne School of Psychological Sciences, The University of Melbourne, Parkville Campus, Melbourne, Victoria, 3010, Australia
| | - Marta I Garrido
- Melbourne School of Psychological Sciences, The University of Melbourne, Parkville Campus, Melbourne, Victoria, 3010, Australia
- Graeme Clark Institute for Biomedical Engineering, The University of Melbourne, Melbourne, Victoria, Australia
| |
Collapse
|
4
|
Mikus N, Lamm C, Mathys C. Computational phenotyping of aberrant belief updating in individuals with schizotypal traits and schizophrenia. Biol Psychiatry 2024:S0006-3223(24)01554-3. [PMID: 39218138 DOI: 10.1016/j.biopsych.2024.08.021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Revised: 08/10/2024] [Accepted: 08/13/2024] [Indexed: 09/04/2024]
Abstract
BACKGROUND Psychotic experiences are thought to emerge from various interrelated patterns of disrupted belief updating, such as overestimating the reliability of sensory information and misjudging task volatility. Yet, these substrates have never been jointly addressed under one computational framework and it is not clear to what degree they reflect trait-like computational patterns. METHODS We introduced a novel hierarchical Bayesian model that describes how individuals simultaneously update their beliefs about the task volatility and noise in observation. We applied this model to data from a modified Predictive inference task in a test-retest study with healthy volunteers (N=45, 4 sessions) and examined the relationship between model parameters and schizotypal traits in a larger online sample (N = 437) and in a cohort of patients with schizophrenia (N = 100). RESULTS The interclass correlations were moderate to high for model parameters and excellent for averaged belief trajectories and precision-weighted learning rates estimated through hierarchical Bayesian inference. We found that uncertainty about the task volatility was related to schizotypal traits and to positive symptoms in patients, when learning to gain rewards. In contrast, negative symptoms in patients were associated with more rigid beliefs about observational noise, when learning to avoid losses. CONCLUSION These findings suggest that individuals with schizotypal traits across the psychosis continuum are less likely to learn or utilize higher-order statistical regularities of the environment and showcase the potential of clinically relevant computational phenotypes for differentiating symptom groups in a transdiagnostic manner.
Collapse
Affiliation(s)
- Nace Mikus
- Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Austria; Interacting Minds Centre, Aarhus University, Denmark.
| | - Claus Lamm
- Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Austria
| | - Christoph Mathys
- Interacting Minds Centre, Aarhus University, Denmark; Translational Neuromodeling Unit, University of Zurich and ETH Zurich, Zurich, Switzerland;; Scuola Internazionale Superiore di Studi Avanzati (SISSA), Trieste, Italy
| |
Collapse
|
5
|
Zika O, Appel J, Klinge C, Shkreli L, Browning M, Wiech K, Reinecke A. Reduction of Aversive Learning Rates in Pavlovian Conditioning by Angiotensin II Antagonist Losartan: A Randomized Controlled Trial. Biol Psychiatry 2024; 96:247-255. [PMID: 38309320 DOI: 10.1016/j.biopsych.2024.01.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 01/12/2024] [Accepted: 01/22/2024] [Indexed: 02/05/2024]
Abstract
BACKGROUND Angiotensin receptor blockade has been linked to aspects of aversive learning and memory formation and to the prevention of posttraumatic stress disorder symptom development. METHODS We investigated the influence of the angiotensin receptor blocker losartan on aversive Pavlovian conditioning using a probabilistic learning paradigm. In a double-blind, randomized, placebo-controlled design, we tested 45 (18 female) healthy volunteers during a baseline session, after application of losartan or placebo (drug session), and during a follow-up session. During each session, participants engaged in a task in which they had to predict the probability of an electrical stimulation on every trial while the true shock contingencies switched repeatedly between phases of high and low shock threat. Computational reinforcement learning models were used to investigate learning dynamics. RESULTS Acute administration of losartan significantly reduced participants' adjustment during both low-to-high and high-to-low threat changes. This was driven by reduced aversive learning rates in the losartan group during the drug session compared with baseline. The 50-mg drug dose did not induce reduction of blood pressure or change in reaction times, ruling out a general reduction in attention and engagement. Decreased adjustment of aversive expectations was maintained at a follow-up session 24 hours later. CONCLUSIONS This study shows that losartan acutely reduces Pavlovian learning in aversive environments, thereby highlighting a potential role of the renin-angiotensin system in anxiety development.
Collapse
Affiliation(s)
- Ondrej Zika
- Max Planck Institute for Human Development, Berlin, Germany
| | - Judith Appel
- Behavioural Science Institute, Radboud University Nijmegen, Nijmegen, the Netherlands
| | - Corinna Klinge
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom
| | - Lorika Shkreli
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom
| | - Michael Browning
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom; Oxford Health NHS Trust, Warneford Hospital, Oxford, United Kingdom
| | - Katja Wiech
- Wellcome Centre for Integrative Functional Neuroimaging, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - Andrea Reinecke
- Department of Psychiatry, University of Oxford, Oxford, United Kingdom; Oxford Health NHS Trust, Warneford Hospital, Oxford, United Kingdom.
| |
Collapse
|
6
|
Schaaf JV, Weidinger L, Molleman L, van den Bos W. Test-retest reliability of reinforcement learning parameters. Behav Res Methods 2024; 56:4582-4599. [PMID: 37684495 PMCID: PMC11289054 DOI: 10.3758/s13428-023-02203-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/18/2023] [Indexed: 09/10/2023]
Abstract
It has recently been suggested that parameter estimates of computational models can be used to understand individual differences at the process level. One area of research in which this approach, called computational phenotyping, has taken hold is computational psychiatry. One requirement for successful computational phenotyping is that behavior and parameters are stable over time. Surprisingly, the test-retest reliability of behavior and model parameters remains unknown for most experimental tasks and models. The present study seeks to close this gap by investigating the test-retest reliability of canonical reinforcement learning models in the context of two often-used learning paradigms: a two-armed bandit and a reversal learning task. We tested independent cohorts for the two tasks (N = 69 and N = 47) via an online testing platform with a between-test interval of five weeks. Whereas reliability was high for personality and cognitive measures (with ICCs ranging from .67 to .93), it was generally poor for the parameter estimates of the reinforcement learning models (with ICCs ranging from .02 to .52 for the bandit task and from .01 to .71 for the reversal learning task). Given that simulations indicated that our procedures could detect high test-retest reliability, this suggests that a significant proportion of the variability must be ascribed to the participants themselves. In support of that hypothesis, we show that mood (stress and happiness) can partly explain within-participant variability. Taken together, these results are critical for current practices in computational phenotyping and suggest that individual variability should be taken into account in the future development of the field.
Collapse
Affiliation(s)
- Jessica V Schaaf
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands.
- Cognitive Neuroscience Department, Radboud University Medical Centre, Nijmegen, the Netherlands.
- Donders Institute for Brain, Cognition and Behaviour, Nijmegen, the Netherlands.
| | - Laura Weidinger
- DeepMind, London, United Kingdom
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany
| | - Lucas Molleman
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany
| | - Wouter van den Bos
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany
| |
Collapse
|
7
|
Waltmann M, Herzog N, Reiter AMF, Villringer A, Horstmann A, Deserno L. Neurocomputational Mechanisms Underlying Differential Reinforcement Learning From Wins and Losses in Obesity With and Without Binge Eating. BIOLOGICAL PSYCHIATRY. COGNITIVE NEUROSCIENCE AND NEUROIMAGING 2024:S2451-9022(24)00160-5. [PMID: 38909896 DOI: 10.1016/j.bpsc.2024.06.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Revised: 05/24/2024] [Accepted: 06/09/2024] [Indexed: 06/25/2024]
Abstract
BACKGROUND Binge-eating disorder (BED) is thought of as a disorder of cognitive control, but evidence regarding its neurocognitive mechanisms is inconclusive. Key limitations of previous research include a lack of consistent separation between effects of BED and obesity and a disregard for self-report evidence suggesting that neurocognitive alterations may emerge primarily in loss- or harm-avoidance contexts. METHODS To address these gaps, in this longitudinal study we investigated behavioral flexibility and its underlying neurocomputational processes in reward-seeking and loss-avoidance contexts. Obese participants with BED, obese participants without BED, and healthy normal-weight participants (n = 96) performed a probabilistic reversal learning task during functional imaging, with different blocks focused on obtaining wins or avoiding losses. They were reinvited for a 6-month follow-up assessment. RESULTS Analyses informed by computational models of reinforcement learning showed that unlike obese participants with BED, obese participants without BED performed worse in the win than in the loss condition. Computationally, this was explained by differential learning sensitivities in the win versus loss conditions in the groups. In the brain, this was echoed in differential neural learning signals in the ventromedial prefrontal cortex per condition. The differences were subtle but scaled with BED symptoms, such that more severe BED symptoms were associated with increasing bias toward improved learning from wins versus losses. Across conditions, obese participants with BED switched more between choice options than healthy normal-weight participants. This was reflected in diminished representation of choice certainty in the ventromedial prefrontal cortex. CONCLUSIONS Our study highlights the importance of distinguishing between obesity with and without BED to identify unique neurocomputational alterations underlying different styles of maladaptive eating behavior.
Collapse
Affiliation(s)
- Maria Waltmann
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Würzburg, Germany; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
| | - Nadine Herzog
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Andrea M F Reiter
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Würzburg, Germany; CRC-940 Volition and Cognitive Control, Faculty of Psychology, Technical University of Dresden, Dresden, Germany; Department of Psychology, Julius-Maximilians-University of Würzburg, Würzburg, Germany
| | - Arno Villringer
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; MindBrainBody Institute, Berlin School of Mind and Brain, Charité-Universitätsmedizin Berlin and Humboldt, Universität zu Berlin, Berlin, Germany
| | - Annette Horstmann
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
| | - Lorenz Deserno
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Würzburg, Germany; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; Department of Psychiatry and Psychotherapy, Technical University of Dresden, Dresden, Germany
| |
Collapse
|
8
|
Sohail A, Zhang L. Informing the treatment of social anxiety disorder with computational and neuroimaging data. PSYCHORADIOLOGY 2024; 4:kkae010. [PMID: 38841558 PMCID: PMC11152174 DOI: 10.1093/psyrad/kkae010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Revised: 04/15/2024] [Accepted: 04/25/2024] [Indexed: 06/07/2024]
Affiliation(s)
- Aamir Sohail
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, B15 2TT, UK
| | - Lei Zhang
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, B15 2TT, UK
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, B15 2TT, UK
| |
Collapse
|
9
|
Schurr R, Reznik D, Hillman H, Bhui R, Gershman SJ. Dynamic computational phenotyping of human cognition. Nat Hum Behav 2024; 8:917-931. [PMID: 38332340 PMCID: PMC11132988 DOI: 10.1038/s41562-024-01814-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2023] [Accepted: 12/21/2023] [Indexed: 02/10/2024]
Abstract
Computational phenotyping has emerged as a powerful tool for characterizing individual variability across a variety of cognitive domains. An individual's computational phenotype is defined as a set of mechanistically interpretable parameters obtained from fitting computational models to behavioural data. However, the interpretation of these parameters hinges critically on their psychometric properties, which are rarely studied. To identify the sources governing the temporal variability of the computational phenotype, we carried out a 12-week longitudinal study using a battery of seven tasks that measure aspects of human learning, memory, perception and decision making. To examine the influence of state effects, each week, participants provided reports tracking their mood, habits and daily activities. We developed a dynamic computational phenotyping framework, which allowed us to tease apart the time-varying effects of practice and internal states such as affective valence and arousal. Our results show that many phenotype dimensions covary with practice and affective factors, indicating that what appears to be unreliability may reflect previously unmeasured structure. These results support a fundamentally dynamic understanding of cognitive variability within an individual.
Collapse
Affiliation(s)
- Roey Schurr
- Department of Psychology, Center for Brain Sciences, Harvard University, Cambridge, MA, USA.
| | - Daniel Reznik
- Department of Psychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
| | - Hanna Hillman
- Department of Psychology, Yale University, New Haven, CT, USA
| | - Rahul Bhui
- Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA
- Institute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Samuel J Gershman
- Department of Psychology, Center for Brain Sciences, Harvard University, Cambridge, MA, USA
- Center for Brains, Minds, and Machines, Massachusetts Institute of Technology, Cambridge, MA, USA
| |
Collapse
|
10
|
Aster HC, Waltmann M, Busch A, Romanos M, Gamer M, Maria van Noort B, Beck A, Kappel V, Deserno L. Impaired flexible reward learning in ADHD patients is associated with blunted reinforcement sensitivity and neural signals in ventral striatum and parietal cortex. Neuroimage Clin 2024; 42:103588. [PMID: 38471434 PMCID: PMC10943992 DOI: 10.1016/j.nicl.2024.103588] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 02/06/2024] [Accepted: 02/28/2024] [Indexed: 03/14/2024]
Abstract
Reward-based learning and decision-making are prime candidates to understand symptoms of attention deficit hyperactivity disorder (ADHD). However, only limited evidence is available regarding the neurocomputational underpinnings of the alterations seen in ADHD. This concerns flexible behavioral adaption in dynamically changing environments, which is challenging for individuals with ADHD. One previous study points to elevated choice switching in adolescent ADHD, which was accompanied by disrupted learning signals in medial prefrontal cortex. Here, we investigated young adults with ADHD (n = 17) as compared to age- and sex-matched controls (n = 17) using a probabilistic reversal learning experiment during functional magnetic resonance imaging (fMRI). The task requires continuous learning to guide flexible behavioral adaptation to changing reward contingencies. To disentangle the neurocomputational underpinnings of the behavioral data, we used reinforcement learning (RL) models, which informed the analysis of fMRI data. ADHD patients performed worse than controls particularly in trials before reversals, i.e., when reward contingencies were stable. This pattern resulted from 'noisy' choice switching regardless of previous feedback. RL modelling showed decreased reinforcement sensitivity and enhanced learning rates for negative feedback in ADHD patients. At the neural level, this was reflected in a diminished representation of choice probability in the left posterior parietal cortex in ADHD. Moreover, modelling showed a marginal reduction of learning about the unchosen option, which was paralleled by a marginal reduction in learning signals incorporating the unchosen option in the left ventral striatum. Taken together, we show that impaired flexible behavior in ADHD is due to excessive choice switching ('hyper-flexibility'), which can be detrimental or beneficial depending on the learning environment. Computationally, this resulted from blunted sensitivity to reinforcement of which we detected neural correlates in the attention-control network, specifically in the parietal cortex. These neurocomputational findings remain preliminary due to the relatively small sample size.
Collapse
Affiliation(s)
- Hans-Christoph Aster
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University Hospital Würzburg, Würzburg, Germany.
| | - Maria Waltmann
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University Hospital Würzburg, Würzburg, Germany; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Anika Busch
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University Hospital Würzburg, Würzburg, Germany
| | - Marcel Romanos
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University Hospital Würzburg, Würzburg, Germany
| | - Matthias Gamer
- Department of Psychology, University of Würzburg, Würzburg, Germany
| | - Betteke Maria van Noort
- Department of Child and Adolescent Psychiatry, Charité University Medicine, Campus Virchow Klinikum, Berlin, Germany; MSB Medical School Berlin, Department of Psychology, Germany
| | - Anne Beck
- Department of Psychiatry and Neurosciences, Charité University Medicine, Berlin, Germany; Department of Psychology, Faculty of Health, Health and Medical University, Potsdam, Germany
| | - Viola Kappel
- Department of Child and Adolescent Psychiatry, Charité University Medicine, Campus Virchow Klinikum, Berlin, Germany
| | - Lorenz Deserno
- Department of Child and Adolescent Psychiatry, Psychotherapy and Psychosomatics, University Hospital Würzburg, Würzburg, Germany; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; Department of Psychiatry and Psychotherapy, Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
11
|
Colas JT, O’Doherty JP, Grafton ST. Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts. PLoS Comput Biol 2024; 20:e1011950. [PMID: 38552190 PMCID: PMC10980507 DOI: 10.1371/journal.pcbi.1011950] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 02/26/2024] [Indexed: 04/01/2024] Open
Abstract
Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.
Collapse
Affiliation(s)
- Jaron T. Colas
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, California, United States of America
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, United States of America
- Computation and Neural Systems Program, California Institute of Technology, Pasadena, California, United States of America
| | - John P. O’Doherty
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, United States of America
- Computation and Neural Systems Program, California Institute of Technology, Pasadena, California, United States of America
| | - Scott T. Grafton
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, California, United States of America
| |
Collapse
|
12
|
Yamamori Y, Robinson OJ, Roiser JP. Approach-avoidance reinforcement learning as a translational and computational model of anxiety-related avoidance. eLife 2023; 12:RP87720. [PMID: 37963085 PMCID: PMC10645421 DOI: 10.7554/elife.87720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2023] Open
Abstract
Although avoidance is a prevalent feature of anxiety-related psychopathology, differences in the measurement of avoidance between humans and non-human animals hinder our progress in its theoretical understanding and treatment. To address this, we developed a novel translational measure of anxiety-related avoidance in the form of an approach-avoidance reinforcement learning task, by adapting a paradigm from the non-human animal literature to study the same cognitive processes in human participants. We used computational modelling to probe the putative cognitive mechanisms underlying approach-avoidance behaviour in this task and investigated how they relate to subjective task-induced anxiety. In a large online study (n = 372), participants who experienced greater task-induced anxiety avoided choices associated with punishment, even when this resulted in lower overall reward. Computational modelling revealed that this effect was explained by greater individual sensitivities to punishment relative to rewards. We replicated these findings in an independent sample (n = 627) and we also found fair-to-excellent reliability of measures of task performance in a sub-sample retested 1 week later (n = 57). Our findings demonstrate the potential of approach-avoidance reinforcement learning tasks as translational and computational models of anxiety-related avoidance. Future studies should assess the predictive validity of this approach in clinical samples and experimental manipulations of anxiety.
Collapse
Affiliation(s)
- Yumeya Yamamori
- Institute of Cognitive Neuroscience, University College LondonLondonUnited Kingdom
| | - Oliver J Robinson
- Institute of Cognitive Neuroscience, University College LondonLondonUnited Kingdom
- Research Department of Clinical, Educational and Health Psychology, University College LondonLondonUnited Kingdom
| | - Jonathan P Roiser
- Institute of Cognitive Neuroscience, University College LondonLondonUnited Kingdom
| |
Collapse
|
13
|
Young MK, Conn KA, Das J, Zou S, Alexander S, Burne TH, Kesby JP. Activity in the Dorsomedial Striatum Underlies Serial Reversal Learning Performance Under Probabilistic Uncertainty. BIOLOGICAL PSYCHIATRY GLOBAL OPEN SCIENCE 2023; 3:1030-1041. [PMID: 37881585 PMCID: PMC10593872 DOI: 10.1016/j.bpsgos.2022.08.005] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 08/16/2022] [Accepted: 08/17/2022] [Indexed: 11/17/2022] Open
Abstract
Background Corticostriatal circuits, particularly the dorsomedial striatum (DMS) and lateral orbitofrontal cortex, are critical for navigating reversal learning under probabilistic uncertainty. These same areas are implicated in the reversal learning impairments observed in individuals with psychosis as well as their psychotic symptoms, suggesting that they may share a common neurobiological substrate. To address this question, we used psychostimulant exposure and specific activation of the DMS during reversal learning in mice to assess corticostriatal activity. Methods We used amphetamine treatment to induce psychosis-relevant neurobiology in male mice during reversal learning and to examine pathway-specific corticostriatal activation. To determine the causal role of DMS activity, we used chemogenetics to drive midbrain inputs during a range of probabilistic contingencies. Results Mice treated with amphetamine showed altered punishment learning, which was associated with decreased shifting after losses and increased perseverative errors after reversals. Reversal learning performance and strategies were dependent on increased activity in lateral orbitofrontal cortex to DMS circuits as well as in the DMS itself. Specific activation of midbrain to DMS circuits also decreased shifting after losses and reversal learning performance. However, these alterations were dependent on the probabilistic contingency. Conclusions Our work suggests that the DMS plays a multifaceted role in reversal learning. Increasing DMS activity impairs multiple reversal learning processes dependent on the level of uncertainty, confirming its role in the maintenance and selection of incoming cortical inputs. Together, these outcomes suggest that elevated dopamine levels in the DMS could contribute to decision-making impairments in individuals with psychosis.
Collapse
Affiliation(s)
- Madison K. Young
- Queensland Brain Institute, The University of Queensland, Brisbane, Queensland, Australia
| | - Kyna-Anne Conn
- Queensland Brain Institute, The University of Queensland, Brisbane, Queensland, Australia
| | - Joyosmita Das
- Queensland Brain Institute, The University of Queensland, Brisbane, Queensland, Australia
| | - Simin Zou
- Queensland Brain Institute, The University of Queensland, Brisbane, Queensland, Australia
| | - Suzy Alexander
- Queensland Brain Institute, The University of Queensland, Brisbane, Queensland, Australia
- Queensland Centre for Mental Health Research, Brisbane, Queensland, Australia
| | - Thomas H.J. Burne
- Queensland Brain Institute, The University of Queensland, Brisbane, Queensland, Australia
- Queensland Centre for Mental Health Research, Brisbane, Queensland, Australia
| | - James P. Kesby
- Queensland Brain Institute, The University of Queensland, Brisbane, Queensland, Australia
- Queensland Centre for Mental Health Research, Brisbane, Queensland, Australia
- QIMR Berghofer Medical Research Institute, Brisbane, Queensland, Australia
| |
Collapse
|
14
|
Zech HG, Gable P, van Dijk WW, van Dillen LF. Test-retest reliability of a smartphone-based approach-avoidance task: Effects of retest period, stimulus type, and demographics. Behav Res Methods 2023; 55:2652-2668. [PMID: 35915356 PMCID: PMC9342838 DOI: 10.3758/s13428-022-01920-6] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/25/2022] [Indexed: 11/25/2022]
Abstract
The approach-avoidance task (AAT) is an implicit task that measures people's behavioral tendencies to approach or avoid stimuli in the environment. In recent years, it has been used successfully to help explain a variety of health problems (e.g., addictions and phobias). Unfortunately, more recent AAT studies have failed to replicate earlier promising findings. One explanation for these replication failures could be that the AAT does not reliably measure approach-avoidance tendencies. Here, we first review existing literature on the reliability of various versions of the AAT. Next, we examine the AAT's reliability in a large and diverse sample (N = 1077; 248 of whom completed all sessions). Using a smartphone-based, mobile AAT, we measured participants' approach-avoidance tendencies eight times over a period of seven months (one measurement per month) in two distinct stimulus sets (happy/sad expressions and disgusting/neutral stimuli). The mobile AAT's split-half reliability was adequate for face stimuli (r = .85), but low for disgust stimuli (r = .72). Its test-retest reliability based on a single measurement was poor for either stimulus set (all ICC1s < .3). Its test-retest reliability based on the average of all eight measurements was moderately good for face stimuli (ICCk = .73), but low for disgust stimuli (ICCk = .5). Results suggest that single-measurement AATs could be influenced by unexplained temporal fluctuations of approach-avoidance tendencies. These fluctuations could be examined in future studies. Until then, this work suggests that future research using the AAT should rely on multiple rather than single measurements.
Collapse
Affiliation(s)
- Hilmar G Zech
- Leiden University, Leiden, The Netherlands.
- Technical University Dresden, Dresden, Germany.
| | | | - Wilco W van Dijk
- Leiden University, Leiden, The Netherlands
- Knowledge Centre Psychology and Economic Behaviour, Leiden, The Netherlands
| | - Lotte F van Dillen
- Leiden University, Leiden, The Netherlands
- Knowledge Centre Psychology and Economic Behaviour, Leiden, The Netherlands
| |
Collapse
|
15
|
Lopez KL, Monachino AD, Vincent KM, Peck FC, Gabard-Durnam LJ. Stability, change, and reliable individual differences in electroencephalography measures: a lifespan perspective on progress and opportunities. Neuroimage 2023; 275:120116. [PMID: 37169118 DOI: 10.1016/j.neuroimage.2023.120116] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 03/27/2023] [Accepted: 04/13/2023] [Indexed: 05/13/2023] Open
Abstract
Electroencephalographic (EEG) methods have great potential to serve both basic and clinical science approaches to understand individual differences in human neural function. Importantly, the psychometric properties of EEG data, such as internal consistency and test-retest reliability, constrain their ability to differentiate individuals successfully. Rapid and recent technological and computational advancements in EEG research make it timely to revisit the topic of psychometric reliability in the context of individual difference analyses. Moreover, pediatric and clinical samples provide some of the most salient and urgent opportunities to apply individual difference approaches, but the changes these populations experience over time also provide unique challenges from a psychometric perspective. Here we take a developmental neuroscience perspective to consider progress and new opportunities for parsing the reliability and stability of individual differences in EEG measurements across the lifespan. We first conceptually map the different profiles of measurement reliability expected for different types of individual difference analyses over the lifespan. Next, we summarize and evaluate the state of the field's empirical knowledge and need for testing measurement reliability, both internal consistency and test-retest reliability, across EEG measures of power, event-related potentials, nonlinearity, and functional connectivity across ages. Finally, we highlight how standardized pre-processing software for EEG denoising and empirical metrics of individual data quality may be used to further improve EEG-based individual differences research moving forward. We also include recommendations and resources throughout that individual researchers can implement to improve the utility and reproducibility of individual differences analyses with EEG across the lifespan.
Collapse
Affiliation(s)
- K L Lopez
- Northeastern University, 360 Huntington Ave, Boston, MA, United States
| | - A D Monachino
- Northeastern University, 360 Huntington Ave, Boston, MA, United States
| | - K M Vincent
- Northeastern University, 360 Huntington Ave, Boston, MA, United States
| | - F C Peck
- University of California, Los Angeles, Los Angeles, CA, United States
| | - L J Gabard-Durnam
- Northeastern University, 360 Huntington Ave, Boston, MA, United States.
| |
Collapse
|
16
|
Eckstein MK, Master SL, Xia L, Dahl RE, Wilbrecht L, Collins AGE. The interpretation of computational model parameters depends on the context. eLife 2022; 11:e75474. [PMID: 36331872 PMCID: PMC9635876 DOI: 10.7554/elife.75474] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Accepted: 09/09/2022] [Indexed: 11/06/2022] Open
Abstract
Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning to complex problem solving, to shed light on developmental and individual differences, and to anchor cognitive processes in specific brain mechanisms. However, the RL literature increasingly reveals contradictory results, which might cast doubt on these claims. We hypothesized that many contradictions arise from two commonly-held assumptions about computational model parameters that are actually often invalid: That parameters generalize between contexts (e.g. tasks, models) and that they capture interpretable (i.e. unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8-30 years to complete three learning tasks in one experimental session, and fitted RL models to each. We found that some parameters (exploration / decision noise) showed significant generalization: they followed similar developmental trajectories, and were reciprocally predictive between tasks. Still, generalization was significantly below the methodological ceiling. Furthermore, other parameters (learning rates, forgetting) did not show evidence of generalization, and sometimes even opposite developmental trajectories. Interpretability was low for all parameters. We conclude that the systematic study of context factors (e.g. reward stochasticity; task volatility) will be necessary to enhance the generalizability and interpretability of computational cognitive models.
Collapse
Affiliation(s)
| | - Sarah L Master
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
- Department of Psychology, New York UniversityNew YorkUnited States
| | - Liyu Xia
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
- Department of Mathematics, University of California, BerkeleyBerkeleyUnited States
| | - Ronald E Dahl
- Institute of Human Development, University of California, BerkeleyBerkeleyUnited States
| | - Linda Wilbrecht
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
- Helen Wills Neuroscience Institute, University of California, BerkeleyBerkeleyUnited States
| | - Anne GE Collins
- Department of Psychology, University of California, BerkeleyBerkeleyUnited States
- Helen Wills Neuroscience Institute, University of California, BerkeleyBerkeleyUnited States
| |
Collapse
|