1
|
Hitchcock PF, Frank MJ. The challenge of learning adaptive mental behavior. JOURNAL OF PSYCHOPATHOLOGY AND CLINICAL SCIENCE 2024; 133:413-426. [PMID: 38815082 PMCID: PMC11229419 DOI: 10.1037/abn0000924] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/01/2024]
Abstract
Many psychotherapies aim to help people replace maladaptive mental behaviors (such as those leading to unproductive worry) with more adaptive ones (such as those leading to active problem solving). Yet, little is known empirically about how challenging it is to learn adaptive mental behaviors. Mental behaviors entail taking mental operations and thus may be more challenging to perform than motor actions; this challenge may enhance or impair learning. In particular, challenge when learning is often desirable because it improves retention. Yet, it is also plausible that the necessity of carrying out mental operations interferes with learning the expected values of mental actions by impeding credit assignment: the process of updating an action's value after reinforcement. Then, it may be more challenging not only to perform-but also to learn the consequences of-mental (vs. motor) behaviors. We designed a task to assess learning to take adaptive mental versus motor actions via matched probabilistic feedback. In two experiments (N = 300), most participants found it more difficult to learn to select optimal mental (vs. motor) actions, as evident in worse accuracy not only in a learning but also test (retention) phase. Computational modeling traced this impairment to an indicator of worse credit assignment (impaired construction and maintenance of expected values) when learning mental actions, accounting for worse accuracy in the learning and retention phases. The results suggest that people have particular difficulty learning adaptive mental behavior and pave the way for novel interventions to scaffold credit assignment and promote adaptive thinking. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
Collapse
Affiliation(s)
- Peter F. Hitchcock
- Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, RI
- Department of Psychology, Emory University, Atlanta GA
| | - Michael J. Frank
- Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, RI
- Carney Institute for Brain Science, Brown University, Providence, RI
| |
Collapse
|
2
|
Guo Z, Yu J, Wang W, Lockwood P, Wu Z. Reinforcement learning of altruistic punishment differs between cultures and across the lifespan. PLoS Comput Biol 2024; 20:e1012274. [PMID: 38990982 PMCID: PMC11288421 DOI: 10.1371/journal.pcbi.1012274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Revised: 07/30/2024] [Accepted: 06/24/2024] [Indexed: 07/13/2024] Open
Abstract
Altruistic punishment is key to establishing cooperation and maintaining social order, yet its developmental trends across cultures remain unclear. Using computational reinforcement learning models, we provided the first evidence of how social feedback dynamically influences group-biased altruistic punishment across cultures and the lifespan. Study 1 (n = 371) found that Chinese participants exhibited higher learning rates than Americans when socially incentivized to punish unfair allocations. Additionally, Chinese adults showed slower learning and less exploration when punishing ingroups than outgroups, a pattern absent in American counterparts, potentially reflecting a tendency towards ingroup favoritism that may contribute to reinforcing collectivist values. Study 2 (n = 430, aged 12-52) further showed that such ingroup favoritism develops with age. Chinese participants' learning rates for ingroup punishment decreased from adolescence into adulthood, while outgroup rates stayed constant, implying a process of cultural learning. Our findings highlight cultural and age-related variations in altruistic punishment learning, with implications for social reinforcement learning and culturally sensitive educational practices promoting fairness and altruism.
Collapse
Affiliation(s)
- Ziyan Guo
- Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing, China
- Lab for Lifelong Learning, Tsinghua University, Beijing, China
| | - Jialu Yu
- Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing, China
- Lab for Lifelong Learning, Tsinghua University, Beijing, China
| | - Wenxin Wang
- Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing, China
- Lab for Lifelong Learning, Tsinghua University, Beijing, China
| | - Patricia Lockwood
- Centre for Human Brain Health and Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, United Kingdom
| | - Zhen Wu
- Department of Psychological and Cognitive Sciences, Tsinghua University, Beijing, China
- Lab for Lifelong Learning, Tsinghua University, Beijing, China
| |
Collapse
|
3
|
Bellucci G, Park SQ. Loneliness is associated with more trust but worse trustworthiness expectations. Br J Psychol 2024. [PMID: 38807533 DOI: 10.1111/bjop.12713] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Accepted: 05/07/2024] [Indexed: 05/30/2024]
Abstract
Subjective feelings of loneliness emerge due to unsatisfactory social relationships, representing a major risk for mental and physical well-being. Despite its social nature, evidence on how loneliness affects social behaviours and expectations is lacking. Using Bayesian analyses and economic games, we show in three different studies that lonelier individuals trusted their partners to a greater extent despite less favourable trustworthiness expectations, showing a greater discrepancy between their trusting behaviours and their expectations of others' trustworthiness. Such discrepancy was reversed in extravert individuals who also reported to be less lonely. These results provide evidence on two opposing effects of loneliness as a motivator for social connections and promoter of social withdrawal, and demonstrate the moderating role of personality traits. This work contributes to a better understanding of how loneliness impacts social behaviour and social expectations, with important downstream clinical implications for varying health conditions associated with heightened feelings of loneliness.
Collapse
Affiliation(s)
- Gabriele Bellucci
- Department of Psychology, Royal Holloway, University of London, Egham, UK
- Department of Psychology I, University of Lübeck, Lübeck, Germany
| | - Soyoung Q Park
- Department of Psychology I, University of Lübeck, Lübeck, Germany
- Department of Decision Neuroscience and Nutrition, German Institute of Human Nutrition (DIfE), Potsdam-Rehbruecke, Germany
- Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität Zu Berlin and Berlin Institute of Health, Neuroscience Research Center, Berlin, Germany
- German Center for Diabetes Research (DZD), Neuherberg, Germany
| |
Collapse
|
4
|
Kirkland JM, Edgar EL, Patel I, Feustel P, Belin S, Kopec AM. Synaptic pruning during adolescence shapes adult social behavior in both males and females. Dev Psychobiol 2024; 66:e22473. [PMID: 38433422 DOI: 10.1002/dev.22473] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2023] [Revised: 01/23/2024] [Accepted: 01/30/2024] [Indexed: 03/05/2024]
Abstract
Evolutionarily conserved, peer-directed social behaviors are essential to participate in many aspects of human society. These behaviors directly impact psychological, physiological, and behavioral maturation. Adolescence is an evolutionarily conserved period during which reward-related behaviors, including social behaviors, develop via developmental plasticity in the mesolimbic dopaminergic "reward" circuitry of the brain. The nucleus accumbens (NAc) is an intermediate reward relay center that develops during adolescence and mediates both social behaviors and dopaminergic signaling. In several developing brain regions, synaptic pruning mediated by microglia, the resident immune cells of the brain, is important for normal behavioral development. We previously demonstrated that during adolescence, in rats, microglial synaptic pruning shapes the development of NAc and social play behavior in males and females. In this report, we hypothesize that interrupting microglial pruning in NAc during adolescence will have persistent effects on male and female social behavior in adulthood. We found that inhibiting microglial pruning in the NAc during adolescence had different effects on social behavior in males and females. In males, inhibiting pruning increased familiar exploration and increased nonsocial contact. In females, inhibiting pruning did not change familiar exploration behavior but increased active social interaction. This leads us to infer that naturally occurring NAc pruning serves to reduce social behaviors toward a familiar conspecific in both males and females.
Collapse
Affiliation(s)
- Julia M Kirkland
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, Albany, New York, USA
| | - Erin L Edgar
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, Albany, New York, USA
| | - Ishan Patel
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, Albany, New York, USA
| | - Paul Feustel
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, Albany, New York, USA
| | - Sophie Belin
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, Albany, New York, USA
| | - Ashley M Kopec
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, Albany, New York, USA
| |
Collapse
|
5
|
Jin Y, Gao Q, Wang Y, Dietz M, Xiao L, Cai Y, Bliksted V, Zhou Y. Impaired social learning in patients with major depressive disorder revealed by a reinforcement learning model. Int J Clin Health Psychol 2023; 23:100389. [PMID: 37829189 PMCID: PMC10564931 DOI: 10.1016/j.ijchp.2023.100389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Accepted: 05/03/2023] [Indexed: 10/14/2023] Open
Abstract
Background/objective Patients with major depressive disorder (MDD) have altered learning rates for rewards and losses in non-social learning paradigms. However, it is not well understood whether the ability to learn from social interactions is altered in MDD patients. Using reinforcement learning during the repeated Trust Game (rTG), we investigated how MDD patients learn to trust newly-met partners in MDD patients. Method Sixty-eight MDD patients and fifty-four controls each played as 'investor' and interacted with ten different partners. We manipulated both the level of trustworthiness by varying the chance of reciprocity (10, 30, 50, 70 and 90%) and reputation disclosure, where partners' reputation was either pre-disclosed or hidden. Results Our reinforcement learning model revealed that MDD patients had significantly higher learning rates for losses than the controls in both the reputation disclosure and non-disclosure condition. The difference was larger when reputation was not disclosed than disclosed. We observed no difference in learning rates for gains in either condition. Conclusions Our findings highlight that abnormal learning for losses underlies the social learning process in MDD patients. This abnormality is higher when situational unpredictability is high versus low. Our findings provide novel insights into social rehabilitation of MDD.
Collapse
Affiliation(s)
- Yuening Jin
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China
- Department of Psychology, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Qinglin Gao
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China
- Department of Psychology, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Yun Wang
- The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China
| | - Martin Dietz
- Center of Functionally Integrative Neuroscience, Institute of Clinical Medicine, Aarhus University, Universitetsbyen 3, Aarhus C 8000, Denmark
| | - Le Xiao
- The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China
| | - Yuyang Cai
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China
- Department of Psychology, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Vibeke Bliksted
- Department of Clinical Medicine, Aarhus University, Palle Juul-Jensens Boulevard 82, Aarhus N 8200, Denmark
- Centre for Interacting Minds, Aarhus University, Jens Chr. Skous Vej 4, Building 1483, Aarhus C 8000, Denmark
| | - Yuan Zhou
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, China
- Department of Psychology, University of Chinese Academy of Sciences, Beijing 100049, China
- The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China
| |
Collapse
|
6
|
Björlin Avdic H, Strannegård C, Engberg H, Willfors C, Nordgren I, Frisén L, Hirschberg AL, Guath M, Nordgren A, Kleberg JL. Reduced effects of social feedback on learning in Turner syndrome. Sci Rep 2023; 13:15858. [PMID: 37739980 PMCID: PMC10516979 DOI: 10.1038/s41598-023-42628-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 09/12/2023] [Indexed: 09/24/2023] Open
Abstract
Turner syndrome is a genetic condition caused by a complete or partial loss of one of the X chromosomes. Previous studies indicate that Turner syndrome is associated with challenges in social skills, but the underlying mechanisms remain largely unexplored. A possible mechanism is a reduced social influence on learning. The current study examined the impact of social and non-social feedback on learning in women with Turner syndrome (n = 35) and a sex- and age-matched control group (n = 37). Participants were instructed to earn points by repeatedly choosing between two stimuli with unequal probabilities of resulting in a reward. Mastering the task therefore required participants to learn through feedback which of the two stimuli was more likely to be rewarded. Data were analyzed using computational modeling and analyses of choice behavior. Social feedback led to a more explorative choice behavior in the control group, resulting in reduced learning compared to non-social feedback. No effects of social feedback on learning were found in Turner syndrome. The current study thus indicates that women with Turner syndrome may be less sensitive to social influences on reinforcement learning, than the general population.
Collapse
Affiliation(s)
- Hanna Björlin Avdic
- Centre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institutet & Stockholm Health Care Services, Region Stockholm, Stockholm, Sweden.
| | - Claes Strannegård
- Department of Molecular Medicine and Surgery, Center for Molecular Medicine, Karolinska Institutet, Stockholm, Sweden
- Division of Cognition and Communication, Department of Applied IT, University of Gothenburg, Gothenburg, Sweden
| | - Hedvig Engberg
- Department of Women's and Children's Health, Karolinska Institutet & Department of Gynaecology and Reproductive Medicine, Karolinska University Hospital, Stockholm, Sweden
| | - Charlotte Willfors
- Department of Molecular Medicine and Surgery, Center for Molecular Medicine, Karolinska Institutet, Stockholm, Sweden
| | - Ida Nordgren
- Department of Molecular Medicine and Surgery, Center for Molecular Medicine, Karolinska Institutet, Stockholm, Sweden
| | - Louise Frisén
- Centre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institutet & Stockholm Health Care Services, Region Stockholm, Stockholm, Sweden
| | - Angelica Lindén Hirschberg
- Department of Women's and Children's Health, Karolinska Institutet & Department of Gynaecology and Reproductive Medicine, Karolinska University Hospital, Stockholm, Sweden
| | - Mona Guath
- Department of Psychology, Uppsala University, Uppsala, Sweden
| | - Ann Nordgren
- Department of Molecular Medicine and Surgery, Center for Molecular Medicine, Karolinska Institutet, Stockholm, Sweden
- Department of Laboratory Medicine, Institute of Biomedicine, University of Gothenburg, Gothenburg, Sweden
- Department of Clinical Genetics and Genomics, Sahlgrenska University Hospital, Gothenburg, Sweden
- Department of Clinical Genetics, Karolinska University Hospital, Stockholm, Sweden
| | - Johan Lundin Kleberg
- Centre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institutet & Stockholm Health Care Services, Region Stockholm, Stockholm, Sweden
- Department of Psychology, Stockholm University, Stockholm, Sweden
| |
Collapse
|
7
|
Kirkland JM, Edgar EL, Patel I, Kopec AM. Impaired microglia-mediated synaptic pruning in the nucleus accumbens during adolescence results in persistent dysregulation of familiar, but not novel social interactions in sex-specific ways. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.02.539115. [PMID: 37205324 PMCID: PMC10187149 DOI: 10.1101/2023.05.02.539115] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
Evolutionarily conserved, peer-directed social behaviors are essential to participate in many aspects of human society. These behaviors directly impact psychological, physiological, and behavioral maturation. Adolescence is an evolutionarily conserved period during which reward-related behaviors, including social behaviors, develop via developmental plasticity in the mesolimbic dopaminergic 'reward' circuitry of the brain. The nucleus accumbens (NAc) is an intermediate reward relay center that develops during adolescence and mediates both social behaviors and dopaminergic signaling. In several developing brain regions, synaptic pruning mediated by microglia, the resident immune cells of the brain, is important for normal behavioral development. In rats, we previously demonstrated that microglial synaptic pruning also mediates NAc and social development during sex-specific adolescent periods and via sex-specific synaptic pruning targets. In this report, we demonstrate that interrupting microglial pruning in NAc during adolescence persistently dysregulates social behavior towards a familiar, but not novel social partner in both sexes, via sex-specific behavioral expression. This leads us to infer that naturally occurring NAc pruning serves to reduce social behaviors primarily directed toward a familiar conspecific in both sexes, but in sex-specific ways.
Collapse
Affiliation(s)
- Julia M. Kirkland
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College
| | - Erin L. Edgar
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College
| | - Ishan Patel
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College
| | - Ashley M. Kopec
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College
| |
Collapse
|
8
|
Zabag R, Azoulay R, Rinck M, Becker E, Levy-Gigi E, Gilboa-Schechtman E. You never get a chance to undo a negative first impression: Social anxiety is associated with impaired positive updating of social information. PERSONALITY AND INDIVIDUAL DIFFERENCES 2023. [DOI: 10.1016/j.paid.2022.111993] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/02/2022]
|
9
|
Kleberg JL, Willfors C, Björlin Avdic H, Riby D, Galazka MA, Guath M, Nordgren A, Strannegård C. Social feedback enhances learning in Williams syndrome. Sci Rep 2023; 13:164. [PMID: 36599864 PMCID: PMC9813264 DOI: 10.1038/s41598-022-26055-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Accepted: 12/08/2022] [Indexed: 01/06/2023] Open
Abstract
Williams syndrome (WS) is a rare genetic condition characterized by high social interest and approach motivation as well as intellectual disability and anxiety. Despite the fact that social stimuli are believed to have an increased intrinsic reward value in WS, it is not known whether this translates to learning and decision making. Genes homozygously deleted in WS are linked to sociability in the general population, making it a potential model condition for understanding the social brain. Probabilistic reinforcement learning was studied with either social or non-social rewards for correct choices. Social feedback improved learning in individuals with Williams syndrome but not in typically developing controls or individuals with other intellectual disabilities. Computational modeling indicated that these effects on social feedback were mediated by a shift towards higher weight given to rewards relative to punishments and increased choice consistency. We conclude that reward learning in WS is characterized by high volatility and a tendency to learn how to avoid punishment rather than how to gain rewards. Social feedback can partly normalize this pattern and promote adaptive reward learning.
Collapse
Affiliation(s)
- Johan Lundin Kleberg
- grid.10548.380000 0004 1936 9377Department of Psychology, Stockholm University, Stockholm, Sweden ,grid.4714.60000 0004 1937 0626Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institute, Stockholm, Sweden
| | - Charlotte Willfors
- grid.4714.60000 0004 1937 0626Department of Molecular Medicine and Surgery, Karolinska Institute, Stockholm, Sweden
| | - Hanna Björlin Avdic
- grid.4714.60000 0004 1937 0626Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institute, Stockholm, Sweden
| | - Deborah Riby
- grid.8250.f0000 0000 8700 0572Department of Psychology, Centre for Developmental Disorders, Durham University, Durham, UK
| | - Martyna A. Galazka
- grid.8761.80000 0000 9919 9582Gillberg Neuropsychiatry Centre, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
| | - Mona Guath
- grid.8993.b0000 0004 1936 9457Department of Psychology, Uppsala University, Uppsala, Sweden
| | - Ann Nordgren
- grid.4714.60000 0004 1937 0626Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institute, Stockholm, Sweden ,grid.24381.3c0000 0000 9241 5705Department of Clinical Genetics, Karolinska University Hospital, Stockholm, Sweden ,grid.8761.80000 0000 9919 9582Department of Laboratory Medicine, Institute of Biomedicine, University of Gothenburg, Gothenburg, Sweden ,grid.1649.a000000009445082XDepartment of Clinical Genetics and Genomics, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Claes Strannegård
- grid.4714.60000 0004 1937 0626Department of Clinical Neuroscience, Centre for Psychiatry Research, Karolinska Institute, Stockholm, Sweden ,grid.8761.80000 0000 9919 9582Division of Cognition and Communication, Department of Applied IT, University of Gothenburg, Gothenburg, Sweden
| |
Collapse
|
10
|
Chen C, Mochizuki Y, Hagiwara K, Hirotsu M, Matsubara T, Nakagawa S. Computational markers of experience- but not description-based decision-making are associated with future depressive symptoms in young adults. J Psychiatr Res 2022; 154:307-314. [PMID: 35973300 DOI: 10.1016/j.jpsychires.2022.08.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Revised: 07/07/2022] [Accepted: 08/03/2022] [Indexed: 11/29/2022]
Abstract
BACKGROUND Early prediction of high depressive symptoms is crucial for selective intervention and the minimization of functional impairment. Recent cross-sectional studies indicated decision-making deficits in depression, which may be an important contributor to the disorder. Our goal was to test whether description- and experience-based decision making, two major neuroeconomic paradigms of decision-making under uncertainty, predict future depressive symptoms in young adults. METHODS One hundred young adults performed two decision-making tasks, one description-based, in which subjects chose between two gambling options given explicitly stated rewards and their probabilities, and the other experience-based, in which subjects were shown rewards but had to learn the probability of those rewards (or cue-outcome contingencies) via trial-and-error experience. We evaluated subjects' depressive symptoms with BDI-II at baseline (T1) and half a year later (T2). RESULTS Comparing subjects with low versus high levels of depressive symptoms at T2 showed that the latter performed worse on the experience- but not description-based task at T1. Computational modeling of the decision-making process suggested that subjects with high levels of depressive symptoms had a more concave utility function, indicating enhanced risk aversion. Furthermore, a more concave utility function at T1 increased the odds of high depressive symptoms at T2, even after controlling depressive symptoms at T1, perceived stress at T2, and several covariates (OR = 0.251, 95% CI [0.085, 0.741]). CONCLUSIONS This is the first study to demonstrate a prospective link between experience-based decision-making and depressive symptoms. Our results suggest that enhanced risk aversion in experience-based decision-making may be an important contributor to the development of depressive symptoms.
Collapse
Affiliation(s)
- Chong Chen
- Division of Neuropsychiatry, Department of Neuroscience, Yamaguchi University Graduate School of Medicine, Ube, Yamaguchi, 755-8505, Japan.
| | - Yasuhiro Mochizuki
- Center for Data Science, Waseda University, 1-6-1 Nishiwaseda, Shinjuku-ku, Tokyo, 169-8050, Japan
| | - Kosuke Hagiwara
- Division of Neuropsychiatry, Department of Neuroscience, Yamaguchi University Graduate School of Medicine, Ube, Yamaguchi, 755-8505, Japan
| | - Masako Hirotsu
- Division of Neuropsychiatry, Department of Neuroscience, Yamaguchi University Graduate School of Medicine, Ube, Yamaguchi, 755-8505, Japan
| | - Toshio Matsubara
- Division of Neuropsychiatry, Department of Neuroscience, Yamaguchi University Graduate School of Medicine, Ube, Yamaguchi, 755-8505, Japan
| | - Shin Nakagawa
- Division of Neuropsychiatry, Department of Neuroscience, Yamaguchi University Graduate School of Medicine, Ube, Yamaguchi, 755-8505, Japan
| |
Collapse
|
11
|
Pike AC, Robinson OJ. Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals: A Systematic Review and Meta-analysis. JAMA Psychiatry 2022; 79:313-322. [PMID: 35234834 PMCID: PMC8892374 DOI: 10.1001/jamapsychiatry.2022.0051] [Citation(s) in RCA: 43] [Impact Index Per Article: 21.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
IMPORTANCE Computational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent. OBJECTIVE To assess whether there are consistent differences in reinforcement-learning parameters between patients with depression or anxiety and control individuals. DATA SOURCES Web of Knowledge, PubMed, Embase, and Google Scholar searches were performed between November 15, 2019, and December 6, 2019, and repeated on December 3, 2020, and February 23, 2021, with keywords (reinforcement learning) AND (computational OR model) AND (depression OR anxiety OR mood). STUDY SELECTION Studies were included if they fit reinforcement-learning models to human choice data from a cognitive task with rewards or punishments, had a case-control design including participants with mood and/or anxiety disorders and healthy control individuals, and included sufficient information about all parameters in the models. DATA EXTRACTION AND SYNTHESIS Articles were assessed for inclusion according to MOOSE guidelines. Participant-level parameters were extracted from included articles, and a conventional meta-analysis was performed using a random-effects model. Subsequently, these parameters were used to simulate choice performance for each participant on benchmarking tasks in a simulation meta-analysis. Models were fitted, parameters were extracted using bayesian model averaging, and differences between patients and control individuals were examined. Overall effect sizes across analytic strategies were inspected. MAIN OUTCOMES AND MEASURES The primary outcomes were estimated reinforcement-learning parameters (learning rate, inverse temperature, reward learning rate, and punishment learning rate). RESULTS A total of 27 articles were included (3085 participants, 1242 of whom had depression and/or anxiety). In the conventional meta-analysis, patients showed lower inverse temperature than control individuals (standardized mean difference [SMD], -0.215; 95% CI, -0.354 to -0.077), although no parameters were common across all studies, limiting the ability to infer differences. In the simulation meta-analysis, patients showed greater punishment learning rates (SMD, 0.107; 95% CI, 0.107 to 0.108) and slightly lower reward learning rates (SMD, -0.021; 95% CI, -0.022 to -0.020) relative to control individuals. The simulation meta-analysis showed no meaningful difference in inverse temperature between patients and control individuals (SMD, 0.003; 95% CI, 0.002 to 0.004). CONCLUSIONS AND RELEVANCE The simulation meta-analytic approach introduced in this article for inferring meta-group differences from heterogeneous computational psychiatry studies indicated elevated punishment learning rates in patients compared with control individuals. This difference may promote and uphold negative affective bias symptoms and hence constitute a potential mechanistic treatment target for mood and anxiety disorders.
Collapse
Affiliation(s)
- Alexandra C. Pike
- Anxiety Lab, Neuroscience and Mental Health Group, Institute of Cognitive Neuroscience, University College London, London, United Kingdom
| | - Oliver J. Robinson
- Anxiety Lab, Neuroscience and Mental Health Group, Institute of Cognitive Neuroscience, University College London, London, United Kingdom,Research Department of Clinical, Educational and Health Psychology, University College London, London, United Kingdom
| |
Collapse
|
12
|
Abstract
Why has computational psychiatry yet to influence routine clinical practice? One reason may be that it has neglected context and temporal dynamics in the models of certain mental health problems. We develop three heuristics for estimating whether time and context are important to a mental health problem: Is it characterized by a core neurobiological mechanism? Does it follow a straightforward natural trajectory? And is intentional mental content peripheral to the problem? For many problems the answers are no, suggesting that modeling time and context is critical. We review computational psychiatry advances toward this end, including modeling state variation, using domain-specific stimuli, and interpreting differences in context. We discuss complementary network and complex systems approaches. Novel methods and unification with adjacent fields may inspire a new generation of computational psychiatry. Expected final online publication date for the Annual Review of Psychology, Volume 73 is January 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
Collapse
Affiliation(s)
- Peter F Hitchcock
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, Rhode Island 02912, USA; ,
| | - Eiko I Fried
- Department of Clinical Psychology, Leiden University, 2333 AK Leiden, The Netherlands;
| | - Michael J Frank
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, Rhode Island 02912, USA; , .,Carney Institute for Brain Science, Brown University, Providence, Rhode Island 02192
| |
Collapse
|
13
|
Zhang D, Shen J, Li S, Gao K, Gu R. I, robot: depression plays different roles in human-human and human-robot interactions. Transl Psychiatry 2021; 11:438. [PMID: 34420040 PMCID: PMC8380250 DOI: 10.1038/s41398-021-01567-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/14/2021] [Revised: 08/04/2021] [Accepted: 08/13/2021] [Indexed: 02/07/2023] Open
Abstract
Socially engaging robots have been increasingly applied to alleviate depressive symptoms and to improve the quality of social life among different populations. Seeing that depression negatively influences social reward processing in everyday interaction, we investigate this influence during simulated interactions with humans or robots. In this study, 35 participants with mild depression and 35 controls (all from nonclinical populations) finished the social incentive delay task with event-related potential recording, in which they received performance feedback from other persons or from a robot. Compared to the controls, the mild depressive symptom (MDS) group represented abnormalities of social reward processing in the human feedback condition: first, the MDS group showed a lower hit rate and a smaller contingent-negative variation (correlated with each other) during reward anticipation; second, depression level modulated both the early phase (indexed by the feedback-related negativity (FRN)) and the late phase (indexed by the P3) of reward consumption. In contrast, the effect of depression was evident only on FRN amplitude in the robot feedback condition. We suggest that compared to human-human interaction, the rewarding properties of human-robot interaction are less likely to be affected by depression. These findings have implications for the utilization of robot-assisted intervention in clinical practice.
Collapse
Affiliation(s)
- Dandan Zhang
- School of Psychology, Shenzhen University, 518060, Shenzhen, China.
- Magnetic Resonance Imaging Center, Shenzhen University, 518060, Shenzhen, China.
| | - Junshi Shen
- School of Psychology, Shenzhen University, 518060, Shenzhen, China
| | - Sijin Li
- School of Psychology, Shenzhen University, 518060, Shenzhen, China
| | - Kexiang Gao
- School of Psychology, Shenzhen University, 518060, Shenzhen, China
| | - Ruolei Gu
- CAS Key Laboratory of Behavioral Science, Institute of Psychology, 100101, Beijing, China.
- Department of Psychology, University of Chinese Academy of Sciences, 100049, Beijing, China.
| |
Collapse
|