1
|
Brown VM, Price R, Dombrovski AY. Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2023; 23:844-868. [PMID: 36869259 PMCID: PMC10475148 DOI: 10.3758/s13415-023-01080-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 02/14/2023] [Indexed: 03/05/2023]
Abstract
In cognitive-behavioral conceptualizations of anxiety, exaggerated threat expectancies underlie maladaptive anxiety. This view has led to successful treatments, notably exposure therapy, but is not consistent with the empirical literature on learning and choice alterations in anxiety. Empirically, anxiety is better described as a disorder of uncertainty learning. How disruptions in uncertainty lead to impairing avoidance and are treated with exposure-based methods, however, is unclear. Here, we integrate concepts from neurocomputational learning models with clinical literature on exposure therapy to propose a new framework for understanding maladaptive uncertainty functioning in anxiety. Specifically, we propose that anxiety disorders are fundamentally disorders of uncertainty learning and that successful treatments, particularly exposure therapy, work by remediating maladaptive avoidance from dysfunctional explore/exploit decisions in uncertain, potentially aversive situations. This framework reconciles several inconsistencies in the literature and provides a path forward to better understand and treat anxiety.
Collapse
Affiliation(s)
- Vanessa M Brown
- Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, USA.
| | - Rebecca Price
- Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, USA
| | | |
Collapse
|
2
|
Scholl J, Trier HA, Rushworth MFS, Kolling N. The effect of apathy and compulsivity on planning and stopping in sequential decision-making. PLoS Biol 2022; 20:e3001566. [PMID: 35358177 PMCID: PMC8970514 DOI: 10.1371/journal.pbio.3001566] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2019] [Accepted: 02/03/2022] [Indexed: 11/21/2022] Open
Abstract
Real-life decision-making often comprises sequences of successive decisions about whether to take opportunities as they are encountered or keep searching for better ones instead. We investigated individual differences related to such sequential decision-making and link them especially to apathy and compulsivity in a large online sample (discovery sample: n = 449 and confirmation sample: n = 756). Our cognitive model revealed distinct changes in the way participants evaluated their environments and planned their own future behaviour. Apathy was linked to decision inertia, i.e., automatically persisting with a sequence of searches for longer than appropriate given the value of searching. Thus, despite being less motivated, they did not avoid the effort associated with longer searches. In contrast, compulsivity was linked to self-reported insensitivity to the cost of continuing with a sequence of searches. The objective measures of behavioural cost insensitivity were clearly linked to compulsivity only in the discovery sample. While the confirmation sample showed a similar effect, it did not reach significance. Nevertheless, in both samples, participants reported awareness of such bias (experienced as "overchasing"). In addition, this awareness made them report preemptively avoiding situations related to the bias. However, we found no evidence of them actually preempting more in the task, which might mean a misalignment of their metacognitive beliefs or that our behavioural measures were incomplete. In summary, individual variation in distinct, fundamental aspects of sequential decision-making can be linked to variation in 2 measures of behavioural traits associated with psychological illness in the normal population.
Collapse
Affiliation(s)
- Jacqueline Scholl
- Lyon Neuroscience Research Center, INSERM U1028, CNRS UMR5292, PSYR2 Team, University Lyon 1, Lyon, France
- Centre Hospitalier Le Vinatier, Pôle EST, Bron, France
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Oxford Centre of Human Brain Activity, Wellcome Integrative Neuroimaging (WIN), Department of Psychiatry, University of Oxford, Oxford, United Kingdom
| | - Hailey A. Trier
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Matthew F. S. Rushworth
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Centre for Functional MRI of the Brain (FMRIB), Wellcome Integrative Neuroimaging (WIN), Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - Nils Kolling
- Oxford Centre of Human Brain Activity, Wellcome Integrative Neuroimaging (WIN), Department of Psychiatry, University of Oxford, Oxford, United Kingdom
- Univ Lyon, Université Lyon 1, Inserm, Stem Cell and Brain Research Institute U1208, Bron, France
| |
Collapse
|
3
|
Voigt K, Giddens E, Stark R, Frisch E, Moskovsky N, Kakoschke N, Stout JC, Bellgrove MA, Andrews ZB, Verdejo-Garcia A. The Hunger Games: Homeostatic State-Dependent Fluctuations in Disinhibition Measured with a Novel Gamified Test Battery. Nutrients 2021; 13:nu13062001. [PMID: 34200678 PMCID: PMC8230368 DOI: 10.3390/nu13062001] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2021] [Revised: 05/28/2021] [Accepted: 06/04/2021] [Indexed: 11/16/2022] Open
Abstract
Food homeostatic states (hunger and satiety) influence the cognitive systems regulating impulsive responses, but the direction and specific mechanisms involved in this effect remain elusive. We examined how fasting, and satiety, affect cognitive mechanisms underpinning disinhibition using a novel framework and a gamified test-battery. Thirty-four participants completed the test-battery measuring three cognitive facets of disinhibition: attentional control, information gathering and monitoring of feedback, across two experimental sessions: one after overnight fasting and another after a standardised meal. Homeostatic state was assessed using subjective self-reports and biological markers (i.e., blood-derived liver-expressed antimicrobial protein 2 (LEAP-2), insulin and leptin). We found that participants who experienced greater subjective hunger during the satiety session were more impulsive in the information gathering task; results were not confounded by changes in mood or anxiety. Homeostatic state did not significantly influence disinhibition mechanisms linked to attentional control or feedback monitoring. However, we found a significant interaction between homeostatic state and LEAP-2 on attentional control, with higher LEAP-2 associated with faster reaction times in the fasted condition only. Our findings indicate lingering hunger after eating increases impulsive behaviour via reduced information gathering. These findings identify a novel mechanism that may underpin the tendency to overeat and/or engage in broader impulsive behaviours.
Collapse
Affiliation(s)
- Katharina Voigt
- Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Clayton, VIC 3800, Australia; (K.V.); (E.G.); (E.F.); (N.M.); (N.K.); (J.C.S.); (M.A.B.)
| | - Emily Giddens
- Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Clayton, VIC 3800, Australia; (K.V.); (E.G.); (E.F.); (N.M.); (N.K.); (J.C.S.); (M.A.B.)
| | - Romana Stark
- Department of Physiology and Biomedicine Discovery Institute, Monash University, Clayton, VIC 3800, Australia; (R.S.); (Z.B.A.)
| | - Emma Frisch
- Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Clayton, VIC 3800, Australia; (K.V.); (E.G.); (E.F.); (N.M.); (N.K.); (J.C.S.); (M.A.B.)
| | - Neda Moskovsky
- Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Clayton, VIC 3800, Australia; (K.V.); (E.G.); (E.F.); (N.M.); (N.K.); (J.C.S.); (M.A.B.)
| | - Naomi Kakoschke
- Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Clayton, VIC 3800, Australia; (K.V.); (E.G.); (E.F.); (N.M.); (N.K.); (J.C.S.); (M.A.B.)
| | - Julie C. Stout
- Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Clayton, VIC 3800, Australia; (K.V.); (E.G.); (E.F.); (N.M.); (N.K.); (J.C.S.); (M.A.B.)
| | - Mark A. Bellgrove
- Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Clayton, VIC 3800, Australia; (K.V.); (E.G.); (E.F.); (N.M.); (N.K.); (J.C.S.); (M.A.B.)
| | - Zane B. Andrews
- Department of Physiology and Biomedicine Discovery Institute, Monash University, Clayton, VIC 3800, Australia; (R.S.); (Z.B.A.)
| | - Antonio Verdejo-Garcia
- Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Clayton, VIC 3800, Australia; (K.V.); (E.G.); (E.F.); (N.M.); (N.K.); (J.C.S.); (M.A.B.)
- Correspondence: ; Tel.: +61-3-9905-5374
| |
Collapse
|
4
|
Farahani HK, Ashouri A, Abroon P, Pierre JS, van Baaren J. Wolbachia manipulate fitness benefits of olfactory associative learning in a parasitoid wasp. J Exp Biol 2021; 224:269008. [PMID: 34086908 DOI: 10.1242/jeb.240549] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2020] [Accepted: 04/20/2021] [Indexed: 11/20/2022]
Abstract
Upon encountering a host, a female parasitoid wasp has to decide whether to learn positive or negative cues related to the host. The optimal female decision will depend on the fitness costs and benefits of learned stimuli. Reward quality is positively related to the rate of behavioral acquisition in processes such as associative learning. Wolbachia, an endosymbiotic bacterium, often plays an impressive role in the manipulation of its arthropod host's biology. Here, we studied the responses of two natural Wolbachia infected/uninfected Trichogramma brassicae wasp populations to theoretically high- and low-reward values during a conditioning process and the consequences of their responses in terms of memory duration. According to our results, uninfected wasps showed an attraction response to high-value rewards, but showed aversive learning in response to low-value rewards. The memory span of uninfected wasps after conditioning by low-value rewards was significantly shorter than that for high-value rewards. As our results revealed, responses to high-quality hosts will bring more benefits (bigger size, increased fecundity and enhanced survival) than those to low-quality hosts for uninfected wasps. Infected wasps were attracted to conditioned stimuli with the same memory duration after conditioning by both types of hosts. This was linked to the fact that parasitoids emerging from both types of hosts present the same life-history traits. Therefore, these hosts represent the same quality reward for infected wasps. According to the obtained results, it can be concluded that Wolbachia manipulates the learning ability of its host, resulting in the wasp responding to all reward values similarly.
Collapse
Affiliation(s)
- Hossein Kishani Farahani
- Department of Plant Protection, Faculty of Agriculture and Natural Resources, University of Tehran, Karaj, Iran
| | - Ahmad Ashouri
- Department of Plant Protection, Faculty of Agriculture and Natural Resources, University of Tehran, Karaj, Iran
| | - Pouria Abroon
- Department of Plant Protection, College of Agriculture, University of Bu Ali, Hamadan, Iran
| | - Jean-Sebastien Pierre
- University of Rennes 1, UMR-CNRS 6553 EcoBio, Avenue du Général Leclerc, Campus de Beaulieu, 35042 Rennes Cedex, France
| | - Joan van Baaren
- University of Rennes 1, UMR-CNRS 6553 EcoBio, Avenue du Général Leclerc, Campus de Beaulieu, 35042 Rennes Cedex, France
| |
Collapse
|
5
|
Abivardi A, Khemka S, Bach DR. Hippocampal Representation of Threat Features and Behavior in a Human Approach-Avoidance Conflict Anxiety Task. J Neurosci 2020; 40:6748-6758. [PMID: 32719163 PMCID: PMC7455211 DOI: 10.1523/jneurosci.2732-19.2020] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2019] [Revised: 06/15/2020] [Accepted: 06/20/2020] [Indexed: 12/13/2022] Open
Abstract
Decisions under threat are crucial to survival and require integration of distinct situational features, such as threat probability and magnitude. Recent evidence from human lesion and neuroimaging studies implicated anterior hippocampus (aHC) and amygdala in approach-avoidance decisions under threat, and linked their integrity to cautious behavior. Here we sought to elucidate how threat dimensions and behavior are represented in these structures. Twenty human participants (11 female) completed an approach-avoidance conflict task during high-resolution fMRI. Participants could gather tokens under threat of capture by a virtual predator, which would lead to token loss. Threat probability (predator wake-up rate) and magnitude (amount of token loss) varied on each trial. To disentangle effects of threat features, and ensuing behavior, we performed a multifold parametric analysis. We found that high threat probability and magnitude related to BOLD signal in left aHC/entorhinal cortex. However, BOLD signal in this region was better explained by avoidance behavior than by these threat features. A priori ROI analysis confirmed the relation of aHC BOLD response with avoidance. Exploratory subfield analysis revealed that this relation was specific to anterior CA2/3 but not CA1. Left lateral amygdala responded to low and high, but not intermediate, threat probability. Our results suggest that aHC BOLD signal is better explained by avoidance behavior than by threat features in approach-avoidance conflict. Rather than representing threat features in a monotonic manner, it appears that aHC may compute approach-avoidance decisions based on integration of situational threat features represented in other neural structures.SIGNIFICANCE STATEMENT An effective threat anticipation system is crucial to survival across species. Natural threats, however, are diverse and have distinct features. To be able to adapt to different modes of danger, the brain needs to recognize these features, integrate them, and use them to modify behavior. Our results disclose the human anterior hippocampus as a likely arbiter of approach-avoidance decisions harnessing compound environmental information while partially replicating previous findings and blending into recent efforts to illuminate the neural basis of approach-avoidance conflict in humans.
Collapse
Affiliation(s)
- Aslan Abivardi
- Computational Psychiatry Research, Department of Psychiatry Psychotherapy and Psychosomatics, Psychiatric Hospital, University of Zurich, Zurich, 8032, Switzerland
- Zurich, Neuroscience Center Zurich, University of Zurich, Zurich, 8057, Switzerland
| | - Saurabh Khemka
- Computational Psychiatry Research, Department of Psychiatry Psychotherapy and Psychosomatics, Psychiatric Hospital, University of Zurich, Zurich, 8032, Switzerland
- Zurich, Neuroscience Center Zurich, University of Zurich, Zurich, 8057, Switzerland
| | - Dominik R Bach
- Computational Psychiatry Research, Department of Psychiatry Psychotherapy and Psychosomatics, Psychiatric Hospital, University of Zurich, Zurich, 8032, Switzerland
- Zurich, Neuroscience Center Zurich, University of Zurich, Zurich, 8057, Switzerland
- Wellcome Centre for Human Neuroimaging and Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, WC1N 3BG, United Kingdom
| |
Collapse
|
6
|
Bach DR, Moutoussis M, Bowler A, Dolan RJ. Predictors of risky foraging behaviour in healthy young people. Nat Hum Behav 2020; 4:832-843. [PMID: 32393840 PMCID: PMC7115941 DOI: 10.1038/s41562-020-0867-0] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2019] [Accepted: 03/19/2020] [Indexed: 12/17/2022]
Abstract
During adolescence and early adulthood, learning when to avoid threats and when to pursue rewards becomes crucial. Using a risky foraging task, we investigated individual differences in this dynamic across 781 individuals aged 14-24 years who were split into a hypothesis-generating discovery sample and a hold-out confirmation sample. Sex was the most important predictor of cautious behaviour and performance. Males earned one standard deviation (or 20%) more reward than females, collected more reward when there was little to lose and reduced foraging to the same level as females when potential losses became high. Other independent predictors of cautiousness and performance were self-reported daringness, IQ and self-reported cognitive complexity. We found no evidence for an impact of age or maturation. Thus, maleness, a high IQ or self-reported cognitive complexity, and self-reported daringness predicted greater success in risky foraging, possibly due to better exploitation of low-risk opportunities in high-risk environments.
Collapse
Affiliation(s)
- Dominik R Bach
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK.
- Wellcome Centre for Human Neuroimaging, University College London, London, UK.
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy and Psychosomatics, Psychiatric Hospital, University of Zurich, Zurich, Switzerland.
| | - Michael Moutoussis
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Aislinn Bowler
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Raymond J Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| |
Collapse
|
7
|
Stimulation of the vagus nerve reduces learning in a go/no-go reinforcement learning task. Eur Neuropsychopharmacol 2020; 35:17-29. [PMID: 32404279 DOI: 10.1016/j.euroneuro.2020.03.023] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/22/2019] [Revised: 02/06/2020] [Accepted: 03/27/2020] [Indexed: 02/06/2023]
Abstract
When facing decisions to approach rewards or to avoid punishments, we often figuratively go with our gut, and the impact of metabolic states such as hunger on motivation are well documented. However, whether and how vagal feedback signals from the gut influence instrumental actions is unknown. Here, we investigated the effect of non-invasive transcutaneous auricular vagus nerve stimulation (taVNS) vs. sham (randomized cross-over design) on approach and avoidance behavior using an established go/no-go reinforcement learning paradigm in 39 healthy human participants (23 female) after an overnight fast. First, mixed-effects logistic regression analysis of choice accuracy showed that taVNS acutely impaired decision-making, p = .041. Computational reinforcement learning models identified the cause of this as a reduction in the learning rate through taVNS (∆α = -0.092, pboot = .002), particularly after punishment (∆αPun = -0.081, pboot = .012 vs. ∆αRew =-0.031, pboot = .22). However, taVNS had no effect on go biases, Pavlovian response biases or response time. Hence, taVNS appeared to influence learning rather than action execution. These results highlight a novel role of vagal afferent input in modulating reinforcement learning by tuning the learning rate according to homeostatic needs.
Collapse
|
8
|
Castegnetti G, Tzovara A, Khemka S, Melinščak F, Barnes GR, Dolan RJ, Bach DR. Representation of probabilistic outcomes during risky decision-making. Nat Commun 2020; 11:2419. [PMID: 32415145 PMCID: PMC7229012 DOI: 10.1038/s41467-020-16202-y] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2019] [Accepted: 04/21/2020] [Indexed: 12/19/2022] Open
Abstract
Goal-directed behaviour requires prospectively retrieving and evaluating multiple possible action outcomes. While a plethora of studies suggested sequential retrieval for deterministic choice outcomes, it remains unclear whether this is also the case when integrating multiple probabilistic outcomes of the same action. We address this question by capitalising on magnetoencephalography (MEG) in humans who made choices in a risky foraging task. We train classifiers to distinguish MEG field patterns during presentation of two probabilistic outcomes (reward, loss), and then apply these to decode such patterns during deliberation. First, decoded outcome representations have a temporal structure, suggesting alternating retrieval of the outcomes. Moreover, the probability that one or the other outcome is being represented depends on loss magnitude, but not on loss probability, and it predicts the chosen action. In summary, we demonstrate decodable outcome representations during probabilistic decision-making, which are sequentially structured, depend on task features, and predict subsequent action.
Collapse
Affiliation(s)
- Giuseppe Castegnetti
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, University of Zurich, Zurich, Switzerland.
- Neuroscience Centre Zurich, University of Zurich, Zurich, Switzerland.
- Institute of Cognitive Neuroscience, University College London, London, UK.
| | - Athina Tzovara
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, University of Zurich, Zurich, Switzerland
- Neuroscience Centre Zurich, University of Zurich, Zurich, Switzerland
- Department of Computer Science & Faculty of Medicine, University of Bern, Bern, Switzerland
- Helen Wills Neuroscience Institute, University of California, Berkeley, USA
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Saurabh Khemka
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, University of Zurich, Zurich, Switzerland
- Neuroscience Centre Zurich, University of Zurich, Zurich, Switzerland
| | - Filip Melinščak
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, University of Zurich, Zurich, Switzerland
- Neuroscience Centre Zurich, University of Zurich, Zurich, Switzerland
| | - Gareth R Barnes
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Raymond J Dolan
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Max Planck UCL Centre for Computational Psychiatry and Ageing, University College London, London, UK
| | - Dominik R Bach
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, University of Zurich, Zurich, Switzerland
- Neuroscience Centre Zurich, University of Zurich, Zurich, Switzerland
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Max Planck UCL Centre for Computational Psychiatry and Ageing, University College London, London, UK
| |
Collapse
|
9
|
Abstract
How do our valuation systems change to homeostatically correct undesirable psychological or physiological states, such as those caused by hunger? There is evidence that hunger increases discounting for food rewards, biasing choices towards smaller but sooner food reward over larger but later reward. However, it is not understood how hunger modulates delay discounting for non-food items. We outline and quantitatively evaluate six possible models of how our valuation systems modulate discounting of various commodities in the face of the undesirable state of being hungry. With a repeated-measures design, an experimental hunger manipulation, and quantitative modeling, we find strong evidence that hunger causes large increases in delay discounting for food, with an approximately 25% spillover effect to non-food commodities. The results provide evidence that in the face of hunger, our valuation systems increase discounting for commodities, which cannot achieve a desired state change as well as for those commodities that can. Given that strong delay discounting can cause negative outcomes in many non-food (consumer, investment, medical, or inter-personal) domains, the present findings suggest caution may be necessary when making decisions involving non-food outcomes while hungry.
Collapse
Affiliation(s)
- Jordan Skrynka
- Division of Psychology, School of Social Sciences, University of Dundee, Dundee, DD1 4HN, UK
| | - Benjamin T Vincent
- Division of Psychology, School of Social Sciences, University of Dundee, Dundee, DD1 4HN, UK.
| |
Collapse
|
10
|
Korn CW, Bach DR. Minimizing threat via heuristic and optimal policies recruits hippocampus and medial prefrontal cortex. Nat Hum Behav 2019; 3:733-745. [PMID: 31110338 PMCID: PMC6629544 DOI: 10.1038/s41562-019-0603-9] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2018] [Accepted: 04/03/2019] [Indexed: 11/30/2022]
Abstract
Jointly minimizing multiple threats over extended time horizons enhances survival. Consequently, many tests of approach-avoidance conflicts incorporate multiple threats for probing corollaries of animal and human anxiety. To facilitate computations necessary for threat minimization, the human brain may concurrently harness multiple decision policies and associated neural controllers, but it is unclear which. We combine a task that mimics foraging under predation with behavioural modelling and functional neuroimaging. Human choices rely on immediate predator probability-a myopic heuristic policy-and on the optimal policy, which integrates all relevant variables. Predator probability relates positively and the associated choice uncertainty relates negatively to activations in the anterior hippocampus, amygdala and dorsolateral prefrontal cortex. The optimal policy is positively associated with dorsomedial prefrontal cortex activity. We thus provide a decision-theoretic outlook on the role of the human hippocampus, amygdala and prefrontal cortex in resolving approach-avoidance conflicts relevant for anxiety and integral for survival.
Collapse
Affiliation(s)
- Christoph W Korn
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics; Psychiatric Hospital, University of Zurich, Zurich, Switzerland.
- Neuroscience Center Zurich, University of Zurich, Zurich, Switzerland.
- Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
| | - Dominik R Bach
- Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics; Psychiatric Hospital, University of Zurich, Zurich, Switzerland
- Neuroscience Center Zurich, University of Zurich, Zurich, Switzerland
- Wellcome Trust Centre for Neuroimaging and Max-Planck UCL Centre for Computational Psychiatry and Ageing, University College London, London, UK
| |
Collapse
|
11
|
A Network for Computing Value Equilibrium in the Human Medial Prefrontal Cortex. Neuron 2019; 101:977-987.e3. [DOI: 10.1016/j.neuron.2018.12.029] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2018] [Revised: 10/31/2018] [Accepted: 12/20/2018] [Indexed: 12/14/2022]
|
12
|
Schulz E, Wu CM, Huys QJM, Krause A, Speekenbrink M. Generalization and Search in Risky Environments. Cogn Sci 2018; 42:2592-2620. [DOI: 10.1111/cogs.12695] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2017] [Revised: 09/26/2018] [Accepted: 09/26/2018] [Indexed: 12/01/2022]
Affiliation(s)
| | - Charley M. Wu
- Center for Adaptive Rationality Max Planck Institute for Human Development
| | | | | | | |
Collapse
|
13
|
Korn CW, Bach DR. Heuristic and optimal policy computations in the human brain during sequential decision-making. Nat Commun 2018; 9:325. [PMID: 29362449 PMCID: PMC5780427 DOI: 10.1038/s41467-017-02750-3] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2016] [Accepted: 12/25/2017] [Indexed: 01/22/2023] Open
Abstract
Optimal decisions across extended time horizons require value calculations over multiple probabilistic future states. Humans may circumvent such complex computations by resorting to easy-to-compute heuristics that approximate optimal solutions. To probe the potential interplay between heuristic and optimal computations, we develop a novel sequential decision-making task, framed as virtual foraging in which participants have to avoid virtual starvation. Rewards depend only on final outcomes over five-trial blocks, necessitating planning over five sequential decisions and probabilistic outcomes. Here, we report model comparisons demonstrating that participants primarily rely on the best available heuristic but also use the normatively optimal policy. FMRI signals in medial prefrontal cortex (MPFC) relate to heuristic and optimal policies and associated choice uncertainties. Crucially, reaction times and dorsal MPFC activity scale with discrepancies between heuristic and optimal policies. Thus, sequential decision-making in humans may emerge from integration between heuristic and optimal policies, implemented by controllers in MPFC. Alhough humans often make a series of related decisions, it is unknown whether this is done by relying on optimal or heuristic strategies. Here, the authors show that humans rely on both the best heuristic and the optimal policy, and that these strategies are controlled by parts of the medial prefrontal cortex.
Collapse
Affiliation(s)
- Christoph W Korn
- Division of Clinical Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics; Psychiatric Hospital, University of Zurich, Lengstrasse 31, 8032, Zurich, Switzerland. .,Neuroscience Center Zurich, University of Zurich, Winterthurerstrasse 190, 8057, Zurich, Switzerland. .,Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Martinistrasse 52, 20246, Hamburg, Germany.
| | - Dominik R Bach
- Division of Clinical Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics; Psychiatric Hospital, University of Zurich, Lengstrasse 31, 8032, Zurich, Switzerland.,Neuroscience Center Zurich, University of Zurich, Winterthurerstrasse 190, 8057, Zurich, Switzerland.,Wellcome Trust Centre for Neuroimaging, University College London, 12 Queen Square, London, WC1N 3BG, United Kingdom
| |
Collapse
|
14
|
Addicott MA, Pearson JM, Sweitzer MM, Barack DL, Platt ML. A Primer on Foraging and the Explore/Exploit Trade-Off for Psychiatry Research. Neuropsychopharmacology 2017; 42:1931-1939. [PMID: 28553839 PMCID: PMC5561336 DOI: 10.1038/npp.2017.108] [Citation(s) in RCA: 93] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/23/2016] [Revised: 05/19/2017] [Accepted: 05/24/2017] [Indexed: 12/18/2022]
Abstract
Foraging is a fundamental behavior, and many types of animals appear to have solved foraging problems using a shared set of mechanisms. Perhaps the most common foraging problem is the choice between exploiting a familiar option for a known reward and exploring unfamiliar options for unknown rewards-the so-called explore/exploit trade-off. This trade-off has been studied extensively in behavioral ecology and computational neuroscience, but is relatively new to the field of psychiatry. Explore/exploit paradigms can offer psychiatry research a new approach to studying motivation, outcome valuation, and effort-related processes, which are disrupted in many mental and emotional disorders. In addition, the explore/exploit trade-off encompasses elements of risk-taking and impulsivity-common behaviors in psychiatric disorders-and provides a novel framework for understanding these behaviors within an ecological context. Here we explain relevant concepts and some common paradigms used to measure explore/exploit decisions in the laboratory, review clinically relevant research on the neurobiology and neuroanatomy of explore/exploit decision making, and discuss how computational psychiatry can benefit from foraging theory.
Collapse
Affiliation(s)
- M A Addicott
- Department of Psychiatry and Behavioral Sciences, Duke University, Durham, NC, USA
| | - J M Pearson
- Center for Cognitive Neuroscience, Duke University, Durham, NC, USA
| | - M M Sweitzer
- Department of Psychiatry and Behavioral Sciences, Duke University, Durham, NC, USA
| | - D L Barack
- Department of Philosophy and Neuroscience, Columbia University, New York, NY, USA
| | - M L Platt
- Departments of Psychology, Neuroscience, and Marketing, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
15
|
Abstract
The nature and neural implementation of emotions is the subject of vigorous debate. Here, we use Bayesian decision theory to address key complexities in this field and conceptualize emotions in terms of their relationship to survival-relevant behavioural choices. Decision theory indicates which behaviours are optimal in a given situation; however, the calculations required are radically intractable. We therefore conjecture that the brain uses a range of pre-programmed algorithms that provide approximate solutions. These solutions seem to produce specific behavioural manifestations of emotions and can also be associated with core affective dimensions. We identify principles according to which these algorithms are implemented in the brain and illustrate our approach by considering decision making in the face of proximal threat.
Collapse
|
16
|
Abstract
The combination of reward and potential threat is termed approach/avoidance conflict and elicits specific behaviors, including passive avoidance and behavioral inhibition (BI). Anxiety-relieving drugs reduce these behaviors, and a rich psychological literature has addressed how personality traits dominated by BI predispose for anxiety disorders. Yet, a formal understanding of the cognitive inference and planning processes underlying anxiety-like BI is lacking. Here, we present and empirically test such formalization in the terminology of reinforcement learning. We capitalize on a human computer game in which participants collect sequentially appearing monetary tokens while under threat of virtual “predation.” First, we demonstrate that humans modulate BI according to experienced consequences. This suggests an instrumental implementation of BI generation rather than a Pavlovian mechanism that is agnostic about action outcomes. Second, an internal model that would make BI adaptive is expressed in an independent task that involves no threat. The existence of such internal model is a necessary condition to conclude that BI is under model-based control. These findings relate a plethora of human and nonhuman observations on BI to reinforcement learning theory, and crucially constrain the quest for its neural implementation.
Collapse
|
17
|
Kroemer NB, Burrasch C, Hellrung L. To work or not to work: Neural representation of cost and benefit of instrumental action. PROGRESS IN BRAIN RESEARCH 2016; 229:125-157. [PMID: 27926436 DOI: 10.1016/bs.pbr.2016.06.009] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
By definition, instrumental actions are performed in order to obtain certain goals. Nevertheless, the attainment of goals typically implies obstacles, and response vigor is known to reflect an integration of subjective benefit and cost. Whereas several brain regions have been associated with cost/benefit ratio decision-making, trial-by-trial fluctuations in motivation are not well understood. We review recent evidence supporting the motivational implications of signal fluctuations in the mesocorticolimbic system. As an extension of "set-point" theories of instrumental action, we propose that response vigor is determined by a rapid integration of brain signals that reflect value and cost on a trial-by-trial basis giving rise to an online estimate of utility. Critically, we posit that fluctuations in key nodes of the network can predict deviations in response vigor and that variability in instrumental behavior can be accounted for by models devised from optimal control theory, which incorporate the effortful control of noise. Notwithstanding, the post hoc analysis of signaling dynamics has caveats that can effectively be addressed in future research with the help of two novel fMRI imaging techniques. First, adaptive fMRI paradigms can be used to establish a time-order relationship, which is a prerequisite for causality, by using observed signal fluctuations as triggers for stimulus presentation. Second, real-time fMRI neurofeedback can be employed to induce predefined brain states that may facilitate benefit or cost aspects of instrumental actions. Ultimately, understanding temporal dynamics in brain networks subserving response vigor holds the promise for targeted interventions that could help to readjust the motivational balance of behavior.
Collapse
Affiliation(s)
- N B Kroemer
- Technische Universität Dresden, Dresden, Germany.
| | - C Burrasch
- Technische Universität Dresden, Dresden, Germany; University of Lübeck, Lübeck, Germany
| | - L Hellrung
- Technische Universität Dresden, Dresden, Germany
| |
Collapse
|