A Computational Analysis of Aberrant Delay Discounting in Psychiatric Disorders

Computational modeling of decision-making in substance abusers: testing Bechara's hypotheses

One of the cognitive abilities most affected by substance abuse is decision-making. Behavioral tasks such as the Iowa Gambling Task (IGT) provide a means to measure the learning process involved in decision-making. To comprehend this process, three hypotheses have emerged: (1) participants prioritize gains over losses, (2) they exhibit insensitivity to losses, and (3) the capacity of operational storage or working memory comes into play. A dynamic model was developed to examine these hypotheses, simulating sensitivity to gains and losses. The Linear Operator model served as the learning rule, wherein net gains depend on the ratio of gains to losses, weighted by the sensitivity to both. The study further proposes a comparison between the performance of simulated agents and that of substance abusers (n = 20) and control adults (n = 20). The findings indicate that as the memory factor increases, along with high sensitivity to losses and low sensitivity to gains, agents prefer advantageous alternatives, particularly those with a lower frequency of punishments. Conversely, when sensitivity to gains increases and the memory factor decreases, agents prefer disadvantageous alternatives, especially those that result in larger losses. Human participants confirmed the agents' performance, particularly when contrasting optimal and sub-optimal outcomes. In conclusion, we emphasize the importance of evaluating the parameters of the linear operator model across diverse clinical and community samples.

Different topological patterns in structural covariance networks between high and low delay discounters

Introduction

People prefer immediate over future rewards because they discount the latter's value (a phenomenon termed "delay discounting," used as an index of impulsivity). However, little is known about how the preferences are implemented in brain in terms of the coordinated pattern of large-scale structural brain networks.

Methods

To examine this question, we classified high discounting group (HDG) and low discounting group (LDG) in young adults by assessing their propensity for intertemporal choice. We compared global and regional topological properties in gray matter volume-based structural covariance networks between two groups using graph theoretical analysis.

Results

HDG had less clustering coefficient and characteristic path length over the wide sparsity range than LDG, indicating low network segregation and high integration. In addition, the degree of small-worldness was more significant in HDG. Locally, HDG showed less betweenness centrality (BC) in the parahippocampal gyrus and amygdala than LDG.

Discussion

These findings suggest the involvement of structural covariance network topology on impulsive choice, measured by delay discounting, and extend our understanding of how impulsive choice is associated with brain morphological features.

Investigating associations of delay discounting with brain structure, working memory, and episodic memory

INTRODUCTION

Delay discounting (DD), the preference for smaller and sooner rewards over larger and later ones, is an important behavioural phenomenon for daily functioning of increasing interest within psychopathology. The neurobiological mechanisms behind DD are not well understood and the literature on structural correlates of DD shows inconsistencies.

METHODS

Here we leveraged a large openly available dataset (n = 1196) to investigate associations with memory performance and gray and white matter correlates of DD using linked independent component analysis.

RESULTS

Greater DD was related to smaller anterior temporal gray matter volume. Associations of DD with total cortical volume, subcortical volumes, markers of white matter microscopic organization, working memory, and episodic memory scores were not significant after controlling for education and income.

CONCLUSION

Effects of size comparable to the one we identified would be unlikely to be replicated with sample sizes common in many previous studies in this domain, which may explain the incongruities in the literature. The paucity and small size of the effects detected in our data underscore the importance of using large samples together with methods that accommodate their statistical structure and appropriate control for confounders, as well as the need to devise paradigms with improved task parameter reliability in studies relating brain structure and cognitive abilities with DD.

Revealing human sensitivity to a latent temporal structure of changes

Precisely timed behavior and accurate time perception plays a critical role in our everyday lives, as our wellbeing and even survival can depend on well-timed decisions. Although the temporal structure of the world around us is essential for human decision making, we know surprisingly little about how representation of temporal structure of our everyday environment impacts decision making. How does the representation of temporal structure affect our ability to generate well-timed decisions? Here we address this question by using a well-established dynamic probabilistic learning task. Using computational modeling, we found that human subjects' beliefs about temporal structure are reflected in their choices to either exploit their current knowledge or to explore novel options. The model-based analysis illustrates a large within-group and within-subject heterogeneity. To explain these results, we propose a normative model for how temporal structure is used in decision making, based on the semi-Markov formalism in the active inference framework. We discuss potential key applications of the presented approach to the fields of cognitive phenotyping and computational psychiatry.

A Neuroscientific and Cognitive Literary Approach to the Treatment of Time in Calderón's Autos sacramentales

Time processing is a fundamental subject in cognitive sciences and neuroscience. Current research is deepening how our brains process time, revealing its essential role in human functionality and survival. In his autos sacramentales, Early Modern Spanish playwright Pedro Calderón de la Barca portrays the relationships between human inner workings and the Christian concept of time. These plays portray the experience of the present, the perception of the flow of time, the measure of time raging from seconds to eternity, and the mental travel necessary to inhabit the past and future with the help of memory and imagination. Calderón explores how the dramatic form can portray all these temporal phenomena and how that portrait of time can constrain the dramatic structure. The different parts of the brain in charge of executive decisions, projections, memories, computation, and calibration are the basis that leads these characters to make the choices that will take them to the future they have cast for themselves. This paper analyzes how the processes that Calderón ascribed to the soul of his characters in the 17th century relate to ongoing cognitive and neuroscientific findings.

The problems with delay discounting: a critical review of current practices and clinical applications

Delay discounting paradigms have gained widespread popularity across clinical research. Given the prevalence in the field, researchers have set lofty expectations for the importance of delay discounting as a key transdiagnostic process and a 'core' process underlying specific domains of dysfunction (e.g. addiction). We believe delay discounting has been prematurely reified as, in and of itself, a core process underlying psychological dysfunction, despite significant concerns with the construct validity of discounting rates. Specifically, high delay discounting rates are only modestly related to measures of psychological dysfunction and therefore are not 'core' to these more complex behavioral problems. Furthermore, discounting rates do not appear to be specifically related to any disorder(s) or dimension(s) of psychopathology. This raises fundamental concerns about the utility of discounting, if the measure is only loosely associated with most forms of psychopathology. This stands in striking contrast to claims that discounting can serve as a 'marker' for specific disorders, despite never demonstrating adequate sensitivity or specificity for any disorder that we are aware of. Finally, empirical evidence does not support the generalizability of discounting rates to other decisions made either in the lab or in the real-world, and therefore discounting rates cannot and should not serve as a summary measure of an individual's decision-making patterns. We provide recommendations for improving future delay discounting research, but also strongly encourage researchers to consider whether the empirical evidence supports the field's hyper-focus on discounting.

Neural Representations of Death in the Cortical Midline Structures Promote Temporal Discounting

Death is an important reminder that our lives are finite. Although some studies have shown that thinking about one’s own death increases temporal discounting (i.e., the devaluing of future rewards), the underlying neural mechanisms are still unknown. In a functional magnetic resonance imaging experiment, we compared the neural and behavioral processes of temporal discounting across four conditions involving distinct types of future thinking (death related, negative, neutral, and positive). Replicating prior research, the behavioral evidence showed that temporal discounting increased when thinking about one’s own future death. Multivoxel pattern analysis showed that death-related future thinking was decoded in default mode regions, including the inferior parietal lobule, precuneus, and medial prefrontal cortex (MPFC). When future thinking was death related (vs. negative), increased temporal discounting was associated with a higher decoding accuracy in the precuneus and MPFC. The present findings suggest that death-related neural representations are distributed across default mode regions, and neural populations in the cortical midline structures play a crucial role in the integration of one's own death into economic decision-making.

Hypothalamic-Extended Amygdala Circuit Regulates Temporal Discounting

Choice behavior is characterized by temporal discounting, i.e., preference for immediate rewards given a choice between immediate and delayed rewards. Agouti-related peptide (AgRP)-expressing neurons located in the arcuate nucleus of the hypothalamus (ARC) regulate food intake and energy homeostasis, yet whether AgRP neurons influence choice behavior and temporal discounting is unknown. Here, we demonstrate that motivational state potently modulates temporal discounting. Hungry mice (both male and female) strongly preferred immediate food rewards, yet sated mice were largely indifferent to reward delay. More importantly, selective optogenetic activation of AgRP-expressing neurons or their axon terminals within the posterior bed nucleus of stria terminalis (BNST) produced temporal discounting in sated mice. Furthermore, activation of neuropeptide Y (NPY) type 1 receptors (Y1Rs) within the BNST is sufficient to produce temporal discounting. These results demonstrate a profound influence of hypothalamic signaling on temporal discounting for food rewards and reveal a novel circuit that determine choice behavior.SIGNIFICANCE STATEMENT Temporal discounting is a universal phenomenon found in many species, yet the underlying neurocircuit mechanisms are still poorly understood. Our results revealed a novel neural pathway from agouti-related peptide (AgRP) neurons in the hypothalamus to the bed nucleus of stria terminalis (BNST) that regulates temporal discounting in decision-making.

Advances in the computational understanding of mental illness

Computational psychiatry is a rapidly growing field attempting to translate advances in computational neuroscience and machine learning into improved outcomes for patients suffering from mental illness. It encompasses both data-driven and theory-driven efforts. Here, recent advances in theory-driven work are reviewed. We argue that the brain is a computational organ. As such, an understanding of the illnesses arising from it will require a computational framework. The review divides work up into three theoretical approaches that have deep mathematical connections: dynamical systems, Bayesian inference and reinforcement learning. We discuss both general and specific challenges for the field, and suggest ways forward.

Delay discounting abnormalities are seen in first-episode schizophrenia but not in bipolar disorder

Delay discounting (DD) is the phenomenon of individuals discounting future rewards as a function of time. It has been studied extensively in chronic schizophrenia (SZ) and the results of these studies have been variable. Comorbidity in chronic samples could be one reason for the mixed findings and studies in first-episode (FE) samples are surprisingly lacking. Bipolar disorder (BP) which shares some genetic and symptom features with SZ could serve as an interesting comparison group for DD but has been underexplored. Here we present the first study that combines FE SZ, FE BP with psychotic features, as well as healthy controls and study DD with two versions of the task. We found that SZ showed steeper discounting than HC and BP on the well-validated Kirby DD task. SZ showed no difference than HC on a separate DD task with smaller rewards presented with decimal places and shorter delays. As a preliminary finding, DD was found to be positively related to positive symptoms in FE SZ, while no relationship was found between negative symptoms and DD. In addition, we found comparable DD in BP compared to HC. Ultimately, our data may help elucidate the psychopathology in SZ and BP during intertemporal decision making.

Neurocognitive Dysfunctions and Their Therapeutic Modulation in Patients With Methamphetamine Dependence: A Pilot Study

AIM

Methamphetamine (MA) abuse and dependence are increasing worldwide and are commonly associated with cognitive deficits. Some studies indicate that such impairments can improve if users become abstinent, but overall results remain inconclusive. Hence, we have performed a longitudinal case-control study investigating key surrogates for attention and impulsive decision-making before and after treatment.

METHODS

Thirty patients with MA dependence and 24 non-substance-abusing control participants were recruited. Groups were matched on age, sex and education. All subjects performed a baseline assessment to obtain neurocognitive measures of sustained attention and delay discounting. Patients subsequently participated in an MA-specific relapse prevention program including repeated monitoring of relapse status. After 3 months, participants of both groups were reevaluated for neurocognitive performance.

RESULTS

At baseline, MA patients showed a significantly higher number of omissions compared to controls, indicative of lower sustained attention. Interestingly, we observed a steep decrease of omissions in MA patients to control-group level post treatment. On the other hand, MA patients discounted delayed rewards significantly stronger than controls, indicating a more impulsive choice behavior both before and after treatment.

LIMITATION

The results should be interpreted with care because of the small sample and short follow-up period.

CONCLUSION

Our data support earlier findings on partial recovery of cognitive deficits in MA patients. They also strengthen the indication for recently recommended psychotherapeutic interventions and may provide a behavioral monitoring tool to inform treatment progress.

Delay discounting and reward sensitivity in a 2 × 2 study of bipolar disorder and alcohol dependence

BACKGROUND AND AIMS

Separate studies have shown increased delay discounting in people with bipolar disorder (BD) and people with alcohol dependence (AD) relative to people without mental health problems. Delay discounting was compared in people with no mental health problems, AD, BD and AD plus BD. Associations of delay discounting with self-reported impulsivity and reward sensitivity were also assessed.

DESIGN

The study was a two-by-two factorial comparative observational design.

SETTING

Data were collected at baseline diagnostic visits as part of a neuroimaging study at a medical university in South Carolina, USA.

PARTICIPANTS

Twenty-two BD + AD, 33 BD, 28 AD and 27 people without mental health problems participated.

MEASUREMENTS

Diagnostic and clinician-rated symptom measures, self-report questionnaires and a computerized delay discounting task were administered. Two-by-two general linear univariate models were tested to examine between-group differences on discounting rates, and bivariate correlations and hierarchical regression analyses were performed to examine associations between discounting rates and self-reported reward sensitivity and impulsivity.

FINDINGS

There was a significant main effect of AD (P = 0.006, η2  = 0.068). The main effect of BD and the BD × AD interaction terms were non-significant (P ≥ 0.293, η2  ≤ 0.010). Reward sensitivity and impulsivity were not significantly associated with discounting rates after adjustment for the other (P ≥ 0.089).

CONCLUSIONS

People with alcohol dependence appear to have higher delay discounting, while previously found associations between bipolar disorder and delay discounting may be secondary to alcohol use disorder.

Delay discounting and affective priming in individuals with negative schizotypy

Previous studies have demonstrated that patients with schizophrenia and individuals with schizotypy experience decreased anticipatory pleasure. However, it is unclear whether this decrease is contributed by altered reward processing at the proximal or distal future. In order to investigate the preference for receiving rewards in the proximal or distal future for individuals with schizophrenia spectrum disorders, individuals with either high or low levels of negative schizotypy performed a delay discounting task under positive, neutral and negative affective priming conditions. Compared with individuals with low levels of negative schizotypy, individuals with high levels of schizotypy exhibited increased delay discounting, preferring to choose immediate but smaller rewards instead of delayed but larger rewards across all three affective priming conditions. Negative affective priming elevated discounting for both groups compared with both the positive and neutral affective conditions. After dividing delayed temporal distance into the proximal and distal future, the results showed that individuals with high levels of negative schizotypy exhibited more preference for immediate but smaller rewards in the distal instead of proximal future compared with controls. Our results suggest that individuals with high levels of negative schizotypy have altered anticipatory reward processing, which is mainly attributed to alterations in representing rewards in the distal future. These findings extend the alterations in representing reward values from schizophrenia patients to schizotypal individuals, and suggest that diminished anticipatory pleasure in schizophrenia spectrum disorders may be due to changes in processing anticipatory rewards in the distal future.

Computational psychiatry of impulsivity and risk: how risk and time preferences interact in health and disease

Choice impulsivity is an important subcomponent of the broader construct of impulsivity and is a key feature of many psychiatric disorders. Choice impulsivity is typically quantified as temporal discounting, a well-documented phenomenon in which a reward's subjective value diminishes as the delay to its delivery is increased. However, an individual's proclivity to-or more commonly aversion to- risk can influence nearly all of the standard experimental tools available for measuring temporal discounting. Despite this interaction, risk preference is a behaviourally and neurobiologically distinct construct that relates to the economic notion of utility or subjective value. In this opinion piece, we discuss the mathematical relationship between risk preferences and time preferences, their neural implementation, and propose ways that research in psychiatry could, and perhaps should, aim to account for this relationship experimentally to better understand choice impulsivity and its clinical implications. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications'.

Risk preferences impose a hidden distortion on measures of choice impulsivity

Measuring temporal discounting through the use of intertemporal choice tasks is now the gold standard method for quantifying human choice impulsivity (impatience) in neuroscience, psychology, behavioral economics, public health and computational psychiatry. A recent area of growing interest is individual differences in discounting levels, as these may predispose to (or protect from) mental health disorders, addictive behaviors, and other diseases. At the same time, more and more studies have been dedicated to the quantification of individual attitudes towards risk, which have been measured in many clinical and non-clinical populations using closely related techniques. Economists have pointed to interactions between measurements of time preferences and risk preferences that may distort estimations of the discount rate. However, although becoming standard practice in economics, discount rates and risk preferences are rarely measured simultaneously in the same subjects in other fields, and the magnitude of the imposed distortion is unknown in the assessment of individual differences. Here, we show that standard models of temporal discounting —such as a hyperbolic discounting model widely present in the literature which fails to account for risk attitudes in the estimation of discount rates— result in a large and systematic pattern of bias in estimated discounting parameters. This can lead to the spurious attribution of differences in impulsivity between individuals when in fact differences in risk attitudes account for observed behavioral differences. We advance a model which, when applied to standard choice tasks typically used in psychology and neuroscience, provides both a better fit to the data and successfully de-correlates risk and impulsivity parameters. This results in measures that are more accurate and thus of greater utility to the many fields interested in individual differences in impulsivity.

Tobacco demand, delay discounting, and smoking topography among smokers with and without psychopathology

INTRODUCTION

Tobacco demand (i.e., relative value attributed to a given reinforcer) and delay discounting (i.e., relative preference for smaller immediate rewards over larger delayed rewards) are two behavioral economic processes that are linked to the progression of problematic substance use. These processes have not been studied among those with psychopathology, a vulnerable group of smokers. The current study examined differences in tobacco demand and delay discounting, and their association with smoking topography among smokers with (n=43) and without (n=64) past-year psychopathology.

METHOD

Adult daily smokers (n=107,Mage=43.5; SD=9.7) participated in a study on "smoking behavior." Past-year psychological disorders were assessed via a clinician-administered diagnostic assessment. All subjects participated in an ad libitum smoking trial and then completed an assessment of delay discounting (Monetary Choice Questionnaire) and tobacco demand (Cigarette Purchase Task) approximately 45-60min post-smoking.

RESULTS

Smokers with psychopathology, compared to those without, had significantly higher demand intensity and maximum expenditure on tobacco (Omax), but did not differ on other demand indices or delay discounting. Smokers with psychopathology had shorter average inter-puff intervals and shorter time to cigarette completion than smokers without psychopathology. Tobacco demand and delay discounting measures were significantly intercorrelated among smokers with psychopathology, but not those without. Both behavioral economic measures were associated with specific aspects of smoking topography in smokers with psychopathology.

DISCUSSION

The association between tobacco demand and delay discounting is evident among smokers with psychopathology and both measures were most consistently related to smoking behavior.

Genetic and Modeling Approaches Reveal Distinct Components of Impulsive Behavior

Impulsivity is an endophenotype found in many psychiatric disorders including substance use disorders, pathological gambling, and attention deficit hyperactivity disorder. Two behavioral features often considered in impulsive behavior are behavioral inhibition (impulsive action) and delayed gratification (impulsive choice). However, the extent to which these behavioral constructs represent distinct facets of behavior with discrete biological bases is unclear. To test the hypothesis that impulsive action and impulsive choice represent statistically independent behavioral constructs in mice, we collected behavioral measures of impulsivity in a single cohort of mice using well-validated operant behavioral paradigms. Mice with manipulation of serotonin 1B receptor (5-HT_1BR) expression were included as a model of disordered impulsivity. A factor analysis was used to characterize correlations between the measures of impulsivity and to identify covariates. Using two approaches, we dissociated impulsive action from impulsive choice. First, the absence of 5-HT_1BRs caused increased impulsive action, but not impulsive choice. Second, based on an exploratory factor analysis, a two-factor model described the data well, with measures of impulsive action and choice separating into two independent factors. A multiple-indicator multiple-causes analysis showed that 5-HT_1BR expression and sex were significant covariates of impulsivity. Males displayed increased impulsivity in both dimensions, whereas 5-HT_1BR expression was a predictor of increased impulsive action only. These data support the conclusion that impulsive action and impulsive choice are distinct behavioral phenotypes with dissociable biological influences that can be modeled in mice. Our work may help inform better classification, diagnosis, and treatment of psychiatric disorders, which present with disordered impulsivity.