An adaptive, individualized fMRI delay discounting procedure to increase flexibility and optimize scanner time

Not all discounts are created equal: Regional activity and brain networks in temporal and effort discounting

Reward outcomes associated with costs like time delay and effort investment are generally discounted in decision-making. Standard economic models predict rewards associated with different types of costs are devalued in a similar manner. However, our review of rodent lesion studies indicated partial dissociations between brain regions supporting temporal- and effort-based decision-making. Another debate is whether options involving low and high costs are processed in different brain substrates (dual-system) or in the same regions (single-system). This research addressed these issues using coordinate-based, connectivity-based, and activation network-based meta-analyses to identify overlapping and separable neural systems supporting temporal (39 studies) and effort (20 studies) discounting. Coordinate-based activation likelihood estimation and resting-state connectivity analyses showed immediate-small reward and delayed-large reward choices engaged distinct regions with unique connectivity profiles, but their activation network mapping was found to engage the default mode network. For effort discounting, salience and sensorimotor networks supported low-effort choices, while the frontoparietal network supported high-effort choices. There was little overlap between the temporal and effort networks. Our findings underscore the importance of differentiating different types of costs in decision-making and understanding discounting at both regional and network levels.

The Hippocampus Contributes to Temporal Discounting When Delays and Rewards Are Experienced in the Moment

Temporal discounting (TD) represents the mental devaluation of rewards that are available after a delay. Whether the hippocampus is critical for TD remains unclear, with marked discrepancies between animal and human studies: although animals with discrete hippocampal lesions display impaired TD, human participants with similar lesions show intact performance on classic intertemporal choice tasks. A candidate explanation for this discrepancy is that delays and rewards are experienced in the moment in animal studies but tend to be hypothetical in human studies. We tested this hypothesis by examining the performance of amnesic participants with hippocampal lesions (one female, six males) on a novel experiential intertemporal choice task that used interesting photographs occluded by thick lines as rewards (Patt et al., 2021). Using a logistic function to model indifference points data, we compared performance to that on a classic intertemporal choice task with hypothetical outcomes. Participants with hippocampal lesions displayed impaired patterns of choices in the experiential task but not in the hypothetical task. Specifically, hippocampal lesions were associated with decreased amplitude of the delay-reward trade-off, with persistent choice of the delayed option despite delay increase. These results help explain previous discrepancies across animal and human studies, indicating that the hippocampus plays a critical role in temporal discounting when the outcomes of decisions are experienced in the moment, but not necessarily when they are hypothetical.SIGNIFICANCE STATEMENT Impaired temporal discounting (TD) has been related to maladaptive behaviors, including substance dependence and nonadherence to medical treatment. There is consensus that TD recruits the brain valuation network but whether the hippocampal memory system is additionally recruited remains unclear. This study examined TD in hippocampal amnesia, providing a unique opportunity to explore the role of the hippocampus in cognition. Whereas most human studies have used hypothetical outcomes, this study used a novel experiential task with real-time delays and rewards. Results demonstrated hippocampal involvement in the experiential task, but not in a classic hypothetical task administered for comparison. These findings elucidate previous discrepancies between animal and human TD studies. This reconciliation is critical as animals serve as models of human neurocognition.

Model-based experimental manipulation of probabilistic behavior in interpretable behavioral latent variable models

Introduction

Interpretable latent variable models that probabilistically link behavioral observations to an underlying latent process have increasingly been used to draw inferences on cognition from observed behavior. The latent process usually connects experimental variables to cognitive computation. While such models provide important insights into the latent processes generating behavior, one important aspect has often been overlooked. They may also be used to generate precise and falsifiable behavioral predictions as a function of the modeled experimental variables. In doing so, they pinpoint how experimental conditions must be designed to elicit desired behavior and generate adaptive experiments.

Methods

These ideas are exemplified on the process of delay discounting (DD). After inferring DD models from behavior on a typical DD task, the models are leveraged to generate a second adaptive DD task. Experimental trials in this task are designed to elicit 9 graded behavioral discounting probabilities across participants. Models are then validated and contrasted to competing models in the field by assessing the ouf-of-sample prediction error.

Results

The proposed framework induces discounting probabilities on nine levels. In contrast to several alternative models, the applied model exhibits high validity as indicated by a comparably low prediction error. We also report evidence for inter-individual differences with respect to the most suitable models underlying behavior. Finally, we outline how to adapt the proposed method to the investigation of other cognitive processes including reinforcement learning.

Discussion

Inducing graded behavioral frequencies with the proposed framework may help to highly resolve the underlying cognitive construct and associated neuronal substrates.

Comparing Discounting of Potentially Real Rewards and Losses by Means of Functional Magnetic Resonance Imaging

Aim

Delay discounting (DD) has often been investigated in the context of decision making whereby individuals attribute decreasing value to rewards in the distant future. Less is known about DD in the context of negative consequences. The aim of this pilot study was to identify commonalities and differences between reward and loss discounting on the behavioral as well as the neural level by means of computational modeling and functional Magnetic Resonance Imaging (fMRI). We furthermore compared the neural activation between anticipation of rewards and losses.

Method

We conducted a study combining an intertemporal choice task for potentially real rewards and losses (decision-making) with a monetary incentive/loss delay task (reward/loss anticipation). Thirty healthy participants (age 18-35, 14 female) completed the study. In each trial, participants had to choose between a smaller immediate loss/win and a larger loss/win at a fixed delay of two weeks. Task-related brain activation was measured with fMRI.

Results

Hyperbolic discounting parameters of loss and reward conditions were correlated (r = 0.56). During decision-making, BOLD activation was observed in the parietal and prefrontal cortex, with no differences between reward and loss conditions. During reward and loss anticipation, dissociable activation was observed in the striatum, the anterior insula and the anterior cingulate cortex.

Conclusion

We observed behavior concurrent with DD in both the reward and loss condition, with evidence for similar behavioral and neural patterns in the two conditions. Intertemporal decision-making recruited the fronto-parietal network, whilst reward and loss anticipation were related to activation in the salience network. The interpretation of these findings may be limited to short delays and small monetary outcomes.

A Model Guided Approach to Evoke Homogeneous Behavior During Temporal Reward and Loss Discounting

Background

The tendency to devaluate future options as a function of time, known as delay discounting, is associated with various factors such as psychiatric illness and personality. Under identical experimental conditions, individuals may therefore strongly differ in the degree to which they discount future options. In delay discounting tasks, this inter-individual variability inevitably results in an unequal number of discounted trials per subject, generating difficulties in linking delay discounting to psychophysiological and neural correlates. Many studies have therefore focused on assessing delay discounting adaptively. Here, we extend these approaches by developing an adaptive paradigm which aims at inducing more comparable and homogeneous discounting frequencies across participants on a dimensional scale.

Method

The proposed approach probabilistically links a (common) discounting function to behavior to obtain a probabilistic model, and then exploits the model to obtain a formal condition which defines how to construe experimental trials so as to induce any desired discounting probability. We first infer subject-level models on behavior on a non-adaptive delay discounting task and then use these models to generate adaptive trials designed to evoke graded relative discounting frequencies of 0.3, 0.5, and 0.7 in each participant. We further compare and evaluate common models in the field through out-of-sample prediction error estimates, to iteratively improve the trial-generating model and paradigm.

Results

The developed paradigm successfully increases discounting behavior during both reward and loss discounting. Moreover, it evokes graded relative choice frequencies in line with model-based expectations (i.e., 0.3, 0.5, and 0.7) suggesting that we can successfully homogenize behavior. Our model comparison analyses indicate that hyperboloid models are superior in predicting unseen discounting behavior to more conventional hyperbolic and exponential models. We report out-of-sample error estimates as well as commonalities and differences between reward and loss discounting, demonstrating for instance lower discounting rates, as well as differences in delay perception in loss discounting.

Conclusion

The present work proposes a model-based framework to evoke graded responses linked to cognitive function at a single subject level. Such a framework may be used in the future to measure cognitive functions on a dimensional rather than dichotomous scale.

Transdiagnostic neuroimaging of reward system phenotypes in ADHD and comorbid disorders

ADHD is a disorder characterized by changes in the reward system and which is highly comorbid with other mental disorders, suggesting common neurobiological pathways. Transdiagnostic neuroimaging findings could help to understand whether a dysregulated reward pathway might be the actual link between ADHD and its comorbidities. We here synthesize ADHD neuroimaging findings on the reward system with findings in obesity, depression, and substance use disorder including their comorbid appearance regarding neuroanatomical features (structural MRI) and activation patterns (resting-state and functional MRI). We focus on findings from monetary-incentive-delay (MID) and delay-discounting (DD) tasks and then review data on striatal connectivity and volumetry. Next, for better understanding of comorbidity in adult ADHD, we discuss these neuroimaging features in ADHD, obesity, depression and substance use disorder and ask whether ADHD heterogeneity and comorbidity are reflected by a common dysregulation in the reward system. Finally, we highlight conceptual issues related to heterogeneous paradigms, different phenotyping, longitudinal prediction and highlight some promising future directions for using striatal reward functioning as a clinical biomarker.

Temporal discounting when outcomes are experienced in the moment: Validation of a novel paradigm and comparison with a classic hypothetical intertemporal choice task

When faced with intertemporal choices, people typically devalue rewards available in the future compared to rewards more immediately available, a phenomenon known as temporal discounting. Decisions involving intertemporal choices arise daily, with critical impact on health and financial wellbeing. Although many such decisions are "experiential" in that they involve delays and rewards that are experienced in real-time and can inform subsequent choices, most studies have focused on intertemporal choices with hypothetical outcomes (or outcomes delivered after all decisions are made). The present study focused on experiential intertemporal choices. First, a novel intertemporal choice task was developed and validated, using delays experienced in real time and artistic photographs as consumable perceptual rewards. Second, performance on the experiential task was compared to performance on a classic intertemporal choice task with hypothetical outcomes. Involvement of distinct processes across tasks was probed by examining differential relations to state and trait anxiety. A two-parameter logistic function framework was proposed to fit indifference point data. This approach accounts for individual variability not only in the delay at which an individual switches from choosing the delayed to more immediate option, but also in the slope of that switch. Fit results indicated that the experiential task elicited temporal discounting, with effective trade-off between delay and perceptual reward. Comparison with the hypothetical intertemporal choice task suggested distinct mechanisms: first, temporal discounting across the two tasks was not correlated; and second, state and trait anxiety both were associated with choice behavior in the experiential task, albeit in distinct ways, whereas neither was significantly associated with choice behavior in the hypothetical task. The engagement of different processes in the experiential compared to hypothetical task may align with neural evidence for the recruitment of the hippocampus in animal but not in classic human intertemporal choice studies.

The neural correlates of delay discounting in obesity and binge eating disorder

BACKGROUND AND AIMS

Increased delay discounting is associated with obesity and binge eating disorder (BED). Although BED and obesity frequently co-occur, the neural mechanisms underlying delay discounting in these conditions remain poorly understood.

METHODS

Thirtyfive women with obesity, including 10 participants with obesity and BED and 31 controls completed a monetary delay discounting task during functional magnetic resonance imaging.

RESULTS

We identified that increased discounting rates were associated with decreased activity in the left anterior insula in participants with obesity compared to controls when choosing immediate rewards over delayed rewards (PFWE < 0.05). An exploratory analysis comparing the BED subsample to the other groups did not detect significant differences.

DISCUSSION AND CONCLUSIONS

Our findings suggest decreased activity in the anterior insula may underlie heightened delay discounting in individuals with obesity, contributing the probability of choosing immediate rewards over delayed rewards based on emotional states. Future studies including larger, more diverse samples are required to confirm these effects.

The relationship between delay discounting and Internet addiction: A systematic review and meta-analysis

AIMS

To estimate the difference in delay discounting (DD) between subjects with Internet addiction (IA) and those without as well as to identify significant variables involved in DD.

METHODS

Using the keywords related to IA (e.g., "excessive Internet use", "Internet dependence") AND "delayed reward discounting" OR "delay discounting" OR "temporal discounting" OR "delayed gratification" OR time discounting OR intertemporal choice OR impulsive choice, the PubMed, Embase, and PsycINFO databases were searched from inception to June 2020 for English articles with comparison between subjects with IA and those without. Effect sizes were calculated by group means from the k value or area under the curve (AUC). The random-effects models were used.

RESULTS

Fourteen studies in total were eligible for the current meta-analysis that involved 696 subjects with IA (mean age = 22.71) and 2,394 subjects without (mean age = 21.91). Subjects with IA had a steeper DD rate (g = 1.10, 95% CI: 0.57-1.64; p ≤ 0.01) compared with that in those without. Regarding DD data, the difference between k value and AUC was significant (p < 0.01; AUC > k). Additionally, the estimation of DD by the paper-and-pencil task was larger than that by the computerized task (p < 0.01). Significant difference in the DD rate was also noted between subjects with Internet gaming disorder (IGD) and those with unspecified IA (p = 0.00; IGD > IA). The percentage of men and task variables were significantly associated with the DD rate (all p < 0.01), suggesting impaired DD in subjects with IA.

CONCLUSIONS

Our results suggested the feasibility of utilizing the DD rate as a therapeutic index for cognitive control in IA. Nevertheless, judicious use is recommended taking into consideration the significant difference between k value and AUC.

Strengthened Default Mode Network Activation During Delay Discounting in Adolescents with Anorexia Nervosa After Partial Weight Restoration: A Longitudinal fMRI Study

The capacity of patients with anorexia nervosa (AN) to resist food-based rewards is often assumed to reflect excessive self-control. Previous cross-sectional functional magnetic resonance imaging (fMRI) studies utilizing the delay discounting (DD) paradigm, an index of impulsivity and self-control, suggested altered neural efficiency of decision-making in acutely underweight patients (acAN) and a relative normalization in long-term, weight-recovered individuals with a history of AN (recAN). The current longitudinal study tested for changes in functional magnetic resonance imaging (fMRI) activation during DD associated with intensive weight restoration treatment. A predominately adolescent cohort of 22 female acAN patients (mean age—15.5 years) performed an established DD paradigm during fMRI at the beginning of hospitalization and again after partial weight restoration (≥12% body mass index (BMI) increase). Analyses investigated longitudinal changes in both reward valuation and executive decision-making processes. Additional exploratory analyses included comparisons with data acquired in aged-matched healthy controls (HC) as well as probes of functional connectivity between empirically identified nodes of the “task-positive” frontoparietal control network (FPN) and “task-negative” default-mode network (DMN). While treatment was not associated with changes in behavioral DD parameters or activation, specific to reward processing, deactivation of the DMN during decision-making was significantly less pronounced following partial weight restoration. Strengthened DMN activation during DD might reflect a relative relaxation of cognitive overcontrol or improved self-referential, decision-making. Together, our findings present further evidence that aberrant decision-making in AN might be remediable by treatment and, therefore, might constitute an acute effect rather than a core trait variable of the disorder.

Intact value-based decision-making during intertemporal choice in women with remitted anorexia nervosa? An fMRI study

BACKGROUND

Extreme restrictive food choice in anorexia nervosa is thought to reflect excessive self-control and/or abnormal reward sensitivity. Studies using intertemporal choice paradigms have suggested an increased capacity to delay reward in anorexia nervosa, and this may explain an unusual ability to resist immediate temptation and override hunger in the long-term pursuit of thinness. It remains unclear, however, whether altered delay discounting in anorexia nervosa constitutes a state effect of acute illness or a trait marker observable after recovery.

METHODS

We repeated the analysis from our previous fMRI investigation of intertemporal choice in acutely underweight patients with anorexia nervosa in a sample of weight-recovered women with anorexia nervosa (n = 36) and age-matched healthy controls (n = 36) who participated in the same study protocol. Follow-up analyses explored functional connectivity separately in both the weight-recovered/healthy controls sample and the acute/healthy controls sample.

RESULTS

In contrast to our previous findings in acutely underweight patients with anorexia nervosa, we found no differences between weight-recovered patients with anorexia nervosa and healthy controls at either behavioural or neural levels. New analysis of data from the acute/healthy controls sample sample revealed increased coupling between dorsal anterior cingulate cortex and posterior brain regions as a function of decision difficulty, supporting the hypothesis of altered neural efficiency in the underweight state.

LIMITATIONS

This was a cross-sectional study, and the results may be task-specific.

CONCLUSION

Although our results underlined previous demonstrations of divergent temporal reward discounting in acutely underweight patients with anorexia nervosa, we found no evidence of alteration in patients with weight-recovered anorexia nervosa. Together, these findings suggest that impaired valuebased decision-making may not constitute a defining trait variable or “scar” of the disorder.

Using an Experimental Medicine Approach to Identify Novel Determinants of Addiction

Our understanding and effectiveness in treating addiction is not fully adequate. Therefore, perhaps developing a pragmatic theory for identifying novel determinants and potential interventions is needed. The experimental medicine approach, derived from Claude Bernard, proposes a methodology for inductive theory development and suggests interventions directed at targets closely aligned with the underlying mechanisms of the disorder. The steps of theory development under this approach are intended to (1) identify an intervention target; (2) develop assays to verify target measurement; (3) engage the target via experiment or intervention; and (4) test the degree to which target engagement produces other therapeutically useful changes in the disorder. In this article, we review these steps in detail using an example from our work. That is, shortened temporal windows (target) is frequently observed among those who are addicted. Delay discounting is an assay used to measure that target. We and others have demonstrated manipulation of the target, delay discounting, is associated with changing the drug valuation among those with addiction. We conclude with a culmination of the experimental medicine approach by proposing a recently developed hypothesis of substance use disorder, Reinforcer Pathology 2.0.

Reinforcer pathology: Common neural substrates for delay discounting and snack purchasing in prediabetics

Reinforcer pathology theory stipulates that individuals with both (a) high preference for smaller, immediate over larger, delayed rewards; and (b) high demand for unhealthy commodities are uniquely susceptible to poor health outcomes. Specifically, two behavioral economic tasks (delay discounting, assessing preference for smaller, immediate or larger, delayed rewards; and purchasing, assessing purchases of commodities over changes in price) have been independently associated with conditions such as overweight/obesity and problem substance use. In the present study, we examined possible shared neural regions involved in the processes of delay discounting and demand for snack foods in a prediabetic sample. Fifty-four participants completed both of these tasks. Conjunction between delay discounting and purchasing task results indicates substantial common neural substrates recruited during these two tasks, consistent with interpretations of executive control, interoception, and attention, in the prefrontal cortex, insula, and frontoparietal cortex (superior/middle frontal cortex and superior/inferior parietal lobules), respectively. Collectively, these results suggest possible neural substrates in which the two behavioral risk factors of reinforcer pathology may interact during real-world decision-making in prediabetes.

Addressing the reliability fallacy in fMRI: Similar group effects may arise from unreliable individual effects

To cast valid predictions of future behavior or diagnose disorders, the reliable measurement of a "biomarker" such as the brain activation to prospective reward is a prerequisite. Surprisingly, only a small fraction of functional magnetic resonance imaging (fMRI) studies report or cite the reliability of brain activation maps involved in group analyses. Here, using simulations and exemplary longitudinal data of 126 healthy adolescents performing an intertemporal choice task, we demonstrate that reproducing a group activation map over time is not a sufficient indication of reliable measurements at the individual level. Instead, selecting regions based on significant main effects at the group level may yield estimates that fail to reliably capture individual variance in the subjective evaluation of an offer. Collectively, our results call for more attention on the reliability of supposed biomarkers at the level of the individual. Thus, caution is warranted in employing brain activation patterns prematurely for clinical applications such as diagnosis or tailored interventions before their reliability has been conclusively established by large-scale studies. To facilitate assessing and reporting of the reliability of fMRI contrasts in future studies, we provide a toolbox that incorporates common measures of global and local reliability.

Leave-One-Trial-Out, LOTO, a general approach to link single-trial parameters of cognitive models to neural data

A key goal of model-based cognitive neuroscience is to estimate the trial-by-trial fluctuations of cognitive model parameters in order to link these fluctuations to brain signals. However, previously developed methods are limited by being difficult to implement, time-consuming, or model-specific. Here, we propose an easy, efficient and general approach to estimating trial-wise changes in parameters: Leave-One-Trial-Out (LOTO). The rationale behind LOTO is that the difference between parameter estimates for the complete dataset and for the dataset with one omitted trial reflects the parameter value in the omitted trial. We show that LOTO is superior to estimating parameter values from single trials and compare it to previously proposed approaches. Furthermore, the method makes it possible to distinguish true variability in a parameter from noise and from other sources of variability. In our view, the practicability and generality of LOTO will advance research on tracking fluctuations in latent cognitive variables and linking them to neural data.

Excessive discounting of delayed reinforcers as a trans-disease process: Update on the state of the science

Delay discounting, or the process by which reinforcers lose value with delay to their receipt, has been identified as a trans-disease process underlying addiction, other disorders, and maladaptive health behaviors. Delay discounting has been identified as an endophenotype for multiple psychiatric disorders including substance use disorder, ADHD, and major depressive disorder, with this endophenotype being linked to deficits in dopaminergic and serotonergic neurotransmission. In addition, neuroanatomical and neurophysiological deficits in areas of the executive and impulsive systems have been associated with both steeper discounting and substance use disorders. Delay discounting constitutes a novel target for interventions to change health behaviors. A new theory, termed reinforcer pathology, has been developed uniting these findings and setting the stage for future research.

Neuronal correlates of delay discounting in healthy subjects and its implication for addiction: an ALE meta-analysis study

BACKGROUND

Delay discounting (DD) describes the phenomenon of devaluing future rewards in favor of immediate rewards. Increased DD is a key behavioral marker of addiction, and has been suggested as a target for interventions to alleviate addiction symptoms (e.g., preference for immediate drug use over larger-and-later rewards, and relapses) in patients with substance use disorders (SUD).

OBJECTIVES

Performed a meta-analysis on neuroimaging results of DD regarding specific contrasts in healthy participants. Reviewed the results of existing patient studies in light of the meta-analyses results.

METHODS

We conducted activation likelihood estimation meta-analyses on DD neuroimaging studies (25 studies, n = 583; 354 males and 229 females) regarding six analytic strategies.

RESULTS

The meta-analyses revealed various subdivisions of the cortical-basal ganglia circuits that are associated with different aspects of DD in healthy subjects. By comparing the meta-analyses results and patients' studies regarding each contrast, we highlighted three brain regions that may underlie excessive DD in patients. Decreased left inferior frontal gyrus (IFG) activity was related to less preference for delayed choices; reduced ventral striatum (VS) activity was associated with impaired valuation processes; and declined anterior cingulate cortex (ACC)/medial prefrontal cortex (mPFC) activity was associated with alterations in processing difficult choices.

CONCLUSIONS

We propose that neuromodulation (e.g. deep brain simulation) or behavioral interventions (e.g. episodic future imagination) targeting these key brain regions (IFG, VS, ACC/mPFC) may be effective for improving DD function in patients with SUD, enhancing valuations of future rewards and helping to resist the temptation of immediate drug use.

Individual differences in rate of acquiring stable neural representations of tasks in fMRI

Task-related functional magnetic resonance imaging (fMRI) is a widely-used tool for studying the neural processing correlates of human behavior in both healthy and clinical populations. There is growing interest in mapping individual differences in fMRI task behavior and neural responses. By utilizing neuroadaptive task designs accounting for such individual differences, task durations can be personalized to potentially optimize neuroimaging study outcomes (e.g., classification of task-related brain states). To test this hypothesis, we first retrospectively tracked the volume-by-volume changes of beta weights generated from general linear models (GLM) for 67 adult subjects performing a stop-signal task (SST). We then modeled the convergence of the volume-by-volume changes of beta weights according to their exponential decay (ED) in units of half-life. Our results showed significant differences in beta weight convergence estimates of optimal stopping times (OSTs) between go following successful stop trials and failed stop trials for both cocaine dependent (CD) and control group (Con), and between go following successful stop trials and go following failed stop trials for Con group. Further, we implemented support vector machine (SVM) classification for 67 CD/Con labeled subjects and compared the classification accuracies of fMRI-based features derived from (1) the full fMRI task versus (2) the fMRI task truncated to multiples of the unit of half-life. Among the computed binary classification accuracies, two types of task durations based on 2 half-lives significantly outperformed the accuracies using fully acquired trials, supporting this length as the OST for the SST. In conclusion, we demonstrate the potential of a neuroadaptive task design that can be widely applied to personalizing other task-based fMRI experiments in either dynamic real-time fMRI applications or within fMRI preprocessing pipelines.