Neurocomputational mechanisms of reinforcement-guided learning in humans: a review

Prediction Errors for Aversive Events Shape Long-Term Memory Formation through a Distinct Neural Mechanism

Prediction errors (PEs) have been known for decades to guide associative learning, but their role in episodic memory formation has been discovered only recently. To identify the neural mechanisms underlying the impact of aversive PEs on long-term memory formation, we used functional magnetic resonance imaging, while participants saw a series of unique stimuli and estimated the probability that an aversive shock would follow. Our behavioral data showed that negative PEs (i.e., omission of an expected outcome) were associated with superior recognition of the predictive stimuli, whereas positive PEs (i.e., presentation of an unexpected outcome) impaired subsequent memory. While medial temporal lobe (MTL) activity during stimulus encoding was overall associated with enhanced memory, memory-enhancing effects of negative PEs were linked to even decreased MTL activation. Additional large-scale network analyses showed PE-related increases in crosstalk between the "salience network" and a frontoparietal network commonly implicated in memory formation for expectancy-congruent events. These effects could not be explained by mere changes in physiological arousal or the prediction itself. Our results suggest that the superior memory for events associated with negative aversive PEs is driven by a potentially distinct neural mechanism that might serve to set these memories apart from those with expected outcomes.

Ageing is associated with disrupted reinforcement learning whilst learning to help others is preserved

Reinforcement learning is a fundamental mechanism displayed by many species. However, adaptive behaviour depends not only on learning about actions and outcomes that affect ourselves, but also those that affect others. Using computational reinforcement learning models, we tested whether young (age 18-36) and older (age 60-80, total n = 152) adults learn to gain rewards for themselves, another person (prosocial), or neither individual (control). Detailed model comparison showed that a model with separate learning rates for each recipient best explained behaviour. Young adults learned faster when their actions benefitted themselves, compared to others. Compared to young adults, older adults showed reduced self-relevant learning rates but preserved prosocial learning. Moreover, levels of subclinical self-reported psychopathic traits (including lack of concern for others) were lower in older adults and the core affective-interpersonal component of this measure negatively correlated with prosocial learning. These findings suggest learning to benefit others is preserved across the lifespan with implications for reinforcement learning and theories of healthy ageing.

Separating Probability and Reversal Learning in a Novel Probabilistic Reversal Learning Task for Mice

The exploration/exploitation tradeoff – pursuing a known reward vs. sampling from lesser known options in the hope of finding a better payoff – is a fundamental aspect of learning and decision making. In humans, this has been studied using multi-armed bandit tasks. The same processes have also been studied using simplified probabilistic reversal learning (PRL) tasks with binary choices. Our investigations suggest that protocols previously used to explore PRL in mice may prove beyond their cognitive capacities, with animals performing at a no-better-than-chance level. We sought a novel probabilistic learning task to improve behavioral responding in mice, whilst allowing the investigation of the exploration/exploitation tradeoff in decision making. To achieve this, we developed a two-lever operant chamber task with levers corresponding to different probabilities (high/low) of receiving a saccharin reward, reversing the reward contingencies associated with levers once animals reached a threshold of 80% responding at the high rewarding lever. We found that, unlike in existing PRL tasks, mice are able to learn and behave near optimally with 80% high/20% low reward probabilities. Altering the reward contingencies towards equality showed that some mice displayed preference for the high rewarding lever with probabilities as close as 60% high/40% low. Additionally, we show that animal choice behavior can be effectively modelled using reinforcement learning (RL) models incorporating learning rates for positive and negative prediction error, a perseveration parameter, and a noise parameter. This new decision task, coupled with RL analyses, advances access to investigate the neuroscience of the exploration/exploitation tradeoff in decision making.

The neural encoding of information prediction errors during non-instrumental information seeking

In a dynamic world, accurate beliefs about the environment are vital for survival, and individuals should therefore regularly seek out new information with which to update their beliefs. This aspect of behaviour is not well captured by standard theories of decision making, and the neural mechanisms of information seeking remain unclear. One recent theory posits that valuation of information results from representation of informative stimuli within canonical neural reward-processing circuits, even if that information lacks instrumental use. We investigated this question by recording EEG from twenty-three human participants performing a non-instrumental information-seeking task. In this task, participants could pay a monetary cost to receive advance information about the likelihood of receiving reward in a lottery at the end of each trial. Behavioural results showed that participants were willing to incur considerable monetary costs to acquire early but non-instrumental information. Analysis of the event-related potential elicited by informative cues revealed that the feedback-related negativity independently encoded both an information prediction error and a reward prediction error. These findings are consistent with the hypothesis that information seeking results from processing of information within neural reward circuits, and suggests that information may represent a distinct dimension of valuation in decision making under uncertainty.

Examining Procedural Learning and Corticostriatal Pathways for Individual Differences in Language: Testing Endophenotypes of DRD2/ANKK1

The aim of the study was to explore whether genetic variation in the dopaminergic system is associated with procedural learning and the corticostriatal pathways in individuals with developmental language impairment (DLI). We viewed these two systems as endophenotypes and hypothesized that they would be more sensitive indicators of genetic effects than the language phenotype itself. Thus, we genotyped two SNPs in the DRD2/ANKK1 gene complex, and tested for their associations to the phenotype of DLI and the two endophenotypes. Results showed that individuals with DLI revealed poor procedural learning abilities and abnormal structures of the basal ganglia. Genetic variation in DRD2/ANKK1 was associated with procedural learning abilities and with microstructural differences of the caudate nucleus. The association of the language phenotype with these DRD2/ANKK1 polymorphisms was not significant, but the phenotype was significantly associated with the two endophenotypes. We suggest that procedural learning and the corticostriatal pathways could be used as effective endophenotypes to aid molecular genetic studies searching for genes predisposing to DLI.

Impulsivity and predictive control are associated with suboptimal action-selection and action-value learning in regular gamblers

Heightened impulsivity and cognitive biases are risk factors for gambling problems. However, little is known about precisely how these factors increase the risks of gambling-related harm in vulnerable individuals. Here, we modelled the behaviour of eighty-seven community-recruited regular, but not clinically problematic, gamblers during a binary-choice reinforcement-learning game, to characterise the relationships between impulsivity, cognitive biases, and the capacity to make optimal action selections and learn about action-values. Impulsive gamblers showed diminished use of an optimal (Bayesian-derived) probability estimate when selecting between candidate actions, and showed slower learning rates and enhanced non-linear probability weighting while learning action values. Critically, gamblers who believed that it is possible to predict winning outcomes (as 'predictive control') failed to use the game's reinforcement history to guide their action selections. Extensive evidence attests to the ease with which gamblers can erroneously perceive structure in the reinforcement history of games when there is none. Our findings demonstrate that the generic and specific risk factors of impulsivity and cognitive biases can interfere with the capacity of some gamblers to utilise structure when it is available in the reinforcement history of games, potentially increasing their risks of sustaining gambling-related harms.

Early effects of reward anticipation are modulated by dopaminergic stimulation

The abilities to predict future rewards and assess the value of reward delivery are crucial aspects of adaptive behavior. While the mesolimbic system, including dopaminergic midbrain, ventral striatum and prefrontal cortex have long been associated with reward processing, recent studies also indicate a prominent role of early visual brain regions. However, the precise underlying neural mechanisms still remain unclear. To address this issue, we presented participants with visual cues predicting rewards of high and low magnitudes and probability (2×2 factorial design), while neural activity was scanned using magnetoencephalography. Importantly, one group of participants received 150 mg of the dopamine precursor levodopa prior to the experiment, while another group received a placebo. For the placebo group, neural signals of reward probability (but not magnitude) emerged at ∼100 ms after cue presentation at occipital sensors in the event-related magnetic fields. Importantly, these probability signals were absent in the levodopa group indicating a close link. Moreover, levodopa administration reduced oscillatory power in the high (20–30 Hz) and low (13–20 Hz) beta band during both reward anticipation and delivery. Taken together, our findings indicate that visual brain regions are involved in coding prospective reward probability but not magnitude and that these effects are modulated by dopamine.

A universal role of the ventral striatum in reward-based learning: evidence from human studies

Reinforcement learning enables organisms to adjust their behavior in order to maximize rewards. Electrophysiological recordings of dopaminergic midbrain neurons have shown that they code the difference between actual and predicted rewards, i.e., the reward prediction error, in many species. This error signal is conveyed to both the striatum and cortical areas and is thought to play a central role in learning to optimize behavior. However, in human daily life rewards are diverse and often only indirect feedback is available. Here we explore the range of rewards that are processed by the dopaminergic system in human participants, and examine whether it is also involved in learning in the absence of explicit rewards. While results from electrophysiological recordings in humans are sparse, evidence linking dopaminergic activity to the metabolic signal recorded from the midbrain and striatum with functional magnetic resonance imaging (fMRI) is available. Results from fMRI studies suggest that the human ventral striatum (VS) receives valuation information for a diverse set of rewarding stimuli. These range from simple primary reinforcers such as juice rewards over abstract social rewards to internally generated signals on perceived correctness, suggesting that the VS is involved in learning from trial-and-error irrespective of the specific nature of provided rewards. In addition, we summarize evidence that the VS can also be implicated when learning from observing others, and in tasks that go beyond simple stimulus-action-outcome learning, indicating that the reward system is also recruited in more complex learning tasks.

Cost-benefit analysis: the first real rule of fight club?

Competition is ubiquitous among social animals. Vying against a conspecific to achieve a particular outcome often requires one to act aggressively, but this is a costly and inherently risky behavior. So why do we aggressively compete, or at the extreme, fight against others? Early work suggested that competitive aggression might stem from an innate aggressive tendency, emanating from subcortical structures. Later work highlighted key cortical regions that contribute toward an instrumental aggression network, one that is recruited or suppressed as needed to achieve a goal. Recent neuroimaging work hints that competitive aggression is upmost a cost-benefit decision, in that it appears to recruit many components of traditional, non-social decision-making networks. This review provides a historical glimpse into the neuroscience of competitive aggression, and proposes a conceptual advancement for studying competitive behavior by outlining how utility calculations of contested-for resources are skewed, pre- and post-competition. A basic multi-factorial model of utility assessment is proposed to account for competitive endowment effects that stem from the presence of peers, peer salience and disposition, and the tactical effort required for victory. In part, competitive aggression is a learned behavior that should only be repeated if positive outcomes are achieved. However, due to skewed utility assessments, deviations of associative learning occur. Hence truly careful cost-benefit analysis is warranted before choosing to vie against another.

High-learners present larger mid-frontal theta power and connectivity in response to incorrect performance feedback

A crucial aspect of cognitive control and learning is the ability to integrate feedback, that is, to evaluate action outcomes and their deviations from the intended goals and to adjust behavior accordingly. However, how high-learners differ from low-learners in relation to feedback processing has not been characterized. Further, little is known about the underlying brain connectivity patterns during feedback processing. This study aimed to fill these gaps by analyzing electrical brain responses from healthy adult human participants while they performed a time estimation task with correct and incorrect feedback. As compared with low-learners, high-learners presented larger mid-frontal theta (4-8 Hz) oscillations and lower sensorimotor beta (17-24 Hz) oscillations in response to incorrect feedback. Further, high-learners showed larger theta connectivity from left central, associated with motor activity, to mid-frontal, associated with performance monitoring, immediately after feedback (0-0.3 s), followed by (from 0.3 to 0.6 s after feedback) a flux from mid-frontal to prefrontal, associated with executive functioning. We suggest that these results reflect two cognitive processes related to successful feedback processing: first, the obtained feedback is compared with the expected one, and second, the feedback history is updated based on this information. Our results also indicate that high- and low-learners differ not only on how they react to incorrect feedback, but also in relation to how their distant brain areas interact while processing both correct and incorrect feedback. This study demonstrates the neural underpinnings of individual differences in goal-directed adaptive behavior.

Neural correlates of reinforcement learning and social preferences in competitive bidding

In competitive social environments, people often deviate from what rational choice theory prescribes, resulting in losses or suboptimal monetary gains. We investigate how competition affects learning and decision-making in a common value auction task. During the experiment, groups of five human participants were simultaneously scanned using MRI while playing the auction task. We first demonstrate that bidding is well characterized by reinforcement learning with biased reward representations dependent on social preferences. Indicative of reinforcement learning, we found that estimated trial-by-trial prediction errors correlated with activity in the striatum and ventromedial prefrontal cortex. Additionally, we found that individual differences in social preferences were related to activity in the temporal-parietal junction and anterior insula. Connectivity analyses suggest that monetary and social value signals are integrated in the ventromedial prefrontal cortex and striatum. Based on these results, we argue for a novel mechanistic account for the integration of reinforcement history and social preferences in competitive decision-making.

When, what, and how much to reward in reinforcement learning-based models of cognition

Reinforcement learning approaches to cognitive modeling represent task acquisition as learning to choose the sequence of steps that accomplishes the task while maximizing a reward. However, an apparently unrecognized problem for modelers is choosing when, what, and how much to reward; that is, when (the moment: end of trial, subtask, or some other interval of task performance), what (the objective function: e.g., performance time or performance accuracy), and how much (the magnitude: with binary, categorical, or continuous values). In this article, we explore the problem space of these three parameters in the context of a task whose completion entails some combination of 36 state-action pairs, where all intermediate states (i.e., after the initial state and prior to the end state) represent progressive but partial completion of the task. Different choices produce profoundly different learning paths and outcomes, with the strongest effect for moment. Unfortunately, there is little discussion in the literature of the effect of such choices. This absence is disappointing, as the choice of when, what, and how much needs to be made by a modeler for every learning model.

Dissociation of response and feedback negativity in schizophrenia: electrophysiological and computational evidence for a deficit in the representation of value

Contrasting theories of schizophrenia propose that the disorder is characterized by a deficit in phasic changes in dopamine activity in response to ongoing events or, alternatively, by a weakness in the representation of the value of responses. Schizophrenia patients have reliably reduced brain activity following incorrect responses but other research suggests that they may have intact feedback-related potentials, indicating that the impairment may be specifically response-related. We used event-related brain potentials and computational modeling to examine this issue by comparing the neural response to outcomes with the neural response to behaviors that predict outcomes in patients with schizophrenia and psychiatrically healthy comparison subjects. We recorded feedback-related activity in a passive gambling task and a time estimation task and error-related activity in a flanker task. Patients' brain activity following an erroneous response was reduced compared to comparison subjects but feedback-related activity did not differ between groups. To test hypotheses about the possible causes of this pattern of results, we used computational modeling of the electrophysiological data to simulate the effects of an overall reduction in patients' sensitivity to feedback, selective insensitivity to positive or negative feedback, reduced learning rate, and a decreased representation of the value of the response given the stimulus on each trial. The results of the computational modeling suggest that schizophrenia patients exhibit weakened representation of response values, possibly due to failure of the basal ganglia to strongly associate stimuli with appropriate response alternatives.

The microstructure of white matter in male to female transsexuals before cross-sex hormonal treatment. A DTI study

BACKGROUND

Diffusion tensor imaging (DTI) has been shown to be sensitive in detecting white matter differences between sexes. Before cross-sex hormone treatment female to male transsexuals (FtM) differ from females but not from males in several brain fibers. The purpose of this paper is to investigate whether white matter patterns in male to female (MtF) transsexuals before commencing cross-sex hormone treatment are also more similar to those of their biological sex or whether they are more similar to those of their gender identity.

METHOD

DTI was performed in 18 MtF transsexuals and 19 male and 19 female controls scanned with a 3 T Trio Tim Magneton. Fractional anisotropy (FA) was performed on white matter of the whole brain, which was spatially analyzed using Tract-Based Spatial Statistics.

RESULTS

MtF transsexuals differed from both male and female controls bilaterally in the superior longitudinal fasciculus, the right anterior cingulum, the right forceps minor, and the right corticospinal tract.

CONCLUSIONS

Our results show that the white matter microstructure pattern in untreated MtF transsexuals falls halfway between the pattern of male and female controls. The nature of these differences suggests that some fasciculi do not complete the masculinization process in MtF transsexuals during brain development.

Model-based analyses: Promises, pitfalls, and example applications to the study of cognitive control

We discuss a recent approach to investigating cognitive control, which has the potential to deal with some of the challenges inherent in this endeavour. In a model-based approach, the researcher defines a formal, computational model that performs the task at hand and whose performance matches that of a research participant. The internal variables in such a model might then be taken as proxies for latent variables computed in the brain. We discuss the potential advantages of such an approach for the study of the neural underpinnings of cognitive control and its pitfalls, and we make explicit the assumptions underlying the interpretation of data obtained using this approach.

A symbiotic brain-machine interface through value-based decision making

Background

In the development of Brain Machine Interfaces (BMIs), there is a great need to enable users to interact with changing environments during the activities of daily life. It is expected that the number and scope of the learning tasks encountered during interaction with the environment as well as the pattern of brain activity will vary over time. These conditions, in addition to neural reorganization, pose a challenge to decoding neural commands for BMIs. We have developed a new BMI framework in which a computational agent symbiotically decoded users' intended actions by utilizing both motor commands and goal information directly from the brain through a continuous Perception-Action-Reward Cycle (PARC).

Methodology

The control architecture designed was based on Actor-Critic learning, which is a PARC-based reinforcement learning method. Our neurophysiology studies in rat models suggested that Nucleus Accumbens (NAcc) contained a rich representation of goal information in terms of predicting the probability of earning reward and it could be translated into an evaluative feedback for adaptation of the decoder with high precision. Simulated neural control experiments showed that the system was able to maintain high performance in decoding neural motor commands during novel tasks or in the presence of reorganization in the neural input. We then implanted a dual micro-wire array in the primary motor cortex (M1) and the NAcc of rat brain and implemented a full closed-loop system in which robot actions were decoded from the single unit activity in M1 based on an evaluative feedback that was estimated from NAcc.

Conclusions

Our results suggest that adapting the BMI decoder with an evaluative feedback that is directly extracted from the brain is a possible solution to the problem of operating BMIs in changing environments with dynamic neural signals. During closed-loop control, the agent was able to solve a reaching task by capturing the action and reward interdependency in the brain.

Decoding different roles for vmPFC and dlPFC in multi-attribute decision making

In everyday life, successful decision making requires precise representations of expected values. However, for most behavioral options more than one attribute can be relevant in order to predict the expected reward. Thus, to make good or even optimal choices the reward predictions of multiple attributes need to be integrated into a combined expected value. Importantly, the individual attributes of such multi-attribute objects can agree or disagree in their reward prediction. Here we address where the brain encodes the combined reward prediction (averaged across attributes) and where it encodes the variability of the value predictions of the individual attributes. We acquired fMRI data while subjects performed a task in which they had to integrate reward predictions from multiple attributes into a combined value. Using time-resolved pattern recognition techniques (support vector regression) we find that (1) the combined value is encoded in distributed fMRI patterns in the ventromedial prefrontal cortex (vmPFC) and that (2) the variability of value predictions of the individual attributes is encoded in the dorsolateral prefrontal cortex (dlPFC). The combined value could be used to guide choices, whereas the variability of the value predictions of individual attributes indicates an ambiguity that results in an increased difficulty of the value-integration. These results demonstrate that the different features defining multi-attribute objects are encoded in non-overlapping brain regions and therefore suggest different roles for vmPFC and dlPFC in multi-attribute decision making.

Life span differences in electrophysiological correlates of monitoring gains and losses during probabilistic reinforcement learning

By recording the feedback-related negativity (FRN) in response to gains and losses, we investigated the contribution of outcome monitoring mechanisms to age-associated differences in probabilistic reinforcement learning. Specifically, we assessed the difference of the monitoring reactions to gains and losses to investigate the monitoring of outcomes according to task-specific goals across the life span. The FRN and the behavioral indicators of learning were measured in a sample of 44 children, 45 adolescents, 46 younger adults, and 44 older adults. The amplitude of the FRN after gains and losses was found to decrease monotonically from childhood to old age. Furthermore, relative to adolescents and younger adults, both children and older adults (a) showed smaller differences between the FRN after losses and the FRN after gains, indicating a less differentiated classification of outcomes on the basis of task-specific goals; (b) needed more trials to learn from choice outcomes, particularly when differences in reward likelihood between the choices were small; and (c) learned less from gains than from losses. We suggest that the relatively greater loss sensitivity among children and older adults may reflect ontogenetic changes in dopaminergic neuromodulation.

Dissociating the contributions of independent corticostriatal systems to visual categorization learning through the use of reinforcement learning modeling and Granger causality modeling

We dissociated the contributions to learning of four corticostriatal "loops" (interacting striatal and cortical regions): motor (putamen and motor cortex), visual (posterior caudate and visual cortex), executive (anterior caudate and prefrontal cortex), and motivational (ventral striatum and ventromedial frontal cortex). Subjects learned to categorize individual repeated images into one of two arbitrary categories via trial and error. We identified (1) regions sensitive to correct categorization, categorization learning, and feedback valence; (2) regions sensitive to prediction error (violation of feedback expectancy) and reward prediction (expected feedback associated with category response) using reinforcement learning modeling; and (3) directed influences between regions using Granger causality modeling. Each loop showed a unique pattern of sensitivity to each of these factors. Both the motor and visual loops were involved in acquisition of categorization ability: activity during correct categorization increased across learning and was sensitive to reward prediction. However, the posterior caudate received directed influence from visual cortex, whereas the putamen exerted directed influence on motor cortex. The motivational and executive loops were involved in feedback processing: both regions were sensitive to feedback valence, which interacted with learning across scans. However, the motivational loop activity reflected prediction error, whereas executive loop activity reflected reward prediction, consistent with the executive loop role in integrating reward and action. Granger causality modeling found directed influences between striatal and cortical regions within each loop. Across loops, the motor loop exerted directed influence on the executive loop which is consistent with the role of the executive loop in integrating feedback with stimulus-response history.

Better than expected or as bad as you thought? The neurocognitive development of probabilistic feedback processing

Learning from feedback lies at the foundation of adaptive behavior. Two prior neuroimaging studies have suggested that there are qualitative differences in how children and adults use feedback by demonstrating that dorsolateral prefrontal cortex (DLPFC) and parietal cortex were more active after negative feedback for adults, but after positive feedback for children. In the current study we used functional magnetic resonance imaging (fMRI) to test whether this difference is related to valence or informative value of the feedback by examining neural responses to negative and positive feedback while applying probabilistic rules. In total, 67 healthy volunteers between ages 8 and 22 participated in the study (8–11 years, n = 18; 13–16 years, n = 27; 18–22 years, n = 22). Behavioral comparisons showed that all participants were able to learn probabilistic rules equally well. DLPFC and dorsal anterior cingulate cortex were more active in younger children following positive feedback and in adults following negative feedback, but only when exploring alternative rules, not when applying the most advantageous rules. These findings suggest that developmental differences in neural responses to feedback are not related to valence per se, but that there is an age-related change in processing learning signals with different informative value.

Unconscious errors enhance prefrontal-occipital oscillatory synchrony

The medial prefrontal cortex (MFC) is critical for our ability to learn from previous mistakes. Here we provide evidence that neurophysiological oscillatory long-range synchrony is a mechanism of post-error adaptation that occurs even without conscious awareness of the error. During a visually signaled Go/No-Go task in which half of the No-Go cues were masked and thus not consciously perceived, response errors enhanced tonic (i.e., over 1–2 s) oscillatory synchrony between MFC and occipital cortex (OCC) leading up to and during the subsequent trial. Spectral Granger causality analyses demonstrated that MFC → OCC directional synchrony was enhanced during trials following both conscious and unconscious errors, whereas transient stimulus-induced occipital → MFC directional synchrony was independent of errors in the previous trial. Further, the strength of pre-trial MFC-occipital synchrony predicted individual differences in task performance. Together, these findings suggest that synchronous neurophysiological oscillations are a plausible mechanism of MFC-driven cognitive control that is independent of conscious awareness.

Brain and autonomic association accompanying stochastic decision-making

To examine the functional association between brain and autonomic activities accompanying decision-making, we simultaneously recorded regional cerebral blood flow using (15)O-water positron emission tomography and event-related brain potentials (ERPs) time-locked to feedback of reward and punishment, as well as cardiovascular parameters, during a stochastic decision-making task. We manipulated the uncertainty of outcomes in the task; specifically, we compared a condition with high predictability of reward/punishment (contingent-reward condition) and a condition with low predictability of reward/punishment (random-reward condition). The anterior cingulate cortex (ACC) was commonly activated in both conditions. Compared with the contingent-reward condition, the orbitofrontal and right dorsolateral prefrontal cortices and dorsal striatum were activated in the random-reward condition, where subjects had to continue to seek contingency between stimuli and reward/punishment. Activation of these brain regions correlated with a positive component of ERPs locked to feedback signals (feedback-related positivity), which showed an association with behavioral decision-making in the contingent-reward condition. Furthermore, cardiovascular responses were attenuated in the random-reward condition, where continuous attention and contingency monitoring were needed, and such attenuation of cardiovascular responses was mediated by vagal activity that was governed by the rostral ACC. These findings suggest that the prefrontal-striatal network provides a neural basis for decision-making and modulation over the peripheral autonomic activity accompanying decision-making.

CNTRICS final task selection: long-term memory

Long-term memory (LTM) is a multifactorial construct, composed of different stages of information processing and different cognitive operations that are mediated by distinct neural systems, some of which may be more responsible for the marked memory problems that limit the daily function of individuals with schizophrenia. From the outset of the CNTRICS initiative, this multidimensionality was appreciated, and an effort was made to identify the specific memory constructs and task paradigms that hold the most promise for immediate translational development. During the second CNTRICS meeting, the LTM group identified item encoding and retrieval and relational encoding and retrieval as key constructs. This article describes the process that the LTM group went through in the third and final CNTRICS meeting to select nominated tasks within the 2 LTM constructs and within a reinforcement learning construct that were judged most promising for immediate development. This discussion is followed by each nominating authors' description of their selected task paradigm, ending with some thoughts about future directions.