Reward-related learning via multiple memory systems

Decisional brain of lawyers at the workplace. A neurolaw pilot study

This pilot study investigated legal and non-legal professionals' decision process during a typical working day. During self-evaluated highly relevant decisions (rated through a daily diary), the two groups were asked to wear the Muse™ Headband to record their electrophysiological (EEG) activity in terms of frequency bands (delta, theta, alpha and beta). EEG cognitive findings displayed a generally increased beta power in the anterior frontal region (mainly in the right than left) for both groups during highly relevant decisions. Significantly results were also found for the legal professionals' group, for which a decrease of alpha power was found in the left compared to right frontal cortex. Furthermore, a decreased alpha power and increased delta and theta power in the right compared to left Temporo-Parietal Junction was observed in the legal professionals when taking highly relevant decisions. This pilot study suggested a specific EEG pattern for legal professionals while taking highly relevant decisions.

The balance of hippocampal and caudate network functional connectivity is associated with episodic memory performance and its decline across adulthood

The hippocampal and caudate networks interact to support episodic memory, but the relationship between hippocampal and caudate connectivity strength and episodic memory is unclear. In general, cognition is optimally supported when connectivity within a functional network dominates connectivity from other networks. For example, episodic memory may be optimally supported when the hippocampal and caudate networks express this pattern of connectivity, consistent with research showing that the two networks are organized competitively. Alternatively, episodic memory may be optimally supported when connectivity in both networks is more balanced, consistent with fMRI reports showing cooperation between networks. Using cross-sectional behavioral and resting state fMRI data from a diverse sample (N = 347; Ages 18-85), I tested the hypothesis that reduced hippocampal and caudate network dominance would be associated with reduced episodic memory across individuals and age. Consistent with this hypothesis, lower caudate network dominance in bilateral thalamic regions was associated with worse episodic memory regardless of age. Age-related differences in caudate network dominance in the pallidum and putamen were also associated with worse episodic memory performance, but through their shared variance with age. I found no evidence that network dominance was related to processing speed or executive function, or that hippocampal network dominance was relate to episodic memory performance. These results show that ongoing biological dynamics between the hippocampal and caudate networks throughout adulthood are related to episodic memory performance and support a growing literature specifying the role of the caudate network in episodic memory.

Learning strategy impacts medical diagnostic reasoning in early learners

Relating learned information to similar yet new scenarios, transfer of learning, is a key characteristic of expert reasoning in many fields including medicine. Psychological research indicates that transfer of learning is enhanced via active retrieval strategies. For diagnostic reasoning, this finding suggests that actively retrieving diagnostic information about patient cases could improve the ability to engage in transfer of learning to later diagnostic decisions. To test this hypothesis, we conducted an experiment in which two groups of undergraduate student participants learned symptom lists of simplified psychiatric diagnoses (e.g., Schizophrenia; Mania). Next, one group received written patient cases and actively retrieved the cases from memory and the other group read these written cases twice, engaging in a passive rehearsal learning strategy. Both groups then diagnosed test cases that had two equally valid diagnoses-one supported by "familiar" symptoms described in learned patient cases, and one by novel symptom descriptions. While all participants were more likely to assign higher diagnostic probability to those supported by the familiar symptoms, this effect was significantly larger for participants that engaged in active retrieval compared to passive rehearsal. There were also significant differences in performance across the given diagnoses, potentially due to differences in established knowledge of the disorders. To test this prediction, Experiment 2 compared performance on the described experiment between a participant group that received the standard diagnostic labels to a group that received fictional diagnostic labels, nonsense words designed to remove prior knowledge with each diagnosis. As predicted, there was no effect of diagnosis on task performance for the fictional label group. These results provide new insight on the impact of learning strategy and prior knowledge in fostering transfer of learning, potentially contributing to expert development in medicine.

Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T

The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.

Response Flexibility: The Role of the Lateral Habenula

The ability to make appropriate decisions that result in an optimal outcome is critical for survival. This process involves assessing the environment as well as integrating prior knowledge about the environment with information about one's current internal state. There are many neural structures that play critical roles in mediating these processes, but it is not yet known how such information coalesces to influence behavioral output. The lateral habenula (LHb) has often been cited as a structure critical for adaptive and flexible responding when environmental contexts and internal state changes. A challenge, however, has been understanding how LHb promotes response flexibility. In this review, we hypothesize that the LHb enables flexible responding following the integration of context memory and internal state information by signaling downstream brainstem structures known to drive hippocampal theta. In this way, animals respond more flexibly in a task situation not because the LHb selects a particular action, but rather because LHb enhances a hippocampal neural state that is often associated with greater attention, arousal, and exploration. In freely navigating animals, these are essential conditions that are needed to discover and implement appropriate alternative choices and behaviors. As a corollary to our hypothesis, we describe short- and intermediate-term functions of the LHb. Finally, we discuss the effects on the behavior of LHb dysfunction in short- and intermediate-timescales, and then suggest that new therapies may act on the LHb to alleviate the behavioral impairments following long-term LHb disruption.

How children learn to communicate discriminatively

How do children learn to communicate, and what do they learn? Traditionally, most theories have taken an associative, compositional approach to these questions, supposing children acquire an inventory of form-meaning associations, and procedures for composing / decomposing them; into / from messages in production and comprehension. This paper presents an alternative account of human communication and its acquisition based on the systematic, discriminative approach embodied in psychological and computational models of learning, and formally described by communication theory. It describes how discriminative learning theory offers an alternative perspective on the way that systems of semantic cues are conditioned onto communicative codes, while information theory provides a very different view of the nature of the codes themselves. It shows how the distributional properties of languages satisfy the communicative requirements described in information theory, enabling language learners to align their expectations despite the vastly different levels of experience among language users, and to master communication systems far more abstract than linguistic intuitions traditionally assume. Topics reviewed include morphological development, the acquisition of verb argument structures, and the functions of linguistic systems that have proven to be stumbling blocks for compositional theories: grammatical gender and personal names.

Functional connectivity of dorsolateral prefrontal cortex predicts cocaine relapse: implications for neuromodulation treatment

Relapse is one of the most perplexing problems of addiction. The dorsolateral prefrontal cortex is crucially involved in numerous cognitive and affective processes that are implicated in the phenotypes of both substance use disorders and other neuropsychiatric diseases and has become the principal site to deliver transcranial magnetic stimulation for their treatment. However, the dorsolateral prefrontal cortex is an anatomically large and functionally heterogeneous region, and the specific dorsolateral prefrontal cortex locus and dorsolateral prefrontal cortex-based functional circuits that contribute to drug relapse and/or treatment outcome remain unknown. We systematically investigated the relationship of cocaine relapse with functional circuits from 98 dorsolateral prefrontal cortex regions-of-interest defined by evenly sampling the entire surface of bilateral dorsolateral prefrontal cortex in a cohort of cocaine dependent patients (n = 43, 5 Fr) following a psychosocial treatment intervention. Cox regression models were utilized to predict relapse likelihood based on dorsolateral prefrontal cortex functional connectivity strength. Functional connectivity from only 3 of the 98 dorsolateral prefrontal cortex loci, one in the left and two in the right hemisphere, significantly predicted cocaine relapse with an accuracy of 83.9%, 84.6% and 85.4%, respectively. Combining all three loci significantly improved prediction validity to 87.5%. Protective and risk circuits related to these dorsolateral prefrontal cortex loci were identified that have previously been implicated to support 'bottom up' drive to use drug and 'top down' control over behaviour together with social emotional, learning and memory processing. Three dorsolateral prefrontal cortex-centric circuits were identified that predict relapse to cocaine use with high accuracy. These functionally distinct dorsolateral prefrontal cortex-based circuits provide insights into the multiple roles played by the dorsolateral prefrontal cortex in cognitive and affective functioning that affects treatment outcome. The identified dorsolateral prefrontal cortex loci may serve as potential neuromodulation targets to be tested in subsequent clinical studies for addiction treatment and as clinically relevant biomarkers of its efficacy. Zhai et al. identify three dorsolateral prefrontal cortex (dlPFC)-centric circuits that predict cocaine relapse with high accuracy, providing insights into the multiple roles of the dlPFC in brain functioning that affects treatment outcome and suggesting the dlPFC loci as potential neuromodulation targets for addiction treatment.

Psychiatric Neural Networks and Precision Therapeutics by Machine Learning

Learning and environmental adaptation increase the likelihood of survival and improve the quality of life. However, it is often difficult to judge optimal behaviors in real life due to highly complex social dynamics and environment. Consequentially, many different brain regions and neuronal circuits are involved in decision-making. Many neurobiological studies on decision-making show that behaviors are chosen through coordination among multiple neural network systems, each implementing a distinct set of computational algorithms. Although these processes are commonly abnormal in neurological and psychiatric disorders, the underlying causes remain incompletely elucidated. Machine learning approaches with multidimensional data sets have the potential to not only pathologically redefine mental illnesses but also better improve therapeutic outcomes than DSM/ICD diagnoses. Furthermore, measurable endophenotypes could allow for early disease detection, prognosis, and optimal treatment regime for individuals. In this review, decision-making in real life and psychiatric disorders and the applications of machine learning in brain imaging studies on psychiatric disorders are summarized, and considerations for the future clinical translation are outlined. This review also aims to introduce clinicians, scientists, and engineers to the opportunities and challenges in bringing artificial intelligence into psychiatric practice.

Functional and Structural Alteration of Default Mode, Executive Control, and Salience Networks in Alcohol Use Disorder

Alcohol use disorder (AUD) has been related to aberrant functional connectivity (FC) in the salience network (SN), executive control network (ECN), and default mode network (DMN). However, there is a lack of comprehensive and simultaneous examination of these networks in patients with AUD and of their relation to potential anatomical changes. We aimed to comprehensively examine the alteration in FC in the three networks in AUD patients, and the correlation of the alteration with anatomical/structural changes (volume) in the neural areas implicated in these networks, by applying voxel-based morphometry (VBM) and region of interest-to-region of interest connectivity analysis simultaneously. In all, 22 patients with AUD and 22 healthy adults participated in the study and underwent T1 magnetic resonance imaging. Patients with AUD showed increased FCs within the DMN and SN networks, especially in terms of connectivity of the frontal areas and bilateral hippocampi. They also showed decreased FCs in the ECN. In addition, there was significant volume reduction in these areas (frontal areas and hippocampus). The increased FCs within the frontal areas or bilateral hippocampi showed a negative correlation with gray matter volume of these areas in AUD patients. Our findings add to the empirical evidence that the frontal lobe and hippocampi are critical areas that are vulnerable to functional and structural changes due to AUD.

Self-regulation of the dopaminergic reward circuit in cocaine users with mental imagery and neurofeedback

BACKGROUND

Enhanced drug-related reward sensitivity accompanied by impaired sensitivity to non-drug related rewards in the mesolimbic dopamine system are thought to underlie the broad motivational deficits and dysfunctional decision-making frequently observed in cocaine use disorder (CUD). Effective approaches to modify this imbalance and reinstate non-drug reward responsiveness are urgently needed. Here, we examined whether cocaine users (CU) can use mental imagery of non-drug rewards to self-regulate the ventral tegmental area and substantia nigra (VTA/SN). We expected that obsessive and compulsive thoughts about cocaine consumption would hamper the ability to self-regulate the VTA/SN activity and tested if real-time fMRI (rtfMRI) neurofeedback (NFB) can improve self-regulation of the VTA/SN.

METHODS

Twenty-two CU and 28 healthy controls (HC) were asked to voluntarily up-regulate VTA/SN activity with non-drug reward imagery alone, or combined with rtfMRI NFB.

RESULTS

On a group level, HC and CU were able to activate the dopaminergic midbrain and other reward regions with reward imagery. In CU, the individual ability to self-regulate the VTA/SN was reduced in those with more severe obsessive-compulsive drug use. NFB enhanced the effect of reward imagery but did not result in transfer effects at the end of the session.

CONCLUSION

CU can voluntary activate their reward system with non-drug reward imagery and improve this ability with rtfMRI NFB. Combining mental imagery and rtFMRI NFB has great potential for modifying the maladapted reward sensitivity and reinstating non-drug reward responsiveness. This motivates further work to examine the use of rtfMRI NFB in the treatment of CUD.

Role of glutamatergic system and mesocorticolimbic circuits in alcohol dependence

Emerging evidence demonstrates that alcohol dependence is associated with dysregulation of several neurotransmitters. Alterations in dopamine, glutamate and gamma-aminobutyric acid release are linked to chronic alcohol exposure. The effects of alcohol on the glutamatergic system in the mesocorticolimbic areas have been investigated extensively. Several studies have demonstrated dysregulation in the glutamatergic systems in animal models exposed to alcohol. Alcohol exposure can lead to an increase in extracellular glutamate concentrations in mesocorticolimbic brain regions. In addition, alcohol exposure affects the expression and functions of several glutamate receptors and glutamate transporters in these brain regions. In this review, we discussed the effects of alcohol exposure on glutamate receptors, glutamate transporters and glutamate homeostasis in each area of the mesocorticolimbic system. In addition, we discussed the genetic aspect of alcohol associated with glutamate and reward circuitry. We also discussed the potential therapeutic role of glutamate receptors and glutamate transporters in each brain region for the treatment of alcohol dependence. Finally, we provided some limitations on targeting the glutamatergic system for potential therapeutic options for the treatment alcohol use disorders.

The network architecture of value learning

Value guides behavior. With knowledge of stimulus values and action consequences, behaviors that maximize expected reward can be selected. Prior work has identified several brain structures critical for representing both stimuli and their values. Yet, it remains unclear how these structures interact with one another and with other regions of the brain to support the dynamic acquisition of value-related knowledge. Here, we use a network neuroscience approach to examine how BOLD functional networks change as 20 healthy human subjects learn the values of novel visual stimuli over the course of four consecutive days. We show that connections between regions of the visual, frontal, and cingulate cortices become stronger as learning progresses, with some of these changes being specific to the type of feedback received during learning. These results demonstrate that functional networks dynamically track behavioral improvement in value judgments, and that interactions between network communities form predictive biomarkers of learning.

Rational human behavior is the pursuit of actions that maximize expected reward. These rewards can be understood as stimulus-value contingencies, learned by experience throughout our lives. Various structures have been recognized to participate in these learning processes. Yet, an understanding of how these structures interact with one another and with other brain regions remains vastly unexplored. Here, we propose a novel analytical framework utilizing and extending techniques from the dynamic network neuroscience to ask “How do our brains change when we learn values?” We find that interactions between sensory and fronto-cingulate structures grow stronger as learning progresses, bringing together several isolated findings in the cognitive neuroscience of value-based behavior and extending our understanding of human learning in general.

Transcranial Stimulation Over the Dorsolateral Prefrontal Cortex Increases the Impact of Past Expenses on Decision-Making

Goal-directed choices should be guided by the expected value of the available options. However, people are often influenced by past costs in their decisions, thus succumbing to a bias known as the "sunk-cost effect." Recent functional magnetic resonance imaging data show that the sunk-cost effect is associated with increased activity in dorsolateral prefrontal cortex (dlPFC) and altered crosstalk of the dlPFC with other prefrontal areas. Are these correlated neural processes causally involved in the sunk-cost effect? Here, we employed transcranial direct current stimulation (tDCS) to examine the role of the dlPFC for biasing choices in line with the cost of past expenses. Specifically, we applied different types of tDCS over the right dlPFC while participants performed an investment task designed to assess the impact of past investments on current choices. Our results show a pronounced sunk-cost effect that was significantly increased by anodal tDCS, but left unaltered by cathodal or sham stimulation. Importantly, choices were not affected by stimulation when no prior investments had been made, underlining the specificity of the obtained effect. Our findings suggest a critical role of the dlPFC in the sunk-cost effect and thus elucidate neural mechanisms by which past investments may influence current decision-making.

Memory states influence value-based decisions

Using memory to guide decisions allows past experience to improve future outcomes. However, the circumstances that modulate how and when memory influences decisions are not well understood. Here, we report that the use of memories to guide decisions depends on the context in which these decisions are made. We show that decisions made in the context of familiar images are more likely to be influenced by past events than are decisions made in the context of novel images (Experiment 1), that this bias persists even when a temporal gap is introduced between the image presentation and the decision (Experiment 2), and that contextual novelty facilitates value learning whereas familiarity facilitates the retrieval and use of previously learned values (Experiment 3). These effects are consistent with neurobiological and computational models of memory, which propose that familiar images evoke a lingering "retrieval state" that facilitates the recollection of other episodic memories. Together, these experiments highlight the importance of episodic memory for decision-making and provide an example of how computational and neurobiological theories can lead to new insights into how and when different types of memories guide our choices. (PsycINFO Database Record

Mechanisms of Choice Behavior Shift Using Cue-approach Training

Cue-approach training has been shown to effectively shift choices for snack food items by associating a cued button-press motor response to particular food items. Furthermore, attention was biased toward previously cued items, even when the cued item is not chosen for real consumption during a choice phase. However, the exact mechanism by which preferences shift during cue-approach training is not entirely clear. In three experiments, we shed light on the possible underlying mechanisms at play during this novel paradigm: (1) Uncued, wholly predictable motor responses paired with particular food items were not sufficient to elicit a preference shift; (2) Cueing motor responses early – concurrently with food item onset – and thus eliminating the need for heightened top–down attention to the food stimulus in preparation for a motor response also eliminated the shift in food preferences. This finding reinforces our hypothesis that heightened attention at behaviorally relevant points in time is key to changing choice behavior in the cue-approach task; (3) Crucially, indicating choice using eye movements rather than manual button presses preserves the effect, thus demonstrating that the shift in preferences is not governed by a learned motor response but more likely via modulation of subjective value in higher associative regions, consistent with previous neuroimaging results. Cue-approach training drives attention at behaviorally relevant points in time to modulate the subjective value of individual items, providing a mechanism for behavior change that does not rely on external reinforcement and that holds great promise for developing real world behavioral interventions.

Ongoing behavioral state information signaled in the lateral habenula guides choice flexibility in freely moving rats

The lateral habenula (LHb) plays a role in a wide variety of behaviors ranging from maternal care, to sleep, to various forms of cognition. One prominent theory with ample supporting evidence is that the LHb serves to relay basal ganglia and limbic signals about negative outcomes to midbrain monoaminergic systems. This makes it likely that the LHb is critically involved in behavioral flexibility as all of these systems have been shown to contribute when flexible behavior is required. Behavioral flexibility is commonly examined across species and is impaired in various neuropsychiatric conditions including autism, depression, addiction, and schizophrenia; conditions in which the LHb is thought to play a role. Therefore, a thorough examination of the role of the LHb in behavioral flexibility serves multiple functions including understanding possible connections with neuropsychiatric illnesses and additional insight into its role in cognition in general. Here, we assess the LHb’s role in behavioral flexibility through comparisons of the roles its afferent and efferent pathways are known to play. Additionally, we provide new evidence supporting the LHb contributions to behavioral flexibility through organization of specific goal directed actions under cognitively demanding conditions. Specifically, in the first experiment, a majority of neurons recorded from the LHb were found to correlate with velocity on a spatial navigation task and did not change significantly when reward outcomes were manipulated. Additionally, measurements of local field potential (LFP) in the theta band revealed significant changes in power relative to velocity and reward location. In a second set of experiments, inactivation of the LHb with the gamma-aminobutyric acid (GABA) agonists baclofen and muscimol led to an impairment in a spatial/response based repeated probabilistic reversal learning task. Control experiments revealed that this impairment was likely due to the demands of repeated switching behaviors as rats were unimpaired on initial discrimination acquisition or retention of probabilistic learning. Taken together, these novel findings compliment other work discussed supporting a role for the LHb in action selection when cognitive or emotional demands are increased. Finally, we discuss future mechanisms by which a superior understanding of the LHb can be obtained through additional examination of behavioral flexibility tasks.

Basal Hippocampal Activity and Its Functional Connectivity Predicts Cocaine Relapse

BACKGROUND

Cocaine-induced neuroplastic changes may result in a heightened propensity for relapse. Using regional cerebral blood flow (rCBF) as a marker of basal neuronal activity, this study assessed alterations in rCBF and related resting state functional connectivity (rsFC) to prospectively predict relapse in patients following treatment for cocaine use disorder (CUD).

METHODS

Pseudocontinuous arterial spin labeling functional magnetic resonance imaging and resting blood oxygen level-dependent functional magnetic resonance imaging data were acquired in the same scan session in abstinent participants with CUD before residential treatment discharge and in 20 healthy matched control subjects. Substance use was assessed twice weekly following discharge. Relapsed participants were defined as those who used stimulants within 30 days following treatment discharge (n = 22); early remission participants (n = 18) did not.

RESULTS

Voxel-wise, whole-brain analysis revealed enhanced rCBF only in the left posterior hippocampus (pHp) in the relapsed group compared with the early remission and control groups. Using this pHp as a seed, increased rsFC strength with the posterior cingulate cortex (PCC)/precuneus was seen in the relapsed versus early remission subgroups. Together, both increased pHp rCBF and strengthened pHp-PCC rsFC predicted relapse with 75% accuracy at 30, 60, and 90 days following treatment.

CONCLUSIONS

In CUD participants at risk of early relapse, increased pHp basal activity and pHp-PCC circuit strength may reflect the propensity for heightened reactivity to cocaine cues and persistent cocaine-related ruminations. Mechanisms to mute hyperactivated brain regions and delink dysregulated neural circuits may prove useful to prevent relapse in patients with CUD.

Large-scale intrinsic functional network organization along the long axis of the human medial temporal lobe

The medial temporal lobe (MTL), encompassing the hippocampus and parahippocampal gyrus (PHG), is a heterogeneous structure which plays a critical role in memory and cognition. Here, we investigate functional architecture of the human MTL along the long axis of the hippocampus and PHG. The hippocampus showed stronger connectivity with striatum, ventral tegmental area and amygdala-regions important for integrating reward and affective signals, whereas the PHG showed stronger connectivity with unimodal and polymodal association cortices. In the hippocampus, the anterior node showed stronger connectivity with the anterior medial temporal lobe and the posterior node showed stronger connectivity with widely distributed cortical and subcortical regions including those involved in sensory and reward processing. In the PHG, differences were characterized by a gradient of increasing anterior-to-posterior connectivity with core nodes of the default mode network. Left and right MTL connectivity patterns were remarkably similar, except for stronger left than right MTL connectivity with regions in the left MTL, the ventral striatum and default mode network. Graph theoretical analysis of MTL-based networks revealed higher node centrality of the posterior, compared to anterior and middle hippocampus. The PHG showed prominent gradients in both node degree and centrality along its anterior-to-posterior axis. Our findings highlight several novel aspects of functional heterogeneity in connectivity along the long axis of the human MTL and provide new insights into how its network organization supports integration and segregation of signals from distributed brain areas. The implications of our findings for a principledunderstanding of distributed pathways that support memory and cognition are discussed.

Striatal and Pallidal Activation during Reward Modulated Movement Using a Translational Paradigm

Human neuroimaging studies of reward processing typically involve tasks that engage decision-making processes in the dorsal striatum or focus upon the ventral striatum's response to feedback expectancy. These studies are often compared to the animal literature; however, some animal studies include both feedback and nonfeedback events that activate the dorsal striatum during feedback expectancy. Differences in task parameters, movement complexity, and motoric effort to attain rewards may partly explain ventral and dorsal striatal response differences across species. We, therefore, used a target capture task during functional neuroimaging that was inspired by a study of single cell modulation in the internal globus pallidus during reward-cued, rotational arm movements in nonhuman primates. In this functional magnetic resonance imaging study, participants used a fiberoptic joystick to make a rotational response to an instruction stimulus that indicated both a target location for a capture movement and whether or not the trial would end with feedback indicating either a small financial gain or a neutral outcome. Portions of the dorsal striatum and pallidum demonstrated greater neural activation to visual cues predicting potential gains relative to cues with no associated outcome. Furthermore, both striatal and pallidal regions displayed a greater response to financial gains relative to neutral outcomes. This reward-dependent modulation of dorsal striatal and pallidal activation in a target-capture task is consistent with findings from reward studies in animals, supporting the use of motorically complex tasks as translational paradigms to investigate the neural substrates of reward expectancy and outcome in humans.

Normative development of ventral striatal resting state connectivity in humans

Incentives play a crucial role in guiding behavior throughout our lives, but perhaps no more so than during the early years of life. The ventral striatum is a critical piece of an incentive-based learning circuit, sharing robust anatomical connections with subcortical (e.g., amygdala, hippocampus) and cortical structures (e.g., medial prefrontal cortex (mPFC), insula) that collectively support incentive valuation and learning. Resting-state functional connectivity (rsFC) is a powerful method that provides insight into the development of the functional architecture of these connections involved in incentive-based learning. We employed a seed-based correlation approach to investigate ventral striatal rsFC in a cross-sectional sample of typically developing individuals between the ages of 4.5 and 23-years old (n=66). Ventral striatal rsFC with the mPFC showed regionally specific linear age-related changes in connectivity that were associated with age-related increases in circulating testosterone levels. Further, ventral striatal connectivity with the posterior hippocampus and posterior insula demonstrated quadratic age-related changes characterized by negative connectivity in adolescence. Finally, across this age range, the ventral striatum demonstrated positive coupling with the amygdala beginning during childhood and remaining consistently positive across age. In sum, our findings suggest that normative ventral striatal rsFC development is dynamic and characterized by early establishment of connectivity with medial prefrontal and limbic structures supporting incentive-based learning, as well as substantial functional reorganization with later developing regions during transitions into and out of adolescence.

Episodic memory encoding interferes with reward learning and decreases striatal prediction errors

Learning is essential for adaptive decision making. The striatum and its dopaminergic inputs are known to support incremental reward-based learning, while the hippocampus is known to support encoding of single events (episodic memory). Although traditionally studied separately, in even simple experiences, these two types of learning are likely to co-occur and may interact. Here we sought to understand the nature of this interaction by examining how incremental reward learning is related to concurrent episodic memory encoding. During the experiment, human participants made choices between two options (colored squares), each associated with a drifting probability of reward, with the goal of earning as much money as possible. Incidental, trial-unique object pictures, unrelated to the choice, were overlaid on each option. The next day, participants were given a surprise memory test for these pictures. We found that better episodic memory was related to a decreased influence of recent reward experience on choice, both within and across participants. fMRI analyses further revealed that during learning the canonical striatal reward prediction error signal was significantly weaker when episodic memory was stronger. This decrease in reward prediction error signals in the striatum was associated with enhanced functional connectivity between the hippocampus and striatum at the time of choice. Our results suggest a mechanism by which memory encoding may compete for striatal processing and provide insight into how interactions between different forms of learning guide reward-based decision making.

Effects of (R)-(-)-5-methyl-1-nicotinoyl-2-pyrazoline on glutamate transporter 1 and cysteine/glutamate exchanger as well as ethanol drinking behavior in male, alcohol-preferring rats

Alcohol consumption is largely associated with alterations in the extracellular glutamate concentrations in several brain reward regions. We recently showed that glutamate transporter 1 (GLT-1) is downregulated following chronic exposure to ethanol for 5 weeks in alcohol-preferring (P) rats and that upregulation of the GLT-1 levels in nucleus accumbens and prefrontal cortex results, in part, in attenuating ethanol consumption. Cystine glutamate antiporter (xCT) is also downregulated after chronic ethanol exposure in P rats, and its upregulation could be valuable in attenuating ethanol drinking. This study examines the effect of a synthetic compound, (R)-(-)-5-methyl-1-nicotinoyl-2-pyrazoline (MS-153), on ethanol drinking and expressions of GLT-1 and xCT in the amygdala and the hippocampus of P rats. P rats were exposed to continuous free-choice access to water, 15% and 30% ethanol, and food for 5 weeks, after which they received treatments of MS-153 or vehicle for 5 days. The results show that MS-153 treatment significantly reduces ethanol consumption. It was revealed that GLT-1 and xCT expressions were downregulated in both the amygdala and the hippocampus of ethanol-vehicle-treated rats (ethanol-vehicle group) compared with water-control animals. MS-153 treatment upregulated GLT-1 and xCT expressions in these brain regions. These findings demonstrate an important role for MS-153 in these glutamate transporters for the attenuation of ethanol-drinking behavior.

System Level spatial-frequency EEG changes coincident with a 90-day cognitive-behavioral therapy program for couples in relationship distress

Evaluating relationship intervention programs traditionally involves the use of self-report surveys or observational studies to assess changes in behavior. Instead, to investigate intervention-related changes in behavior, our study evaluates spatial-frequency electroencephalography (EEG) patterns from the brains of couples participating in an Imago Relationship workshop and 12 weeks of group counseling sessions lasting approximately 90 days. This explorative study recorded 32-channel EEGs from nine committed distressed couples prior to, during and immediately following the Imago Relationship Therapy program. A repeated measures t-Test approach was applied to investigate if significant group level brain pattern changes could be identified in key resting state networks in the brains of the participants that could be correlated with changes in relationship outcome. The study results show that significant reductions in EEG power in the alpha2, beta3 and gamma bands were evident in the averaged brain activity in the pre-frontal, frontal and temporal-parietal cortices that are anatomically associated with the frontal executive, default mode and salience networks of the human brain. Our current understanding of system level neural connectivity and network dynamics strongly indicates that each of these systems is integrally required in learning and implementing a complex communication process taught in the Imago intervention. Thus, a high degree of hemispheric lateralization is consistent with our understanding of language function and mood regulation in the brain and is consistent with recent research into the use of resting frontal EEG asymmetry as an indicator of behavioral changes in distressed couples undergoing a program for relationship improvement. Although preliminary, these results further indicate that the EEG is an inexpensive and easily quantifiable measure, and possibly predictor, of behavioral changes in response to a cognitive behavioral intervention.

Altered intrinsic hippocmapus declarative memory network and its association with impulsivity in abstinent heroin dependent subjects

Converging evidence suggests that addiction can be considered a disease of aberrant learning and memory with impulsive decision-making. In the past decades, numerous studies have demonstrated that drug addiction is involved in multiple memory systems such as classical conditioned drug memory, instrumental learning memory and the habitual learning memory. However, most of these studies have focused on the contributions of non-declarative memory, and declarative memory has largely been neglected in the research of addiction. Based on a recent finding that hippocampus, as a core functioning region of declarative memory, was proved biased the decision-making process based on past experiences by spreading associated reward values throughout memory. Our present study focused on the hippocampus. By utilizing seed-based network analysis on the resting-state functional MRI datasets with the seed hippocampus we tested how the intrinsic hippocampal memory network altered toward drug addiction, and examined how the functional connectivity strength within the altered hippocampal network correlated with behavioral index 'impulsivity'. Our results demonstrated that HD group showed enhanced coherence between hippocampus which represents declarative memory system and non-declarative reward-guided learning memory system, and also showed attenuated intrinsic functional link between hippocampus and top-down control system, compared to the CN group. This alteration was furthered found to have behavioral significance over the behavioral index 'impulsivity' measured with Barratt Impulsiveness Scale (BIS). These results provide insights into the mechanism of declarative memory underlying the impulsive behavior in drug addiction.

Covert rapid action-memory simulation (CRAMS): a hypothesis of hippocampal-prefrontal interactions for adaptive behavior

Effective choices generally require memory, yet little is known regarding the cognitive or neural mechanisms that allow memory to influence choices. We outline a new framework proposing that covert memory processing of hippocampus interacts with action-generation processing of prefrontal cortex in order to arrive at optimal, memory-guided choices. Covert, rapid action-memory simulation (CRAMS) is proposed here as a framework for understanding cognitive and/or behavioral choices, whereby prefrontal-hippocampal interactions quickly provide multiple simulations of potential outcomes used to evaluate the set of possible choices. We hypothesize that this CRAMS process is automatic, obligatory, and covert, meaning that many cycles of action-memory simulation occur in response to choice conflict without an individual's necessary intention and generally without awareness of the simulations, leading to adaptive behavior with little perceived effort. CRAMS is thus distinct from influential proposals that adaptive memory-based behavior in humans requires consciously experienced memory-based construction of possible future scenarios and deliberate decisions among possible future constructions. CRAMS provides an account of why hippocampus has been shown to make critical contributions to the short-term control of behavior, and it motivates several new experimental approaches and hypotheses that could be used to better understand the ubiquitous role of prefrontal-hippocampal interactions in situations that require adaptively using memory to guide choices. Importantly, this framework provides a perspective that allows for testing decision-making mechanisms in a manner that translates well across human and nonhuman animal model systems.

Hippocampal Homer1 levels influence motivational behavior in an operant conditioning task

Loss of motivation and learning impairments are commonly accepted core symptoms of psychiatric disorders such as depression and schizophrenia. Reward-motivated learning is dependent on the hippocampal formation but the molecular mechanisms that lead to functional incentive motivation in this brain region are still largely unknown. Recent evidence implicates neurotransmission via metabotropic glutamate receptors and Homer1, their interaction partner in the postsynaptic density, in drug addiction and motivational learning. As previous reports mainly focused on the prefrontal cortex and the nucleus accumbens, we now investigated the role of hippocampal Homer1 in operant reward learning in the present study. We therefore tested either Homer1 knockout mice or mice that overexpress Homer1 in the hippocampus in an operant conditioning paradigm. Our results show that deletion of Homer1 leads to a diverging phenotype that either displays an inability to perform the task or outstanding hyperactivity in both learning and motivational sessions. Due to the apparent bimodal distribution of this phenotype, the overall effect of Homer1 deletion in this paradigm is not significantly altered. Overexpression of hippocampal Homer1 did not lead to a significantly altered learning performance in any stage of the testing paradigm, yet may subtly contribute to emerging motivational deficits. Our results indicate an involvement of Homer1-mediated signaling in the hippocampus in motivation-based learning tasks and encourage further investigations regarding the specific molecular underpinnings of the phenotypes observed in this study. We also suggest to cautiously interpret the results of this and other studies regarding the phenotype following Homer1 manipulations in animals, since their behavioral phenotype appears to be highly diverse. Future studies would benefit from larger group sizes that would allow splitting the experimental groups in responders and non-responders.

How musical training affects cognitive development: rhythm, reward and other modulating variables

Musical training has recently gained additional interest in education as increasing neuroscientific research demonstrates its positive effects on brain development. Neuroimaging revealed plastic changes in the brains of adult musicians but it is still unclear to what extent they are the product of intensive music training rather than of other factors, such as preexisting biological markers of musicality. In this review, we synthesize a large body of studies demonstrating that benefits of musical training extend beyond the skills it directly aims to train and last well into adulthood. For example, children who undergo musical training have better verbal memory, second language pronunciation accuracy, reading ability and executive functions. Learning to play an instrument as a child may even predict academic performance and IQ in young adulthood. The degree of observed structural and functional adaptation in the brain correlates with intensity and duration of practice. Importantly, the effects on cognitive development depend on the timing of musical initiation due to sensitive periods during development, as well as on several other modulating variables. Notably, we point to motivation, reward and social context of musical education, which are important yet neglected factors affecting the long-term benefits of musical training. Further, we introduce the notion of rhythmic entrainment and suggest that it may represent a mechanism supporting learning and development of executive functions. It also hones temporal processing and orienting of attention in time that may underlie enhancements observed in reading and verbal memory. We conclude that musical training uniquely engenders near and far transfer effects, preparing a foundation for a range of skills, and thus fostering cognitive development.

Dysfunctions of decision-making and cognitive control as transdiagnostic mechanisms of mental disorders: advances, gaps, and needs in current research

Disadvantageous decision-making and impaired volitional control over actions, thoughts, and emotions are characteristics of a wide range of mental disorders such as addiction, eating disorders, depression, and anxiety disorders and may reflect transdiagnostic core mechanisms and possibly vulnerability factors. Elucidating the underlying neurocognitive mechanisms is a precondition for moving from symptom-based to mechanism-based disorder classifications and ultimately mechanism-targeted interventions. However, despite substantial advances in basic research on decision-making and cognitive control, there are still profound gaps in our current understanding of dysfunctions of these processes in mental disorders. Central unresolved questions are: (i) to which degree such dysfunctions reflect transdiagnostic mechanisms or disorder-specific patterns of impairment; (ii) how phenotypical features of mental disorders relate to dysfunctional control parameter settings and aberrant interactions between large-scale brain systems involved in habit and reward-based learning, performance monitoring, emotion regulation, and cognitive control; (iii) whether cognitive control impairments are consequences or antecedent vulnerability factors of mental disorders; (iv) whether they reflect generalized competence impairments or context-specific performance failures; (v) whether not only impaired but also chronic over-control contributes to mental disorders. In the light of these gaps, needs for future research are: (i) an increased focus on basic cognitive-affective mechanisms underlying decision and control dysfunctions across disorders; (ii) longitudinal-prospective studies systematically incorporating theory-driven behavioural tasks and neuroimaging protocols to assess decision-making and control dysfunctions and aberrant interactions between underlying large-scale brain systems; (iii) use of latent-variable models of cognitive control rather than single tasks; (iv) increased focus on the interplay of implicit and explicit cognitive-affective processes; (v) stronger focus on computational models specifying neurocognitive mechanisms underlying phenotypical expressions of mental disorders.

Decision making: from neuroscience to psychiatry

Adaptive behaviors increase the likelihood of survival and reproduction and improve the quality of life. However, it is often difficult to identify optimal behaviors in real life due to the complexity of the decision maker's environment and social dynamics. As a result, although many different brain areas and circuits are involved in decision making, evolutionary and learning solutions adopted by individual decision makers sometimes produce suboptimal outcomes. Although these problems are exacerbated in numerous neurological and psychiatric disorders, their underlying neurobiological causes remain incompletely understood. In this review, theoretical frameworks in economics and machine learning and their applications in recent behavioral and neurobiological studies are summarized. Examples of such applications in clinical domains are also discussed for substance abuse, Parkinson's disease, attention-deficit/hyperactivity disorder, schizophrenia, mood disorders, and autism. Findings from these studies have begun to lay the foundations necessary to improve diagnostics and treatment for various neurological and psychiatric disorders.

Neural predictors of individual differences in response to math tutoring in primary-grade school children

Now, more than ever, the ability to acquire mathematical skills efficiently is critical for academic and professional success, yet little is known about the behavioral and neural mechanisms that drive some children to acquire these skills faster than others. Here we investigate the behavioral and neural predictors of individual differences in arithmetic skill acquisition in response to 8-wk of one-to-one math tutoring. Twenty-four children in grade 3 (ages 8-9 y), a critical period for acquisition of basic mathematical skills, underwent structural and resting-state functional MRI scans pretutoring. A significant shift in arithmetic problem-solving strategies from counting to fact retrieval was observed with tutoring. Notably, the speed and accuracy of arithmetic problem solving increased with tutoring, with some children improving significantly more than others. Next, we examined whether pretutoring behavioral and brain measures could predict individual differences in arithmetic performance improvements with tutoring. No behavioral measures, including intelligence quotient, working memory, or mathematical abilities, predicted performance improvements. In contrast, pretutoring hippocampal volume predicted performance improvements. Furthermore, pretutoring intrinsic functional connectivity of the hippocampus with dorsolateral and ventrolateral prefrontal cortices and the basal ganglia also predicted performance improvements. Our findings provide evidence that individual differences in morphometry and connectivity of brain regions associated with learning and memory, and not regions typically involved in arithmetic processing, are strong predictors of responsiveness to math tutoring in children. More generally, our study suggests that quantitative measures of brain structure and intrinsic brain organization can provide a more sensitive marker of skill acquisition than behavioral measures.

Neuroeconomics: a bridge for translational research

Neuroeconomic methods combine behavioral economic experiments to parameterize aspects of reward-related decision-making with neuroimaging techniques to record corresponding brain activity. In this introductory article to the current special issue, we propose that neuroeconomics is a potential bridge for translational research in psychiatry for several reasons. First, neuroeconomics-derived theoretical predictions about optimal adaptation in a changing environment provide an objective metric to examine psychopathology. Second, neuroeconomics provides a "multilevel" research approach that combines performance (behavioral) measures with intermediate measures between behavior and neurobiology (e.g., neuroimaging) and uses a common metaphor to describe decision-making across multiple levels of explanation. As such, ecologically valid behavioral paradigms closely mirror the physical mechanisms of reward processing. Third, neuroeconomics provides a platform for investigators from neuroscience, economics, psychiatry, and social and clinical psychology to develop a common language for studying reward-related decision making in psychiatric disorders. Therefore, neuroeconomics can provide promising candidate endophenotypes that might help clarify the basis of high heritability associated with psychiatric disorders and that might, in turn, inform treatment.