Lesions of ventrolateral striatum eliminate lose-shift but not win-stay behaviour in rats

Voluntary wheel running exercise rescues behaviorally-evoked acetylcholine efflux in the medial prefrontal cortex and epigenetic changes in ChAT genes following adolescent intermittent ethanol exposure

Adolescent intermittent ethanol (AIE) exposure, which models heavy binge ethanol intake in adolescence, leads to a variety of deficits that persist into adulthood-including suppression of the cholinergic neuron phenotype within the basal forebrain. This is accompanied by a reduction in acetylcholine (ACh) tone in the medial prefrontal cortex (mPFC). Voluntary wheel running exercise (VEx) has been shown to rescue AIE-induced suppression of the cholinergic phenotype. Therefore, the goal of the current study is to determine if VEx will also rescue ACh efflux in the mPFC during spontaneous alternation, attention set shifting performance, and epigenetic silencing of the cholinergic phenotype following AIE. Male and female rats were subjected to 16 intragastric gavages of 20% ethanol or tap water on a two-day on/two-day off schedule from postnatal day (PD) 25-54, before being assigned to either VEx or stationary control groups. In Experiment 1, rats were tested on a four-arm spontaneous alternation maze with concurrent in vivo microdialysis for ACh in the mPFC. An operant attention set-shifting task was used to measure changes in cognitive and behavioral flexibility. In Experiment 2, a ChIP analysis of choline acetyltransferase (ChAT) genes was performed on basal forebrain tissue. It was found that VEx increased ACh efflux in the mPFC in both AIE and control male and female rats, as well as rescued the AIE-induced epigenetic methylation changes selectively at the Chat promoter CpG island across sexes. Overall, these data support the restorative effects of exercise on damage to the cholinergic projections to the mPFC and demonstrate the plasticity of cholinergic system for recovery after alcohol induced brain damage.

Heterogeneity of decision-making strategies for preschoolers on a variant of the IGT

Adaptive decision-making strategies are critical for dealing with the complexity of the social world. The present study investigated the use of decision-making strategies in preschoolers and their association to prosocial behavior and peer problems. Eighty-six preschoolers aged 3- and 4-years completed the preschool decision-making task (PGT), a child variant of the Iowa Gambling task . Win-stay/lose-shift responses along with exploration (consecutive choices from the advantageous deck) and exploitation (shifting between options) were examined. Preschoolers showed a range of strategies, with 4-year-olds adapting their approach as the game progressed and making better use of feedback in comparison to 3-year-olds. Children who differed in terms of choices from the advantageous deck were distinguished by different combinations of exploration and exploitation. Furthermore, unique combinations of decision-making strategies also distinguished children who were rated as high versus low in prosocial behavior as well as children rated as having a high versus low level of peer problems. The findings suggest that consideration of strategies used in decision-making tasks could provide useful insight in a clinical setting, particularly for populations with social difficulties.

Cannabis use is associated with sexually dimorphic changes in executive control of visuospatial decision-making

When the outcome of a choice is less favorable than expected, humans and animals typically shift to an alternate choice option on subsequent trials. Several lines of evidence indicate that this “lose-shift” responding is an innate sensorimotor response strategy that is normally suppressed by executive function. Therefore, the lose-shift response provides a covert gauge of cognitive control over choice mechanisms. We report here that the spatial position, rather than visual features, of choice targets drives the lose-shift effect. Furthermore, the ability to inhibit lose-shift responding to gain reward is different among male and female habitual cannabis users. Increased self-reported cannabis use was concordant with suppressed response flexibility and an increased tendency to lose-shift in women, which reduced performance in a choice task in which random responding is the optimal strategy. On the other hand, increased cannabis use in men was concordant with reduced reliance on spatial cues during decision-making, and had no impact on the number of correct responses. These data (63,600 trials from 106 participants) provide strong evidence that spatial-motor processing is an important component of economic decision-making, and that its governance by executive systems is different in men and women who use cannabis frequently.

Assessing behavioural profiles following neutral, positive and negative feedback

Previous data suggest zero-value, neutral outcomes (draw) are subjectively assigned negative rather than positive valence. The combined observations of faster rather than slower reaction times, subsequent actions defined by shift rather than stay behaviour, reduced flexibility, and, larger rather than smaller deviations from optimal performance following draws all align with the consequences of explicitly negative outcomes such as losses. We further tested the relationships between neutral, positive and negative outcomes by manipulating value salience and observing their behavioural profiles. Despite speeded reaction times and a non-significant bias towards shift behaviour similar to losses when draws were assigned the value of 0 (Experiment 1), the degree of shift behaviour approached an approximation of optimal performance when the draw value was explicitly positive (+1). This was in contrast to when the draw value was explicitly negative (-1), which led to a significant increase in the degree of shift behaviour (Experiment 2). Similar modifications were absent when the same value manipulations were applied to win or lose trials (Experiment 3). Rather than viewing draws as neutral and valence-free outcomes, the processing cascade generated by draws produces a complex behavioural profile containing elements found in response to both explicitly positive and explicitly negative results.

Breaking the bonds of reinforcement: Effects of trial outcome, rule consistency and rule complexity against exploitable and unexploitable opponents

In two experiments, we used the simple zero-sum game Rock, Paper and Scissors to study the common reinforcement-based rules of repeating choices after winning (win-stay) and shifting from previous choice options after losing (lose-shift). Participants played the game against both computer opponents who could not be exploited and computer opponents who could be exploited by making choices that would at times conflict with reinforcement. Against unexploitable opponents, participants achieved an approximation of random behavior, contrary to previous research commonly finding reinforcement biases. Against exploitable opponents, the participants learned to exploit the opponent regardless of whether optimal choices conflicted with reinforcement or not. The data suggest that learning a rule that allows one to exploit was largely determined by the outcome of the previous trial.

Chronic unpredictable stress shifts rat behavior from exploration to exploitation

Chronic stress evokes wide-ranging behavioral alterations, including risk avoidance, increased motoric output, and reduced consummatory behaviors. These are often interpreted as dysfunctions, but they may subserve adaptations for coping with existential threats. We tested this in a cohort of rats previously exposed to mild unpredictable stress for 5 weeks. Previously stressed rats exhibited the typically increased avoidance of open field and altered responses to predator odor, suggesting enhanced sensitivity to threatening contexts and cues. Interestingly, these animals collected rewards at a higher rate than controls, because they locomoted faster, spent less time in off-task (exploratory) behavior, and committed fewer licks at feeders. Further, they were not impaired in flexibly shifting choice as reward probabilities changed among feeders, suggesting that behavioral adaptations are not simply of transference to behavioral control to neural systems insensitive to reward (e.g. habits). These data add to a small but growing body of evidence indicating that stress shifts responses away from exploration and toward exploitation of resources, possibly to reduce threat exposure.HighlightsRats with a history of stress collected reward at a higher rate than controls on an operant task, owing to increase locomotion speed, reduced off-task behavior, and reduced time licking at feeders.Previously stressed rats exhibited increased win-stay responses than controls, suggesting the involvement of neural circuits related to goal-directed responding.Previously stressed rats performed equally to controls on a task requiring a shift of preferences based on reward probability, suggesting that they are not simply relying more on habit-based neural systems.

A micro-genesis account of longer-form reinforcement learning in structured and unstructured environments

We explored the possibility that in order for longer-form expressions of reinforcement learning (win-calmness, loss-restlessness) to manifest across tasks, they must first develop because of micro-transactions within tasks. We found no evidence of win-calmness or loss-restlessness when wins could not be maximised (unexploitable opponents), nor when the threat of win minimisation was presented (exploiting opponents), but evidence of win-calmness (but not loss-restlessness) when wins could be maximised (exploitable opponents).

Discovering the Lost Reward: Critical Locations for Endocannabinoid Modulation of the Cortico-Striatal Loop That Are Implicated in Major Depression

Depression, the most prevalent psychiatric disorder in the Western world, is characterized by increased negative affect (i.e., depressed mood, cost value increase) and reduced positive affect (i.e., anhedonia, reward value decrease), fatigue, loss of appetite, and reduced psychomotor activity except for cases of agitative depression. Some forms, such as post-partum depression, have a high risk for suicidal attempts. Recent studies in humans and in animal models relate major depression occurrence and reoccurrence to alterations in dopaminergic activity, in addition to other neurotransmitter systems. Imaging studies detected decreased activity in the brain reward circuits in major depression. Therefore, the location of dopamine receptors in these circuits is relevant for understanding major depression. Interestingly, in cortico-striatal-dopaminergic pathways within the reward and cost circuits, the expression of dopamine and its contribution to reward are modulated by endocannabinoid receptors. These receptors are enriched in the striosomal compartment of striatum that selectively projects to dopaminergic neurons of substantia nigra compacta and is vulnerable to stress. This review aims to show the crosstalk between endocannabinoid and dopamine receptors and their vulnerability to stress in the reward circuits, especially in corticostriatal regions. The implications for novel treatments of major depression are discussed.

Variability in competitive decision-making speed and quality against exploiting and exploitative opponents

A presumption in previous work has been that sub-optimality in competitive performance following loss is the result of a reduction in decision-making time (i.e., post-error speeding). The main goal of this paper is to test the relationship between decision-making speed and quality, with the hypothesis that slowing down decision-making should increase the likelihood of successful performance in cases where a model of opponent domination can be implemented. Across Experiments 1–3, the speed and quality of competitive decision-making was examined in a zero-sum game as a function of the nature of the opponent (unexploitable, exploiting, exploitable). Performance was also examined against the nature of a credit (or token) system used as a within-experimental manipulation (no credit, fixed credit, variable credit). To compliment reaction time variation as a function of outcome, both the fixed credit and variable credit conditions were designed to slow down decision-making, relative to a no credit condition where the game could be played in quick succession and without interruption. The data confirmed that (a) self-imposed reductions in processing time following losses (post-error speeding) were causal factors in determining poorer-quality behaviour, (b) the expression of lose-shift was less flexible than the expression of win-stay, and, (c) the use of a variable credit system may enhance the perceived control participants have against exploitable opponents. Future work should seek to disentangle temporal delay and response interruption as determinants of decision-making quality against numerous styles of opponency.

Lesions of lateral habenula attenuate win-stay but not lose-shift responses in a competitive choice task

Multiple neural systems contribute to choice adaptation following reinforcement. Recent evidence suggests that the lateral habenula (LHb) plays a key role in such adaptations, particularly when reinforcements are worse than expected. Here, we investigated the effects of bilateral LHb lesions on responding in a binary choice task with no discriminatory cues. LHb lesions in rats decreased win-stay responses but surprisingly left lose-shift responses intact. This same dissociated effect was also observed after systemic administration of d-amphetamine in a separate cohort of animals. These results suggest that at least some behavioural responses triggered by reward omission do not depend on an intact LHb.