Role of CAMKII in reinforcement learning: a computational model of glutamate and dopamine signaling pathways

Individual Differences in Ethanol Drinking and Seeking Behaviors in Rats Exposed to Chronic Intermittent Ethanol Vapor Exposure is Associated with Altered CaMKII Autophosphorylation in the Nucleus Accumbens Shell

Chronic intermittent ethanol vapor exposure (CIE) in rodents produces reliable and high blood ethanol concentration and behavioral symptoms associated with moderate to severe alcohol use disorder (AUD)—for example, escalation of operant ethanol self-administration, a feature suggestive of transition from recreational to addictive use, is a widely replicated behavior in rats that experience CIE. Herein, we present evidence from a subset of rats that do not demonstrate escalation of ethanol self-administration following seven weeks of CIE. These low responders (LR) maintain low ethanol self-administration during CIE, demonstrate lower relapse to drinking during abstinence and reduced reinstatement of ethanol seeking triggered by ethanol cues when compared with high responders (HR). We examined the blood ethanol levels in LR and HR rats during CIE and show higher levels in LR compared with HR. We also examined peak corticosterone levels during CIE and show that LR rats have higher levels compared with HR rats. Lastly, we evaluated the levels of Ca²⁺/calmodulin-dependent protein kinase II (CaMKII) in the nucleus accumbens shell and reveal that the activity of CaMKII, which is autophosphorylated at site Tyr-286, is significantly reduced in HR rats compared with LR rats. These findings demonstrate that dysregulation of the hypothalamic–pituitary–adrenal axis activity and plasticity-related proteins regulating molecular memory in the nucleus accumbens shell are associated with higher ethanol-drinking and -seeking in HR rats. Future mechanistic studies should evaluate CaMKII autophosphorylation-dependent remodeling of glutamatergic synapses in the ventral striatum as a plausible mechanism for the CIE-induced enhanced ethanol drinking and seeking behaviors.

Dopamine, affordance and active inference

The role of dopamine in behaviour and decision-making is often cast in terms of reinforcement learning and optimal decision theory. Here, we present an alternative view that frames the physiology of dopamine in terms of Bayes-optimal behaviour. In this account, dopamine controls the precision or salience of (external or internal) cues that engender action. In other words, dopamine balances bottom-up sensory information and top-down prior beliefs when making hierarchical inferences (predictions) about cues that have affordance. In this paper, we focus on the consequences of changing tonic levels of dopamine firing using simulations of cued sequential movements. Crucially, the predictions driving movements are based upon a hierarchical generative model that infers the context in which movements are made. This means that we can confuse agents by changing the context (order) in which cues are presented. These simulations provide a (Bayes-optimal) model of contextual uncertainty and set switching that can be quantified in terms of behavioural and electrophysiological responses. Furthermore, one can simulate dopaminergic lesions (by changing the precision of prediction errors) to produce pathological behaviours that are reminiscent of those seen in neurological disorders such as Parkinson's disease. We use these simulations to demonstrate how a single functional role for dopamine at the synaptic level can manifest in different ways at the behavioural level.