Simulating bout-and-pause patterns with reinforcement learning

Habit formation viewed as structural change in the behavioral network

Habit formation is a process in which an action becomes involuntary. While goal-directed behavior is driven by its consequences, habits are elicited by a situation rather than its consequences. Existing theories have proposed that actions are controlled by corresponding two distinct systems. Although canonical theories based on such distinctions are starting to be challenged, there are a few theoretical frameworks that implement goal-directed behavior and habits within a single system. Here, we propose a novel theoretical framework by hypothesizing that behavior is a network composed of several responses. With this framework, we have shown that the transition of goal-directed actions to habits is caused by a change in a single network structure. Furthermore, we confirmed that the proposed network model behaves in a manner consistent with the existing experimental results reported in animal behavioral studies. Our results revealed that habit could be formed under the control of a single system rather than two distinct systems. By capturing the behavior as a single network change, this framework provides a new perspective on studying the structure of the behavior for experimental and theoretical research.

Reinforcement History Dependent Effects of Low Dose Ethanol on Reward Motivation in Male and Female Mice

Alcohol use disorders (AUDs) are more prevalent in men than in women, though AUD diagnoses in women are growing rapidly, making an understanding of sex differences in alcohol-related behaviors increasingly important. The development of AUDs involves the transition from casual, low levels of alcohol drinking to higher, maladaptive levels. The ability of low dose alcohol to drive reward and drug seeking may differ in males and females, and this could underlie differences in susceptibility to AUD. In this study we sought to determine whether a history of chronic, low dose ethanol exposure (0.5 g/kg; i.p.) could drive sucrose reward seeking and motivation, and whether this differed between male and female mice. Adult mice were trained to lever press for a liquid sucrose reward on two reinforcement schedules: a random interval (RI) schedule and a variable ratio (VR) schedule. After training, mice were tested on each of these levers for reward motivation using a progressive ratio test. We found that a history of low dose ethanol exposure increased sucrose reward motivation in male mice, but only on the RI lever and only when exposure occurred proximal to learning. Female mice were more motivated for sucrose on the RI lever than the VR lever regardless of ethanol exposure condition. These findings indicate that training on different reinforcement schedules affects reward motivation. Further, we show that males are more susceptible to the effects of low dose ethanol on sucrose reward motivation than females. These data broaden our understanding of sex differences in reward seeking as a result of ethanol exposure.

Optogenetic induction of orbitostriatal long-term potentiation in the dorsomedial striatum elicits a persistent reduction of alcohol-seeking behavior in rats

Uncontrolled drug-seeking and -taking behaviors are generally driven by maladaptive corticostriatal synaptic plasticity. The orbital frontal cortex (OFC) and its projections to the dorsomedial striatum (DMS) have been extensively implicated in drug-seeking and relapse behaviors. The influence of the synaptic plasticity of OFC projections to the DMS (OFC→DMS) on drug-seeking and -taking behaviors has not been fully characterized. To investigate this, we trained rats to self-administer 20% alcohol and then delivered an in vivo optogenetic protocol designed to induce long-term potentiation (LTP) selectively at OFC→DMS synapses. We selected LTP induction because we found that voluntary alcohol self-administration suppressed OFC→DMS transmission and LTP may normalize this transmission, consequently reducing alcohol-seeking behavior. Importantly, ex vivo slice electrophysiology studies confirmed that this in vivo optical stimulation protocol resulted in a significant increase in excitatory OFC→DMS transmission strength on day two after stimulation, suggesting that LTP was induced in vivo. Rat alcohol-seeking and -taking behaviors were significantly reduced on days 1-3, but not on days 7-11, after LTP induction. Striatal synaptic plasticity is modulated by several critical neurotransmitter receptors, including dopamine D1 receptors (D1Rs) and adenosine A2A receptors (A2ARs). We found that delivery of in vivo optical stimulation in the presence of a D1R antagonist abolished the LTP-associated decrease in alcohol-seeking behavior, whereas delivery in the presence of an A2AR antagonist may facilitate this LTP-induced behavioral change. These results demonstrate that alcohol-seeking behavior was negatively regulated by the potentiation of excitatory OFC→DMS neurotransmission. Our findings provide direct evidence that the OFC exerts "top-down" control of alcohol-seeking behavior via the DMS.