Striatal Activity and Reward Relativity: Neural Signals Encoding Dynamic Outcome Valuation

Altered reward sensitivity to sucrose outcomes prior to drug exposure in alcohol preferring rats

Addiction involves key impairments in reward sensitivity (RS). The current study explored impaired RS to natural reward as a predisposing factor to addictive-like behavior. Alcohol preferring (P) rats are selectively bred based on significantly greater ethanol consumption and preference and offer the ability to inspect differences in subjects with a positive family history of addictive-like behavior. P rat's RS was compared to RS in the well-used Sprague-Dawley (SD) strain. To assess RS in a novel manner, instrumental incentive contrast, discrimination and consumption of sucrose solution were examined. Animals performed in a free operant situation for different sucrose concentration solutions using a block of 'mixed' trials with alternating outcome concentrations (e.g., 5 and 10 % sucrose) to change outcome value in a predictable manner. Animals also performed for reward in blocks of single outcome trials (5 or 10 or 20 or 40 % sucrose daily exposure) surrounding the mixed block. RS (e.g., reward discrimination and contrast effects between and within-sessions) was measured by changes in trials completed, instrumental response latency and consumption. P rats expressed an altered profile of RS with a greater tendency toward equivalent responding to different outcomes within the same session and an absence of incentive contrast from diverse reward comparisons. In contrast, SD animals expressed within-session reward discrimination and a subset of incentive contrast effects. These effects were moderated by food deprivation more consistently in SD compared to P rats. P rat alterations in processing natural rewards could predispose them to addictive-like behaviors including greater alcohol consumption and preference.

Context changes judgments of liking and predictability for melodies

Predictability plays an important role in the experience of musical pleasure. By leveraging expectations, music induces pleasure through tension and surprise. However, musical predictions draw on both prior knowledge and immediate context. Similarly, musical pleasure, which has been shown to depend on predictability, may also vary relative to the individual and context. Although research has demonstrated the influence of both long-term knowledge and stimulus features in influencing expectations, it is unclear how perceptions of a melody are influenced by comparisons to other music pieces heard in the same context. To examine the effects of context we compared how listeners' judgments of two distinct sets of stimuli differed when they were presented alone or in combination. Stimuli were excerpts from a repertoire of Western music and a set of experimenter created melodies. Separate groups of participants rated liking and predictability for each set of stimuli alone and in combination. We found that when heard together, the Repertoire stimuli were more liked and rated as less predictable than if they were heard alone, with the opposite pattern being observed for the Experimental stimuli. This effect was driven by a change in ratings between the Alone and Combined conditions for each stimulus set. These findings demonstrate a context-based shift of predictability ratings and derived pleasure, suggesting that judgments stem not only from the physical properties of the stimulus, but also vary relative to other options available in the immediate context.

Effort-reward balance and work motivation in rats: Effects of context and order of experience

Being motivated means exerting effort toward a goal. The 'law of least work' emphasizes a preference for exerting relatively less effort. The law crosses boundaries among species and between physical and mental work. Organisms should be highly sensitive to shifts in effort-reward balance (ERB) in order to make optimal choices. We used a free operant-foraging task to investigate changes in ERB on choice between options requiring more or less effort. Results showed a consistent preference for the option with less effort and insensitivity to shifts in ERB. A second aim explored the influence of order of experience on effort choice. Choice for the more effortful option significantly increased after experiencing an equal effort-reward relationship during the initial free operant-foraging session. This relative increase in choice for the effortful option persisted even after effort-reward imbalance. The findings highlight the importance of contextual factors such as order of experience when examining the impact of shifting effort-reward associations. Instead of ignoring or reducing order effects, the sequence of experience (e.g. for shifts in ERB) could be manipulated to enhance or reduce value of outcomes or effort itself.

High resolution behavioral and neural activity representation using a geometrical approach

Available tools for recording neuronal activity are limited and reductive due to massive data arising from high-frequency measurements. We have developed a method that utilizes variance within the physiological activity and includes all data points per measurement. Data is expressed geometrically in a physiologically meaningful manner, to represent a precise and detailed view of the recorded neural activity. The recorded raw data from any pair of electrodes was plotted and following a covariance calculation, an eigenvalues and chi-square distribution were used to define the ellipse which bounds 95% of the raw data. We validated our method by correlating specific behavioral observation and physiological activity with behavioral tasks that require similar body movement but potentially involve significantly different neuronal activity. Graphical representation of telemetrically recorded data generates a scatter plot with distinct elliptic geometrical properties that clearly and significantly correlated with behavior. Our reproducible approach improves on existing methods by allowing a dynamic, accurate and comprehensive correlate using an intuitive output. Long-term, it may serve as the basis for advanced machine learning applications and animal-based artificial intelligence models aimed at predicting or characterizing behavior.

Feeding Behavior of Mice under Different Food Allocation Regimens

Social interaction, a basic survival strategy for many animal species, helps maintain a social environment that has limited conflict. Social dominance has a dramatic effect on motivation. Recent evidence suggests that some primate and nonprimate species display aversive behavior toward food allocation regimens that differ from their peers. Thus, we examined the behaviors displayed by mice under different food allocation regimens. We analyzed changes in food intake using several parameters. In the same food condition, the mice received the same food; in the quality different condition, the mice received different foods; in the quantity different condition, one mouse did not receive food; and in the no food condition, none of the mice received food. To test differences based on food quality, one mouse received normal solid food as a less preferred reward, and the other received chocolate chips as a high-level reward. No behavioral change was observed in comparison to the same food condition. To test differences based on food quantity, one mouse received chocolate chips while the other received nothing. Mice who received nothing spent more time on the other side of the reward throughout the experiment. Interestingly, highly rewarded mice required more time to consume the chocolate chips. Thus, under different food allocation regimens, mice changed their behavior by being more hesitant. Moreover, mice alter food intake behavior according to the social environment. The findings help elucidate potential evolutionary aspects that help maintain social cohesion while providing insights into potential mechanisms underlying socially anxious behavior.

Translating striatal activity from brain slice to whole animal neurophysiology: A guide for neuroscience research integrating diverse levels of analysis

An important goal of this review is highlighting research in neuroscience as examples of multilevel functional and anatomical analyses addressing basic science issues and applying results to the understanding of diverse disorders. The research of Dr. Michael Levine, a leader in neuroscience, exemplifies this approach by uncovering fundamental properties of basal ganglia function and translating these findings to clinical applications. The review focuses on neurophysiological research connecting results from in vitro and in vivo recordings. A second goal is to utilize these research connections to produce novel, accurate descriptions for corticostriatal processing involved in varied, complex functions. Medium spiny neurons in striatum act as integrators combining input with baseline activity creating motivational "events." Basic research on corticostriatal synapses is described and links developed to issues with clinical relevance such as inhibitory gating, self-injurious behavior, and relative reward valuation. Work is highlighted on dopamine-glutamate interactions. Individual medium spiny neurons express both D₁ and D₂ receptors and encode information in a bivalent manner depending upon the mix of receptors involved. Current work on neurophysiology of reward processing has taken advantage of these basic approaches at the cellular and molecular levels. Future directions in studying physiology of reward processing and action sequencing could profit by incorporating the divergent ways dopamine modulates incoming neurochemical signals. Primary investigators leading research teams should mirror Mike Levine's efforts in "climbing the mountain" of scientific inquiry by performing analyses at different levels of inquiry, integrating the findings, and building comprehensive answers to problems unsolvable without this bold approach.

An investigation of variety effects during operant responding in the rat utilizing different reward flavors

Humans and nonhuman animals respond to food diversity by increasing intake and appetitive behaviors, reflecting enhanced valuation for items presented in the context of variety. Previous work on food variety effects has posited two main explanatory mechanisms. Variety could slow habituation processes by decreasing exposure to a single food item or could elicit contrast effects in which comparisons between items impact relative valuation. This study used three flavors of sucrose rewards to investigate rats' responses to qualitative reward variety in different variety contexts: low (2 flavors) and high (3 flavors) conditions. Control sessions used only a single flavored pellet (no variety). Animals were tested in low (10 trials), moderate (20 trials) and high consumption (30 trials) sessions. A trial within each session was defined as completion of the operant response and acquisition of the reward pellet. Cues associated with flavors were used to examine predictability and between-trial ('micro') variety. Indicators of a variety effect were found including faster responding for rewards during the variety context compared to an initial control (no variety) context. This decrease in response latency continued to be observed for some measures in post-variety control contexts. The most robust statistical finding of variety effects was found using trial-by-trial analysis, with shorter response latencies obtained for trials with outcomes differing from the preceding trial compared to successive trials with identical outcomes. These results have implications for understanding how a general reward context like variety impacts behavior, and for informing clinical approaches focusing on motivation and eating disorders.

Ventral pallidum encodes relative reward value earlier and more robustly than nucleus accumbens

The ventral striatopallidal system, a basal ganglia network thought to convert limbic information into behavioral action, includes the nucleus accumbens (NAc) and the ventral pallidum (VP), typically described as a major output of NAc. Here, to investigate how reward-related information is transformed across this circuit, we measure the activity of neurons in NAc and VP when rats receive two highly palatable but differentially preferred rewards, allowing us to track the reward-specific information contained within the neural activity of each region. In VP, we find a prominent preference-related signal that flexibly reports the relative value of reward outcomes across multiple conditions. This reward-specific firing in VP is present in a greater proportion of the population and arises sooner following reward delivery than in NAc. Our findings establish VP as a preeminent value signaler and challenge the existing model of information flow in the ventral basal ganglia.

A possible social relative reward effect: Influences of outcome inequity between rats during operant responding

Social interactions/situations have dramatic influences on motivation. Creating animal models examining these influences promotes a better understanding of the psychological and biological underpinnings of social motivation. Rodents are sensitive to social history/experience during associative conditioning and food-sharing tasks. Would reward-oriented operant behavior be sensitive to social influences by showing a negative contrast-like effect when another organism obtains a greater value outcome? We used a side-by-side arrangement of operant response chambers wherein one animal obtained consistently high reward signaled by a discrete cue. The neighboring, experimental rat experienced different combinations of high and low reward trial sequences. Control conditions included distraction from a conspecific in the neighboring chamber (rat distractor) or cue/food dispenser operating without a conspecific (program distractor) in addition to testing subjects alone. Results support an influence of the other animal actively performing the task on the experimental subject's behavior. Primarily, responding was significantly slower for the low reward trials while the neighboring rat was receiving the higher magnitude reward. The lever-press and not food-cup retrieval latency was significantly slower during exposure to a conspecific neighbor performing the operant task. The effect was not obtained in all session sequences and was more pronounced using longer series of consecutive low reward trials. The slowing effect was also obtained with the program-distractor experience in a different trial sequence. These findings suggest a social-induced negative incentive contrast effect in rats possibly mediated by an outcome inequity process that could have key similarities to complex situational-affective effects on motivation involving frustration or jealously.

Neural encoding of choice during a delayed response task in primate striatum and orbitofrontal cortex

Reward outcomes are available in many diverse situations and all involve choice. If there are multiple outcomes each rewarding, then decisions regarding relative value lead to choosing one over another. Important factors related to choice context should be encoded and utilized for this form of adaptive choosing. These factors can include the number of alternatives, the pacing of choice behavior and the possibility to reverse one's choice. An essential step in understanding if the context of choice is encoded is to directly compare choice with a context in which choice is absent. Neural activity in orbitofrontal cortex and striatum encodes potential value parameters related to reward quality and quantity as well as relative preference. We examined how neural activations in these brain regions are sensitive to choice situations and potentially involved in a prediction for the upcoming outcome selection. Neural activity was recorded and compared between a two-choice spatial delayed response task and an imperative 'one-option' task. Neural activity was obtained that extended from the instruction cue to the movement similar to previous work utilizing the identical imperative task. Orbitofrontal and striatal neural responses depended upon the decision about the choice of which reward to collect. Moreover, signals to predictive instruction cues that precede choice were selective for the choice situation. These neural responses could reflect chosen value with greater information on relative value of individual options as well as encode choice context itself embedded in the task as a part of the post-decision variable.

The effects of ethanol on diverse components of choice in the rat: reward discrimination, preference and relative valuation

Alcohol consumption impairs judgment and choice. How alcohol alters these crucial processes is primarily unknown. Choice can be fractionated into different components including reward discrimination, preference and relative valuation that can function together or in isolation depending upon diverse factors including choice context. We examined the diverse components and contextual effects by analyzing the effects of alcohol drinking on choice behavior in a task with a reduced level of temporal and spatial constraints. Rats were trained to drink 10% ethanol during 6 weeks of behavior testing using a combined sucrose-fade and two-bottle free-choice procedure. Two different sucrose pellet outcomes (e.g., constant vs. variable) were presented each week to examine the impact of voluntary drinking on reward-based decision-making. Behavioral contexts of single option, free choice and extinction were examined for each outcome set. Comparisons were made between alcohol and control groups and within the alcohol group over time to inspect choice profiles. Between-group results showed alcohol drinking animals expressed altered place preference and modified sucrose reward approach latencies. The within-group profile showed that alcohol drinking animals can express adequate reward discrimination, preference and incentive contrast during free choice. All of these components were significantly reduced during the context of extinction. Control animals were also impacted by extinction but not as severely. The findings point to a need for a greater focus on the context and the diverse components of choice when examining external and internal factors influencing decision-making during alcohol or other substance of abuse exposure.

Effects of anandamide administration on components of reward processing during free choice

Previous research has implicated the positive modulation of anandamide, an endocannabinoid neurotransmitter, on feeding behavior. Anandamide is particularly noteworthy as it acts as an endogenous ligand of the CB1 receptor, the same receptor that is activated by tetrahydrocannabinol, the primary psychoactive component in Cannabis sativa. Cannabis legalization in North America has presented with a need to study endocannabinoid agonists and their effects on behavior. Much has yet to be determined in terms of the role of the endocannabinoid system in decision-making scenarios. The research presented here tested the hypothesis that anandamide would augment motivation and reward processing via appetitive and consummatory measures during an operant, foraging task. A three-box design was used in order to provide the animals with a free choice, exploratory foraging environment. Discrimination, preference, and incentive contrast were analyzed as discrete measures of decision-making in the three-box paradigm. Anandamide administration (1mg/kg) was found to significantly increase motivation for the optimal foraging outcome and alter basic processing of reward information involved in discrimination and relative valuation. The positive effects of anandamide on eating behavior and motivation have implications toward possible treatment modalities for patient populations presenting with disorders of motivation. These findings suggest the need for continued investigation of the endocannabinoid system as a central component of motivated behavior.