Appetitively motivated tasks in the IntelliCage reveal a higher motivational cost of spatial learning in male than female mice

The IntelliCage (IC) permits the assessment of the behavior and learning abilities of mice in a social home cage context. To overcome water deprivation as an aversive driver of learning, we developed protocols in which spatial learning is motivated appetitively by the preference of mice for sweetened over plain water. While plain water is available at all times, only correct task responses give access to sweetened water rewards. Under these conditions, C57BL/6J mice successfully mastered a corner preference task with the reversal and also learned a more difficult time-place task with reversal. However, the rate of responding to sweetened water decreased strongly with increasing task difficulty, indicating that learning challenges and reduced success in obtaining rewards decreased the motivation of the animals to seek sweetened water. While C57BL/6J mice of both sexes showed similar initial taste preferences and learned similarly well in simple learning tasks, the rate of responding to sweetened water and performance dropped more rapidly in male than in female mice in response to increasing learning challenges. Taken together, our data indicate that male mice can have a disadvantage relative to females in mastering difficult, appetitively motivated learning tasks, likely due to sex differences in value-based decision-making.

Refinement of IntelliCage protocols for complex cognitive tasks through replacement of drinking restrictions by incentive-disincentive paradigms

The IntelliCage allows automated testing of cognitive abilities of mice in a social home cage environment without handling by human experimenters. Restricted water access in combination with protocols in which only correct responses give access to water is a reliable learning motivator for hippocampus-dependent tasks assessing spatial memory and executive function. However, water restriction may negatively impact on animal welfare, especially in poor learners. To better comply with the 3R principles, we previously tested protocols in which water was freely available but additional access to sweetened water could be obtained by learning a task rule. While this purely appetitive motivation worked for simple tasks, too many mice lost interest in the sweet reward during more difficult hippocampus-dependent tasks. In the present study, we tested a battery of increasingly difficult spatial tasks in which water was still available without learning the task rule, but rendered less attractive either by adding bitter tasting quinine or by increasing the amount of work to obtain it. As in previous protocols, learning of the task rule provided access to water sweetened with saccharin. The two approaches of dual motivation were tested in two cohorts of female C57BL/6 N mice. Compared to purely appetitive motivation, both novel protocols strongly improved task engagement and increased task performance. Importantly, neither of the added disincentives had an adverse impact on liquid consumption, health status or body weight of the animals. Our results show that it is possible to refine test protocols in the IntelliCage so that they challenge cognitive functions without restricting access to water.

The basolateral amygdala-medial prefrontal cortex circuitry regulates behavioral flexibility during appetitive reversal learning

Environmental cues can become predictors of food availability through Pavlovian conditioning. Two forebrain regions important in this associative learning are the basolateral amygdala (BLA) and medial prefrontal cortex (mPFC). Recent work showed the BLA-mPFC pathway is activated when a cue reliably signals food, suggesting the BLA informs the mPFC of the cue's value. The current study tested this hypothesis by altering the value of 2 food cues using reversal learning and illness-induced devaluation paradigms. Rats that received unilateral excitotoxic lesions of the BLA and mPFC contralaterally placed, along with ipsilateral and sham controls, underwent discriminative conditioning, followed by reversal learning and then devaluation. All groups successfully discriminated between 2 auditory stimuli that were followed by food delivery (conditional stimulus [CS] +) or not rewarded (CS-), demonstrating this learning does not require BLA-mPFC communication. When the outcomes of the stimuli were reversed, the rats with disconnected BLA-mPFC (contralateral condition) showed increased responding to the CSs, especially to the rCS + (original CS-) during the first session, suggesting impaired cue memory recall and behavioral inhibition compared to the other groups. For devaluation, all groups successfully learned conditioned taste aversion; however, there was no evidence of cue devaluation or differences between groups. Interestingly, at the end of testing, the nondevalued contralateral group was still responding more to the original CS + (rCS-) compared to the devalued contralateral group. These results suggest a potential role for BLA-mPFC communication in guiding appropriate responding during periods of behavioral flexibility when the outcomes, and thus the values, of learned cues are altered. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Functional Compartmentalization of the Contribution of Hippocampal Subfields to Context-Dependent Extinction Learning

During extinction learning (EL), an individual learns that a previously learned behavior no longer fulfills its original purpose, or is no longer relevant. Recent studies have contradicted earlier theories that EL comprises forgetting, or the inhibition of the previously learned behavior, and indicate that EL comprises new associative learning. This suggests that the hippocampus is involved in this process. Empirical evidence is lacking however. Here, we used fluorescence in situ hybridization of somatic immediate early gene (IEG) expression to scrutinize if the hippocampus processes EL. Rodents engaged in context-dependent EL and were also tested for renewal of (the original behavioral response to) a spatial appetitive task in a T-maze. Whereas distal and proximal CA1 subfields processed both EL and renewal, effects in the proximal CA1 were more robust consistent with a role of this subfield in processing context. The lower blade of the dentate gyrus (DG) and the proximal CA3 subfields were particularly involved in renewal. Responses in the distal and proximal CA3 subfields suggest that this hippocampal subregion may also contribute to the evaluation of the reward outcome. Taken together, our findings provide novel and direct evidence for the involvement of distinct hippocampal subfields in context-dependent EL and renewal.

The Post-mating Switch in the Pheromone Response of Nasonia Females Is Mediated by Dopamine and Can Be Reversed by Appetitive Learning

The olfactory sense is of crucial importance for animals, but their response to chemical stimuli is plastic and depends on their physiological state and prior experience. In many insect species, mating status influences the response to sex pheromones, but the underlying neuromodulatory mechanisms are poorly understood. After mating, females of the parasitic wasp Nasonia vitripennis are no longer attracted to the male sex pheromone. Here we show that this post-mating behavioral switch is mediated by dopamine (DA). Females fed a DA-receptor antagonist prior to mating maintained their attraction to the male pheromone after mating while virgin females injected with DA became unresponsive. However, the switch is reversible as mated females regained their pheromone preference after appetitive learning. Feeding mated N. vitripennis females with antagonists of either octopamine- (OA) or DA-receptors prevented relearning of the pheromone preference suggesting that both receptors are involved in appetitive learning. Moreover, DA injection into mated females was sufficient to mimic the oviposition reward during odor conditioning with the male pheromone. Our data indicate that DA plays a key role in the plastic pheromone response of N. vitripennis females and reveal some striking parallels between insects and mammals in the neuromodulatory mechanisms underlying olfactory plasticity.

Roles of Octopamine and Dopamine Neurons for Mediating Appetitive and Aversive Signals in Pavlovian Conditioning in Crickets

Revealing neural systems that mediate appetite and aversive signals in associative learning is critical for understanding the brain mechanisms controlling adaptive behavior in animals. In mammals, it has been shown that some classes of dopamine neurons in the midbrain mediate prediction error signals that govern the learning process, whereas other classes of dopamine neurons control execution of learned actions. In this review, based on the results of our studies on Pavlovian conditioning in the cricket Gryllus bimaculatus and by referring to the findings in honey bees and fruit-flies, we argue that comparable aminergic systems exist in the insect brain. We found that administrations of octopamine (the invertebrate counterpart of noradrenaline) and dopamine receptor antagonists impair conditioning to associate an olfactory or visual conditioned stimulus (CS) with water or sodium chloride solution (appetitive or aversive unconditioned stimulus, US), respectively, suggesting that specific octopamine and dopamine neurons mediate appetitive and aversive signals, respectively, in conditioning in crickets. These findings differ from findings in fruit-flies. In fruit-flies, appetitive and aversive signals are mediated by different dopamine neuron subsets, suggesting diversity in neurotransmitters mediating appetitive signals in insects. We also found evidences of “blocking” and “auto-blocking” phenomena, which suggested that the prediction error, the discrepancy between actual US and predicted US, governs the conditioning in crickets and that octopamine neurons mediate prediction error signals for appetitive US. Our studies also showed that activations of octopamine and dopamine neurons are needed for the execution of an appetitive conditioned response (CR) and an aversive CR, respectively, and we, thus, proposed that these neurons mediate US prediction signals that drive appetitive and aversive CRs. Our findings suggest that the basic principles of functioning of aminergic systems in associative learning, i.e., to transmit prediction error signals for conditioning and to convey US prediction signals for execution of CR, are conserved among insects and mammals, on account of the fact that the organization of the insect brain is much simpler than that of the mammalian brain. Further investigation of aminergic systems that govern associative learning in insects should lead to a better understanding of commonalities and diversities of computational rules underlying associative learning in animals.

Changes in Appetitive Associative Strength Modulates Nucleus Accumbens, But Not Orbitofrontal Cortex Neuronal Ensemble Excitability

Cues that predict the availability of food rewards influence motivational states and elicit food-seeking behaviors. If a cue no longer predicts food availability, then animals may adapt accordingly by inhibiting food-seeking responses. Sparsely activated sets of neurons, coined "neuronal ensembles," have been shown to encode the strength of reward-cue associations. Although alterations in intrinsic excitability have been shown to underlie many learning and memory processes, little is known about these properties specifically on cue-activated neuronal ensembles. We examined the activation patterns of cue-activated orbitofrontal cortex (OFC) and nucleus accumbens (NAc) shell ensembles using wild-type and Fos-GFP mice, which express green fluorescent protein (GFP) in activated neurons, after appetitive conditioning with sucrose and extinction learning. We also investigated the neuronal excitability of recently activated, GFP+ neurons in these brain areas using whole-cell electrophysiology in brain slices. Exposure to a sucrose cue elicited activation of neurons in both the NAc shell and OFC. In the NAc shell, but not the OFC, these activated GFP+ neurons were more excitable than surrounding GFP- neurons. After extinction, the number of neurons activated in both areas was reduced and activated ensembles in neither area exhibited altered excitability. These data suggest that learning-induced alterations in the intrinsic excitability of neuronal ensembles is regulated dynamically across different brain areas. Furthermore, we show that changes in associative strength modulate the excitability profile of activated ensembles in the NAc shell.SIGNIFICANCE STATEMENT Sparsely distributed sets of neurons called "neuronal ensembles" encode learned associations about food and cues predictive of its availability. Widespread changes in neuronal excitability have been observed in limbic brain areas after associative learning, but little is known about the excitability changes that occur specifically on neuronal ensembles that encode appetitive associations. Here, we reveal that sucrose cue exposure recruited a more excitable ensemble in the nucleus accumbens, but not orbitofrontal cortex, compared with their surrounding neurons. This excitability difference was not observed when the cue's salience was diminished after extinction learning. These novel data provide evidence that the intrinsic excitability of appetitive memory-encoding ensembles is regulated differentially across brain areas and adapts dynamically to changes in associative strength.

Distinct Contributions of Ventromedial and Dorsolateral Subregions of the Human Substantia Nigra to Appetitive and Aversive Learning

The role of neurons in the substantia nigra (SN) and ventral tegmental area (VTA) of the midbrain in contributing to the elicitation of reward prediction errors during appetitive learning has been well established. Less is known about the differential contribution of these midbrain regions to appetitive versus aversive learning, especially in humans. Here we scanned human participants with high-resolution fMRI focused on the SN and VTA while they participated in a sequential Pavlovian conditioning paradigm involving an appetitive outcome (a pleasant juice), as well as an aversive outcome (an unpleasant bitter and salty flavor). We found a degree of regional specialization within the SN: Whereas a region of ventromedial SN correlated with a temporal difference reward prediction error during appetitive Pavlovian learning, a dorsolateral area correlated instead with an aversive expected value signal in response to the most distal cue, and to a reward prediction error in response to the most proximal cue to the aversive outcome. Furthermore, participants' affective reactions to both the appetitive and aversive conditioned stimuli more than 1 year after the fMRI experiment was conducted correlated with activation in the ventromedial and dorsolateral SN obtained during the experiment, respectively. These findings suggest that, whereas the human ventromedial SN contributes to long-term learning about rewards, the dorsolateral SN may be particularly important for long-term learning in aversive contexts.

SIGNIFICANCE STATEMENT

The role of the substantia nigra (SN) and ventral tegmental area (VTA) in appetitive learning is well established, but less is known about their contribution to aversive compared with appetitive learning, especially in humans. We used high-resolution fMRI to measure activity in the SN and VTA while participants underwent higher-order Pavlovian learning. We found a regional specialization within the SN: a ventromedial area was selectively engaged during appetitive learning, and a dorsolateral area during aversive learning. Activity in these areas predicted affective reactions to appetitive and aversive conditioned stimuli over 1 year later. These findings suggest that, whereas the human ventromedial SN contributes to long-term learning about rewards, the dorsolateral SN may be particularly important for long-term learning in aversive contexts.

Updating appetitive memory during reconsolidation window: critical role of cue-directed behavior and amygdala central nucleus

When presented with a light cue followed by food, some rats simply approach the foodcup (Nonorienters), while others first orient to the light in addition to displaying the food-cup approach behavior (Orienters). Cue-directed orienting may reflect enhanced attentional and/or emotional processing of the cue, suggesting divergent natures of cue-information processing in Orienters and Nonorienters. The current studies investigate how differences in cue processing might manifest in appetitive memory retrieval and updating using a paradigm developed to persistently attenuate fear responses (Retrieval-extinction paradigm; Monfils et al., 2009). First, we examined whether the retrieval-extinction paradigm could attenuate appetitive responses in Orienters and Nonorienters. Next, we investigated if the appetitive memory could be updated using reversal learning (fear conditioning) during the reconsolidation window (as opposed to repeated unreinforced trials, i.e., extinction). Both extinction and new fear learning given within the reconsolidation window were effective at persistently updating the initial appetitive memory in the Orienters, but not the Nonorienters. Since conditioned orienting is mediated by the amygdala central nucleus (CeA), our final experiment examined the CeA's role in the retrieval-extinction process. Bilateral CeA lesions interfered with the retrieval-extinction paradigm-did not prevent spontaneous recovery of food-cup approach. Together, our studies demonstrate the critical role of conditioned orienting behavior and the CeA in updating appetitive memory during the reconsolidation window.

Individual differences in the conditioned and unconditioned rat 50-kHz ultrasonic vocalizations elicited by repeated amphetamine exposure

RATIONALE

Adult rats often produce 50-kHz ultrasonic vocalizations (USVs), particularly the frequency-modulated varieties, in appetitive situations. These calls are thought by some to reflect positive affective states and the reinforcing value of drugs such as amphetamine and cocaine.

OBJECTIVE

The objective of this study was to determine whether the number of unconditioned 50-kHz USVs elicited by amphetamine predicts the development and/or magnitude of drug-conditioned motivation.

METHODS

In three experiments, we recorded USVs before and after injections of 1 mg/kg amphetamine (i.v. or i.p.) administered once per session. Rats were categorized as "high callers" or "low callers" according to individual differences in the number of 50-kHz USVs elicited by their first amphetamine injection. We examined the conditioned appetitive behavior and conditioned place preference (CPP) that emerged in high and low callers after repeated pairings of amphetamine with specific contexts. We also examined whether amphetamine-induced calling was affected by treatment within an unfamiliar (test chamber) versus familiar (home cage) context.

RESULTS

Within an unfamiliar environment, the high callers consistently produced more amphetamine-induced 50-kHz USVs than the low callers. Compared to the low callers, high callers showed significantly greater amphetamine CPP as well as enhanced conditioned 50-kHz USVs and locomotor activity during anticipation of amphetamine. Individual differences were stable when amphetamine was administered in test chambers, but when it was administered in home cages, low callers showed an increase in 50-kHz calling that matched the high callers.

CONCLUSIONS

These findings suggest that individual differences in drug-induced USVs can reveal environment-sensitive traits involved in drug-related appetitive motivation.

CASK regulates CaMKII autophosphorylation in neuronal growth, calcium signaling, and learning

Calcium (Ca²⁺)/calmodulin (CaM)-dependent kinase II (CaMKII) activity plays a fundamental role in learning and memory. A key feature of CaMKII in memory formation is its ability to be regulated by autophosphorylation, which switches its activity on and off during synaptic plasticity. The synaptic scaffolding protein CASK (calcium (Ca²⁺)/calmodulin (CaM) associated serine kinase) is also important for learning and memory, as mutations in CASK result in intellectual disability and neurological defects in humans. We show that in Drosophila larvae, CASK interacts with CaMKII to control neuronal growth and calcium signaling. Furthermore, deletion of the CaMK-like and L27 domains of CASK (CASK β null) or expression of overactive CaMKII (T287D) produced similar effects on synaptic growth and Ca²⁺ signaling. CASK overexpression rescues the effects of CaMKII overactivity, consistent with the notion that CASK and CaMKII act in a common pathway that controls these neuronal processes. The reduction in Ca²⁺ signaling observed in the CASK β null mutant caused a decrease in vesicle trafficking at synapses. In addition, the decrease in Ca²⁺ signaling in CASK mutants was associated with an increase in Ether-à-go-go (EAG) potassium (K⁺) channel localization to synapses. Reducing EAG restored the decrease in Ca²⁺ signaling observed in CASK mutants to the level of wildtype, suggesting that CASK regulates Ca²⁺ signaling via EAG. CASK knockdown reduced both appetitive associative learning and odor evoked Ca²⁺ responses in Drosophila mushroom bodies, which are the learning centers of Drosophila. Expression of human CASK in Drosophila rescued the effect of CASK deletion on the activity state of CaMKII, suggesting that human CASK may also regulate CaMKII autophosphorylation.

Chemical structure of odorants and perceptual similarity in ants

Animals are often immersed in a chemical world consisting of mixtures of many compounds rather than of single substances, and they constantly face the challenge of extracting relevant information out of the chemical landscape. To this purpose, the ability to discriminate among different stimuli with different valence is essential, but it is also important to be able to generalise, i.e. to treat different but similar stimuli as equivalent, as natural variation does not necessarily affect stimulus valence. Animals can thus extract regularities in their environment and make predictions, for instance about distribution of food resources. We studied perceptual similarity of different plant odours by conditioning individual carpenter ants to one odour, and subsequently testing their response to another, structurally different odour. We found that asymmetry in generalisation, where ants generalise from odour A to B, but not from B to A, is dependent on both chain length and functional group. By conditioning ants to a binary mixture, and testing their reaction to the individual components of the mixture, we show that overshadowing, where parts of a mixture are learned better than others, is rare. Additionally, generalisation is dependent not only on the structural similarity of odorants, but also on their functional value, which might play a crucial role. Our results provide insight into how ants make sense of the complex chemical world around them, for example in a foraging context, and provide a basis with which to investigate the neural mechanisms behind perceptual similarity.

Saccharin pre-exposure enhances appetitive flavor learning in pre-weanling rats

In adult rats, data suggest that consumption of sweet tastes that do not deliver anticipated caloric consequences using high-intensity, non-caloric sweeteners, such as saccharin, interferes with learned relations that may contribute to energy balance. The goal of the present study was to assess the development of learning about sweet taste and calories by assessing whether pre-exposure to saccharin solutions reduces cue competition in pre-weanling rats. The results demonstrated that rats pre-exposed to saccharin and then trained with a novel grape flavor paired with a glucose-sweetened solution consumed more of the novel grape flavor presented alone than rats that had been pre-exposed to saccharin and given the grape flavor paired with water alone. No differences in intake of the novel grape flavor were observed in groups given pre-exposure to water or glucose solutions. Thus, by 15 days of age, rats appear to have established an association between sweet tastes and calories, and this association can be weakened by exposure to saccharin.

A cocaine cue is more preferred and evokes more frequency-modulated 50-kHz ultrasonic vocalizations in rats prone to attribute incentive salience to a food cue

RATIONALE

Individuals vary considerably in the extent to which they attribute incentive salience to food-associated cues.

OBJECTIVES

We asked whether individuals prone to attribute incentive salience to a food cue are also prone to attribute incentive properties to a stimulus associated with a drug of abuse-cocaine.

METHODS

We first identified those rats that attributed incentive salience to a food cue by quantifying the extent to which they came to approach and engage a food cue. We then used a conditioned place preference procedure to pair an injection of 10 mg/kg cocaine (i.p.) with one distinct floor texture (grid or holes) and saline with another. Following 8 days of conditioning, each rat was given a saline injection and placed into a chamber that had both floors present. We measured the time spent on each floor, and also 50-kHz ultrasonic vocalizations, which have been associated with positive affective states.

RESULTS

Rats that vigorously engaged the food cue ("sign trackers") expressed a preference for the cocaine-paired floor compared to those that did not ("goal trackers"). In addition, sign trackers made substantially more frequency-modulated 50-kHz vocalizations when injected with cocaine and when later exposed to the cocaine cue.

CONCLUSIONS

Rats prone to attribute incentive salience to a food cue are also prone to attribute incentive motivational properties to a tactile cue associated with cocaine. We suggest that individuals prone to attribute incentive salience to reward cues will have difficulty resisting them and, therefore, may be especially vulnerable to develop impulse control disorders, including addiction.

Behavioral and neurophysiological study of olfactory perception and learning in honeybees

The honeybee Apis mellifera has been a central insect model in the study of olfactory perception and learning for more than a century, starting with pioneer work by Karl von Frisch. Research on olfaction in honeybees has greatly benefited from the advent of a range of behavioral and neurophysiological paradigms in the Lab. Here I review major findings about how the honeybee brain detects, processes, and learns odors, based on behavioral, neuroanatomical, and neurophysiological approaches. I first address the behavioral study of olfactory learning, from experiments on free-flying workers visiting artificial flowers to laboratory-based conditioning protocols on restrained individuals. I explain how the study of olfactory learning has allowed understanding the discrimination and generalization ability of the honeybee olfactory system, its capacity to grant special properties to olfactory mixtures as well as to retain individual component information. Next, based on the impressive amount of anatomical and immunochemical studies of the bee brain, I detail our knowledge of olfactory pathways. I then show how functional recordings of odor-evoked activity in the brain allow following the transformation of the olfactory message from the periphery until higher-order central structures. Data from extra- and intracellular electrophysiological approaches as well as from the most recent optical imaging developments are described. Lastly, I discuss results addressing how odor representation changes as a result of experience. This impressive ensemble of behavioral, neuroanatomical, and neurophysiological data available in the bee make it an attractive model for future research aiming to understand olfactory perception and learning in an integrative fashion.