Post-trial reinforcing hypothalamic stimulation can facilitate avoidance learning

The role of dopamine in the context of aversive stimuli with particular reference to acoustically signaled avoidance learning

Learning from punishment is a powerful means for behavioral adaptation with high relevance for various mechanisms of self-protection. Several studies have explored the contribution of released dopamine (DA) or responses of DA neurons on reward seeking using rewards such as food, water, and sex. Phasic DA signals evoked by rewards or conditioned reward predictors are well documented, as are modulations of these signals by such parameters as reward magnitude, probability, and deviation of actually occurring from expected rewards. Less attention has been paid to DA neuron firing and DA release in response to aversive stimuli, and the prediction and avoidance of punishment. In this review, we first focus on DA changes in response to aversive stimuli as measured by microdialysis and voltammetry followed by the change in electrophysiological signatures by aversive stimuli and fearful events. We subsequently focus on the role of DA and effect of DA manipulations on signaled avoidance learning, which consists of learning the significance of a warning cue through Pavlovian associations and the execution of an instrumental avoidance response. We present a coherent framework utilizing the data on microdialysis, voltammetry, electrophysiological recording, electrical brain stimulation, and behavioral analysis. We end by outlining current gaps in the literature and proposing future directions aimed at incorporating technical and conceptual progress to understand the involvement of reward circuit on punishment based decisions.

Effects of ventral tegmental area stimulation on the acquisition and long-term retention of active avoidance learning

The development of avoidance learning depends on dopamine release in forebrain regions. Previous studies indicated that rewarding brain stimulation facilitated two-way active avoidance learning. However, it is not clear whether the temporal relationship of brain stimulation to the training session (before, during or after) is important. To investigate the role of stimulation condition (no stimulation, self-stimulation only, or self-stimulation plus avoidance stimulation) and sequence of self-stimulation training (before or after avoidance training), we used a 3×2 factorial design, in which every level of stimulation was paired with every level of sequence for a total of 6 different groups. The results suggest that self-stimulation either before or after avoidance learning improved acquisition performance, but acquisition was maximal when stimulation was also given during acquisition trials. Importantly, the sequence of self-stimulation (before or after each acquisition session) was irrelevant to this beneficial effect. However, stimulation had no apparent effect on long-term retention when tested 10 days later under conditions of no stimulation, except that the performance of the group that had previously received avoidance-contingent stimulation deteriorated over the course of 60 trials. This may reflect frustration from the omission of expected reward. These results are relevant for optimizing brain stimulation to improve learning.

Roles for nigrostriatal--not just mesocorticolimbic--dopamine in reward and addiction

Forebrain dopamine circuitry has traditionally been studied by two largely independent specialist groups: students of Parkinson's disease who study the nigrostriatal dopamine system that originates in the substantia nigra (SN), and students of motivation and addiction who study the role of the mesolimbic and mesocortical dopamine systems that originate in the ventral tegmental area (VTA). The anatomical evidence for independent nigrostriatal and mesolimbic dopamine systems has, however, long been obsolete. There is now compelling evidence that both nominal "systems" participate in reward function and addiction. Electrical stimulation of both SN and VTA is rewarding, blockade of glutamatergic or cholinergic input to either SN or VTA attenuates the habit-forming effects of intravenous cocaine, and dopamine in both nigrostriatal and mesocorticolimbic terminal fields participates in the defining property of rewarding events: the reinforcement of memory consolidation. Thus, the similarities between nigrostriatal and mesolimbic dopamine systems can be as important as their differences.

Brain electrical stimulation: kindling and memory aspects

After short introduction in which mile-stones of brain electrical stimulation are mentioned, author discusses kindling as a model of epilepsy and learning. The plastic nature of these two neuronal processes is discussed from the point of view: 1. similarities of these two processes and 2. relation between epileptic discharges (seizures) and learning (memory). The lack of quantitative analysis of afterdischarge duration and its effect on retention is emphasized. Author presents his own study of relation between performance of formed motor avoidance conditioned reflex (CR) and epileptic spontaneous spike activity in hippocampi and duration of afterdischarges (AD) in hippocampi and amygdala in hippocampal kindling in cats. It was shown that spontaneous hippocampal spikes, dissipated over a time, have no effect on performance or retention of the CR. However, there is clear relation between duration of AD and retention of the CR. When duration of the AD was to 15 seconds, correct CRs were obtained in 75% and in 25% the result was negative CR. However, when duration of the AD was longer, the CS presentation between 16-30 sec. resulted in 75% of negative CR and 25% of positive CRs. When the CS was presented during ADs which were longer than 30 sec. almost in all cases the CR was negative, however, in a few instances still possible. Complex effect of brain stimulation are discussed and value of weak stimulation like in kindling technique is emphasized.

The effect of rewarding hypothalamic stimulation on behavioral and neural hippocampal responses during trace eyeblink conditioning in rabbit (Oryctolagus cuniculus)

Rabbits were trace-conditioned with a tone as a conditioned stimulus and an airpuff as an unconditioned stimulus. Electrical stimulation to the medial forebrain bundle in the lateral hypothalamus was delivered either before or after the tone-airpuff pair. The purpose of the present study was to test whether the effect of post-trial hypothalamic stimulation differed from the effect of pre-trial hypothalamic stimulation on trace conditioning in the same subjects. Additionally, hippocampal responses were measured during sessions to see if hypothalamic stimulation activated dopaminergic fibres and affected hippocampal cell functioning and thus learning. The results showed that behavioral nictitating membrane conditioned responses were acquired quickly and hippocampal multiple unit activity increased with post-trial hypothalamic stimulation in comparison to almost non-existent conditioned responses and activity changes in the pre-trial hypothalamic stimulation sessions. The results suggest that hypothalamic stimulation affects trace conditioning differently depending on its time of delivery during conditioning.

The H1- and H2-histamine blockers chlorpheniramine and ranitidine applied to the nucleus basalis magnocellularis region modulate anxiety and reinforcement related processes

This study examined the effects of the H1-antagonist chlorpheniramine and the H2-antagonist ranitidine on reinforcement and anxiety-parameters following unilateral injection into the vicinity of the nucleus basalis magnocellularis (NBM). In Experiment 1, rats with chronically implanted cannulae were injected with chlorpheniramine or ranitidine (each at doses of 0.1, 1, 10 and 20 microg) and were placed into one of four restricted quadrants of a circular open field (closed corral) for a single conditioning trial. During the test for conditioned corral preference, when provided a choice between the four quadrants, only those rats injected with 10 or 20 microg chlorpheniramine spent more time in the treatment corral, indicative of a positively reinforcing action. None of the other doses of chlorpheniramine or of the H2-antagonist influenced rats' preference behavior. In Experiment 2, the elevated plus-maze (EPM) was used to gauge possible anxiolytic or anxiogenic effects of intra-basalis injection of chlorpheniramine or ranitidine (each at doses of 0.1, 1, 10 and 20 microg). A single injection of chlorpheniramine at 0.1 or 20 microg as well as ranitidine at 20 microg was found to exert anxiolytic-like effects in the EPM. Both compounds elevated the time spent on the open arms and increased scanning over the edge of an open arm. None of the other doses of the H1- and H2-antagonist influenced rats' behavior in the EPM. In sum, these findings show that H1- and H2-receptor antagonists differentially modulate reinforcement and fear-related processes in the NBM and thus, provide the first evidence for a behavioral relevance for the histaminergic innervation of this brain site.

Effects of rewarding electrical stimulation of lateral hypothalamus on classical conditioning of the nictitating membrane response

1. Adult New Zealand albino rabbits were prepared with chronic hypothalamic stimulating electrodes and hippocampal recording electrodes. 2. Rabbits were restrained and classically conditioned by a tone CS and an airpuff US either followed or preceded by a hypothalamic stimulation (HS). Control rabbits were conditioned without the HS. 3. It was found that HS following the CS facilitated both behavioral and hippocampal responses, while HS preceding the CS inhibited them. 4. Enhanced hippocampal learning-related unit firing to the CS may represent an early indication of conditioning before the behavioral activity produces any observable change.

Behavioral habituation to spatial novelty: interference and noninterference studies

Long-term behavioral habituation (LTH), that is activity decrement upon repeated exposures to spatial novelty, is a relatively simple and ubiquitous form of behavioral plasticity in the animal kingdom, that can be used as a model of nonassociative learning in the freely behaving organism. Several strategies can be followed to tackle upon it. (a) Interference studies pertain to manipulation of the between-exposure interval by a variety of agents of different nature, that are known to interfere with hypothesized "consolidation process(es)" in associative learning paradigms. This approach indicates that LTH is modulated by NMDA receptors, requires polysome aggregation and protein synthesis, a functioning neocortex and both slow wave and paradoxical sleep. Further, it is modulated by endogenously released or exogenously given vasopressin and is not affected by blockade of endogenous opioids, at least through the "mu" receptor type. Moreover, LTH is disrupted by bilateral, electrolytic lesion of the locus coeruleus, but it is only impaired by 6-OH-DA bilateral lesion of the dorsal noradrenergic bundle, and it is facilitated by electrolytic lesion of the medial septal nuclei. (b) Noninterference correlative studies: Individual differences in behavioral variables can be correlated to some components of the architecture of the hippocampus to reveal structure-function relationships. (c) Noninterference maturation studies pertain to the study of the maturation of LTH during postnatal development in a scaled-down Làt-maze in normally reared rats and in rats with deranged rate of body and brain growth by litter size technique, differential stimulation or by perinatal propylthiouracil-induced hypothyroidism. (d) Noninterference development studies pertain to the formation of LTH varying the between-exposure interval. It was studied in albino rats of a Sprague-Dawley, random-bred stock (NRB) and of the Naples High (NHE) and Low-Excitability (NLE) lines. The study was carried out during the light or the dark phase of a 12:12LD cycle, by retesting at different inter-exposure intervals. Multivariate analysis of variance showed significant effects of strain, inter-exposure interval and of postexposure sleep or wakefulness. Furthermore, analysis of the temporal pattern showed the formation of LTH to follow a non linear complex function. Further, behavioral habituation consists of emotional and cognitive components that can be separated across different approaches. In conclusion, long-term habituation to a novel environmental is a useful model to study experience-induced nonassociative behavioral modifications.

In search of the mechanism of action of the nootropics: new insights and potential clinical implications

The positive action of nootropics on the memory has up to now primarily been discussed in the context of effects on energy metabolism and cholinergic or glutaminergic neurotransmission. Recent findings have shown that the memory-enhancing effect is steroid-sensitive. Since corticosteroids are potent modulators of gene transcription, it appears possible that the nootropics may exert a modulatory action on protein synthesis. This assumption is supported on the one hand by the fact that the nootropics improve the memory even if they are administered several hours after the learning trial, and on the other hand by the observation that their memory-enhancing effect does not become detectable until 16-24 hours after the treatment and learning trial. Provided the memory-enhancing effect in animal experiments and the therapeutic effect in patients come about by way of the same mode of action, the fact that high levels of corticosteroids suppress the effects of the nootropics could also have clinical implications: in the light of the observation that the majority of Alzheimer patients have elevated steroid levels it could explain why there is always only a small proportion of patients in clinical trials that respond to treatment with nootropics.

The relationship between reinforcement and memory: parallels in the rewarding and mnemonic effects of the neuropeptide substance P

A theory of reinforcement is presented which accounts for the backward action of a reinforcer on operant behavior in terms of its effect on memory traces left by the operant. Several possible ways in which a reinforcer could strengthen the probability of recurrence of an operant are discussed. Predictions from the model regarding general memory-promoting effects of reinforcers presented posttrial in various learning paradigms are outlined. The theory also predicts a parallelism in reinforcing and memory-promoting effects of stimuli, including drugs. The second part of the chapter outlines experiments investigating memory modulating and reinforcing effects of the neuropeptide substance P. In general, injection of SP is positively reinforcing when injected into parts of the brain where it has been shown to facilitate learning. Peripheral injection of SP is also reinforcing at the dose known to promote passive avoidance learning when presented posttrial.

Facilitation or inhibition of memory by morphine: a question of experimental parameters

The effects of morphine on memory are highly controversial. According to some investigators post-trial injections of morphine facilitate memory. Others, however, have reported impairment of memory after morphine injections. To investigate the extent to which this may be due to different experimental parameters, foot-shock intensity and dosage of morphine were systematically varied in a passive-avoidance task. It was found that post-trial administration of medium and relatively high doses of morphine facilitate retention performance following moderate levels of foot-shock. Under other conditions of dose and shock intensity, the drug was not effective or even impaired retention.

Facilitation of memory processing by posttrial morphine: possible involvement of reinforcement mechanisms?

Posttrial administration of 40 mg/kg and 100 mg/kg, but not of 1 mg/kg, of morphine hydrochloride facilitates learning of a one-trial passive avoidance task in drug-naive mice. The effect does not depend on the punishing properties of the morphine injection, since in injection of LiCl (a strong punisher) fails to enhance learning in a similar way. After the establishment of tolerance by several morphone administrations, the 100 mg/kg, but not the 40 mg/kg, dose level resulted in memory facilitation. The data are discussed in connection with the hypothesis that morphine acts directly on reinforcement mechanisms by activating the opiate receptor.

Enhanced passive avoidance learning and appetitive T-maze learning with post-trial rewarding hypothalamic stimulation

Experiments were carried out to investigate the effects of post-trial reinforcing stimulation of the lateral hypothalamus on learning in rats. The reinforcing stimulation was always presented for a duration of 20--30 sec (0.2 sec on/0.8 sec off), and was administered either immediately, 30 sec delayed or 300 sec delayed after exposure to the learning situation. In experiment 1 post-trial stimulation led to improved passive avoidance learning of an alcove-avoidance task when presented 30 sec compared to immediately after the footshock. In Experiment 2 reversal learning of a one-way shuttle-box avoidance task was facilitated by 30 sec delayed, but not 300 sec delayed post-trial reinforcing stimulation. In Experiment 3 appetitive left-right discrimination was investigated using a T-maze task. Thirty sec delayed post-trial reinforcing stimulation presented contingent on errors facilitated learning of this task. Together, the 3 studies provide further support for the hypothesis that reinforcers directly influence labile memory processes (such as short-term memory) and thereby improve learning.