Appetitive Pavlovian goal-tracking memories reconsolidate only under specific conditions

Drug memory reconsolidation: from molecular mechanisms to the clinical context

Since its rediscovery at the beginning of the 21st Century, memory reconsolidation has been proposed to be a therapeutic target for reducing the impact of emotional memories that can go awry in mental health disorders such as drug addiction (substance use disorder, SUD). Addiction can be conceptualised as a disorder of learning and memory, in which both pavlovian and instrumental learning systems become hijacked into supporting drug-seeking and drug-taking behaviours. The past two decades of research have characterised the details of the molecular pathways supporting the reconsolidation of pavlovian cue-drug memories, with more recent work indicating that the reconsolidation of instrumental drug-seeking memories also relies upon similar mechanisms. This narrative review considers what is known about the mechanisms underlying the reconsolidation of pavlovian and instrumental memories associated with drug use, how these approaches have translated to experimental medicine studies, and the challenges and opportunities for the clinical use of reconsolidation-based therapies.

Making Leaps and Hitting Boundaries in Reconsolidation: Overcoming Boundary Conditions to Increase Clinical Translatability of Reconsolidation-based Therapies

Reconsolidation results in the restabilisation, and thus persistence, of a memory made labile by retrieval, and interfering with this process is thought to enable modification or weakening of the original trace. As such, reconsolidation-blockade has been a focus of research aiming to target the maladaptive memories underlying mental health disorders, including post-traumatic stress disorder and drug addiction. Current first-line therapies are not effective for all patients, and a substantial proportion of those for whom therapies are effective later relapse. A reconsolidation-based intervention would be invaluable as an alternative treatment for these conditions. However, the translation of reconsolidation-based therapies to the clinic presents a number of challenges, with arguably the greatest being the overcoming of the boundary conditions governing the opening of the reconsolidation window. These include factors such as the age and strength of memory, and can broadly be divided into two categories: intrinsic features of the targeted memory itself, and parameters of the reactivation procedure used. With maladaptive memory characteristics inevitably varying amongst individuals, manipulation of the other limitations imposed by procedural variables have been explored to circumvent the boundary conditions on reconsolidation. Although several apparently discrepant results remain to be reconciled and these limitations yet to be truly defined, many studies have produced successful results which encouragingly demonstrate that boundary conditions may be overcome using various proposed strategies to enable translation of a reconsolidation-based intervention to clinical use.

Procedures between training and reactivation influence the destabilization of instrumental sucrose memory

Memory destabilization and reconsolidation is hypothesized to be a fundamental mnemonic process that can underpin memory updating. Instrumental memories have been shown recently to be destabilized following a reactivation session that involves a change in instrumental reward contingency. However, the acquisition and performance of an instrumental response occurs in the presence of the learning of other reward-related memories. This may influence the ability of a given reactivation session to destabilize the previously learned instrumental memory. Here we present a series of experiments in male rats involving an instrumental memory trained on an FR1 schedule over 10 days, and then reactivated in a session that imposed a VR5 schedule of reinforcement. When MK-801 was injected prior to the VR5 reactivation session, it reliably impaired subsequent instrumental performance at test only when the reactivation session occurred 48 h, and not 24 h, after the end of training. The interposition between the end of training and the reactivation session of a context extinction session, an additional VR5 reactivation session, or indeed the simple experience of being handled and injected with vehicle, resulted in MK-801 no longer having an amnestic effect on test performance. While we do not have a clear account for the process and mechanism underpinning this apparent selectivity of the effect of the VR5 session to destabilize the instrumental memory, it does additionally highlight the need for greater understanding of the conditions that facilitate reactivation-induced memory destabilization.

Role of prediction error and the cholinergic system on memory reconsolidation processes in mice

The ability to make predictions based on stored information is a general coding strategy. A prediction error (PE) is a mismatch between expected and current events. Our memories, like ourselves, are subject to change. Thus, an acquired memory can become active and update its content or strength by a labilization-reconsolidation process. Within the reconsolidation framework, PE drives the updating of consolidated memories. In the past our lab has made key progresses showing that a blockade in the central cholinergic system during reconsolidation can cause memory impairment, while reinforcement of cholinergic activity enhances it. In the present work we determined that PE is a necessary condition for memory to reconsolidate in an inhibitory avoidance task using both male and female mice. Depending on the intensity of the unconditioned stimulus (US) used during training, a negative (higher US intensity) or positive (lower US intensity/no US) PE on a retrieval session modified the behavioral response on a subsequent testing session. Furthermore, we demonstrated that the cholinergic system modulates memory reconsolidation only when PE is detected. In this scenario administration of oxotremorine, scopolamine or nicotine after memory reactivation either enhanced or impaired memory reconsolidation in a sex-specific manner.

Memory reconsolidation impairments in sign-tracking to an audiovisual compound stimulus

Studies of memory reconsolidation of pavlovian memories have typically employed unimodal conditioned stimuli, despite the use of multimodal compound stimuli in other settings. Here we studied sign-tracking behaviour to a compound audiovisual stimulus. First, we observed not unexpectedly that sign-tracking was poorer to the audiovisual compound than to unimodal visual stimuli. Then, we showed that, depending on the parameters of compound stimulus re-exposure at memory reactivation, systemic MK-801 treatment either impaired extinction to improve sign-tracking at test, or disrupted reconsolidation to impair test behaviour. When memory reactivation consisted of re-exposure to only the auditory component of the compound stimulus, we observed sign-tracking impairments following MK-801 treatment, but only under certain test conditions. This was in contrast to the consistent impairment following reactivation with the full audiovisual compound. Moreover, the parameters of auditory stimulus presentation to enable MK-801-induced impairment at test varied depending on whether the stimulus was presented within or outside the training context. These findings suggest that behaviour under the control of appetitive pavlovian compound stimuli can be modulated by targeting both extinction and reconsolidation, and that it is not necessary to re-expose to the full compound stimulus in order to achieve a degree of modulation of behaviour.

On the Resistance to Relapse to Cocaine-Seeking Following Impairment of Instrumental Cocaine Memory Reconsolidation

Reconsolidation normally functions to update and maintain memories in the long-term. However, this process can be disrupted pharmacologically to weaken memories. Exploiting such experimental amnesia to disrupt the maladaptive reward memories underpinning addiction may provide a novel therapeutic avenue to prevent relapse. Here, we tested whether targeted disruption of the reconsolidation of instrumental (operant) lever pressing for cocaine resulted in protection against different forms of relapse in a rat self-administration model. We first confirmed that systemic injection of the non-competitive N-methyl–D-aspartate receptor (NMDAR) antagonist MK-801 did impair reconsolidation to reduce spontaneous instrumental drug-seeking memory at test. This deficit was not rescued by pharmacological induction of stress with the anxiogenic α₂-noradrenergic receptor antagonist yohimbine. In contrast, cocaine-seeking was restored to control levels following priming with cocaine itself, or presentation of a cocaine-associated cue. These results suggest that while stress-induced relapse can be reduced by disruption of instrumental memory reconsolidation, the apparent sparing of the pavlovian cue-drug memory permitted other routes to relapse. Therefore, future reconsolidation-based therapeutic strategies for addictive drug-seeking may need to target both instrumental and pavlovian memories.

Acute but Not Permanent Effects of Propranolol on Fear Memory Expression in Humans

Experimental evidence in humans and non-human animals suggests that the administration of propranolol shortly after the retrieval of an emotional memory can lead to an attenuation of its later expression, a phenomenon known as post-reactivation amnesia. Using more potent amnestic drugs, post-reactivation amnesia has been shown in animals to be reversible by re-administration of the drug prior to memory retention testing. The latter finding suggests that, at least under some circumstances, post-reactivation amnesia may not reflect a disruption of reconsolidation (i.e., a memory storage deficit) but an acquired state-dependency of memory expression (i.e., a memory retrieval deficit that is relieved when the drug state is recreated during testing). We conducted a double-blind, placebo-controlled study to investigate whether the previously established amnestic effects of post-reactivation propranolol administration on memory retention in humans may similarly reflect a retrieval deficit. In four groups of participants, fear memories were first established through differential fear conditioning. One day later, a single presentation of the CS+ without shock was used to reactivate the memory in three of the four groups, followed by the administration of 40 mg Propranolol HCl (Groups PrPl and PrPr) or placebo (Group PlPl). Memory was not reactivated in the fourth group (Group NR). Another 24 h later, Propranolol HCl (Group PrPr) or placebo (Groups PrPl, PlPl, and NR) was again administered, followed by a test of memory retention (extinction testing) and recovery (reinstatement testing). We did not observe any effects of post-reactivation propranolol on memory retention; conditioned responding was similar for all groups at the start of retention testing and similarly sensitive to recovery through reinstatement. We did observe an acute effect of propranolol administration on fear-potentiated startle responding during retention testing in Group PrPr, where participants exhibited attenuated startle responses during extinction testing but similar sensitivity to reinstatement as participants in the other groups. While our findings fail to corroborate previous reports of propranolol-induced post-reactivation amnesia in humans, they do point to acute effects of propranolol administration on extinction performance.

Reconsolidation blockade for the treatment of addiction: challenges, new targets, and opportunities

Addiction is a chronic, relapsing disorder. The progression to pathological drug-seeking is thought to be driven by maladaptive learning processes which store and maintain associative memory, linking drug highs with cues and actions in the environment. These memories can encode Pavlovian associations which link predictive stimuli (e.g., people, places, and paraphernalia) with a hedonic drug high, as well as instrumental learning about the actions required to obtain drug-associated incentives. Learned memories are not permanent however, and much recent interest has been generated in exploiting the process of reconsolidation to erase or significantly weaken maladaptive memories to treat several mental health disorders, including addictions. Normally reconsolidation serves to update and maintain the adaptive relevance of memories, however administration of amnestic agents within the critical "reconsolidation window" can weaken or even erase maladaptive memories. Here we discuss recent advances in the field, including ongoing efforts to translate preclinical reconsolidation research in animal models into clinical practice.

Saccharin fading is not required for the acquisition of alcohol self-administration, and can alter the dynamics of cue-alcohol memory reconsolidation

RATIONALE

Animal models of alcohol-seeking are useful for understanding alcohol addiction and for treatment development, but throughput in these models is limited by the extensive pretraining required to overcome the aversive taste of ethanol. Work by Augier et al. (Psychopharmacology 231: 4561-4568, 2014) indicates that Wistar rats will self-administer alcohol without water deprivation, exposure to sweetened ethanol solutions or intermittent access to ethanol.

OBJECTIVES AND METHODS

We sought to replicate and extend the work of Augier et al. by comparing the acquisition of instrumental self-administration of ethanol in Lister-Hooded rats that had been previously saccharin faded (SF group) or not (NSF group). We also aimed to determine whether NMDA receptor antagonism with MK-801, given at memory reactivation, reduced subsequent ethanol-seeking behaviour in both groups of animals. Finally, we assessed the ethanol preference of SF and NSF rats using the two-bottle choice procedure.

RESULTS

Both SF and NSF groups acquired instrumental self-administration of ethanol, though SF rats consumed fewer of the earned reinforcers. MK-801, given at memory reactivation, had different effects on NSF and SF rats: impairing the capacity of an ethanol-paired conditioned stimulus (CS) to support reinstatement in NSF rats, and enhancing it in SF rats. Finally, neither SF nor NSF rats showed a preference for ethanol.

CONCLUSIONS

Our data support those of Augier et al. (Psychopharmacology 231: 4561-4568, 2014) that pretraining is unnecessary for rats to acquire instrumental self-administration of ethanol. Indeed, saccharin fading may produce a weaker memory that extinguishes more readily, thus accounting for the different effects of MK-801 on SF and NSF rats.

An Update on Memory Reconsolidation Updating

The reactivation of a stored memory in the brain can make the memory transiently labile. During the time it takes for the memory to restabilize (reconsolidate) the memory can either be reduced by an amnesic agent or enhanced by memory enhancers. The change in memory expression is related to changes in the brain correlates of long-term memory. Many have suggested that such retrieval-induced plasticity is ideally placed to enable memories to be updated with new information. This hypothesis has been tested experimentally, with a translational perspective, by attempts to update maladaptive memories to reduce their problematic impact. We review here progress on reconsolidation updating studies, highlighting their translational exploitation and addressing recent challenges to the reconsolidation field.

Prediction error and trace dominance determine the fate of fear memories after post-training manipulations

Different mnemonic outcomes have been observed when associative memories are reactivated by CS exposure and followed by amnestics. These outcomes include mere retrieval, destabilization-reconsolidation, a transitional period (which is insensitive to amnestics), and extinction learning. However, little is known about the interaction between initial learning conditions and these outcomes during a reinforced or nonreinforced reactivation. Here we systematically combined temporally specific memories with different reactivation parameters to observe whether these four outcomes are determined by the conditions established during training. First, we validated two training regimens with different temporal expectations about US arrival. Then, using Midazolam (MDZ) as an amnestic agent, fear memories in both learning conditions were submitted to retraining either under identical or different parameters to the original training. Destabilization (i.e., susceptibly to MDZ) occurred when reactivation was reinforced, provided the occurrence of a temporal prediction error about US arrival. In subsequent experiments, both treatments were systematically reactivated by nonreinforced context exposure of different lengths, which allowed to explore the interaction between training and reactivation lengths. These results suggest that temporal prediction error and trace dominance determine the extent to which reactivation produces the different outcomes.

Reduction in Responding for Sucrose and Cocaine Reinforcement by Disruption of Memory Reconsolidation

Stored memories are dynamic and, when reactivated, can undergo a process of destabilization and reconsolidation to update them with new information. Reconsolidation has been shown for a variety of experimental settings; most recently for well-learned instrumental memories, a class of memory previously thought not to undergo reconsolidation. Here we tested, in rats, whether a weakly-trained lever-pressing memory destabilized following a shift in reinforcement contingency. We show that lever-pressing memory for both sucrose and cocaine reinforcement destabilized under appropriate conditions, and that the reconsolidation of this memory was impaired by systemic administration of the NMDA receptor (NMDAR) antagonist [5R,10S]-[+]-5-methyl-10,1-dihydro-5H-dibenzo[a,d]cyclohepten-5,10-imine (MK-801). We went on to investigate the potential role of the nucleus accumbens (NAc) in the reconsolidation of sucrose-reinforced instrumental memories, showing that co-infusion of the NMDAR antagonist 2-amino-5-phosphonopentanoic acid (AP-5) and the dopamine-1 receptor (D1R) antagonist 7-chloro-3-methyl-1-phenyl-1,2,4,5-tetrahydro-3-benzazepin-8-ol (SCH23390) into the NAc prior to memory reactivation impaired reconsolidation; however, there was no effect when these drugs were infused alone. Further investigation of this effect suggests the combined infusion disrupted the reconsolidation of pavlovian components of memory, and we hypothesize that coactivation of accumbal D1Rs and NMDARs may contribute to both the destabilization and reconsolidation of appetitive memory. Our work demonstrates that weakly-trained instrumental memories undergo reconsolidation under similar parameters to well-trained ones, and also suggests that receptor coactivation in the NAc may contribute to memory destabilization. Furthermore, it provides an important demonstration of the therapeutic potential of reconsolidation-based treatments that target the instrumental components of memory in maladaptive drug seeking.

Reconsolidation of a well-learned instrumental memory

Once consolidated, memories are dynamic entities that go through phases of instability in order to be updated with new information, via a process of reconsolidation. The phenomenon of reconsolidation has been demonstrated in a wide variety of experimental paradigms. However, the memories underpinning instrumental behaviors are currently not believed to reconsolidate. We show that well-learned lever pressing in rats does undergo reconsolidation, which can be disrupted by systemic administration of the noncompetitive N-methyl-D-aspartate receptor (NMDAR) antagonist (+)-5-methyl-10,11-dihydro-SH-dibenzo[a,d]cyclohepten-5,10-imine maleate (MK-801) when administered prior to a switch to a variable, but not fixed, ratio schedule. Disruption of reconsolidation resulted in a reduction in long-term lever pressing performance and diminished the sensitivity of behavior to contingency change. Further investigation demonstrated that expression of the reconsolidation impairment was not affected by outcome value, implying a deficit in a stimulus–response (S–R) process. The ability to disrupt the performance of well-learned instrumental behaviors is potentially of great importance in the development of reconsolidation-based clinical treatments for conditions that involve compulsive seeking behaviors.

Acquisition of conditioned responding in a multiple schedule depends on the reinforcement's temporal contingency with each stimulus

Forty mice acquired conditioned responses to stimuli presented in a multiple schedule with variable inter-trial intervals (ITIs). In some trials, reinforcement was preceded by a variable conditioned stimulus (CS), while other trials were reinforced following distinctive fixed-duration CS. A third stimulus was presented but never paired with reinforcement. Subjects in five groups experienced ITIs of different durations. Acquisition of responding to each stimulus depended only on the cycle-to-trial ratio (C/T), and thus on the temporal contingency of each stimulus. Acquisition was unaffected by whether CSs were of fixed or variable duration.

A bout of voluntary running enhances context conditioned fear, its extinction, and its reconsolidation

Three experiments used rats to examine the effect of a single bout of voluntary activity (wheel running) on the acquisition, extinction, and reconsolidation of context conditioned fear. In Experiment 1, rats provided with access to a wheel for 3 h immediately before or after a shocked exposure to a context froze more when tested in that context than rats provided with access to the wheels 6 h after the shocked exposure or rats not provided with access to the wheels. In Experiment 2, rats provided with access to the wheels immediately before or after a nonshocked exposure to the conditioned context froze less when tested in that context than rats provided with access to the wheels 6 h after the nonshocked exposure or rats not provided with access to the wheels. In Experiment 3, rats provided with access to wheels immediately after an extended nonshocked exposure to the conditioned context again froze less, whereas rats provided with access to the wheels after a brief nonshocked exposure froze more on the subsequent test than sedentary controls. These results show that a single bout of running can enhance acquisition, extinction, and reconsolidation of context conditioned fear.

Mechanisms governing the reactivation-dependent destabilization of memories and their role in extinction

The extinction of learned associations has traditionally been considered to involve new learning, which competes with the original memory for control over behavior. However, a recent resurgence of interest in reactivation-dependent amnesia has revealed that the retrieval of fear-related memory (with what is essentially a brief extinction session) can result in its destabilization. This review discusses some of the cellular and molecular mechanisms that are involved in the destabilization of a memory following its reactivation and/or extinction, and investigates the evidence that extinction may involve both new learning as well as a partial destabilization-induced erasure of the original memory trace.

Ventral tegmental dopamine dysregulation prevents appetitive memory destabilization

Through the process of reconsolidation, memories can be updated to maintain their relevance. To reconsolidate, a memory must first be destabilized in a process that we have hypothesized is initiated by a prediction error signal. Here we demonstrate that dysregulation of ventral tegmental area (VTA) signaling, which is thought to mediate prediction errors, prevented the destabilization of an appetitive goal-tracking memory in rats. We additionally show that intra-VTA infusion of either the competitive NMDA antagonist AP5 or the noncompetitive NMDA antagonist MK-801 does not selectively disrupt reconsolidation, indicating that the VTA may not be an important neural locus of reconsolidation-related neural plasticity.

Memory reconsolidation in aversive and appetitive settings

Memory reconsolidation has been observed across species and in a number of behavioral paradigms. The majority of memory reconsolidation studies have been carried out in Pavlovian fear conditioning and other aversive memory settings, with potential implications for the treatment of post-traumatic stress disorder. However, there is a growing literature on memory reconsolidation in appetitive reward-related memory paradigms, including translational models of drug addiction. While there appears to be substantial similarity in the basic phenomenon and underlying mechanisms of memory reconsolidation across unconditioned stimulus valence, there are also notable discrepancies. These arise both when comparing aversive to appetitive paradigms and also across different paradigms within the same valence of memory. We review the demonstration of memory reconsolidation across different aversive and appetitive memory paradigms, the commonalities and differences in underlying mechanisms and the conditions under which each memory undergoes reconsolidation. We focus particularly on whether principles derived from the aversive literature are applicable to appetitive settings, and also whether the expanding literature in appetitive paradigms is informative for fear memory reconsolidation.

Reconsolidation and extinction of an appetitive pavlovian memory

When memories are retrieved, they can enter a labile state during which the memory may be modified and subsequently restabilized through the process of reconsolidation. However, this does not occur in all situations, and certain "boundary conditions" determine whether a memory will undergo reconsolidation. Naïve male lister hooded rats were trained for 5 days to press a lever in order to retrieve a food reward associated with a pavlovian light stimulus. Three days post-training, animals were injected with either MK-801 (0.1 mgkg(-1); i.p.) or saline vehicle, 30 min before they were placed back into the training context for a retrieval session. Lever pressing was reinforced only by the light stimulus and was restricted to either 10, 30 or 50 presentations of the light conditioned stimulus. After 48 h, animals were again returned to the boxes and light-reinforced lever-pressing activity was recorded. MK-801-treated animals in the 10CS group significantly reduced lever pressing at test, compared to saline controls. In contrast, MK-801-treated rats in the 50CS group demonstrated a significant increase. There was no effect of MK-801 in the 30CS group. Additionally, there were no effects of MK-801 in an analogous, pure instrumental, setting when the cue lights were omitted. The opposing effects of MK-801 under different parametric conditions likely reflect impairments of appetitive pavlovian memory reconsolidation and extinction, respectively. These results demonstrate a competition between reconsolidation and extinction. However, there are also conditions under which MK-801 fails to impair either process.