Pain-relief learning in flies, rats, and man: basic research and applied perspectives

An externally validated resting-state brain connectivity signature of pain-related learning

Pain can be conceptualized as a precision signal for reinforcement learning in the brain and alterations in these processes are a hallmark of chronic pain conditions. Investigating individual differences in pain-related learning therefore holds important clinical and translational relevance. Here, we developed and externally validated a novel resting-state brain connectivity-based predictive model of pain-related learning. The pre-registered external validation indicates that the proposed model explains 8-12% of the inter-individual variance in pain-related learning. Model predictions are driven by connections of the amygdala, posterior insula, sensorimotor, frontoparietal, and cerebellar regions, outlining a network commonly described in aversive learning and pain. We propose the resulting model as a robust and highly accessible biomarker candidate for clinical and translational pain research, with promising implications for personalized treatment approaches and with a high potential to advance our understanding of the neural mechanisms of pain-related learning.

65 years of research on dopamine's role in classical fear conditioning and extinction: A systematic review

Dopamine, a catecholamine neurotransmitter, has historically been associated with the encoding of reward, whereas its role in aversion has received less attention. Here, we systematically gathered the vast evidence of the role of dopamine in the simplest forms of aversive learning: classical fear conditioning and extinction. In the past, crude methods were used to augment or inhibit dopamine to study its relationship with fear conditioning and extinction. More advanced techniques such as conditional genetic, chemogenic and optogenetic approaches now provide causal evidence for dopamine's role in these learning processes. Dopamine neurons encode conditioned stimuli during fear conditioning and extinction and convey the signal via activation of D1-4 receptor sites particularly in the amygdala, prefrontal cortex and striatum. The coordinated activation of dopamine receptors allows for the continuous formation, consolidation, retrieval and updating of fear and extinction memory in a dynamic and reciprocal manner. Based on the reviewed literature, we conclude that dopamine is crucial for the encoding of classical fear conditioning and extinction and contributes in a way that is comparable to its role in encoding reward.

Stress relief as a natural resilience mechanism against depression-like behaviors

Relief, the appetitive state after the termination of aversive stimuli, is evolutionarily conserved. Understanding the behavioral role of this well-conserved phenomenon and its underlying neurobiological mechanisms are open and important questions. Here, we discover that the magnitude of relief from physical stress strongly correlates with individual resilience to depression-like behaviors in chronic stressed mice. Notably, blocking stress relief causes vulnerability to depression-like behaviors, whereas natural rewards supplied shortly after stress promotes resilience. Stress relief is mediated by reward-related mesolimbic dopamine neurons, which show minute-long, persistent activation after stress termination. Circuitry-wise, activation or inhibition of circuits downstream of the ventral tegmental area during the transient relief period bi-directionally regulates depression resilience. These results reveal an evolutionary function of stress relief in depression resilience and identify the neural substrate mediating this effect. Importantly, our data suggest a behavioral strategy of augmenting positive valence of stress relief with natural rewards to prevent depression.

Olfactory navigation in arthropods

Using odors to find food and mates is one of the most ancient and highly conserved behaviors. Arthropods from flies to moths to crabs use broadly similar strategies to navigate toward odor sources-such as integrating flow information with odor information, comparing odor concentration across sensors, and integrating odor information over time. Because arthropods share many homologous brain structures-antennal lobes for processing olfactory information, mechanosensors for processing flow, mushroom bodies (or hemi-ellipsoid bodies) for associative learning, and central complexes for navigation, it is likely that these closely related behaviors are mediated by conserved neural circuits. However, differences in the types of odors they seek, the physics of odor dispersal, and the physics of locomotion in water, air, and on substrates mean that these circuits must have adapted to generate a wide diversity of odor-seeking behaviors. In this review, we discuss common strategies and specializations observed in olfactory navigation behavior across arthropods, and review our current knowledge about the neural circuits subserving this behavior. We propose that a comparative study of arthropod nervous systems may provide insight into how a set of basic circuit structures has diversified to generate behavior adapted to different environments.

Chronic unilateral inhibition of GABA synthesis in the amygdala increases specificity of conditioned fear in a discriminative fear conditioning paradigm in rats

Neural activity in the amygdala is critical for fear learning. In anxiety disorder patients, bilateral hyperactivity of the amygdala can be observed. This hyperactivation is often associated with the facilitation of fear learning and/or over-generalization of conditioned fear. In contrast, hypoactivity of the amygdala, e.g. by pharmacological interventions, attenuates or blocks fear learning. To date, little is known about how neural excitability of the amygdala affects specificity or generalization of fear. Therefore, the present study utilized chronic inhibition of GABA synthesis in the amygdala to increase excitability and investigated the effect on the specificity of fear learning. In rats, unilateral cannulas aiming at the amygdala were implanted. The cannulas were connected to subcutaneously implanted osmotic mini pumps that delivered either the GABA synthesis inhibitor L-allylglycine or its inactive enantiomer D-allylglycine. Following one week of chronic GABA synthesis manipulation, the rats were submitted to a discriminative fear conditioning protocol. In addition, anxiety-like behavior in the light-dark box was measured. Our data show that chronic unilateral L-AG infusions into the amygdala improve the specificity of learned fear, support safety learning, and reduce fear generalization and anxiety. This data demonstrates that moderately increased amygdala excitability can be beneficial for the specificity of fear learning and highlights the potential application for therapeutic interventions.

Local 5-HT signaling bi-directionally regulates the coincidence time window for associative learning

The coincidence between conditioned stimulus (CS) and unconditioned stimulus (US) is essential for associative learning; however, the mechanism regulating the duration of this temporal window remains unclear. Here, we found that serotonin (5-HT) bi-directionally regulates the coincidence time window of olfactory learning in Drosophila and affects synaptic plasticity of Kenyon cells (KCs) in the mushroom body (MB). Utilizing GPCR-activation-based (GRAB) neurotransmitter sensors, we found that KC-released acetylcholine (ACh) activates a serotonergic dorsal paired medial (DPM) neuron, which in turn provides inhibitory feedback to KCs. Physiological stimuli induce spatially heterogeneous 5-HT signals, which proportionally gate the intrinsic coincidence time windows of different MB compartments. Artificially reducing or increasing the DPM neuron-released 5-HT shortens or prolongs the coincidence window, respectively. In a sequential trace conditioning paradigm, this serotonergic neuromodulation helps to bridge the CS-US temporal gap. Altogether, we report a model circuitry for perceiving the temporal coincidence and determining the causal relationship between environmental events.

Modulation of aversive value coding in the vertebrate and invertebrate brain

Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.

The association of spouse interactions and emotional learning in interference related to chronic back pain

Social interactions affect individual behaviours, preferences, and attitudes. This is also critical in the context of experiencing pain and expressing pain behaviours, and may relate to learned emotional responses. In this respect, individual variability in the medial prefrontal cortex (mPFC), which is involved in adjusting an organism's behaviour to its environment by evaluating and interpreting information within the context of past experiences, is important. It is critical for selecting suitable behavioural responses within a social environment and may reinforce maladaptation in chronic pain. In our study, we used brain imaging during appetitive and aversive pavlovian conditioning in persons with chronic back pain (CBP), subacute back pain (SABP), and healthy controls (HC), together with information on spouse responses to pain behaviours. We also examined the relationship of these responses with pain-related interference in the patients. Our findings yielded a significant negative association between mPFC responses to appetitive and aversive learning in CBP. We also observed a significant negative association for mPFC responses during aversive learning and distracting spouse responses, and a significant positive association between mPFC responses during appetitive learning and solicitous spouse responses in CBP. Both significantly predicted pain-related interference in the CBP group (explained variance up to 53%). Significant associations were not found for SABP or HC. Our findings support an association between appetitive and aversive pavlovian learning, related brain circuits and spouse responses to pain in CBP, where appetitive and aversive learning processes seem to be differentially involved. This can inform prevention and early intervention in a mechanistic approach.

Multidimensional Pain Modulation by Acupuncture Analgesia: The Reward Effect of Acupuncture on Pain Relief

Pain is an intrinsically unpleasant experience with features that protect an organism by promoting motivation and learning. Pain relief, a negative reinforcement of pain, is considered a reward and activates the brain's reward system. The reward circuit in the brain involves reward and pain. Acupuncture has a multidimensional and comprehensive regulating effect on chronic pain. However, the reward effect of acupuncture in relieving chronic pain and the mechanism of the brain reward circuit involved in acupuncture analgesia are not thoroughly studied. In this article, we have reviewed the definition of pain abnormalities and negative emotions in patients with chronic pain, the conceptual characteristics of analgesic reward, and the new progress in studying brain reward circuits and functions. Moreover, we have expounded on the critical clinical and scientific significance of studying the reward effect of acupuncture analgesia and related brain reward circuits, the pain mechanism obtained from human neuroimaging studies, and the survey results on the effects of acupuncture on reward/motivation circuits. Some viewpoints and suggestions on the reward effect of acupuncture analgesia and related reward circuits have been put forward to clarify the multidimensional characteristics and benign regulation of acupuncture analgesia. Studies on the reward effect of acupuncture in relieving chronic pain and the regulating effect of the brain reward loop on acupuncture analgesia help to deepen the clinical understanding of acupuncture analgesia, innovate the research concept of acupuncture analgesia, and provide help for further studies on the central mechanism of acupuncture in improving chronic pain in the future.

Complex representation of taste quality by second-order gustatory neurons in Drosophila

Sweet and bitter compounds excite different sensory cells and drive opposing behaviors. However, it remains unclear how sweet and bitter tastes are represented by the neural circuits linking sensation to behavior. To investigate this question in Drosophila, we devised trans-Tango(activity), a strategy for calcium imaging of second-order gustatory projection neurons based on trans-Tango, a genetic transsynaptic tracing technique. We found spatial overlap between the projection neuron populations activated by sweet and bitter tastants. The spatial representation of bitter tastants in the projection neurons was consistent, while that of sweet tastants was heterogeneous. Furthermore, we discovered that bitter tastants evoke responses in the gustatory receptor neurons and projection neurons upon both stimulus onset and offset and that bitter offset and sweet onset excite overlapping second-order projections. These findings demonstrate an unexpected complexity in the representation of sweet and bitter tastants by second-order neurons of the gustatory circuit.

Optogenetically induced reward and 'frustration' memory in larval Drosophila

Humans and animals alike form oppositely valenced memories for stimuli that predict the occurrence versus the termination of a reward: appetitive 'reward' memory for stimuli associated with the occurrence of a reward and aversive 'frustration' memory for stimuli that are associated with its termination. We characterize these memories in larval Drosophila using a combination of Pavlovian conditioning, optogenetic activation of the dopaminergic central-brain DAN-i1864 neuron, and high-resolution video-tracking. This reveals their dependency on the number of training trials and the duration of DAN-i1864 activation, their temporal stability, and the parameters of locomotion that are modulated during memory expression. Together with previous results on 'punishment' versus 'relief' learning by DAN-f1 neuron activation, this reveals a 2x2 matrix of timing-dependent memory valence for the occurrence/ termination of reward/ punishment. These findings should aid the understanding and modelling of how brains decipher the predictive, causal structure of events around a target reinforcing occurrence.

Dopamine activity in projection neurons regulates short-lasting olfactory approach memory in Drosophila

Survival in many animals requires the ability to associate certain cues with danger and others with safety. In a Drosophila melanogaster aversive olfactory conditioning paradigm, flies are exposed to two odours, one presented coincidentally with electrical shocks, and a second presented 45 s after shock cessation. When flies are later given a choice between these two odours, they avoid the shock‐paired odour and prefer the unpaired odour. While many studies have examined how flies learn to avoid the shock‐paired odour through formation of odour‐fear associations, here we demonstrate that conditioning also causes flies to actively approach the second odour. In contrast to fear memories, which are longer lasting and requires activity of D1‐like dopamine receptors only in the mushroom bodies, approach memory is short‐lasting and requires activity of D1‐like dopamine receptors in projection neurons originating from the antennal lobes, primary olfactory centers. Further, while recall of fear memories requires activity of the mushroom bodies, recall of approach memories does not. Our data suggest that olfactory approach memory is formed using different mechanisms in different brain locations compared to aversive and appetitive olfactory memories.

Corticostriatal Suppression of Appetitive Pavlovian Conditioned Responding

The capacity to suppress learned responses is essential for animals to adapt in dynamic environments. Extinction is a process by which animals learn to suppress conditioned responding when an expected outcome is omitted. The infralimbic (IL) cortex to nucleus accumbens shell (NAcS) neural circuit is implicated in suppressing conditioned responding after extinction, especially in the context of operant cocaine-seeking behavior. However, the role of the IL-to-NAcS neural circuit in the extinction of responding to appetitive Pavlovian cues is unknown, and the psychological mechanisms involved in response suppression following extinction are unclear. We trained male Long Evans rats to associate a 10 s auditory conditioned stimulus (CS; 14 trials per session) with a sucrose unconditioned stimulus (US; 0.2 ml per CS) in a specific context, and then following extinction in a different context, precipitated a renewal of CS responding by presenting the CS alone in the original Pavlovian conditioning context. Unilateral, optogenetic stimulation of the IL-to-NAcS circuit selectively during CS trials suppressed renewal. In a separate experiment, IL-to-NAcS stimulation suppressed CS responding regardless of prior extinction and impaired extinction retrieval. Finally, IL-to-NAcS stimulation during the CS did not suppress the acquisition of Pavlovian conditioning but was required for the subsequent expression of CS responding. These results are consistent with multiple studies showing that the IL-to-NAcS neural circuit is involved in the suppression of operant cocaine-seeking, extending these findings to appetitive Pavlovian cues. The suppression of appetitive Pavlovian responding following IL-to-NAcS circuit stimulation, however, does not appear to be an extinction-dependent process.SIGNIFICANCE STATEMENT Extinction is a form of inhibitory learning through which animals learn to suppress conditioned responding in the face of nonreinforcement. We investigated the role of the IL cortex inputs to the NAcS in the extinction of responding to appetitive Pavlovian cues and the psychological mechanisms involved in response suppression following extinction. Using in vivo optogenetics, we found that stimulating the IL-to-NAcS neural circuit suppressed context-induced renewal of conditioned responding after extinction. In a separate experiment, stimulating the IL-to-NAcS circuit suppressed conditioned responding in an extinction-independent manner. These findings can be used by future research aimed at understanding how corticostriatal circuits contribute to behavioral flexibility and mental disorders that involve the suppression of learned behaviors.

The skin conductance response indicating pain relief is independent of self or social influence on pain

Pain relief is defined as the ease of pain and is thus highly relevant for clinical applications and everyday life. Given that pain relief is based on the cessation of an aversive pain experience, it is reasonable to assume that pain relief learning would also be shaped by factors that alter subjective and physiological pain responses, such as social presence or a feeling of control. To date, it remains unclear whether and how factors that shape autonomic pain responses might affect pain relief learning. Here, we investigated how pain relief learning is shaped by two important factors known to modulate pain responses, i.e. social influence and controllability of pain. Skin conductance responses (SCRs) were recorded while participants learned to associate a formerly neutral stimulus with pain relief under three different pain conditions. In the social‐influence condition (N = 34), the pain stimulation could be influenced by another person’s decisions. In the self‐influence condition (N = 31), the participants themselves could influence the pain stimulation. Finally, in the no‐influence condition (N = 32), pain stimulation was simply delivered without any influence. According to our results, the SCRs elicited by the stimulus that was associated with pain relief were significantly smaller compared to the SCRs elicited by a neutral control stimulus, indicating pain relief learning. However, there was no significant difference in the pain relief learning effect across the groups. These results suggest that physiological pain relief learning in humans is not significantly influenced by social influence and pain controllability.

The contents of this page will be shown on the eTOC on the online version only. It will not be published as part of the article PDF. Subjective and physiological pain responses can be altered by social support or pain controllability. Pain relief strongly depends on the perceived pain, but we found that it is less influenceable. Neither social nor self‐influence changed the magnitude of physiological pain‐relief responses. We conclude that pain relief is a basic mechanism not easily influenced by circumstantial manipulations.

Individual bitter-sensing neurons in Drosophila exhibit both ON and OFF responses that influence synaptic plasticity

The brain generates internal representations that translate sensory stimuli into appropriate behavior. In the taste system, different tastes activate distinct populations of sensory neurons. We investigated the temporal properties of taste responses in Drosophila and discovered that different types of taste sensory neurons show striking differences in their response dynamics. Strong responses to stimulus onset (ON responses) and offset (OFF responses) were observed in bitter-sensing neurons in the labellum, whereas bitter neurons in the leg and other classes of labellar taste neurons showed only an ON response. Individual labellar bitter neurons generate both ON and OFF responses through a cell-intrinsic mechanism that requires canonical bitter receptors. A single receptor complex likely generates both ON and OFF responses to a given bitter ligand. These ON and OFF responses in the periphery are propagated to dopaminergic neurons that mediate aversive learning, and the presence of the OFF response impacts synaptic plasticity when bitter is used as a reinforcement cue. These studies reveal previously unknown features of taste responses that impact neural circuit function and may be important for behavior. Moreover, these studies show that OFF responses can dramatically influence timing-based synaptic plasticity, which is thought to underlie associative learning.

Aversive teaching signals from individual dopamine neurons in larval Drosophila show qualitative differences in their temporal "fingerprint"

Dopamine serves many functions, and dopamine neurons are correspondingly diverse. We use a combination of optogenetics, behavioral experiments, and high-resolution video-tracking to probe for the functional capacities of two single, identified dopamine neurons in larval Drosophila. The DAN-f1 and the DAN-d1 neuron were recently found to carry aversive teaching signals during Pavlovian olfactory learning. We enquire into a fundamental feature of these teaching signals, namely their temporal "fingerprint". That is, receiving punishment feels bad, whereas being relieved from it feels good, and animals and humans alike learn with opposite valence about the occurrence and the termination of punishment (the same principle applies in the appetitive domain, with opposite sign). We find that DAN-f1 but not DAN-d1 can mediate such timing-dependent valence reversal: presenting an odor before DAN-f1 activation leads to learned avoidance of the odor (punishment memory), whereas presenting the odor upon termination of DAN-f1 activation leads to learned approach (relief memory). In contrast, DAN-d1 confers punishment memory only. These effects are further characterized in terms of the impact of the duration of optogenetic activation, the temporal stability of the memories thus established, and the specific microbehavioral patterns of locomotion through which they are expressed. Together with recent findings in the appetitive domain and from adult Drosophila, our results suggest that heterogeneity in the temporal fingerprint of teaching signals might be a more general principle of reinforcement processing through dopamine neurons.

Foot shock facilitates reward seeking in an experience-dependent manner

Animals organize reward seeking around aversive events. An abundance of research shows that foot shock, as well as a shock-associated cue, can elicit freezing and suppress reward seeking. Yet, there is evidence that experience can flip the effect of foot shock to facilitate reward seeking. Here we examined cue suppression, foot shock suppression and foot shock facilitation of reward seeking in a single behavioural setting. Male Long Evans rats received fear discrimination consisting of danger, uncertainty, and safety cues. Discrimination took place over a baseline of rewarded nose poking. With limited experience (1-2 sessions), all cues and foot shock suppressed reward seeking. With continued experience (10-16 sessions), suppression became specific to shock-associated cues, foot shock briefly suppressed, then facilitated reward seeking. Our results provide a means of assessing positive properties of foot shock, and may provide insight into maladaptive behaviour around aversive events.

Regulation of CREB Phosphorylation in Nucleus Accumbens after Relief Conditioning

Relief learning is the association of environmental cues with the cessation of aversive events. While there is increasing knowledge about the neural circuitry mediating relief learning, the respective molecular pathways are not known. Therefore, the aim of the present study was to examine different putative molecular pathways underlying relief learning. To this purpose, male rats were subjected either to relief conditioning or to a pseudo conditioning procedure. Forty-five minutes or 6 h after conditioning, samples of five different brain regions, namely the prefrontal cortex, nucleus accumbens (NAC), dorsal striatum, dorsal hippocampus, and amygdala, were collected. Using quantitative Western blots, the expression level of CREB, pCREB, ERK1/2, pERK1/2, CaMKIIα, MAP2K, PKA, pPKA, Akt, pAkt, DARPP-32, pDARPP-32, 14-3-3, and neuroligin2 were studied. Our analyses revealed that relief conditioned rats had higher CREB phosphorylation in NAC 6 h after conditioning than pseudo conditioned rats. The data further revealed that this CREB phosphorylation was mainly induced by dopamine D1 receptor-mediated activation of PKA, however, other kinases, downstream of the NMDA receptor, may also contribute. Taken together, the present study suggests that CREB phosphorylation, induced by a combination of different molecular pathways downstream of dopamine D1 and NMDA receptors, is essential for the acquisition and consolidation of relief learning.

Changing memories on the fly: the neural circuits of memory re-evaluation in Drosophila melanogaster

Associative learning leads to modifications in neural networks to assign valence to sensory cues. These changes not only allow the expression of learned behavior but also modulate subsequent learning events. In the brain of the adult fruit fly, Drosophila melanogaster, olfactory memories are established as dopamine-driven plasticity in the output of a highly recurrent network, the mushroom body. Recent findings have highlighted how these changes in the network can steer the strengthening, weakening and formation of parallel memories when flies are exposed to subsequent training trials, conflicting situations or the reversal of contingencies. Together, these processes provide an initial understanding of how learned information can be used to guide the re-evaluation of memories.

The connectome of the adult Drosophila mushroom body provides insights into function

Making inferences about the computations performed by neuronal circuits from synapse-level connectivity maps is an emerging opportunity in neuroscience. The mushroom body (MB) is well positioned for developing and testing such an approach due to its conserved neuronal architecture, recently completed dense connectome, and extensive prior experimental studies of its roles in learning, memory, and activity regulation. Here, we identify new components of the MB circuit in Drosophila, including extensive visual input and MB output neurons (MBONs) with direct connections to descending neurons. We find unexpected structure in sensory inputs, in the transfer of information about different sensory modalities to MBONs, and in the modulation of that transfer by dopaminergic neurons (DANs). We provide insights into the circuitry used to integrate MB outputs, connectivity between the MB and the central complex and inputs to DANs, including feedback from MBONs. Our results provide a foundation for further theoretical and experimental work.

Animal models of pain: Diversity and benefits

Chronic pain is a maladaptive neurological disease that remains a major health problem. A deepening of our knowledge on mechanisms that cause pain is a prerequisite to developing novel treatments. A large variety of animal models of pain has been developed that recapitulate the diverse symptoms of different pain pathologies. These models reproduce different pain phenotypes and remain necessary to examine the multidimensional aspects of pain and understand the cellular and molecular basis underlying pain conditions. In this review, we propose an overview of animal models, from simple organisms to rodents and non-human primates and the specific traits of pain pathologies they model. We present the main behavioral tests for assessing pain and investing the underpinning mechanisms of chronic pathological pain. The validity of animal models is analysed based on their ability to mimic human clinical diseases and to predict treatment outcomes. Refine characterization of pathological phenotypes also requires to consider pain globally using specific procedures dedicated to study emotional comorbidities of pain. We discuss the limitations of pain models when research findings fail to be translated from animal models to human clinics. But we also point to some recent successes in analgesic drug development that highlight strategies for improving the predictive validity of animal models of pain. Finally, we emphasize the importance of using assortments of preclinical pain models to identify pain subtype mechanisms, and to foster the development of better analgesics.

A reinforcement-learning model of active avoidance behavior: Differences between Sprague Dawley and Wistar-Kyoto rats

Avoidance behavior is a typically adaptive response performed by an organism to avert harmful situations. Individuals differ remarkably in their tendency to acquire and perform new avoidance behaviors, as seen in anxiety disorders where avoidance becomes pervasive and inappropriate. In rodent models of avoidance, the inbred Wistar-Kyoto (WKY) rat demonstrates increased learning and expression of avoidance compared to the outbred Sprague Dawley (SD) rat. However, underlying mechanisms that contribute to these differences are unclear. Computational modeling techniques can help identify factors that may not be easily decipherable from behavioral data alone. Here, we utilize a reinforcement learning (RL) model approach to better understand strain differences in avoidance behavior. An actor-critic model, with separate learning rates for action selection (in the actor) and state evaluation (in the critic), was applied to individual data of avoidance acquisition from a large cohort of WKY and SD rats. Latent parameters were extracted, such as learning rate and subjective reinforcement value of foot shock, that were then compared across groups. The RL model was able to accurately represent WKY and SD avoidance behavior, demonstrating that the model could simulate individual performance. The model determined that the perceived negative value of foot shock was significantly higher in WKY than SD rats, whereas learning rate in the actor was lower in WKY than SD rats. These findings demonstrate the utility of computational modeling in identifying underlying processes that could promote strain differences in behavioral performance.

Searching for relief: Drosophila melanogaster navigation in a virtual bitter maze

Animals use relief-based place learning to pinpoint a specific location where noxious stimuli are diminished or abolished. Here we show how the optogenetically-induced activation of bitter-sensing neurons in Drosophila melanogaster elicits pain-like behavioural responses and stimulates the search for a place where this activation is relieved. Under this "virtual" stimulation paradigm it would be feasible to test relief learning several times throughout an animal's lifespan, without the potentially damaging effects which may result from the repeated administration of "real" heat or electrical shock. Furthermore, virtual bitter taste could be used in place of virtual pain stimulation to guide conditioned place preference and study learning processes. We also propose that spatially-specific reduction of locomotor velocity may provide immediate evidence of relief-based place learning and spatial memory.

The Drosophila Mushroom Body: From Architecture to Algorithm in a Learning Circuit

The Drosophila brain contains a relatively simple circuit for forming Pavlovian associations, yet it achieves many operations common across memory systems. Recent advances have established a clear framework for Drosophila learning and revealed the following key operations: a) pattern separation, whereby dense combinatorial representations of odors are preprocessed to generate highly specific, nonoverlapping odor patterns used for learning; b) convergence, in which sensory information is funneled to a small set of output neurons that guide behavioral actions; c) plasticity, where changing the mapping of sensory input to behavioral output requires a strong reinforcement signal, which is also modulated by internal state and environmental context; and d) modularization, in which a memory consists of multiple parallel traces, which are distinct in stability and flexibility and exist in anatomically well-defined modules within the network. Cross-module interactions allow for higher-order effects where past experience influences future learning. Many of these operations have parallels with processes of memory formation and action selection in more complex brains.

Making translation work: Harmonizing cross-species methodology in the behavioural neuroscience of Pavlovian fear conditioning

Translational neuroscience bridges insights from specific mechanisms in rodents to complex functions in humans and is key to advance our general understanding of central nervous function. A prime example of translational research is the study of cross-species mechanisms that underlie responding to learned threats, by employing Pavlovian fear conditioning protocols in rodents and humans. Hitherto, evidence for (and critique of) these cross-species comparisons in fear conditioning research was based on theoretical viewpoints. Here, we provide a perspective to substantiate these theoretical concepts with empirical considerations of cross-species methodology. This meta-research perspective is expected to foster cross-species comparability and reproducibility to ultimately facilitate successful transfer of results from basic science into clinical applications.

Role of the PACAP system of the extended amygdala in the acoustic startle response in rats

Anxiety-related disorders are the most prevalent mental disorders in the world and they are characterized by abnormal responses to stressors. Pituitary adenylate cyclase-activating polypeptide (PACAP) is a neuropeptide highly expressed in the extended amygdala, a brain macrostructure involved in the response to threat that includes the central nucleus of the amygdala (CeA) and the bed nucleus of the stria terminalis (BNST). The aim of this series of experiments was to systematically elucidate the role of the PACAP system of the CeA and BNST under both control, unstressed conditions and after the presentation of a stressor in rats. For this purpose, we used the acoustic startle response (ASR), an unconscious response to sudden acoustic stimuli sensitive to changes in stress which can be used as an operationalization of the hypervigilance present in anxiety- and trauma-related disorders. We found that infusion of PACAP, but not the related peptide vasoactive intestinal peptide (VIP), into either the CeA or the BNST causes a dose-dependent increase in ASR. In addition, while infusion of the antagonist PACAP(6-38) into either the CeA or the BNST does not affect ASR in non-stressed conditions, it prevents the sensitization of ASR induced by an acute footshock stress. Finally, we found that footshock stress induces a significant increase in PACAP, but not VIP, levels in both of these brain areas. Altogether, these data show that the PACAP system of the extended amygdala contributes to stress-induced hyperarousal and suggest it as a potential novel target for the treatment of stress-related disorders.

Pain: A Precision Signal for Reinforcement Learning and Control

Since noxious stimulation usually leads to the perception of pain, pain has traditionally been considered sensory nociception. But its variability and sensitivity to a broad array of cognitive and motivational factors have meant it is commonly viewed as inherently imprecise and intangibly subjective. However, the core function of pain is motivational-to direct both short- and long-term behavior away from harm. Here, we illustrate that a reinforcement learning model of pain offers a mechanistic understanding of how the brain supports this, illustrating the underlying computational architecture of the pain system. Importantly, it explains why pain is tuned by multiple factors and necessarily supported by a distributed network of brain regions, recasting pain as a precise and objectifiable control signal.

Recent advances in understanding the role of phasic dopamine activity

The latest animal neurophysiology has revealed that the dopamine reward prediction error signal drives neuronal learning in addition to behavioral learning and reflects subjective reward representations beyond explicit contingency. The signal complies with formal economic concepts and functions in real-world consumer choice and social interaction. An early response component is influenced by physical impact, reward environment, and novelty but does not fully code prediction error. Some dopamine neurons are activated by aversive stimuli, which may reflect physical stimulus impact or true aversiveness, but they do not seem to code general negative value or aversive prediction error. The reward prediction error signal is complemented by distinct, heterogeneous, smaller and slower changes reflecting sensory and motor contributors to behavioral activation, such as substantial movement (as opposed to precise motor control), reward expectation, spatial choice, vigor, and motivation. The different dopamine signals seem to defy a simple unifying concept and should be distinguished to better understand phasic dopamine functions.

Distinct Dopamine Receptor Pathways Underlie the Temporal Sensitivity of Associative Learning

Animals rely on the relative timing of events in their environment to form and update predictive associations, but the molecular and circuit mechanisms for this temporal sensitivity remain incompletely understood. Here, we show that olfactory associations in Drosophila can be written and reversed on a trial-by-trial basis depending on the temporal relationship between an odor cue and dopaminergic reinforcement. Through the synchronous recording of neural activity and behavior, we show that reversals in learned odor attraction correlate with bidirectional neural plasticity in the mushroom body, the associative olfactory center of the fly. Two dopamine receptors, DopR1 and DopR2, contribute to this temporal sensitivity by coupling to distinct second messengers and directing either synaptic depression or potentiation. Our results reveal how dopamine-receptor signaling pathways can detect the order of events to instruct opposing forms of synaptic and behavioral plasticity, allowing animals to flexibly update their associations in a dynamic environment.

Topography of Reward and Aversion Encoding in the Mesolimbic Dopaminergic System

Dopamine (DA) neurons in the VTA play essential roles in adaptive motivated behavior, which requires rapid discrimination between positive and negative motivational signature. However, the precise functional DA circuitry processing reward and aversive information remain elusive. Here, we report that the encoding of reward and aversion by the DA system in the NAc is tightly associated with its anatomical location. By recording the dynamics of DA release with genetically encoded fluorescent DA sensor using in vivo fiber photometry in freely moving male mice, we found that the DA-sensor signal in the dorsomedial NAc shell and dorsolateral NAc shell were increased during rewarding events and decreased during aversive noxious events. In contrast, the release of DA in the ventromedial NAc shell was increased by both rewarding and aversive stimuli, whereas the DA-sensor signal in the central ventromedial NAc shell and ventrolateral NAc shell showed complex dynamics. Furthermore, the activity of DA fibers in different subregions of NAc measured with calcium sensor largely recapitulated the changes of DA-sensor signal in response to rewarding and aversive stimuli. In addition, correlation analysis showed that the response magnitude of DA-sensor or fibers significantly changed along the DV axis of the NAc. These results revealed the distinct role of the mesolimbic DA system in different subregions of NAc in encoding value and salience.SIGNIFICANCE STATEMENT Adaptive motivated behavior requires rapid discrimination between favorable and harmful events and is dynamically modulated by dopamine (DA) neurons in the VTA. However, the precise relationship between distinct DA circuitry and reward/aversion signal encoding is not well understood. Here, by recording the dynamics of DA release and the activity of DA fibers in each subregion of the NAc using in vivo fiber photometry in freely moving animals, we found that the DA system in the dorsomedial/dorsolateral, ventromedial, and ventrolateral NAc shell plays different roles in encoding value and salience. These results extend our knowledge about how the mesolimbic DA system process motivational information at the circuitry level.

Analgesic Effects of Cnidium officinale Extracts on Postoperative, Neuropathic, and Menopausal Pain in Rat Models

Cnidium officinale, widely cultivated in East Asia, has been reported to exhibit pharmacological efficacy in various disorders. However, little has been reported on its role as a pain killer. In this study, we reveal that the C. officinale extract (COE) has great efficacy as a novel analgesic in various in vivo pain models. Administration of COE attenuated hypersensitivity in all postoperative, neuropathic, and menopausal pain models. Decreased hyperalgesia was confirmed by a mechanical withdrawal threshold assay and ultrasonic vocalization call analysis. In addition, application of COE inhibited the induction of the proinflammatory cytokines and calpain-3 on dorsal root ganglion neurons in a spared nerve injury rat model. Treatment with ferulic acid, which was identified as one of the components of COE by HPLC analysis, alleviated nociceptive behaviors. Our findings suggest that ferulic acid is an active compound from COE, and COE is a potential phytomedical source for pain relief by inhibiting the process of inflammation.

Aversive Training of Honey Bees in an Automated Y-Maze

Honeybees have remarkable learning abilities given their small brains, and have thus been established as a powerful model organism for the study of learning and memory. Most of our current knowledge is based on appetitive paradigms, in which a previously neutral stimulus (e.g., a visual, olfactory, or tactile stimulus) is paired with a reward. Here, we present a novel apparatus, the yAPIS, for aversive training of walking honey bees. This system consists in three arms of equal length and at 120° from each other. Within each arm, colored lights (λ = 375, 465 or 520 nm) or odors (here limonene or linalool) can be delivered to provide conditioned stimuli (CS). A metal grid placed on the floor and roof delivers the punishment in the form of mild electric shocks (unconditioned stimulus, US). Our training protocol followed a fully classical procedure, in which the bee was exposed sequentially to a CS paired with shocks (CS+) and to another CS not punished (CS-). Learning performance was measured during a second phase, which took advantage of the Y-shape of the apparatus and of real-time tracking to present the bee with a choice situation, e.g., between the CS+ and the CS-. Bees reliably chose the CS- over the CS+ after only a few training trials with either colors or odors, and retained this memory for at least a day, except for the shorter wavelength (λ = 375 nm) that produced mixed results. This behavior was largely the result of the bees avoiding the CS+, as no evidence was found for attraction to the CS-. Interestingly, trained bees initially placed in the CS+ spontaneously escaped to a CS- arm if given the opportunity, even though they could never do so during the training. Finally, honey bees trained with compound stimuli (color + odor) later avoided either components of the CS+. Thus, the yAPIS is a fast, versatile and high-throughput way to train honey bees in aversive paradigms. It also opens the door for controlled laboratory experiments investigating bimodal integration and learning, a field that remains in its infancy.

Bed nucleus of the stria terminalis regulates fear to unpredictable threat signals

The bed nucleus of the stria terminalis (BNST) has been implicated in conditioned fear and anxiety, but the specific factors that engage the BNST in defensive behaviors are unclear. Here we examined whether the BNST mediates freezing to conditioned stimuli (CSs) that poorly predict the onset of aversive unconditioned stimuli (USs) in rats. Reversible inactivation of the BNST selectively reduced freezing to CSs that poorly signaled US onset (e.g., a backward CS that followed the US), but did not eliminate freezing to forward CSs even when they predicted USs of variable intensity. Additionally, backward (but not forward) CSs selectively increased Fos in the ventral BNST and in BNST-projecting neurons in the infralimbic region of the medial prefrontal cortex (mPFC), but not in the hippocampus or amygdala. These data reveal that BNST circuits regulate fear to unpredictable threats, which may be critical to the etiology and expression of anxiety.

Self-Criticism Impacts Emotional Responses to Pain

Nonsuicidal self-injury (NSSI) is associated with numerous negative outcomes (e.g., suicide attempts), making it a focus of great clinical concern. Yet, mechanisms reinforcing NSSI remain unclear. The benefits and barriers model proposes that NSSI engagement is determined by both benefits of and barriers to NSSI. Benefits include mood improvement, a function reported by most who engage in NSSI; barriers include a desire to avoid pain and bodily harm. Self-criticism is generally understood as a trait lowering desire to avoid pain and bodily harm, thus decreasing that specific barrier. However, recent research demonstrated that self-criticism may also increase NSSI benefits. Highly self-critical people may view NSSI and pain in the context of feeling deserving of pain and punishment-thus, pain may improve mood for self-critical individuals. We tested whether self-criticism impacted emotional responding to pain among adult females with (n = 44) and without (n = 65) NSSI histories. After a negative mood induction, participants rated their moods before, during, and after self-administered pain. In participants with and without NSSI histories, self-criticism was positively correlated with mood improvements during pain. Thus, regardless of NSSI history, self-criticism impacted emotional responses to pain. Together, results suggest that self-criticism may not only decrease an important NSSI barrier but also enhance NSSI benefits, specifically leading to more mood improvement during pain.

The role of trait anxiety in associative learning during and after an aversive event

Trait anxiety is considered to be a risk factor for anxiety disorders. The aim of the present study was to investigate how trait anxiety affects associative learning during and after an aversive event in laboratory rats. For this, rats were first submitted to a light–dark box test, followed by relief, safety, and fear learning. Our data demonstrate that all types of learning were affected by trait anxiety, both on a group and on an individual level. Whereas high levels of anxiety impaired relief and safety learning, fear learning was more pronounced. These findings help to show how trait anxiety might be involved in the etiology of anxiety disorders and pathological sensation- and risk-seeking.

Sound check, stage design and screen plot - how to increase the comparability of fear conditioning and fear extinction experiments

In the recent decade, fear conditioning has evolved as a standard procedure for testing cognitive abilities such as memory acquisition, consolidation, recall, reconsolidation, and extinction, preferentially in genetically modified mice. The reasons for the popularity of this powerful approach are its ease to perform, the short duration of training and testing, and its well-described neural basis. So why to bother about flaws in standardization of test procedures and analytical routines? Simplicity does not preclude the existence of fallacies. A short survey of the literature revealed an indifferent use of acoustic stimuli in terms of quality (i.e., white noise vs. sine wave), duration, and intensity. The same applies to the shock procedures. In the present article, I will provide evidence for the importance of qualitative and quantitative parameters of conditioned and unconditioned stimuli for the experimental outcome. Moreover, I will challenge frequently applied interpretations of short-term vs. long-term extinction and spontaneous recovery. On the basis of these concerns, I suggest a guideline for standardization of fear conditioning experiments in mice to improve the comparability of the experimental data.

Connectomics and function of a memory network: the mushroom body of larval Drosophila

The Drosophila larva is a relatively simple, 10 000-neuron study case for learning and memory with enticing analytical power, combining genetic tractability, the availability of robust behavioral assays, the opportunity for single-cell transgenic manipulation, and an emerging synaptic connectome of its complete central nervous system. Indeed, although the insect mushroom body is a much-studied memory network, the connectome revealed that more than half of the classes of connection within the mushroom body had escaped attention. The connectome also revealed circuitry that integrates, both within and across brain hemispheres, higher-order sensory input, intersecting valence signals, and output neurons that instruct behavior. Further, it was found that activating individual dopaminergic mushroom body input neurons can have a rewarding or a punishing effect on olfactory stimuli associated with it, depending on the relative timing of this activation, and that larvae form molecularly dissociable short-term, long-term, and amnesia-resistant memories. Together, the larval mushroom body is a suitable study case to achieve a nuanced account of molecular function in a behaviorally meaningful memory network.

Nociceptive Biology of Molluscs and Arthropods: Evolutionary Clues About Functions and Mechanisms Potentially Related to Pain

Important insights into the selection pressures and core molecular modules contributing to the evolution of pain-related processes have come from studies of nociceptive systems in several molluscan and arthropod species. These phyla, and the chordates that include humans, last shared a common ancestor approximately 550 million years ago. Since then, animals in these phyla have continued to be subject to traumatic injury, often from predators, which has led to similar adaptive behaviors (e.g., withdrawal, escape, recuperative behavior) and physiological responses to injury in each group. Comparisons across these taxa provide clues about the contributions of convergent evolution and of conservation of ancient adaptive mechanisms to general nociceptive and pain-related functions. Primary nociceptors have been investigated extensively in a few molluscan and arthropod species, with studies of long-lasting nociceptive sensitization in the gastropod, Aplysia, and the insect, Drosophila, being especially fruitful. In Aplysia, nociceptive sensitization has been investigated as a model for aversive memory and for hyperalgesia. Neuromodulator-induced, activity-dependent, and axotomy-induced plasticity mechanisms have been defined in synapses, cell bodies, and axons of Aplysia primary nociceptors. Studies of nociceptive sensitization in Drosophila larvae have revealed numerous molecular contributors in primary nociceptors and interacting cells. Interestingly, molecular contributors examined thus far in Aplysia and Drosophila are largely different, but both sets overlap extensively with those in mammalian pain-related pathways. In contrast to results from Aplysia and Drosophila, nociceptive sensitization examined in moth larvae (Manduca) disclosed central hyperactivity but no obvious peripheral sensitization of nociceptive responses. Squid (Doryteuthis) show injury-induced sensitization manifested as behavioral hypersensitivity to tactile and especially visual stimuli, and as hypersensitivity and spontaneous activity in nociceptor terminals. Temporary blockade of nociceptor activity during injury subsequently increased mortality when injured squid were exposed to fish predators, providing the first demonstration in any animal of the adaptiveness of nociceptive sensitization. Immediate responses to noxious stimulation and nociceptive sensitization have also been examined behaviorally and physiologically in a snail (Helix), octopus (Adopus), crayfish (Astacus), hermit crab (Pagurus), and shore crab (Hemigrapsus). Molluscs and arthropods have systems that suppress nociceptive responses, but whether opioid systems play antinociceptive roles in these phyla is uncertain.

Role of the mesolimbic dopamine system in relief learning

The relief from an aversive event is rewarding. Since organisms are able to learn which environmental cues can cease an aversive event, relief learning helps to better cope with future aversive events. Literature data suggest that relief learning is affected in various psychopathological conditions, such as anxiety disorders. Here, we investigated the role of the mesolimbic dopamine system in relief learning. Using a relief learning procedure in Sprague Dawley rats, we applied a combination of behavioral experiments with anatomical tracing, c-Fos immunohistochemistry, and local chemogenetic and pharmacological interventions to broadly characterize the role of the mesolimbic dopamine system. The present study shows that a specific part of the mesolimbic dopamine system, the projection from the posterior medial ventral tegmental area (pmVTA) to the nucleus accumbens shell (AcbSh), is activated by aversive electric stimuli. 6-OHDA lesions of the pmVTA blocked relief learning but fear learning and safety learning were not affected. Chemogenetic silencing of the pmVTA-AcbSh projection using the DREADD approach, as well as intra-AcbSh injections of the dopamine D2/3 receptor antagonist raclopride inhibited relief learning. Taken together, the present data demonstrate that the dopaminergic pmVTA-AcbSh projection is critical for relief learning but not for similar learning phenomena. This novel finding may have clinical implications since the processing of signals predicting relief and safety is often impaired in patients suffering from anxiety disorders. Furthermore, it may help to better understand psychological conditions like non-suicidal self-injury, which are associated with pain offset relief.

The interruptive effect of electric shock on odor response requires mushroom bodies in Drosophila melanogaster

Nociceptive stimulus involuntarily interrupts concurrent activities. This interruptive effect is related to the protective function of nociception that is believed to be under stringent evolutionary pressure. To determine whether such interruptive effect is conserved in invertebrate and potentially uncover underlying neural circuits, we examined Drosophila melanogaster. Electric shock (ES) is a commonly used nociceptive stimulus for nociception related research in Drosophila. Here, we showed that background noxious ES dramatically interrupted odor response behaviors in a T-maze, which is termed blocking odor response by electric shock (BOBE). The interruptive effect is not odor specific. ES could interrupt both odor avoidance and odor approach. To identify involved brain areas, we focused on the odor avoidance to 3-OCT. By spatially abolishing neurotransmission with temperature sensitive Shibire^TS1 , we found that mushroom bodies (MBs) are necessary for BOBE. Among the 3 major MB Kenyon cell (KCs) subtypes, α/β neurons and γ neurons but not α'/β' neurons are required for normal BOBE. Specifically, abolishing the neurotransmission of either α/β surface (α/β_s ), α/β core (α/β_c ) or γ dorsal (γ_d ) neurons alone is sufficient to abrogate BOBE. This pattern of MB subset requirement is distinct from that of aversive olfactory learning, indicating a specialized BOBE pathway. Consistent with this idea, BOBE was not diminished in several associative memory mutants and noxious ES interrupted both innate and learned odor avoidance. Overall, our results suggest that MB α/β and γ neurons are parts of a previously unappreciated central neural circuit that processes the interruptive effect of nociception.