Translational research in punishment learning

Punishment learning is learning of the causal relationship between responses and their adverse or undesirable consequences. Here, we review our translational approach for understanding whether, when, and how individuals differ in what they learn during punishment, and how these differences in learning may drive persistent poor or maladaptive decisions. We show that individual differences in punishment insensitivity can emerge from differences between individuals in what they learn about punishment (instrumental contingency knowledge), rather than differences in aversive valuation, reward valuation, general (impulsivity), or specific (habit) behavioral control. These differences in instrumental contingency knowledge are shared with and can be studied in other animals. Our approach has strong construct and predictive validity, providing a robust translational platform for studying how punishment learning and decision making may contribute to neuropsychiatric disorders. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Pathways to the persistence of drug use despite its adverse consequences

The persistence of drug taking despite its adverse consequences plays a central role in the presentation, diagnosis, and impacts of addiction. Eventual recognition and appraisal of these adverse consequences is central to decisions to reduce or cease use. However, the most appropriate ways of conceptualizing persistence in the face of adverse consequences remain unclear. Here we review evidence that there are at least three pathways to persistent use despite the negative consequences of that use. A cognitive pathway for recognition of adverse consequences, a motivational pathway for valuation of these consequences, and a behavioral pathway for responding to these adverse consequences. These pathways are dynamic, not linear, with multiple possible trajectories between them, and each is sufficient to produce persistence. We describe these pathways, their characteristics, brain cellular and circuit substrates, and we highlight their relevance to different pathways to self- and treatment-guided behavior change.

A cognitive pathway to punishment insensitivity

Individuals differ in their sensitivity to the adverse consequences of their actions, leading some to persist in maladaptive behaviors. Two pathways have been identified for this insensitivity: a motivational pathway based on excessive reward valuation and a behavioral pathway based on autonomous stimulus-response mechanisms. Here, we identify a third, cognitive pathway based on differences in punishment knowledge and use of that knowledge to suppress behavior. We show that distinct phenotypes of punishment sensitivity emerge from differences in what people learn about their actions. Exposed to identical punishment contingencies, some people (sensitive phenotype) form correct causal beliefs that they use to guide their behavior, successfully obtaining rewards and avoiding punishment, whereas others form incorrect but internally coherent causal beliefs that lead them to earn punishment they do not like. Incorrect causal beliefs were not inherently problematic because we show that many individuals benefit from information about why they are being punished, revaluing their actions and changing their behavior to avoid further punishment (unaware phenotype). However, one condition where incorrect causal beliefs were problematic was when punishment is infrequent. Under this condition, more individuals show punishment insensitivity and detrimental patterns of behavior that resist experience and information-driven updating, even when punishment is severe (compulsive phenotype). For these individuals, rare punishment acted as a "trap," inoculating maladaptive behavioral preferences against cognitive and behavioral updating.

Role of anterior insula cortex in context-induced relapse of nicotine-seeking

Tobacco use is the leading cause of preventable death worldwide, and relapse during abstinence remains the critical barrier to successful treatment of tobacco addiction. During abstinence, environmental contexts associated with nicotine use can induce craving and contribute to relapse. The insular cortex (IC) is thought to be a critical substrate of nicotine addiction and relapse. However, its specific role in context-induced relapse of nicotine-seeking is not fully known. In this study, we report a novel rodent model of context-induced relapse to nicotine-seeking after punishment-imposed abstinence, which models self-imposed abstinence through increasing negative consequences of excessive drug use. Using the neuronal activity marker Fos we find that the anterior (aIC), but not the middle or posterior IC, shows increased activity during context-induced relapse. Combining Fos with retrograde labeling of aIC inputs, we show projections to aIC from contralateral aIC and basolateral amygdala exhibit increased activity during context-induced relapse. Next, we used fiber photometry in aIC and observed phasic increases in aIC activity around nicotine-seeking responses during self-administration, punishment, and the context-induced relapse tests. Next, we used chemogenetic inhibition in both male and female rats to determine whether activity in aIC is necessary for context-induced relapse. We found that chemogenetic inhibition of aIC decreased context-induced nicotine-seeking after either punishment- or extinction-imposed abstinence. These findings highlight the critical role nicotine-associated contexts play in promoting relapse, and they show that aIC activity is critical for this context-induced relapse following both punishment and extinction-imposed abstinence.

Punishment insensitivity in humans is due to failures in instrumental contingency learning

Punishment maximises the probability of our individual survival by reducing behaviours that cause us harm, and also sustains trust and fairness in groups essential for social cohesion. However, some individuals are more sensitive to punishment than others and these differences in punishment sensitivity have been linked to a variety of decision-making deficits and psychopathologies. The mechanisms for why individuals differ in punishment sensitivity are poorly understood, although recent studies of conditioned punishment in rodents highlight a key role for punishment contingency detection (Jean-Richard-Dit-Bressel et al., 2019). Here, we applied a novel ‘Planets and Pirates’ conditioned punishment task in humans, allowing us to identify the mechanisms for why individuals differ in their sensitivity to punishment. We show that punishment sensitivity is bimodally distributed in a large sample of normal participants. Sensitive and insensitive individuals equally liked reward and showed similar rates of reward-seeking. They also equally disliked punishment and did not differ in their valuation of cues that signalled punishment. However, sensitive and insensitive individuals differed profoundly in their capacity to detect and learn volitional control over aversive outcomes. Punishment insensitive individuals did not learn the instrumental contingencies, so they could not withhold behaviour that caused punishment and could not generate appropriately selective behaviours to prevent impending punishment. These differences in punishment sensitivity could not be explained by individual differences in behavioural inhibition, impulsivity, or anxiety. This bimodal punishment sensitivity and these deficits in instrumental contingency learning are identical to those dictating punishment sensitivity in non-human animals, suggesting that they are general properties of aversive learning and decision-making.

Phasic inhibition of dopamine neurons is an instrumental punisher

It is well established that the activity of VTA dopamine neurons is sufficient to serve as a Pavlovian reinforcer but whether this activity can also serve as instrumental reinforcer is less well understood. Here we studied the effects of optogenetic inhibition of VTA dopamine neurons in instrumental conditioning preparations. We show that optogenetic inhibition of VTA dopamine neurons causes a response-specific, contingency-sensitive suppression of instrumental responding. This suppression was due to instrumental response, not Pavlovian stimulus, learning and could not be attributed to deepened instrumental extinction learning. These effects of optogenetic inhibition of VTA dopamine neurons on instrumental responding are formally similar to the effects of aversive events in instrumental preparations and show that optogenetic inhibition of VTA dopamine neurons is sufficient to serve as an instrumental punisher. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The Mesolimbic Dopamine Activity Signatures of Relapse to Alcohol-Seeking

The mesolimbic dopamine system comprises distinct compartments supporting different functions in learning and motivation. Less well understood is how complex addiction-related behaviors emerge from activity patterns across these compartments. Here we show how different forms of relapse to alcohol-seeking in male rats are assembled from activity across the VTA and the nucleus accumbens. First, we used chemogenetic approaches to show a causal role for VTA TH neurons in two forms of relapse to alcohol-seeking: renewal (context-induced reinstatement) and reacquisition. Then, using gCaMP fiber photometry of VTA TH neurons, we identified medial and lateral VTA TH neuron activity profiles during self-administration, renewal, and reacquisition. Next, we used optogenetic inhibition of VTA TH neurons to show distinct causal roles for VTA subregions in distinct forms of relapse. We then used dLight fiber photometry to measure dopamine binding across the ventral striatum (medial accumbens shell, accumbens core, lateral accumbens shell) and showed complex and heterogeneous profiles of dopamine binding during self-administration and relapse. Finally, we used representational similarity analysis to identify mesolimbic dopamine signatures of self-administration, extinction, and relapse. Our results show that signatures of relapse can be identified from heterogeneous activity profiles across the mesolimbic dopamine system and that these signatures are unique for different forms of relapse.SIGNIFICANCE STATEMENT It is axiomatic that the actions of dopamine are critical to drug addiction. Yet how relapse to drug-seeking is assembled from activity across the mesolimbic dopamine system is poorly understood. Here we show how relapse to alcohol-seeking relates to activity in specific VTA and accumbens compartments, how these change for different forms of relapse, and how relapse-associated activity relates to activity during self-administration and extinction. We report the mesolimbic dopamine activity signatures for relapse and show that these signatures are unique for different forms of relapse.

Analyzing Event-Related Transients: Confidence Intervals, Permutation Tests, and Consecutive Thresholds

Fiber photometry has enabled neuroscientists to easily measure targeted brain activity patterns in awake, freely behaving animal. A focus of this technique is to identify functionally-relevant changes in activity around particular environmental and/or behavioral events, i.e., event-related activity transients (ERT). A simple and popular approach to identifying ERT is to summarize peri-event signal [e.g., area under the curve (AUC), peak activity, etc.,] and perform standard analyses on this summary statistic. We highlight the various issues with this approach and overview straightforward alternatives: waveform confidence intervals (CIs) and permutation tests. We introduce the rationale behind these approaches, describe the results of Monte Carlo simulations evaluating their effectiveness at controlling Type I and Type II error rates, and offer some recommendations for selecting appropriate analysis strategies for fiber photometry experiments.

Punishment insensitivity emerges from impaired contingency detection, not aversion insensitivity or reward dominance

Our behaviour is shaped by its consequences - we seek rewards and avoid harm. It has been reported that individuals vary markedly in their avoidance of detrimental consequences, that is in their sensitivity to punishment. The underpinnings of this variability are poorly understood; they may be driven by differences in aversion sensitivity, motivation for reward, and/or instrumental control. We examined these hypotheses by applying several analysis strategies to the behaviour of rats (n = 48; 18 female) trained in a conditioned punishment task that permitted concurrent assessment of punishment, reward-seeking, and Pavlovian fear. We show that punishment insensitivity is a unique phenotype, unrelated to differences in reward-seeking and Pavlovian fear, and due to a failure of instrumental control. Subjects insensitive to punishment are afraid of aversive events, they are simply unable to change their behaviour to avoid them.

Behavioral and neurobiological mechanisms of punishment: implications for psychiatric disorders

Punishment involves learning about the relationship between behavior and its adverse consequences. Punishment is fundamental to reinforcement learning, decision-making and choice, and is disrupted in psychiatric disorders such as addiction, depression, and psychopathy. However, little is known about the brain mechanisms of punishment and much of what is known is derived from study of superficially similar, but fundamentally distinct, forms of aversive learning such as fear conditioning and avoidance learning. Here we outline the unique conditions that support punishment, the contents of its learning, and its behavioral consequences. We consider evidence implicating GABA and monoamine neurotransmitter systems, as well as corticostriatal, amygdala, and dopamine circuits in punishment. We show how maladaptive punishment processes are implicated in addictions, impulse control disorders, psychopathy, anxiety, and depression and argue that a better understanding of the cellular, circuit, and cognitive mechanisms of punishment will make important contributions to next generation therapeutic approaches.

Lateral, not medial, prefrontal cortex contributes to punishment and aversive instrumental learning

Aversive outcomes punish behaviors that cause their occurrence. The prefrontal cortex (PFC) has been implicated in punishment learning and behavior, although the exact roles for different PFC regions in instrumental aversive learning and decision-making remain poorly understood. Here, we assessed the role of the orbitofrontal (OFC), rostral agranular insular (RAIC), prelimbic (PL), and infralimbic (IL) cortex in instrumental aversive learning and decision-making. Rats that pressed two individually presented levers for pellet rewards rapidly suppressed responding to one lever if it also caused mild punishment (punished lever) but continued pressing the other lever that did not cause punishment (unpunished lever). Inactivations of OFC, RAIC, IL, or PL via the GABA agonists baclofen and muscimol (BM) had no effect on the acquisition of instrumental learning. OFC inactivations increased responding on the punished lever during expression of well-learned instrumental aversive learning, whereas RAIC inactivations increased responding on the punished lever when both levers were presented simultaneously in an unpunished choice test. There were few effects of medial PFC (PL and IL) inactivation. These results suggest that lateral PFC, notably OFC and RAIC, have complementary functions in aversive instrumental learning and decision-making; OFC is important for using established aversive instrumental memories to guide behavior away from actions that cause punishment, whereas RAIC is important for aversive decision-making under conditions of choice.

The role of the basolateral amygdala in punishment

Aversive stimuli not only support fear conditioning to their environmental antecedents, they also punish behaviors that cause their occurrence. The amygdala, especially the basolateral nucleus (BLA), has been critically implicated in Pavlovian fear learning but its role in punishment remains poorly understood. Here, we used a within-subjects punishment task to assess the role of the BLA in the acquisition and expression of punishment as well as aversive choice. Rats that pressed two individually presented levers for pellet rewards rapidly suppressed responding to one lever if it also caused footshock deliveries (punished lever) but continued pressing a second lever that did not cause footshock (unpunished lever). Infusions of GABA agonists baclofen and muscimol (BM) into the BLA significantly impaired the acquisition of this suppression. BLA inactivations using BM also reduced the expression of well-trained punishment. There was anatomical segregation within the BLA so that caudal, not rostral, BLA was implicated in punishment. However, when presented with punished and unpunished levers simultaneously in a choice test without deliveries of shock punisher, rats expressed a preference for unpunished over the punished lever and BLA inactivations had no effect on this preference. Taken together, these findings indicate that the BLA is important for both the acquisition and expression of punishment but not for aversive choice. This role appears to be linked to neurons in the caudal BLA, rather than rostral BLA, although the circuitry that contributes to this functional segregation is currently unknown, and is most parsimoniously interpreted as a role for caudal BLA in determining the aversive value of the shock punisher.