The trade-off between pulse duration and power in optical excitation of midbrain dopamine neurons approximates Bloch's law

Does phasic dopamine release cause policy updates?

Phasic dopamine activity is believed to both encode reward-prediction errors (RPEs) and to cause the adaptations that these errors engender. If so, a rat working for optogenetic stimulation of dopamine neurons will repeatedly update its policy and/or action values, thus iteratively increasing its work rate. Here, we challenge this view by demonstrating stable, non-maximal work rates in the face of repeated optogenetic stimulation of midbrain dopamine neurons. Furthermore, we show that rats learn to discriminate between world states distinguished only by their history of dopamine activation. Comparison of these results to reinforcement learning simulations suggests that the induced dopamine transients acted more as rewards than RPEs. However, pursuit of dopaminergic stimulation drifted upwards over a time scale of days and weeks, despite its stability within trials. To reconcile the results with prior findings, we consider multiple roles for dopamine signalling.

Optogenetic action potentials and intrinsic pacemaker interplay in retrogradely identified midbrain dopamine neurons

Dissecting the diversity of midbrain dopamine (DA) neurons by optotagging is a promising addition to better identify their functional properties and contribution to motivated behavior. Retrograde molecular targeting of DA neurons with specific axonal projection allows further refinement of this approach. Here, we focus on adult mouse DA neurons in the substantia nigra pars compacta (SNc) projecting to dorsal striatum (DS) by demonstrating the selectivity of a floxed AAV9-based retrograde channelrhodopsin-eYFP (ChR-eYFP) labeling approach in DAT-cre mice. Furthermore, we show the utility of a sparse labeling version for anatomical single-cell reconstruction and demonstrate that ChR-eYFR expressing DA neurons retain intrinsic functional properties indistinguishable from conventionally retrogradely red-beads-labeled neurons. We systematically explore the properties of optogenetically evoked action potentials (oAPs) and their interaction with intrinsic pacemaking in this defined subpopulation of DA neurons. We found that the shape of the oAP and its first derivative, as a proxy for extracellularly recorded APs, is highly distinct from spontaneous APs (sAPs) of the same neurons and systematically varies across the pacemaker duty cycle. The timing of the oAP also affects the backbone oscillator of the intrinsic pacemaker by introducing transient "compensatory pauses". Characterizing this systematic interplay between oAPs and sAPs in defined DA neurons will also facilitate a refinement of DA neuron optotagging in vivo.

Comparison of progressive hold and progressive response schedules of reinforcement

Progressive ratio (PR) schedules have been widely used to study motivation to work for a reinforcer. After a post-reinforcer pause, subjects engage in pressing a lever until a reinforcer is obtained. However, the discrete nature of lever presses allows alternative behaviors during inter-response time and has led to several behavioral explanations of pauses and work time. A progressive hold-down (PH) is incompatible with alternative responses and may allow a precise estimation of work time. Performance of rats trained in both PR and PH that received sucrose or intracranial self-stimulation (ICSS) as reinforcer were compared. We observed that rats mastered the PR and PH schedules. Post-reinforcer pauses (PS^RP), work time and inter-reinforcer time increased as a function of the response or hold requirement. However, rats' performance suggested that the PH progression may be experienced by the rats as easier that the PR progression. Elimination of consummatory behavior with ICSS reduced post-reinforcer pause in accordance with predictions of explanatory models of fixed and variable schedules of reinforcement. In the case of PH performance, pauses showed little variation across intermediate requirements but increased rapidly on later requirements; since rats controlled their pause length and work time was close to the hold requirement, time allocation between PR and PH schedules diverged. Finally, the Mathematical Principles of Reinforcement model of Bradshaw and Killeen (Psychopharmacology 2012, 222: 549) rendered a good description of the performance in both PR and PH schedules.

On the Similarity Between the Reinforcing and the Discriminative Properties of Intracranial Self-Stimulation

Rats work very hard for intracranial self-stimulation (ICSS) and tradeoff effort or time allocation for intensity and frequency parameters producing a sigmoidal function of the subjective reward magnitude of ICSS. Previous studies using electrical intracranial stimuli (ICS) as a discriminative cue focused on estimating detection thresholds or on the discrimination between intensities. To our knowledge, there is no direct comparison of the reinforcer tradeoff functions with the discriminative functions. Rats were trained to press and hold the lever for ICSS using the maximum reinforcing intensity below motor alterations or avoidance behavior. First, rats were trained to hold the lever for 1 s; after stability, they undergo trials where intensity or frequency was decreased on 0.1 log step. Thereafter, they undergo further training with a hold of 2 and later of 4 s to determine tradeoff with intensity or frequency. The same rats were trained on a discrimination task where the previously used ICSS signaled a lever where a 1 s hold response was followed by a reinforcing ICSS; on randomly alternating trials, a −0.6 log ICS signaled an alternate lever where a similar hold response led to a reinforcer. After mastering discrimination, generalization tests were carried out with varying intensity or frequency. Rats completed training with 2 and later 4 s hold response. After the completion of each task, the rats had different doses of a pimozide challenge while their intensity and hold-down requirement were varied. With regards to the rats’ tradeoff response time allocation as a function of intensity or frequency, sigmoid functions were displaced to the right when long responses were required. Rats that learned the discrimination task attained a discrimination index of 90–98%. Discrimination accuracy decreased slightly with the increase of hold requirement, but generalization gradients were not displaced to the right as a function of the response requirement. Pimozide induced a dose-dependent displacement of the time-allocation gradients, but it did not affect the generalization gradients. It is concluded that rats integrate response requirements as part of the reinforcement tradeoff function, but the response cost is not integrated into the discriminative function of ICSS.