Does phasic dopamine release cause policy updates?

Phasic dopamine activity is believed to both encode reward-prediction errors (RPEs) and to cause the adaptations that these errors engender. If so, a rat working for optogenetic stimulation of dopamine neurons will repeatedly update its policy and/or action values, thus iteratively increasing its work rate. Here, we challenge this view by demonstrating stable, non-maximal work rates in the face of repeated optogenetic stimulation of midbrain dopamine neurons. Furthermore, we show that rats learn to discriminate between world states distinguished only by their history of dopamine activation. Comparison of these results to reinforcement learning simulations suggests that the induced dopamine transients acted more as rewards than RPEs. However, pursuit of dopaminergic stimulation drifted upwards over a time scale of days and weeks, despite its stability within trials. To reconcile the results with prior findings, we consider multiple roles for dopamine signalling.

Longitudinal CSF proteomics identifies NPTX2 as a prognostic biomarker of Alzheimer's disease

Introduction: Biomarkers that reflect pathologic processes affecting neuronal function during preclinical and early stages of Alzheimer's disease (AD) are needed to aid drug development.

Methods: A targeted, stable isotope, quantitative mass spectrometry‐based investigation of longitudinal changes in concentrations of previously identified candidate biomarkers was performed in cerebrospinal fluid (CSF) of Alzheimer's Disease Neuroimaging Initiative participants who were classified as cognitively normal (CN; n = 76) or with mild cognitive impairment (MCI; n = 111) at baseline.

Results: Of the candidate biomarkers, the CSF concentration of neuronal pentraxin 2 (NPTX2), a protein involved in synaptic function, exhibited rates of change that were significantly different between three comparison groups (i.e., CN vs. MCI participants; AD pathology positive vs. negative defined by phosphorylated tau181/amyloid beta1‐42 ratio; and clinical progressors vs. non‐progressors). The rate of change of NPTX2 also significantly correlated with declining cognition.

Discussion: CSF NPTX2 concentration is a strong prognostic biomarker candidate of accelerated cognitive decline with potential use as a therapeutic target.

Discovery of a putative blood-based protein signature associated with response to ALK tyrosine kinase inhibition

Background

ALK tyrosine kinase inhibition has become a mainstay in the clinical management of ALK fusion positive NSCLC patients. Although ALK mutations can reliably predict the likelihood of response to ALK tyrosine kinase inhibitors (TKIs) such as crizotinib, they cannot reliably predict response duration or intrinsic/extrinsic therapeutic resistance. To further refine the application of personalized medicine in this indication, this study aimed to identify prognostic proteomic biomarkers in ALK fusion positive NSCLC patients to crizotinib.

Methods

Twenty-four patients with advanced NSCLC harboring ALK fusion were administered crizotinib in a phase IV trial which included blood sampling prior to treatment. Targeted proteomics of 327 proteins using MRM-MS was used to measure plasma levels at baseline (including pre-treatment and early treatment blood samples) and assess potential clinical association.

Results

Patients were categorized by duration of response: long-term responders [PFS ≥ 24 months (n = 7)], normal responders [3 < PFS < 24 months (n = 10)] and poor responders [PFS ≤ 3 months (n = 5)]. Several proteins were identified as differentially expressed between long-term responders and poor responders, including DPP4, KIT and LUM. Next, using machine learning algorithms, we evaluated the classification potential of 40 proteins. Finally, by integrating the different analytic methods, we selected 22 proteins as potential candidates for a blood-based prognostic signature of response to crizotinib in NSCLC patients harboring ALK fusion.

Conclusion

In conjunction with ALK mutation, the expression of this proteomic signature may represent a liquid biopsy-based marker of long-term response to crizotinib in NSCLC. Expanding the utility of prognostic biomarkers of response duration could influence choice of therapy, therapeutic sequencing, and potentially the need for alternative or combination therapy.Trial registration ClinicalTrials.gov, NCT02041468. Registered 22 January 2014, https://clinicaltrials.gov/ct2/show/NCT02041468?term=NCT02041468&rank=1.

Learning to use past evidence in a sophisticated world model

Humans and other animals are able to discover underlying statistical structure in their environments and exploit it to achieve efficient and effective performance. However, such structure is often difficult to learn and use because it is obscure, involving long-range temporal dependencies. Here, we analysed behavioural data from an extended experiment with rats, showing that the subjects learned the underlying statistical structure, albeit suffering at times from immediate inferential imperfections as to their current state within it. We accounted for their behaviour using a Hidden Markov Model, in which recent observations are integrated with evidence from the past. We found that over the course of training, subjects came to track their progress through the task more accurately, a change that our model largely attributed to improved integration of past evidence. This learning reflected the structure of the task, decreasing reliance on recent observations, which were potentially misleading.

Humans and other animals possess the remarkable ability to find and exploit patterns and structures in their experience of a complex and varied world. However, such structures are often temporally extended and latent or hidden, being only partially correlated with immediate observations of the world. This makes it essential to integrate current and historical information, and creates a challenging statistical and computational problem. Here, we examine the behaviour of rats facing a version of this challenge posed by a brain-stimulation reward task. We find that subjects learned the general structure of the task, but struggled when immediate observations were misleading. We captured this behaviour with a model in which subjects integrated evidence from recent observations together with evidence from the past. The subjects’ performance improved markedly over successive sessions, allowing them to overcome misleading observations. According to our model, this was made possible by more effective usage of past evidence to better determine the true state of the world.

Aging impairs deliberation and behavioral flexibility in inter-temporal choice

Inter-temporal choice depends on multiple, interacting systems, some of which may be compromised with age. Some of these systems may be responsible for ongoing trial-by-trial choice strategies. Some may represent the consequences of action. Some may be necessary for the coupling between anticipated consequences and strategies currently in use, flexibly guiding behavior. When faced with a difficult decision, rats will orient back and forth, a behavior termed "vicarious trial and error" (VTE). Recent experiments have linked the occurrence of VTE to hippocampal search processes and behavioral flexibility. We tested 5 month (n = 6), 9 month (n = 8) and over-27 month-old (n = 10) rats on a Spatial Adjusting Delay Discounting task to examine how aging impacted lap-by-lap strategies and VTE during inter-temporal choice. Rats chose between spatially separated food goals that provided a smaller-sooner or larger-later reward. On each lap, the delay to the larger-later reward was adjusted as a function of the rat's decisions, increasing by 1 s after delayed-side choices and decreasing by 1 s after non-delayed side choices. The strategies that aged rats used differed from those used in young and adult rats. Moreover, aged rats produced reliably more VTE behaviors, for protracted periods of time, uncoupled from behavioral flexibility.

The effect of probability discounting on reward seeking: a three-dimensional perspective

Rats will work for electrical stimulation of the medial forebrain bundle. The rewarding effect arises from the volleys of action potentials fired by the stimulation and subsequent spatio-temporal integration of their post-synpatic impact. The proportion of time allocated to self-stimulation depends on the intensity of the rewarding effect as well as on other key determinants of decision-making, such as subjective opportunity costs and reward probability. We have proposed that a 3D model relating time allocation to the intensity and cost of reward can distinguish manipulations acting prior to the output of the spatio-temporal integrator from those acting at or beyond it. Here, we test this proposition by varying reward probability, a variable that influences the computation of payoff in the 3D model downstream from the output of the integrator. On riskless trials, reward was delivered on every occasion that the rat held down the lever for a cumulative duration called the “price,” whereas on risky trials, reward was delivered with probability 0.75 or 0.50. According to the model, the 3D structure relating time allocation to reward intensity and price is shifted leftward along the price axis by reductions in reward probability; the magnitude of the shift estimates the change in subjective probability. The predictions were borne out: reducing reward probability shifted the 3D structure systematically along the price axis while producing only small, inconsistent displacements along the pulse-frequency axis. The results confirm that the model can accurately distinguish manipulations acting at or beyond the spatio-temporal integrator and strengthen the conclusions of previous studies showing similar shifts following dopaminergic manipulations. Subjective and objective reward probabilities appeared indistinguishable over the range of 0.5 ≤ p ≤ 1.0.

Validation and extension of the reward-mountain model

The reward-mountain model relates the vigor of reward seeking to the strength and cost of reward. Application of this model provides information about the stage of processing at which manipulations such as drug administration, lesions, deprivation states, and optogenetic interventions act to alter reward seeking. The model has been updated by incorporation of new information about frequency following in the directly stimulated neurons responsible for brain stimulation reward and about the function that maps objective opportunity costs into subjective ones. The behavioral methods for applying the model have been updated and improved as well. To assess the impact of these changes, two related predictions of the model that were supported by earlier work have been retested: (1) altering the duration of rewarding brain stimulation should change the pulse frequency required to produce a reward of half-maximal intensity, and (2) this manipulation should not change the opportunity cost at which half-maximal performance is directed at earning a maximally intense reward. Prediction 1 was supported in all six subjects, but prediction 2 was supported in only three. The latter finding is interpreted to reflect recruitment, at some stimulation sites, of a heterogeneous reward substrate comprising dual, parallel circuits that integrate the stimulation-induced neural signals.

At what stage of neural processing does cocaine act to boost pursuit of rewards?

Dopamine-containing neurons have been implicated in reward and decision making. One element of the supporting evidence is that cocaine, like other drugs that increase dopaminergic neurotransmission, powerfully potentiates reward seeking. We analyze this phenomenon from a novel perspective, introducing a new conceptual framework and new methodology for determining the stage(s) of neural processing at which drugs, lesions and physiological manipulations act to influence reward-seeking behavior. Cocaine strongly boosts the proclivity of rats to work for rewarding electrical brain stimulation. We show that the conventional conceptual framework and methods do not distinguish between three conflicting accounts of how the drug produces this effect: increased sensitivity of brain reward circuitry, increased gain, or decreased subjective reward costs. Sensitivity determines the stimulation strength required to produce a reward of a given intensity (a measure analogous to the K_M of an enzyme) whereas gain determines the maximum intensity attainable (a measure analogous to the v_max of an enzyme-catalyzed reaction). To distinguish sensitivity changes from the other determinants, we measured and modeled reward seeking as a function of both stimulation strength and opportunity cost. The principal effect of cocaine was a two-fourfold increase in willingness to pay for the electrical reward, an effect consistent with increased gain or decreased subjective cost. This finding challenges the long-standing view that cocaine increases the sensitivity of brain reward circuitry. We discuss the implications of the results and the analytic approach for theories of how dopaminergic neurons and other diffuse modulatory brain systems contribute to reward pursuit, and we explore the implications of the conceptual framework for the study of natural rewards, drug reward, and mood.