Subthreshold basis for reward-predictive persistent activity in mouse prefrontal cortex

Sub-threshold neuronal activity and the dynamical regime of cerebral cortex

Cortical neurons exhibit temporally irregular spiking patterns and heterogeneous firing rates. These features arise in model circuits operating in a 'fluctuation-driven regime', in which fluctuations in membrane potentials emerge from the network dynamics. However, it is still debated whether the cortex operates in such a regime. We evaluated the fluctuation-driven hypothesis by analyzing spiking and sub-threshold membrane potentials of neurons in the frontal cortex of mice performing a decision-making task. We showed that while standard fluctuation-driven models successfully account for spiking statistics, they fall short in capturing the heterogeneity in sub-threshold activity. This limitation is an inevitable outcome of bombarding single-compartment neurons with a large number of pre-synaptic inputs, thereby clamping the voltage of all neurons to more or less the same average voltage. To address this, we effectively incorporated dendritic morphology into the standard models. Inclusion of dendritic morphology in the neuronal models increased neuronal selectivity and reduced error trials, suggesting a functional role for dendrites during decision-making. Our work suggests that, during decision-making, cortical neurons in high-order cortical areas operate in a fluctuation-driven regime.

Learning to express reward prediction error-like dopaminergic activity requires plastic representations of time

The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference learning (TD) learning, whereby certain units signal reward prediction errors (RPE). The TD algorithm has been traditionally mapped onto the dopaminergic system, as firing properties of dopamine neurons can resemble RPEs. However, certain predictions of TD learning are inconsistent with experimental results, and previous implementations of the algorithm have made unscalable assumptions regarding stimulus-specific fixed temporal bases. We propose an alternate framework to describe dopamine signaling in the brain, FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, dopamine release is similar, but not identical to RPE, leading to predictions that contrast to those of TD. While FLEX itself is a general theoretical framework, we describe a specific, biophysically plausible implementation, the results of which are consistent with a preponderance of both existing and reanalyzed experimental data.

Distributed and specific encoding of sensory, motor, and decision information in the mouse neocortex during goal-directed behavior

Goal-directed behaviors involve coordinated activity in many cortical areas, but whether the encoding of task variables is distributed across areas or is more specifically represented in distinct areas remains unclear. Here, we compared representations of sensory, motor, and decision information in the whisker primary somatosensory cortex, medial prefrontal cortex, and tongue-jaw primary motor cortex in mice trained to lick in response to a whisker stimulus with mice that were not taught this association. Irrespective of learning, properties of the sensory stimulus were best encoded in the sensory cortex, whereas fine movement kinematics were best represented in the motor cortex. However, movement initiation and the decision to lick in response to the whisker stimulus were represented in all three areas, with decision neurons in the medial prefrontal cortex being more selective, showing minimal sensory responses in miss trials and motor responses during spontaneous licks. Our results reconcile previous studies indicating highly specific vs. highly distributed sensorimotor processing.

In Vivo Whole-Cell Recording from the Mouse Brain

Measuring the membrane potential dynamics of neurons offers a comprehensive understanding of the molecular and cellular mechanisms that form their spiking activity, thus playing a crucial role in unraveling the mechanistic processes governing brain function. Techniques for intracellular recordings of membrane potentials pioneered in the 1940s have witnessed significant advancements since their inception. Among these, whole-cell patch-clamp recording has emerged as a leading method for measuring neuronal membrane potentials due to its high stability and broad applicability ranging from cultured cells to brain slices and even behaving animals. This chapter provides a detailed protocol to acquire stable whole-cell recordings from neurons in the cerebral cortex of awake, head-restrained mice. Significant enhancements to our protocol include implanting a metal head-post using adhesive resin cement and preparing a recording pipette with a long shank for targeting deeper brain regions. This protocol, once implemented, enables whole-cell recordings up to 2.5 mM beneath the cortical surface.

Learning to Express Reward Prediction Error-like Dopaminergic Activity Requires Plastic Representations of Time

The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference (TD) reinforcement learning. The TD framework predicts that some neuronal elements should represent the reward prediction error (RPE), which means they signal the difference between the expected future rewards and the actual rewards. The prominence of the TD theory arises from the observation that firing properties of dopaminergic neurons in the ventral tegmental area appear similar to those of RPE model-neurons in TD learning. Previous implementations of TD learning assume a fixed temporal basis for each stimulus that might eventually predict a reward. Here we show that such a fixed temporal basis is implausible and that certain predictions of TD learning are inconsistent with experiments. We propose instead an alternative theoretical framework, coined FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, feature specific representations of time are learned, allowing for neural representations of stimuli to adjust their timing and relation to rewards in an online manner. In FLEX dopamine acts as an instructive signal which helps build temporal models of the environment. FLEX is a general theoretical framework that has many possible biophysical implementations. In order to show that FLEX is a feasible approach, we present a specific biophysically plausible model which implements the principles of FLEX. We show that this implementation can account for various reinforcement learning paradigms, and that its results and predictions are consistent with a preponderance of both existing and reanalyzed experimental data.

Neural Algorithms and Circuits for Motor Planning

The brain plans and executes volitional movements. The underlying patterns of neural population activity have been explored in the context of movements of the eyes, limbs, tongue, and head in nonhuman primates and rodents. How do networks of neurons produce the slow neural dynamics that prepare specific movements and the fast dynamics that ultimately initiate these movements? Recent work exploits rapid and calibrated perturbations of neural activity to test specific dynamical systems models that are capable of producing the observed neural activity. These joint experimental and computational studies show that cortical dynamics during motor planning reflect fixed points of neural activity (attractors). Subcortical control signals reshape and move attractors over multiple timescales, causing commitment to specific actions and rapid transitions to movement execution. Experiments in rodents are beginning to reveal how these algorithms are implemented at the level of brain-wide neural circuits.

Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making

Nervous systems evolved to effectively navigate the dynamics of the environment to achieve their goals. One framework used to study this fundamental problem arose in the study of learning and decision-making. In this framework, the demands of effective behavior require slow dynamics-on the scale of seconds to minutes-of networks of neurons. Here, we review the phenomena and mechanisms involved. Using vignettes from a few species and areas of the nervous system, we view neuromodulators as key substrates for temporal scaling of neuronal dynamics. Expected final online publication date for the Annual Review of Neuroscience, Volume 45 is July 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.