Learning differentially shapes prefrontal and hippocampal activity during classical conditioning

Reward timescale controls the rate of behavioural and dopaminergic learning

Learning the causes of rewards is necessary for survival. Thus, it is critical to understand the mechanisms of such a vital biological process. Cue-reward learning is controlled by mesolimbic dopamine and improves with spacing of cue-reward pairings. However, whether a mathematical rule governs such improvements in learning rate, and if so, whether a unifying mechanism captures this rule and dopamine dynamics during learning remain unknown. Here, we investigate the behavioral, algorithmic, and dopaminergic mechanisms governing cue-reward learning rate. Across a range of conditions in mice, we show a strong, mathematically proportional relationship between both behavioral and dopaminergic learning rates and the duration between rewards. Due to this relationship, removing up to 19 out of 20 cue-reward pairings over a fixed duration has no influence on overall learning. These findings are explained by a dopamine-based model of retrospective learning, thereby providing a unified account of the biological mechanisms of learning.

Entorhinal cortex represents task-relevant remote locations independent of CA1

Neurons can collectively represent the current sensory experience while an animal is exploring its environment or remote experiences while the animal is immobile. These remote representations can reflect learned associations1-3 and be required for learning4. Neurons in the medial entorhinal cortex (MEC) reflect the animal's current location during movement5, but little is known about what MEC neurons collectively represent during immobility. Here, we recorded thousands of neurons in superficial MEC and dorsal CA1 as mice learned to associate two pairs of rewarded locations. We found that during immobility, the MEC neural population frequently represented positions far from the animal's location, which we defined as 'non-local coding'. Cells with spatial firing fields at remote locations drove non-local coding, even as cells representing the current position remained active. While MEC non-local coding has been reported during sharp-wave ripples in downstream CA16, we observed non-local coding more often outside of ripples. In fact, CA1 activity was less coordinated with MEC during non-local coding. We further observed that non-local coding was pertinent to the task, as MEC preferentially represented remote task-relevant locations at appropriate times, while rarely representing task-irrelevant locations. Together, this work raises the possibility that MEC non-local coding could strengthen associations between locations independently from CA1.

Prefrontal neuronal ensembles link prior knowledge with novel actions during flexible action selection

We make decisions based on currently perceivable information or an internal model of the environment. The medial prefrontal cortex (mPFC) and its interaction with the hippocampus have been implicated in the latter, model-based decision-making; however, the underlying computational properties remain incompletely understood. We have examined mPFC spiking and hippocampal oscillatory activity while rats flexibly select new actions using a known associative structure of environmental cues and outcomes. During action selection, the mPFC reinstates representations of the associative structure. These awake reactivation events are accompanied by synchronous firings among neurons coding the associative structure and those coding actions. Moreover, their functional coupling is strengthened upon the reactivation events leading to adaptive actions. In contrast, only cue-coding neurons improve functional coupling during hippocampal sharp wave ripples. Thus, the lack of direct experience disconnects the mPFC from the hippocampus to independently form self-organized neuronal ensemble dynamics linking prior knowledge with novel actions.

Aversive stimulus-tuned responses in the CA1 of the dorsal hippocampus

Throughout life animals inevitably encounter unforeseen threatening events. Activity of principal cells in the hippocampus is tuned for locations and for salient stimuli in the animals' environment thus forming a map known to be pivotal for guiding behavior. Here, we explored if a code of threatening stimuli exists in the CA1 region of the dorsal hippocampus of mice by recording neuronal response to aversive stimuli delivered at changing locations. We have discovered a rapidly emerging, location independent response to innoxious aversive stimuli composed of the coordinated activation of subgroups of pyramidal cells and connected interneurons. Activated pyramidal cells had higher basal firing rate, more probably participated in ripples, targeted more interneurons than place cells and many of them lacked place fields. We also detected aversive stimulus-coupled assemblies dominated by the activated neurons. Notably, these assemblies could be observed even before the delivery of the first aversive event. Finally, we uncovered the systematic shift of the spatial code from the aversive to, surprisingly, the reward location during the fearful stimulus. Our results uncovered components of the dorsal CA1 circuit possibly key for re-sculpting the spatial map in response to abrupt aversive events.

Hippocampo-cortical circuits for selective memory encoding, routing, and replay

Traditionally considered a homogeneous cell type, hippocampal pyramidal cells have been recently shown to be highly diverse. However, how this cellular diversity relates to the different hippocampal network computations that support memory-guided behavior is not yet known. We show that the anatomical identity of pyramidal cells is a major organizing principle of CA1 assembly dynamics, the emergence of memory replay, and cortical projection patterns in rats. Segregated pyramidal cell subpopulations encoded trajectory and choice-specific information or tracked changes in reward configuration respectively, and their activity was selectively read out by different cortical targets. Furthermore, distinct hippocampo-cortical assemblies coordinated the reactivation of complementary memory representations. These findings reveal the existence of specialized hippocampo-cortical subcircuits and provide a cellular mechanism that supports the computational flexibility and memory capacities of these structures.

Resolving the prefrontal mechanisms of adaptive cognitive behaviors: A cross-species perspective

The prefrontal cortex (PFC) enables a staggering variety of complex behaviors, such as planning actions, solving problems, and adapting to new situations according to external information and internal states. These higher-order abilities, collectively defined as adaptive cognitive behavior, require cellular ensembles that coordinate the tradeoff between the stability and flexibility of neural representations. While the mechanisms underlying the function of cellular ensembles are still unclear, recent experimental and theoretical studies suggest that temporal coordination dynamically binds prefrontal neurons into functional ensembles. A so far largely separate stream of research has investigated the prefrontal efferent and afferent connectivity. These two research streams have recently converged on the hypothesis that prefrontal connectivity patterns influence ensemble formation and the function of neurons within ensembles. Here, we propose a unitary concept that, leveraging a cross-species definition of prefrontal regions, explains how prefrontal ensembles adaptively regulate and efficiently coordinate multiple processes in distinct cognitive behaviors.

Sustained Activity of Hippocampal Parvalbumin-Expressing Interneurons Supports Trace Eyeblink Conditioning in Mice

Although recent studies have revealed an involvement of hippocampal interneurons in learning the association among time-separated events, its underlying cellular mechanisms remained not fully clarified. Here, we combined multichannel recording and optogenetics to elucidate how the hippocampal parvalbumin-expressing interneurons (PV-INs) support associative learning. To address this issue, we trained the mice (both sexes) to learn hippocampus-dependent trace eyeblink conditioning (tEBC) in which they associated a light flash conditioned stimulus (CS) with a corneal air puff unconditioned stimuli (US) separated by a 250 ms time interval. We found that the hippocampal PV-INs exhibited learning-associated sustained activity at the early stage of tEBC acquisition. Moreover, the PV-IN sustained activity was positively correlated with the occurrence of conditioned eyeblink responses at the early learning stage. Suppression of the PV-IN sustained activity impaired the acquisition of tEBC, whereas the PV-IN activity suppression had no effect on the acquisition of delay eyeblink conditioning, a hippocampus-independent learning task. Learning-associated augmentation in the excitatory pyramidal cell-to-PVIN drive may contribute to the formation of PV-IN sustained activity. Suppression of the PV-IN sustained activity disrupted hippocampal gamma but not theta band oscillation during the CS-US interval period. Gamma frequency (40 Hz) activation of the PV-INs during the CS-US interval period facilitated the acquisition of tEBC. Our current findings highlight the involvement of hippocampal PV-INs in tEBC acquisition and reveal insights into the PV-IN activity kinetics which are of key importance for the hippocampal involvement in associative learning.SIGNIFICANCE STATEMENT The cellular mechanisms underlying associative learning have not been fully clarified. Previous studies focused on the involvement of hippocampal pyramidal cells in associative learning, whereas the activity and function of hippocampal interneurons were largely neglected. We herein demonstrated the hippocampal PV-INs exhibited learning-associated sustained activity, which was required for the acquisition of tEBC. Furthermore, we showed evidence that the PV-IN sustained activity might have arisen from the learning-associated augmentation in excitatory pyramidal cell-to-PVIN drive and contributed to learning-associated augmentation in gamma band oscillation during tEBC acquisition. Our findings provide more mechanistic understanding of the cellular mechanisms underlying the hippocampal involvement in associative learning.

The hearing hippocampus

The hippocampus has a well-established role in spatial and episodic memory but a broader function has been proposed including aspects of perception and relational processing. Neural bases of sound analysis have been described in the pathway to auditory cortex, but wider networks supporting auditory cognition are still being established. We review what is known about the role of the hippocampus in processing auditory information, and how the hippocampus itself is shaped by sound. In examining imaging, recording, and lesion studies in species from rodents to humans, we uncover a hierarchy of hippocampal responses to sound including during passive exposure, active listening, and the learning of associations between sounds and other stimuli. We describe how the hippocampus' connectivity and computational architecture allow it to track and manipulate auditory information - whether in the form of speech, music, or environmental, emotional, or phantom sounds. Functional and structural correlates of auditory experience are also identified. The extent of auditory-hippocampal interactions is consistent with the view that the hippocampus makes broad contributions to perception and cognition, beyond spatial and episodic memory. More deeply understanding these interactions may unlock applications including entraining hippocampal rhythms to support cognition, and intervening in links between hearing loss and dementia.

Contingent Amygdala Inputs Trigger Heterosynaptic LTP at Hippocampus-To-Accumbens Synapses

The nucleus accumbens shell (NAcSh) is a key brain region where environmental cues acquire incentive salience to reinforce motivated behaviors. Principal medium spiny neurons (MSNs) in the NAcSh receive extensive glutamatergic projections from limbic regions, among which, the ventral hippocampus (vH) transmits information enriched in contextual cues, and the basolateral amygdala (BLA) encodes real-time arousing states. The vH and BLA project convergently to NAcSh MSNs, both activated in a time-locked manner on a cue-conditioned motivational action. In brain slices prepared from male and female mice, we show that co-activation of the two projections induces long-term potentiation (LTP) at vH-to-NAcSh synapses without affecting BLA-to-NAcSh synapses, revealing a heterosynaptic mechanism through which BLA signals persistently increase the temporally contingent vH-to-NAcSh transmission. Furthermore, this LTP is more prominent in dopamine D1 receptor-expressing (D1) MSNs than D2 MSNs and can be prevented by inhibition of either D1 receptors or dopaminergic terminals in NAcSh. This heterosynaptic LTP may provide a dopamine-guided mechanism through which vH-encoded cue inputs that are contingent to BLA activation acquire increased circuit representation to reinforce behavior.SIGNIFICANCE STATEMENT In motivated behaviors, environmental cues associated with arousing stimuli acquire increased incentive salience, processes mediated in part by the nucleus accumbens (NAc). NAc principal neurons receive glutamatergic projections from the ventral hippocampus (vH) and basolateral amygdala (BLA), which transmit information encoding contextual cues and affective states, respectively. Our results show that co-activation of the two projections induces long-term potentiation (LTP) at vH-to-NAc synapses without affecting BLA-to-NAc synapses, revealing a heterosynaptic mechanism through which BLA signals potentiate the temporally contingent vH-to-NAc transmission. Furthermore, this LTP is prevented by inhibition of either D1 receptors or dopaminergic axons. This heterosynaptic LTP may provide a dopamine-guided mechanism through which vH-encoded cue inputs that are contingent to BLA activation acquire increased circuit representation to reinforce behavior.

Prefrontal projections to the nucleus reuniens signal behavioral relevance of stimuli during associative learning

The nucleus reuniens (RE) is necessary for memories dependent on the interaction between the medial prefrontal cortex (mPFC) and hippocampus (HPC). One example is trace eyeblink conditioning, in which the mPFC exhibits differential activity to neutral conditioned stimuli (CS) depending on their contingency with an aversive unconditioned stimulus (US). To test if this relevancy signal is routed to the RE, we photometrically recorded mPFC axon terminals within the RE and tracked their changes with learning. As a comparison, we measured prefrontal terminal activity in the mediodorsal thalamus (MD), which lacks connectivity with the HPC. In naïve male rats, prefrontal terminals within the RE were not strongly activated by tone or light. As the rats associated one of the stimuli (CS+) with the US, terminals gradually increased their response to the CS+ but not the other stimulus (CS-). In contrast, stimulus-evoked responses of prefrontal terminals within the MD were strong even before conditioning. They also became augmented only to the CS+ in the first conditioning session; however, the degree of activity differentiation did not improve with learning. These findings suggest that associative learning selectively increased mPFC output to the RE, signaling the behavioral relevance of sensory stimuli.

Combining Cortical Voltage Imaging and Hippocampal Electrophysiology for Investigating Global, Multi-Timescale Activity Interactions in the Brain

A new generation of optogenetic tools for analyzing neural activity has been contributing to the elucidation of classical open questions in neuroscience. Specifically, voltage imaging technologies using enhanced genetically encoded voltage indicators have been increasingly used to observe the dynamics of large circuits at the mesoscale. Here, we describe how to combine cortical wide-field voltage imaging with hippocampal electrophysiology in awake, behaving mice. Furthermore, we highlight how this method can be useful for different possible investigations, using the characterization of hippocampal-neocortical interactions as a case study.

Learning differentially shapes prefrontal and hippocampal activity during classical conditioning

The ability to use sensory cues to inform goal-directed actions is a critical component of behavior. To study how sounds guide anticipatory licking during classical conditioning, we employed high-density electrophysiological recordings from the hippocampal CA1 area and the prefrontal cortex (PFC) in mice. CA1 and PFC neurons undergo distinct learning-dependent changes at the single-cell level and maintain representations of cue identity at the population level. In addition, reactivation of task-related neuronal assemblies during hippocampal awake Sharp-Wave Ripples (aSWRs) changed within individual sessions in CA1 and over the course of multiple sessions in PFC. Despite both areas being highly engaged and synchronized during the task, we found no evidence for coordinated single cell or assembly activity during conditioning trials or aSWR. Taken together, our findings support the notion that persistent firing and reactivation of task-related neural activity patterns in CA1 and PFC support learning during classical conditioning.