Reinforcement learning links spontaneous cortical dopamine impulses to reward

Closed-loop experiments and brain machine interfaces with multiphoton microscopy

In the field of neuroscience, the importance of constructing closed-loop experimental systems has increased in conjunction with technological advances in measuring and controlling neural activity in live animals. We provide an overview of recent technological advances in the field, focusing on closed-loop experimental systems where multiphoton microscopy-the only method capable of recording and controlling targeted population activity of neurons at a single-cell resolution in vivo-works through real-time feedback. Specifically, we present some examples of brain machine interfaces (BMIs) using in vivo two-photon calcium imaging and discuss applications of two-photon optogenetic stimulation and adaptive optics to real-time BMIs. We also consider conditions for realizing future optical BMIs at the synaptic level, and their possible roles in understanding the computational principles of the brain.

Genetically encoded sensors for in vivo detection of neurochemicals relevant to depression

Depressive disorders are a common and debilitating form of mental illness with significant impacts on individuals and society. Despite the high prevalence, the underlying causes and mechanisms of depressive disorders are still poorly understood. Neurochemical systems, including serotonin, norepinephrine, and dopamine, have been implicated in the development and perpetuation of depressive symptoms. Current treatments for depression target these neuromodulator systems, but there is a need for a better understanding of their role in order to develop more effective treatments. Monitoring neurochemical dynamics during depressive symptoms is crucial for gaining a better a understanding of their involvement in depressive disorders. Genetically encoded sensors have emerged recently that offer high spatial-temporal resolution and the ability to monitor neurochemical dynamics in real time. This review explores the neurochemical systems involved in depression and discusses the applications and limitations of current monitoring tools for neurochemical dynamics. It also highlights the potential of genetically encoded sensors for better characterizing neurochemical dynamics in depression-related behaviors. Furthermore, potential improvements to current sensors are discussed in order to meet the requirements of depression research.

Past, present, and future of tools for dopamine detection

Dopamine (DA) is a critical neuromodulator involved in various brain functions. To understand how DA regulates neural circuits and behaviors in the physiological and pathological conditions, it is essential to have tools that enable the direct detection of DA dynamics in vivo. Recently, genetically encoded DA sensors based on G protein-coupled receptors revolutionized this field, as it allows us to track in vivo DA dynamic with unprecedented spatial-temporal resolution, high molecular specificity, and sub-second kinetics. In this review, we first summarize traditional DA detection methods. Then we focus on the development of genetically encoded DA sensors and feature its significance to understanding dopaminergic neuromodulation across diverse behaviors and species. Finally, we present our perspectives about the future direction of the next-generation DA sensors and extend their potential applications. Overall, this review offers a comprehensive perspective on the past, present, and future of DA detection tools, with important implications for the study of DA functions in health and disease.

Spontaneous behaviour is structured by reinforcement without explicit reward

Spontaneous animal behaviour is built from action modules that are concatenated by the brain into sequences1,2. However, the neural mechanisms that guide the composition of naturalistic, self-motivated behaviour remain unknown. Here we show that dopamine systematically fluctuates in the dorsolateral striatum (DLS) as mice spontaneously express sub-second behavioural modules, despite the absence of task structure, sensory cues or exogenous reward. Photometric recordings and calibrated closed-loop optogenetic manipulations during open field behaviour demonstrate that DLS dopamine fluctuations increase sequence variation over seconds, reinforce the use of associated behavioural modules over minutes, and modulate the vigour with which modules are expressed, without directly influencing movement initiation or moment-to-moment kinematics. Although the reinforcing effects of optogenetic DLS dopamine manipulations vary across behavioural modules and individual mice, these differences are well predicted by observed variation in the relationships between endogenous dopamine and module use. Consistent with the possibility that DLS dopamine fluctuations act as a teaching signal, mice build sequences during exploration as if to maximize dopamine. Together, these findings suggest a model in which the same circuits and computations that govern action choices in structured tasks have a key role in sculpting the content of unconstrained, high-dimensional, spontaneous behaviour.

Multiscale imaging informs translational mouse modeling of neurological disease

Multiscale neurophysiology reveals that simple motor actions are associated with changes in neuronal firing in virtually every brain region studied. Accordingly, the assessment of focal pathology such as stroke or progressive neurodegenerative diseases must also extend widely across brain areas. To derive mechanistic information through imaging, multiple resolution scales and multimodal factors must be included, such as the structure and function of specific neurons and glial cells and the dynamics of specific neurotransmitters. Emerging multiscale methods in preclinical animal studies that span micro- to macroscale examinations fill this gap, allowing a circuit-based understanding of pathophysiological mechanisms. Combined with high-performance computation and open-source data repositories, these emerging multiscale and large field-of-view techniques include live functional ultrasound, multi- and single-photon wide-scale light microscopy, video-based miniscopes, and tissue-penetrating fiber photometry, as well as variants of post-mortem expansion microscopy. We present these technologies and outline use cases and data pipelines to uncover new knowledge within animal models of stroke, Alzheimer's disease, and movement disorders.

Probing Neuropeptide Volume Transmission In Vivo by Simultaneous Near-Infrared Light-Triggered Release and Optical Sensing

Neuropeptides are abundant signaling molecules in the central nervous system. Yet remarkably little is known about their spatiotemporal spread and biological activity. Here, we developed an integrated optical approach using Plasmonic nAnovesicles and cell-based neurotransmitter fluorescent engineered reporter (CNiFER), or PACE, to probe neuropeptide signaling in the mouse neocortex. Small volumes (fL to pL) of exogenously supplied somatostatin-14 (SST) can be rapidly released under near-infrared light stimulation from nanovesicles implanted in the brain and detected by SST2 CNiFERs with nM sensitivity. Our measurements reveal reduced but synchronized SST transmission within 130 μm, and markedly smaller and delayed transmission at longer distances. These measurements enabled a quantitative estimation of the SST loss rate due to peptide degradation and binding. PACE offers a new tool for determining the spatiotemporal scales of neuropeptide volume transmission and signaling in the brain.

Neurofeedback through the lens of reinforcement learning

Despite decades of experimental and clinical practice, the neuropsychological mechanisms underlying neurofeedback (NF) training remain obscure. NF is a unique form of reinforcement learning (RL) task, during which participants are provided with rewarding feedback regarding desired changes in neural patterns. However, key RL considerations - including choices during practice, prediction errors, credit-assignment problems, or the exploration-exploitation tradeoff - have infrequently been considered in the context of NF. We offer an RL-based framework for NF, describing different internal states, actions, and rewards in common NF protocols, thus fashioning new proposals for characterizing, predicting, and hastening the course of learning. In this way we hope to advance current understanding of neural regulation via NF, and ultimately to promote its effectiveness, personalization, and clinical utility.

Three Water Restriction Schedules Used in Rodent Behavioral Tasks Transiently Impair Growth and Differentially Evoke a Stress Hormone Response without Causing Dehydration

Water restriction is commonly used to motivate rodents to perform behavioral tasks; however, its effects on hydration and stress hormone levels are unknown. Here, we report daily body weight and bi-weekly packed red blood cell volume and corticosterone (CORT) in adult male rats across 80 days for three commonly used water restriction schedules. We also assessed renal adaptation to water restriction using postmortem histologic evaluation of renal medulla. A control group received ad libitum water. After one week of water restriction, rats on all restriction schedules resumed similar levels of growth relative to the control group. Normal hydration was observed, and water restriction did not drive renal adaptation. An intermittent restriction schedule was associated with an increase in CORT relative to the control group. However, intermittent restriction evokes a stress response which could affect behavioral and neurobiological results. Our results also suggest that stable motivation in behavioral tasks may only be achieved after one week of restriction.