Rapid Integration of Artificial Sensory Feedback during Operant Conditioning of Motor Cortex Neurons

Closed-loop experiments and brain machine interfaces with multiphoton microscopy

In the field of neuroscience, the importance of constructing closed-loop experimental systems has increased in conjunction with technological advances in measuring and controlling neural activity in live animals. We provide an overview of recent technological advances in the field, focusing on closed-loop experimental systems where multiphoton microscopy-the only method capable of recording and controlling targeted population activity of neurons at a single-cell resolution in vivo-works through real-time feedback. Specifically, we present some examples of brain machine interfaces (BMIs) using in vivo two-photon calcium imaging and discuss applications of two-photon optogenetic stimulation and adaptive optics to real-time BMIs. We also consider conditions for realizing future optical BMIs at the synaptic level, and their possible roles in understanding the computational principles of the brain.

A combinatorial neural code for long-term motor memory

Motor skill repertoire can be stably retained over long periods, but the neural mechanism underlying stable memory storage remains poorly understood. Moreover, it is unknown how existing motor memories are maintained as new motor skills are continuously acquired. Here we tracked neural representation of learned actions throughout a significant portion of a mouse's lifespan, and we show that learned actions are stably retained in motor memory in combination with context, which protects existing memories from erasure during new motor learning. We used automated home-cage training to establish a continual learning paradigm in which mice learned to perform directional licking in different task contexts. We combined this paradigm with chronic two-photon imaging of motor cortex activity for up to 6 months. Within the same task context, activity driving directional licking was stable over time with little representational drift. When learning new task contexts, new preparatory activity emerged to drive the same licking actions. Learning created parallel new motor memories while retaining the previous memories. Re-learning to make the same actions in the previous task context re-activated the previous preparatory activity, even months later. At the same time, continual learning of new task contexts kept creating new preparatory activity patterns. Context-specific memories, as we observed in the motor system, may provide a solution for stable memory storage throughout continual learning. Learning in new contexts produces parallel new representations instead of modifying existing representations, thus protecting existing motor repertoire from erasure.

Coexistence of state, choice, and sensory integration coding in barrel cortex LII/III

During perceptually guided decisions, correlates of choice are found as upstream as in the primary sensory areas. However, how well these choice signals align with early sensory representations, a prerequisite for their interpretation as feedforward substrates of perception, remains an open question. We designed a two alternative forced choice task (2AFC) in which male mice compared stimulation frequencies applied to two adjacent vibrissae. The optogenetic silencing of individual columns in the primary somatosensory cortex (wS1) resulted in predicted shifts of psychometric functions, demonstrating that perception depends on focal, early sensory representations. Functional imaging of layer II/III single neurons revealed mixed coding of stimuli, choices and engagement in the task. Neurons with multi-whisker suppression display improved sensory discrimination and had their activity increased during engagement in the task, enhancing selectively representation of the signals relevant to solving the task. From trial to trial, representation of stimuli and choice varied substantially, but mostly orthogonally to each other, suggesting that perceptual variability does not originate from wS1 fluctuations but rather from downstream areas. Together, our results highlight the role of primary sensory areas in forming a reliable sensory substrate that could be used for flexible downstream decision processes.

Shining Light in Mechanobiology: Optical Tweezers, Scissors, and Beyond

Mechanobiology helps us to decipher cell and tissue functions by looking at changes in their mechanical properties that contribute to development, cell differentiation, physiology, and disease. Mechanobiology sits at the interface of biology, physics and engineering. One of the key technologies that enables characterization of properties of cells and tissue is microscopy. Combining microscopy with other quantitative measurement techniques such as optical tweezers and scissors, gives a very powerful tool for unraveling the intricacies of mechanobiology enabling measurement of forces, torques and displacements at play. We review the field of some light based studies of mechanobiology and optical detection of signal transduction ranging from optical micromanipulation-optical tweezers and scissors, advanced fluorescence techniques and optogenentics. In the current perspective paper, we concentrate our efforts on elucidating interesting measurements of forces, torques, positions, viscoelastic properties, and optogenetics inside and outside a cell attained when using structured light in combination with optical tweezers and scissors. We give perspective on the field concentrating on the use of structured light in imaging in combination with tweezers and scissors pointing out how novel developments in quantum imaging in combination with tweezers and scissors can bring to this fast growing field.

Brain-machine interface learning is facilitated by specific patterning of distributed cortical feedback

Neuroprosthetics offer great hope for motor-impaired patients. One obstacle is that fine motor control requires near-instantaneous, rich somatosensory feedback. Such distributed feedback may be recreated in a brain-machine interface using distributed artificial stimulation across the cortical surface. Here, we hypothesized that neuronal stimulation must be contiguous in its spatiotemporal dynamics to be efficiently integrated by sensorimotor circuits. Using a closed-loop brain-machine interface, we trained head-fixed mice to control a virtual cursor by modulating the activity of motor cortex neurons. We provided artificial feedback in real time with distributed optogenetic stimulation patterns in the primary somatosensory cortex. Mice developed a specific motor strategy and succeeded to learn the task only when the optogenetic feedback pattern was spatially and temporally contiguous while it moved across the topography of the somatosensory cortex. These results reveal spatiotemporal properties of the sensorimotor cortical integration that set constraints on the design of neuroprosthetics.

Neural Plasticity in Sensorimotor Brain-Machine Interfaces

Brain-machine interfaces (BMIs) aim to treat sensorimotor neurological disorders by creating artificial motor and/or sensory pathways. Introducing artificial pathways creates new relationships between sensory input and motor output, which the brain must learn to gain dexterous control. This review highlights the role of learning in BMIs to restore movement and sensation, and discusses how BMI design may influence neural plasticity and performance. The close integration of plasticity in sensory and motor function influences the design of both artificial pathways and will be an essential consideration for bidirectional devices that restore both sensory and motor function.

Thalamocortical loops as temporal demodulators across senses

Sensory information is coded in space and in time. The organization of neuronal activity in space maintains straightforward relationships with the spatial organization of the perceived environment. In contrast, the temporal organization of neuronal activity is not trivially related to external features due to sensor motion. Still, the temporal organization shares similar principles across sensory modalities. Likewise, thalamocortical circuits exhibit common features across senses. Focusing on touch, vision, and audition, we review their shared coding principles and suggest that thalamocortical systems include circuits that allow analogous recoding mechanisms in all three senses. These thalamocortical circuits constitute oscillations-based phase-locked loops, that translate temporally-coded sensory information to rate-coded cortical signals, signals that can integrate information across sensory and motor modalities. The loop also allows predictive locking to the onset of future modulations of the sensory signal. The paper thus suggests a theoretical framework in which a common thalamocortical mechanism implements temporal demodulation across senses.

High-precision monitoring of and feedback control over drug concentrations in the brains of freely moving rats

Knowledge of drug concentrations in the brains of behaving subjects remains constrained on a number of dimensions, including poor temporal resolution and lack of real-time data. Here, however, we demonstrate the ability of electrochemical aptamer-based sensors to support seconds-resolved, real-time measurements of drug concentrations in the brains of freely moving rats. Specifically, using such sensors, we achieve <4 μM limits of detection and 10-s resolution in the measurement of procaine in the brains of freely moving rats, permitting the determination of the pharmacokinetics and concentration-behavior relations of the drug with high precision for individual subjects. In parallel, we have used closed-loop feedback-controlled drug delivery to hold intracranial procaine levels constant (±10%) for >1.5 hours. These results demonstrate the utility of such sensors in (i) the determination of the site-specific, seconds-resolved neuropharmacokinetics, (ii) enabling the study of individual subject neuropharmacokinetics and concentration-response relations, and (iii) performing high-precision control over intracranial drug levels.

Peripersonal encoding of forelimb proprioception in the mouse somatosensory cortex

Conscious perception of limb movements depends on proprioceptive neural responses in the somatosensory cortex. In contrast to tactile sensations, proprioceptive cortical coding is barely studied in the mammalian brain and practically non-existent in rodent research. To understand the cortical representation of this important sensory modality we developed a passive forelimb displacement paradigm in behaving mice and also trained them to perceptually discriminate where their limb is moved in space. We delineated the rodent proprioceptive cortex with wide-field calcium imaging and optogenetic silencing experiments during behavior. Our results reveal that proprioception is represented in both sensory and motor cortical areas. In addition, behavioral measurements and responses of layer 2/3 neurons imaged with two-photon microscopy reveal that passive limb movements are both perceived and encoded in the mouse cortex as a spatial direction vector that interfaces the limb with the body's peripersonal space.

Restoration of sensory information via bionic hands

Individuals who have lost the use of their hands because of amputation or spinal cord injury can use prosthetic hands to restore their independence. A dexterous prosthesis requires the acquisition of control signals that drive the movements of the robotic hand, and the transmission of sensory signals to convey information to the user about the consequences of these movements. In this Review, we describe non-invasive and invasive technologies for conveying artificial sensory feedback through bionic hands, and evaluate the technologies' long-term prospects.

Daily two-photon neuronal population imaging with targeted single-cell electrophysiology and subcellular imaging in auditory cortex of behaving mice

Quantitative and mechanistic understanding of learning and long-term memory at the level of single neurons in living brains require highly demanding techniques. A specific need is to precisely label one cell whose firing output property is pinpointed amidst a functionally characterized large population of neurons through the learning process and then investigate the distribution and properties of dendritic inputs. Here, we disseminate an integrated method of daily two-photon neuronal population Ca²⁺ imaging through an auditory associative learning course, followed by targeted single-cell loose-patch recording and electroporation of plasmid for enhanced chronic Ca²⁺ imaging of dendritic spines in the targeted cell. Our method provides a unique solution to the demand, opening a solid path toward the hard-cores of how learning and long-term memory are physiologically carried out at the level of single neurons and synapses.

On closed-loop brain stimulation systems for improving the quality of life of patients with neurological disorders

Emerging brain technologies have significantly transformed human life in recent decades. For instance, the closed-loop brain-computer interface (BCI) is an advanced software-hardware system that interprets electrical signals from neurons, allowing communication with and control of the environment. The system then transmits these signals as controlled commands and provides feedback to the brain to execute specific tasks. This paper analyzes and presents the latest research on closed-loop BCI that utilizes electric/magnetic stimulation, optogenetic, and sonogenetic techniques. These techniques have demonstrated great potential in improving the quality of life for patients suffering from neurodegenerative or psychiatric diseases. We provide a comprehensive and systematic review of research on the modalities of closed-loop BCI in recent decades. To achieve this, the authors used a set of defined criteria to shortlist studies from well-known research databases into categories of brain stimulation techniques. These categories include deep brain stimulation, transcranial magnetic stimulation, transcranial direct-current stimulation, transcranial alternating-current stimulation, and optogenetics. These techniques have been useful in treating a wide range of disorders, such as Alzheimer's and Parkinson's disease, dementia, and depression. In total, 76 studies were shortlisted and analyzed to illustrate how closed-loop BCI can considerably improve, enhance, and restore specific brain functions. The analysis revealed that literature in the area has not adequately covered closed-loop BCI in the context of cognitive neural prosthetics and implanted neural devices. However, the authors demonstrate that the applications of closed-loop BCI are highly beneficial, and the technology is continually evolving to improve the lives of individuals with various ailments, including those with sensory-motor issues or cognitive deficiencies. By utilizing emerging techniques of stimulation, closed-loop BCI can safely improve patients' cognitive and affective skills, resulting in better healthcare outcomes.

Learning in a closed-loop brain-machine interface with distributed optogenetic cortical feedback

Objective.Distributed microstimulations at the cortical surface can efficiently deliver feedback to a subject during the manipulation of a prosthesis through a brain-machine interface (BMI). Such feedback can convey vast amounts of information to the prosthesis user and may be key to obtain an accurate control and embodiment of the prosthesis. However, so far little is known of the physiological constraints on the decoding of such patterns. Here, we aimed to test a rotary optogenetic feedback that was designed to encode efficiently the 360° movements of the robotic actuators used in prosthetics. We sought to assess its use by mice that controlled a prosthesis joint through a closed-loop BMI.Approach.We tested the ability of mice to optimize the trajectory of a virtual prosthesis joint in order to solve a rewarded reaching task. They could control the speed of the joint by modulating the activity of individual neurons in the primary motor cortex. During the task, the patterned optogenetic stimulation projected on the primary somatosensory cortex continuously delivered information to the mouse about the position of the joint.Main results.We showed that mice are able to exploit the continuous, rotating cortical feedback in the active behaving context of the task. Mice achieved better control than in the absence of feedback by detecting reward opportunities more often, and also by moving the joint faster towards the reward angular zone, and by maintaining it longer in the reward zone. Mice controlling acceleration rather than speed of the joint failed to improve motor control.Significance.These findings suggest that in the context of a closed-loop BMI, distributed cortical feedback with optimized shapes and topology can be exploited to control movement. Our study has direct applications on the closed-loop control of rotary joints that are frequently encountered in robotic prostheses.

Multiscale imaging informs translational mouse modeling of neurological disease

Multiscale neurophysiology reveals that simple motor actions are associated with changes in neuronal firing in virtually every brain region studied. Accordingly, the assessment of focal pathology such as stroke or progressive neurodegenerative diseases must also extend widely across brain areas. To derive mechanistic information through imaging, multiple resolution scales and multimodal factors must be included, such as the structure and function of specific neurons and glial cells and the dynamics of specific neurotransmitters. Emerging multiscale methods in preclinical animal studies that span micro- to macroscale examinations fill this gap, allowing a circuit-based understanding of pathophysiological mechanisms. Combined with high-performance computation and open-source data repositories, these emerging multiscale and large field-of-view techniques include live functional ultrasound, multi- and single-photon wide-scale light microscopy, video-based miniscopes, and tissue-penetrating fiber photometry, as well as variants of post-mortem expansion microscopy. We present these technologies and outline use cases and data pipelines to uncover new knowledge within animal models of stroke, Alzheimer's disease, and movement disorders.

Real-Time Image Processing Toolbox for All-Optical Closed-Loop Control of Neuronal Activities

The development of in vivo imaging and optogenetic tools makes it possible to control neural circuit activities in an all-optical, closed-loop manner, but such applications are limited by the lack of software for online analysis of neuronal imaging data. We developed an analysis software ORCA (Online Real-time activity and offline Cross-session Analysis), which performs image registration, neuron segmentation, and activity extraction at over 100 frames per second, fast enough to support real-time detection and readout of neural activity. Our active neuron detection algorithm is purely statistical, achieving a much higher speed than previous methods. We demonstrated closed-loop control of neurons that were identified on the fly, without prior recording or image processing. ORCA also includes a cross-session alignment module that efficiently tracks neurons across multiple sessions. In summary, ORCA is a powerful toolbox for fast imaging data analysis and provides a solution for all-optical closed-loop control of neuronal activity.

Advantages, Pitfalls, and Developments of All Optical Interrogation Strategies of Microcircuits in vivo

The holy grail for every neurophysiologist is to conclude a causal relationship between an elementary behaviour and the function of a specific brain area or circuit. Our effort to map elementary behaviours to specific brain loci and to further manipulate neural activity while observing the alterations in behaviour is in essence the goal for neuroscientists. Recent advancements in the area of experimental brain imaging in the form of longer wavelength near infrared (NIR) pulsed lasers with the development of highly efficient optogenetic actuators and reporters of neural activity, has endowed us with unprecedented resolution in spatiotemporal precision both in imaging neural activity as well as manipulating it with multiphoton microscopy. This readily available toolbox has introduced a so called all-optical physiology and interrogation of circuits and has opened new horizons when it comes to precisely, fast and non-invasively map and manipulate anatomically, molecularly or functionally identified mesoscopic brain circuits. The purpose of this review is to describe the advantages and possible pitfalls of all-optical approaches in system neuroscience, where by all-optical we mean use of multiphoton microscopy to image the functional response of neuron(s) in the network so to attain flexible choice of the cells to be also optogenetically photostimulated by holography, in absence of electrophysiology. Spatio-temporal constraints will be compared toward the classical reference of electrophysiology methods. When appropriate, in relation to current limitations of current optical approaches, we will make reference to latest works aimed to overcome these limitations, in order to highlight the most recent developments. We will also provide examples of types of experiments uniquely approachable all-optically. Finally, although mechanically non-invasive, all-optical electrophysiology exhibits potential off-target effects which can ambiguate and complicate the interpretation of the results. In summary, this review is an effort to exemplify how an all-optical experiment can be designed, conducted and interpreted from the point of view of the integrative neurophysiologist.

Existing function in primary visual cortex is not perturbed by new skill acquisition of a non-matched sensory task

Acquisition of new skills has the potential to disturb existing network function. To directly assess whether previously acquired cortical function is altered during learning, mice were trained in an abstract task in which selected activity patterns were rewarded using an optical brain-computer interface device coupled to primary visual cortex (V1) neurons. Excitatory neurons were longitudinally recorded using 2-photon calcium imaging. Despite significant changes in local neural activity during task performance, tuning properties and stimulus encoding assessed outside of the trained context were not perturbed. Similarly, stimulus tuning was stable in neurons that remained responsive following a different, visual discrimination training task. However, visual discrimination training increased the rate of representational drift. Our results indicate that while some forms of perceptual learning may modify the contribution of individual neurons to stimulus encoding, new skill learning is not inherently disruptive to the quality of stimulus representation in adult V1.

Diverse operant control of different motor cortex populations during learning

During motor learning,1 as well as during neuroprosthetic learning,2-4 animals learn to control motor cortex activity in order to generate behavior. Two different populations of motor cortex neurons, intra-telencephalic (IT) and pyramidal tract (PT) neurons, convey the resulting cortical signals within and outside the telencephalon. Although a large amount of evidence demonstrates contrasting functional organization among both populations,5,6 it is unclear whether the brain can equally learn to control the activity of either class of motor cortex neurons. To answer this question, we used a calcium-imaging-based brain-machine interface (CaBMI)3 and trained different groups of mice to modulate the activity of either IT or PT neurons in order to receive a reward. We found that the animals learned to control PT neuron activity faster and better than IT neuron activity. Moreover, our findings show that the advantage of PT neurons is the result of characteristics inherent to this population as well as their local circuitry and cortical depth location. Taken together, our results suggest that the motor cortex is more efficient at controlling the activity of pyramidal tract neurons, which are embedded deep in the cortex, and relaying motor commands outside the telencephalon.

Cognitive Control as a Multivariate Optimization Problem

A hallmark of adaptation in humans and other animals is our ability to control how we think and behave across different settings. Research has characterized the various forms cognitive control can take-including enhancement of goal-relevant information, suppression of goal-irrelevant information, and overall inhibition of potential responses-and has identified computations and neural circuits that underpin this multitude of control types. Studies have also identified a wide range of situations that elicit adjustments in control allocation (e.g., those eliciting signals indicating an error or increased processing conflict), but the rules governing when a given situation will give rise to a given control adjustment remain poorly understood. Significant progress has recently been made on this front by casting the allocation of control as a decision-making problem. This approach has developed unifying and normative models that prescribe when and how a change in incentives and task demands will result in changes in a given form of control. Despite their successes, these models, and the experiments that have been developed to test them, have yet to face their greatest challenge: deciding how to select among the multiplicity of configurations that control can take at any given time. Here, we will lay out the complexities of the inverse problem inherent to cognitive control allocation, and their close parallels to inverse problems within motor control (e.g., choosing between redundant limb movements). We discuss existing solutions to motor control's inverse problems drawn from optimal control theory, which have proposed that effort costs act to regularize actions and transform motor planning into a well-posed problem. These same principles may help shed light on how our brains optimize over complex control configuration, while providing a new normative perspective on the origins of mental effort.

Strengthening the GABAergic system through neurofeedback training suppresses implicit motor learning

Gamma-aminobutyric acid (GABA) activity within the primary motor cortex (M1) is essential for motor learning in cortical plasticity, and a recent study has suggested that real-time neurofeedback training (NFT) can self-regulate GABA activity. Therefore, this study aimed to investigate the effect of GABA activity strengthening via NFT on subsequent motor learning. Thirty-six healthy participants were randomly assigned to either an NFT group or control group, which received sham feedback. GABA activity was assessed for short intracortical inhibition (SICI) within the right M1 using paired-pulse transcranial magnetic stimulation. During the NFT intervention period, the participants tried to modulate the size of a circle, which was altered according to the degree of SICI in the NFT group. However, the size was altered independently of the degree of SICI in the control group. We measured the reaction time before, after (online learning), and 24 h after (offline learning) the finger-tapping task. Results showed the strengthening of GABA activity induced by the NFT intervention, and the suppression of the online but not the offline learning. These findings suggest that prior GABA activity modulation may affect online motor learning.

Volitional control of individual neurons in the human brain

Brain-machine interfaces allow neuroscientists to causally link specific neural activity patterns to a particular behaviour. Thus, in addition to their current clinical applications, brain-machine interfaces can also be used as a tool to investigate neural mechanisms of learning and plasticity in the brain. Decades of research using such brain-machine interfaces have shown that animals (non-human primates and rodents) can be operantly conditioned to self-regulate neural activity in various motor-related structures of the brain. Here, we ask whether the human brain, a complex interconnected structure of over 80 billion neurons, can learn to control itself at the most elemental scale-a single neuron. We used the unique opportunity to record single units in 11 individuals with epilepsy to explore whether the firing rate of a single (direct) neuron in limbic and other memory-related brain structures can be brought under volitional control. To do this, we developed a visual neurofeedback task in which participants were trained to move a block on a screen by modulating the activity of an arbitrarily selected neuron from their brain. Remarkably, participants were able to volitionally modulate the firing rate of the direct neuron in these previously uninvestigated structures. We found that a subset of participants (learners), were able to improve their performance within a single training session. Successful learning was characterized by (i) highly specific modulation of the direct neuron (demonstrated by significantly increased firing rates and burst frequency); (ii) a simultaneous decorrelation of the activity of the direct neuron from the neighbouring neurons; and (iii) robust phase-locking of the direct neuron to local alpha/beta-frequency oscillations, which may provide some insights in to the potential neural mechanisms that facilitate this type of learning. Volitional control of neuronal activity in mnemonic structures may provide new ways of probing the function and plasticity of human memory without exogenous stimulation. Furthermore, self-regulation of neural activity in these brain regions may provide an avenue for the development of novel neuroprosthetics for the treatment of neurological conditions that are commonly associated with pathological activity in these brain structures, such as medically refractory epilepsy.

Probing neural codes with two-photon holographic optogenetics

Optogenetics ushered in a revolution in how neuroscientists interrogate brain function. Because of technical limitations, the majority of optogenetic studies have used low spatial resolution activation schemes that limit the types of perturbations that can be made. However, neural activity manipulations at finer spatial scales are likely to be important to more fully understand neural computation. Spatially precise multiphoton holographic optogenetics promises to address this challenge and opens up many new classes of experiments that were not previously possible. More specifically, by offering the ability to recreate extremely specific neural activity patterns in both space and time in functionally defined ensembles of neurons, multiphoton holographic optogenetics could allow neuroscientists to reveal fundamental aspects of the neural codes for sensation, cognition and behavior that have been beyond reach. This Review summarizes recent advances in multiphoton holographic optogenetics that substantially expand its capabilities, highlights outstanding technical challenges and provides an overview of the classes of experiments it can execute to test and validate key theoretical models of brain function. Multiphoton holographic optogenetics could substantially accelerate the pace of neuroscience discovery by helping to close the loop between experimental and theoretical neuroscience, leading to fundamental new insights into nervous system function and disorder.

Long-term stability of cortical ensembles

Neuronal ensembles, coactive groups of neurons found in spontaneous and evoked cortical activity, are causally related to memories and perception, but it is still unknown how stable or flexible they are over time. We used two-photon multiplane calcium imaging to track over weeks the activity of the same pyramidal neurons in layer 2/3 of the visual cortex from awake mice and recorded their spontaneous and visually evoked responses. Less than half of the neurons remained active across any two imaging sessions. These stable neurons formed ensembles that lasted weeks, but some ensembles were also transient and appeared only in one single session. Stable ensembles preserved most of their neurons for up to 46 days, our longest imaged period, and these 'core' cells had stronger functional connectivity. Our results demonstrate that neuronal ensembles can last for weeks and could, in principle, serve as a substrate for long-lasting representation of perceptual states or memories.

Circuit mechanisms for cortical plasticity and learning

The cerebral cortex integrates sensory information with emotional states and internal representations to produce coherent percepts, form associations, and execute voluntary actions. For the cortex to optimize perception, its neuronal network needs to dynamically retrieve and encode new information. Over the last few decades, research has started to provide insight into how the cortex serves these functions. Building on classical Hebbian plasticity models, the latest hypotheses hold that throughout experience and learning, streams of feedforward, feedback, and modulatory information operate in selective and coordinated manners to alter the strength of synapses and ultimately change the response properties of cortical neurons. Here, we describe cortical plasticity mechanisms that involve the concerted action of feedforward and long-range feedback input onto pyramidal neurons as well as the implication of local disinhibitory circuit motifs in this process.

Operant conditioning reveals task-specific responses of single neurons in a brain-machine interface

OBJECTIVE

Volitional modulation of single cortical neurons holds great potential for the implementation of brain-machine interfaces (BMIs) because it can induce a rapid acquisition of arbitrary associations between machines and neural activity. It can also be used as a framework to study the limits of single-neuron control in BMIs.

APPROACH

We tested the control of a one-dimensional actuator in two BMI tasks which differed only in the neural contingency that determined when a reward was dispensed. A thresholded activity task, commonly implemented in single-neuron BMI control, consisted of reaching or exceeding a neuron activity level, while the second task consisted of reaching and maintaining a narrow neuron activity level (i.e. windowed activity task).

MAIN FINDINGS

Single neurons in layer V of the motor cortex of rats improved performance during both the thresholded activity and windowed activity BMI tasks. However, correct performance during the windowed activity task was accompanied by activation of neighboring neurons, not in direct control of the BMI. In contrast, only neurons in direct control of the BMI were active at the time of reward during the thresholded activity task.

SIGNIFICANCE

These results suggest that thresholded activity single-neuron BMI implementations are more appropriate compared to windowed activity BMI tasks to capitalize on the adaptability of cortical circuits to acquire novel arbitrary skills.

Effects of Optogenetic Stimulation of Primary Somatosensory Cortex and Its Projections to Striatum on Vibrotactile Perception in Freely Moving Rats

Tactile sensation is one of our primary means to collect information about the nearby environment and thus crucial for daily activities and survival. Therefore, it is of high importance to restore sensory feedback after sensory loss. Optogenetic manipulation allows local or pathway-specific write-in of information. However, it remains elusive whether optogenetic stimulation can be interpreted as tactile sensation to guide operant behavior and how it is integrated with tactile stimuli. To address these questions, we employed a vibrotactile detection task combined with optogenetic neuromodulation in freely moving rats. By bidirectionally manipulating the activity of neurons in primary somatosensory cortex (S1), we demonstrated that optical activation as well as inhibition of S1 reduced the detection rate for vibrotactile stimuli. Interestingly, activation of corticostriatal terminals improved the detection of tactile stimuli, while inhibition of corticostriatal terminals did not affect the performance. To manipulate the corticostriatal pathway more specifically, we employed a dual viral system. Activation of corticostriatal cell bodies disturbed the tactile perception while activation of corticostriatal terminals slightly facilitated the detection of vibrotactile stimuli. In the absence of tactile stimuli, both corticostriatal cell bodies as well as terminals caused a reaction. Taken together, our data confirmed the possibility to restore sensation using optogenetics and demonstrated that S1 and its descending projections to striatum play differential roles in the neural processing underlying vibrotactile detection.

The sensory representation of causally controlled objects

Intentional control over external objects is informed by our sensory experience of them. To study how causal relationships are learned and effected, we devised a brain machine interface (BMI) task using wide-field calcium signals. Mice learned to entrain activity patterns in arbitrary pairs of cortical regions to guide a visual cursor to a target location for reward. Brain areas that were normally correlated could be rapidly reconfigured to exert control over the cursor in a sensory-feedback-dependent manner. Higher visual cortex was more engaged when expert but not naive animals controlled the cursor. Individual neurons in higher visual cortex responded more strongly to the cursor when mice controlled it than when they passively viewed it, with the greatest response boosting as the cursor approached the target location. Thus, representations of causally controlled objects are sensitive to intention and proximity to the subject's goal, potentially strengthening sensory feedback to allow more fluent control.

Sensory substitution reveals a manipulation bias

Sensory substitution is a promising therapeutic approach for replacing a missing or diseased sensory organ by translating inaccessible information into another sensory modality. However, many substitution systems are not well accepted by subjects. To explore the effect of sensory substitution on voluntary action repertoires and their associated affective valence, we study deaf songbirds to which we provide visual feedback as a substitute of auditory feedback. Surprisingly, deaf birds respond appetitively to song-contingent binary visual stimuli. They skillfully adapt their songs to increase the rate of visual stimuli, showing that auditory feedback is not required for making targeted changes to vocal repertoires. We find that visually instructed song learning is basal-ganglia dependent. Because hearing birds respond aversively to the same visual stimuli, sensory substitution reveals a preference for actions that elicit sensory feedback over actions that do not, suggesting that substitution systems should be designed to exploit the drive to manipulate.

Operant conditioning of motor cortex neurons reveals neuron-subtype-specific responses in a brain-machine interface task

Operant conditioning is implemented in brain-machine interfaces (BMI) to induce rapid volitional modulation of single neuron activity to control arbitrary mappings with an external actuator. However, intrinsic factors of the volitional controller (i.e. the brain) or the output stage (i.e. individual neurons) might hinder performance of BMIs with more complex mappings between hundreds of neurons and actuators with multiple degrees of freedom. Improved performance might be achieved by studying these intrinsic factors in the context of BMI control. In this study, we investigated how neuron subtypes respond and adapt to a given BMI task. We conditioned single cortical neurons in a BMI task. Recorded neurons were classified into bursting and non-bursting subtypes based on their spike-train autocorrelation. Both neuron subtypes had similar improvement in performance and change in average firing rate. However, in bursting neurons, the activity leading up to a reward increased progressively throughout conditioning, while the response of non-bursting neurons did not change during conditioning. These results highlight the need to characterize neuron-subtype-specific responses in a variety of tasks, which might ultimately inform the design and implementation of BMIs.

Adenosine A2A receptor blockade improves neuroprosthetic learning by volitional control of population calcium signal in M1 cortical neurons

Volitional control is at the core of brain-machine interfaces (BMI) adaptation and neuroprosthetic-driven learning to restore motor function for disabled patients, but neuroplasticity changes and neuromodulation underlying volitional control of neuroprosthetic learning are largely unexplored. To better study volitional control at annotated neural population, we have developed an operant neuroprosthetic task with closed-loop feedback system by volitional conditioning of population calcium signal in the M1 cortex using fiber photometry recording. Importantly, volitional conditioning of the population calcium signal in M1 neurons did not improve within-session adaptation, but specifically enhanced across-session neuroprosthetic skill learning with reduced time-to-target and the time to complete 50 successful trials. With brain-behavior causality of the neuroprosthetic paradigm, we revealed that proficiency of neuroprosthetic learning by volitional conditioning of calcium signal was associated with the stable representational (plasticity) mapping in M1 neurons with the reduced calcium peak. Furthermore, pharmacological blockade of adenosine A_2A receptors facilitated volitional conditioning of neuroprosthetic learning and converted an ineffective volitional conditioning protocol to be the effective for neuroprosthetic learning. These findings may help to harness neuroplasticity for better volitional control of neuroprosthetic training and suggest a novel pharmacological strategy to improve neuroprosthetic learning in BMI adaptation by targeting striatal A_2A receptors.

Abnormal reinforcement learning in a mice model of autism induced by prenatal exposure to valproic acid

Individuals with autism spectrum disorder (ASD) display dysfunction in learning from environmental stimulus that have positive or negative emotional values, posing obstacles to their everyday life. Unfortunately, mechanisms of the dysfunction are still unclear. Although early intervention for ASD victims based on reinforcement learning are commonly used, the mechanisms and characteristics of the improvement are also unknown. By using a mice model of ASD produced by prenatal exposure to valproic acid (VPA), the present work discovered a delayed response-reinforcer forming, and an impaired habit forming in a negative reinforcement learning paradigm in VPA exposure male offspring. But the extinction of the learned skills was found to become faster than normal male animals. Since escape action of nosepoking and the motility remain unchanged in the VPA male offspring, the impaired learning and the accelerated extinction are caused by deficits in higher brain functions underlying association between the animals' behavioral responses and the outcomes of such responses. The results further suggest that the rodent ASD model produced by prenatal exposure to VPA reproduces the deficits in reasoning or building the contingency between one's own behaviors and the consequent outcomes of the behavior seen in ASD patients.

Real-Time Selective Markerless Tracking of Forepaws of Head Fixed Mice Using Deep Neural Networks

Here, we describe a system capable of tracking specific mouse paw movements at high frame rates (70.17 Hz) with a high level of accuracy (mean = 0.95, SD < 0.01). Short-latency markerless tracking of specific body parts opens up the possibility of manipulating motor feedback. We present a software and hardware scheme built on DeepLabCut—a robust movement-tracking deep neural network framework—which enables real-time estimation of paw and digit movements of mice. Using this approach, we demonstrate movement-generated feedback by triggering a USB-GPIO (general-purpose input/output)-controlled LED when the movement of one paw, but not the other, selectively exceeds a preset threshold. The mean time delay between paw movement initiation and LED flash was 44.41 ms (SD = 36.39 ms), a latency sufficient for applying behaviorally triggered feedback. We adapt DeepLabCut for real-time tracking as an open-source package we term DeepCut2RealTime. The ability of the package to rapidly assess animal behavior was demonstrated by reinforcing specific movements within water-restricted, head-fixed mice. This system could inform future work on a behaviorally triggered “closed loop” brain–machine interface that could reinforce behaviors or deliver feedback to brain regions based on prespecified body movements.

Automated task training and longitudinal monitoring of mouse mesoscale cortical circuits using home cages

We report improved automated open-source methodology for head-fixed mesoscale cortical imaging and/or behavioral training of home cage mice using Raspberry Pi-based hardware. Staged partial and probabilistic restraint allows mice to adjust to self-initiated headfixation over 3 weeks' time with ~50% participation rate. We support a cue-based behavioral licking task monitored by a capacitive touch-sensor water spout. While automatically head-fixed, we acquire spontaneous, movement-triggered, or licking task-evoked GCaMP6 cortical signals. An analysis pipeline marked both behavioral events, as well as analyzed brain fluorescence signals as they relate to spontaneous and/or task-evoked behavioral activity. Mice were trained to suppress licking and wait for cues that marked the delivery of water. Correct rewarded go-trials were associated with widespread activation of midline and lateral barrel cortex areas following a vibration cue and delayed frontal and lateral motor cortex activation. Cortical GCaMP signals predicted trial success and correlated strongly with trial-outcome dependent body movements.

Real-time detection of neural oscillation bursts allows behaviourally relevant neurofeedback

Neural oscillations as important information carrier in the brain, are increasingly interpreted as transient bursts rather than as sustained oscillations. Short (<150 ms) bursts of beta-waves (15-30 Hz) have been documented in humans, monkeys and mice. These events were correlated with memory, movement and perception, and were even suggested as the primary ingredient of all beta-band activity. However, a method to measure these short-lived events in real-time and to investigate their impact on behaviour is missing. Here we present a real-time data analysis system, capable to detect short narrowband bursts, and demonstrate its usefulness to increase the beta-band burst-rate in rats. This neurofeedback training induced changes in overall oscillatory power, and bursts could be decoded from the movement of the rats, thus enabling future investigation of the role of oscillatory bursts.

Automatic Adaptation of Model Neurons and Connections to Build Hybrid Circuits with Living Networks

Hybrid circuits built by creating mono- or bi-directional interactions among living cells and model neurons and synapses are an effective way to study neuron, synaptic and neural network dynamics. However, hybrid circuit technology has been largely underused in the context of neuroscience studies mainly because of the inherent difficulty in implementing and tuning this type of interactions. In this paper, we present a set of algorithms for the automatic adaptation of model neurons and connections in the creation of hybrid circuits with living neural networks. The algorithms perform model time and amplitude scaling, real-time drift adaptation, goal-driven synaptic and model tuning/calibration and also automatic parameter mapping. These algorithms have been implemented in RTHybrid, an open-source library that works with hard real-time constraints. We provide validation examples by building hybrid circuits in a central pattern generator. The results of the validation experiments show that the proposed dynamic adaptation facilitates closed-loop communication among living and artificial model neurons and connections, and contributes to characterize system dynamics, achieve control, automate experimental protocols and extend the lifespan of the preparations.

Neural reinforcement: re-entering and refining neural dynamics leading to desirable outcomes

How do organisms learn to do again, on-demand, a behavior that led to a desirable outcome? Dopamine-dependent cortico-striatal plasticity provides a framework for learning behavior's value, but it is less clear how it enables the brain to re-enter desired behaviors and refine them over time. Reinforcing behavior is achieved by re-entering and refining the neural patterns that produce it. We review studies using brain-machine interfaces which reveal that reinforcing cortical population activity requires cortico-basal ganglia circuits. Then, we propose a formal framework for how reinforcement in cortico-basal ganglia circuits acts on the neural dynamics of cortical populations. We propose two parallel mechanisms: i) fast reinforcement which selects the inputs that permit the re-entrance of the particular cortical population dynamics which naturally produced the desired behavior, and ii) slower reinforcement which leads to refinement of cortical population dynamics and more reliable production of neural trajectories driving skillful behavior on-demand.

Causes and consequences of representational drift

The nervous system learns new associations while maintaining memories over long periods, exhibiting a balance between flexibility and stability. Recent experiments reveal that neuronal representations of learned sensorimotor tasks continually change over days and weeks, even after animals have achieved expert behavioral performance. How is learned information stored to allow consistent behavior despite ongoing changes in neuronal activity? What functions could ongoing reconfiguration serve? We highlight recent experimental evidence for such representational drift in sensorimotor systems, and discuss how this fits into a framework of distributed population codes. We identify recent theoretical work that suggests computational roles for drift and argue that the recurrent and distributed nature of sensorimotor representations permits drift while limiting disruptive effects. We propose that representational drift may create error signals between interconnected brain regions that can be used to keep neural codes consistent in the presence of continual change. These concepts suggest experimental and theoretical approaches to studying both learning and maintenance of distributed and adaptive population codes.

Directional Reaching for Water as a Cortex-Dependent Behavioral Framework for Mice

Optogenetic tools and imaging methods for recording and manipulating brain activity have boosted the field of neuroscience in unprecedented ways. However, behavioral paradigms for mice lag behind those of primates, limiting the full potential of such tools. Here, we present an innovative behavioral framework in which head-fixed mice directionally reach for water droplets, similar to the primate "center-out" reaching task. Mice rapidly engaged in the task, performed hundreds of trials, and reached in multiple directions when droplets were presented at different locations. Surprisingly, mice used chemosensation to determine the presence of water droplets. Optogenetic inactivation of the motor cortex halted the initiation and rapidly diverted the trajectory of ongoing movements. Layer 2/3 two-photon imaging revealed robust direction selectivity in most reach-related neurons. Finally, mice performed directional reaching instructed by vibratotactile stimuli, demonstrating the potential of this framework for studying, in addition to motor control, sensory processing, and decision making.

Circuit Investigations With Open-Source Miniaturized Microscopes: Past, Present and Future

The ability to simultaneously image the spatiotemporal activity signatures from many neurons during unrestrained vertebrate behaviors has become possible through the development of miniaturized fluorescence microscopes, or miniscopes, sufficiently light to be carried by small animals such as bats, birds and rodents. Miniscopes have permitted the study of circuits underlying song vocalization, action sequencing, head-direction tuning, spatial memory encoding and sleep to name a few. The foundation for these microscopes has been laid over the last two decades through academic research with some of this work resulting in commercialization. More recently, open-source initiatives have led to an even broader adoption of miniscopes in the neuroscience community. Open-source designs allow for rapid modification and extension of their function, which has resulted in a new generation of miniscopes that now permit wire-free or wireless recording, concurrent electrophysiology and imaging, two-color fluorescence detection, simultaneous optical actuation and read-out as well as wide-field and volumetric light-field imaging. These novel miniscopes will further expand the toolset of those seeking affordable methods to probe neural circuit function during naturalistic behaviors. Here, we will discuss the early development, present use and future potential of miniscopes.

RTHybrid: A Standardized and Open-Source Real-Time Software Model Library for Experimental Neuroscience

Closed-loop technologies provide novel ways of online observation, control and bidirectional interaction with the nervous system, which help to study complex non-linear and partially observable neural dynamics. These protocols are often difficult to implement due to the temporal precision required when interacting with biological components, which in many cases can only be achieved using real-time technology. In this paper we introduce RTHybrid (www.github.com/GNB-UAM/RTHybrid), a free and open-source software that includes a neuron and synapse model library to build hybrid circuits with living neurons in a wide variety of experimental contexts. In an effort to encourage the standardization of real-time software technology in neuroscience research, we compared different open-source real-time operating system patches, RTAI, Xenomai 3 and Preempt-RT, according to their performance and usability. RTHybrid has been developed to run over Linux operating systems supporting both Xenomai 3 and Preempt-RT real-time patches, and thus allowing an easy implementation in any laboratory. We report a set of validation tests and latency benchmarks for the construction of hybrid circuits using this library. With this work we want to promote the dissemination of standardized, user-friendly and open-source software tools developed for open- and closed-loop experimental neuroscience.

Real-Time Prosthetic Digit Actuation by Optical Read-out of Activity-Dependent Calcium Signals in an Ex Vivo Peripheral Nerve

Improved neural interfacing strategies are needed for the full articulation of advanced prostheses. To address limitations of existing control interface designs, the work of our laboratory has presented an optical approach to reading activity from individual nerve fibers using activity-dependent calcium transients. Here, we demonstrate the feasibility of such signals to control prosthesis actuation by using the axonal fluorescence signal in an ex vivo mouse nerve to drive a prosthetic digit in real-time. Additionally, signals of varying action potential frequency are streamed post hoc to the prosthesis, showing graded motor output and the potential for proportional neural control. This proof-of-concept work is a novel demonstration of the functional use of activity-dependent optical read-out in the nerve.

Real-Time Processing of Two-Photon Calcium Imaging Data Including Lateral Motion Artifact Correction

Two-photon calcium imaging has been extensively used to record neural activity in the brain. It has been long used solely with post-hoc analysis, but the recent efforts began to include closed-loop experiments. Closed-loop experiments pose new challenges because they require fast, real-time image processing without iterative parameter tuning. When imaging awake animals, one of the crucial steps of post hoc image analysis is correction of lateral motion artifacts. In most of the closed-loop experiments, this step has not been implemented and ignored due to technical difficulties. We recently reported the first experiments with real-time processing of calcium imaging that included lateral motion correction. Here, we report the details of the implementation of fast motion correction and present performance analysis across several algorithms with different parameters. Additionally, we introduce a novel method to estimate baseline calcium signal using kernel density estimate, which reduces the number of parameters to be tuned. Combined, we propose a novel software pipeline of real-time image processing suited for closed-loop experiments. The pipeline is also useful for rapid post hoc image processing.

Closed-loop all-optical interrogation of neural circuits in vivo

Understanding the causal relationship between neural activity and behavior requires the ability to perform rapid and targeted interventions in ongoing activity. Here we describe a closed-loop all-optical strategy for dynamically controlling neuronal activity patterns in awake mice. We rapidly tailored and delivered two-photon optogenetic stimulation based on online readout of activity using simultaneous two-photon imaging, thus enabling the manipulation of neural circuit activity 'on the fly' during behavior.

Targeted cortical reorganization using optogenetics in non-human primates

Brain stimulation modulates the excitability of neural circuits and drives neuroplasticity. While the local effects of stimulation have been an active area of investigation, the effects on large-scale networks remain largely unexplored. We studied stimulation-induced changes in network dynamics in two macaques. A large-scale optogenetic interface enabled simultaneous stimulation of excitatory neurons and electrocorticographic recording across primary somatosensory (S1) and motor (M1) cortex (Yazdan-Shahmorad et al., 2016). We tracked two measures of network connectivity, the network response to focal stimulation and the baseline coherence between pairs of electrodes; these were strongly correlated before stimulation. Within minutes, stimulation in S1 or M1 significantly strengthened the gross functional connectivity between these areas. At a finer scale, stimulation led to heterogeneous connectivity changes across the network. These changes reflected the correlations introduced by stimulation-evoked activity, consistent with Hebbian plasticity models. This work extends Hebbian plasticity models to large-scale circuits, with significant implications for stimulation-based neurorehabilitation.

From riding a bike to reaching for a cup of coffee, all skilled actions rely on precise connections between the sensory and motor areas of the brain. While sensory areas receive and analyse input from the senses, motor areas plan and trigger muscle contractions. Precisely adjusting the connections between these and other areas enables us to learn new skills, and it also helps us to relearn skills lost as a result of brain injury or stroke.

About 70 years ago, a psychologist named Donald Hebb came up with an idea for how this process might occur. He proposed that whenever two neurons are active at the same time, the connection between them becomes stronger. This idea, that ‘cells that fire together, wire together’, became known as Hebb’s rule. Many studies have since shown that Hebb’s rule can explain changes in the strength of connections between pairs of neurons. But can it also explain how connections between entire brain regions become stronger or weaker?

New results show that it can. The data were obtained using a technique called optogenetics, in which viruses are used to introduce genes for light-sensitive proteins into neurons. Shining light onto the brain will then activate any cells within that area that contain the resulting proteins. Yazdan-Shahmorad, Silversmith et al. used this technique to activate small regions of either sensory or motor brain tissue in live macaque monkeys. Doing so strengthened the overall connectivity between the two areas. The effects were more variable at the level of smaller brain regions, with some connections becoming weaker rather than stronger. However, Yazdan-Shahmorad, Silversmith et al. show that Hebb’s rule explains most of the observed changes.

Many neurological and psychiatric disorders stem from abnormal brain connectivity. Simple forms of brain stimulation are already used to treat certain neurological disorders, such as Parkinson’s disease. Stimulating the brain to induce specific changes in connectivity may ultimately enable us to leverage the brain’s natural learning mechanisms to cure, instead of just treat, these conditions.

Volitional Modulation of Primary Visual Cortex Activity Requires the Basal Ganglia

Animals acquire behaviors through instrumental conditioning. Brain-machine interfaces have used instrumental conditioning to reinforce patterns of neural activity directly, especially in frontal and motor cortices, which are a rich source of signals for voluntary action. However, evidence suggests that activity in primary sensory cortices may also reflect internally driven processes, instead of purely encoding antecedent stimuli. Here, we show that rats and mice can learn to produce arbitrary patterns of neural activity in their primary visual cortex to control an auditory cursor and obtain reward. Furthermore, learning was prevented when neurons in the dorsomedial striatum (DMS), which receives input from visual cortex, were optogenetically inhibited, but not during inhibition of nearby neurons in the dorsolateral striatum. After learning, DMS inhibition did not affect production of the rewarded patterns. These data demonstrate that cortico-basal ganglia circuits play a general role in learning to produce cortical activity that leads to desirable outcomes.

Neural Population Dynamics Underlying Motor Learning Transfer

Covert motor learning can sometimes transfer to overt behavior. We investigated the neural mechanism underlying transfer by constructing a two-context paradigm. Subjects performed cursor movements either overtly using arm movements, or covertly via a brain-machine interface that moves the cursor based on motor cortical activity (in lieu of arm movement). These tasks helped evaluate whether and how cortical changes resulting from "covert rehearsal" affect overt performance. We found that covert learning indeed transfers to overt performance and is accompanied by systematic population-level changes in motor preparatory activity. Current models of motor cortical function ascribe motor preparation to achieving initial conditions favorable for subsequent movement-period neural dynamics. We found that covert and overt contexts share these initial conditions, and covert rehearsal manipulates them in a manner that persists across context changes, thus facilitating overt motor learning. This transfer learning mechanism might provide new insights into other covert processes like mental rehearsal.

Brain-Computer Interface with Inhibitory Neurons Reveals Subtype-Specific Strategies

Brain-computer interfaces have seen an increase in popularity due to their potential for direct neuroprosthetic applications for amputees and disabled individuals. Supporting this promise, animals-including humans-can learn even arbitrary mapping between the activity of cortical neurons and movement of prosthetic devices [1-4]. However, the performance of neuroprosthetic device control has been nowhere near that of limb control in healthy individuals, presenting a dire need to improve the performance. One potential limitation is the fact that previous work has not distinguished diverse cell types in the neocortex, even though different cell types possess distinct functions in cortical computations [5-7] and likely distinct capacities to control brain-computer interfaces. Here, we made a first step in addressing this issue by tracking the plastic changes of three major types of cortical inhibitory neurons (INs) during a neuron-pair operant conditioning task using two-photon imaging of IN subtypes expressing GCaMP6f. Mice were rewarded when the activity of the positive target neuron (N+) exceeded that of the negative target neuron (N-) beyond a set threshold. Mice improved performance with all subtypes, but the strategies were subtype specific. When parvalbumin (PV)-expressing INs were targeted, the activity of N- decreased. However, targeting of somatostatin (SOM)- and vasoactive intestinal peptide (VIP)-expressing INs led to an increase of the N+ activity. These results demonstrate that INs can be individually modulated in a subtype-specific manner and highlight the versatility of neural circuits in adapting to new demands by using cell-type-specific strategies.