Operant conditioning of single-unit response patterns in visual cortex

Recruitment and Differential Firing Patterns of Single Units During Conditioning to a Tone in a Mute Locked-In Human

Single units that are not related to the desired task can become related to the task by conditioning their firing rates. We theorized that, during conditioning of firing rates to a tone, (a) unrelated single units would be recruited to the task; (b) the recruitment would depend on the phase of the task; (c) tones of different frequencies would produce different patterns of single unit recruitment. In our mute locked-in participant, we conditioned single units using tones of different frequencies emitted from a tone generator. The conditioning task had three phases: Listen to the tone for 20 s, then silently sing the tone for 10 s, with a prior control period of resting for 10 s. Twenty single units were recorded simultaneously while feedback of one of the twenty single units was made audible to the mute locked-in participant. The results indicate that (a) some of the non-audible single units were recruited during conditioning, (b) some were recruited differentially depending on the phase of the paradigm (listen, rest, or silent sing), and (c) single unit firing patterns were specific for different tone frequencies such that the tone could be recognized from the pattern of single unit firings. These data are important when conditioning single unit firings in brain-computer interfacing tasks because they provide evidence that increased numbers of previously unrelated single units can be incorporated into the task. This incorporation expands the bandwidth of the recorded single unit population and thus enhances the brain-computer interface. This is the first report of conditioning of single unit firings in a human participant with a brain to computer implant.

Bilateral responses of rat ventral striatum tonically active neurons to unilateral medial forebrain bundle stimulation

Unilateral medial forebrain bundle (MFB) stimulation is an extremely effective promoter of reinforcement learning irrespective of the conditioned cue's laterality. The effectiveness of unilateral MFB stimulation, which activates the mesolimbic pathway connecting the ventral tegmental area to the ventral striatum (vStr), is surprising considering that these fibers rarely cross to the contralateral hemisphere. Specifically, this type of biased fiber distribution entails the activation of brain structures that are primarily ipsilateral to the stimulated MFB, along with weak to negligible activation of the contralateral structures, thus impeding the formation of a cue-outcome association. To better understand the spread of activation of MFB stimulation across hemispheres, we studied whether unilateral MFB stimulation primarily activates the ipsilateral vStr or the vStr of both hemispheres. We simultaneously recorded neuronal activity in the vStr of both hemispheres in response to several sets of unilateral MFB stimulation in anesthetized and freely moving rats. Unilateral MFB stimulation evoked strong stimulus-dependent activation of vStr tonically active neurons (TANs), presumably the cholinergic interneurons, in both hemispheres. However, the TANs' activation patterns and responsiveness depended on whether the stimulus was delivered ipsilaterally or contralaterally to the recorded neuron. These findings indicate that unilateral MFB stimulation effectively activates the vStr in both hemispheres in a stimulus-dependent manner which may serve as neuronal substrate for the formation of cue-outcome associations during reinforcement learning.

How feedback, motor imagery, and reward influence brain self-regulation using real-time fMRI

The learning process involved in achieving brain self-regulation is presumed to be related to several factors, such as type of feedback, reward, mental imagery, duration of training, among others. Explicitly instructing participants to use mental imagery and monetary reward are common practices in real-time fMRI (rtfMRI) neurofeedback (NF), under the assumption that they will enhance and accelerate the learning process. However, it is still not clear what the optimal strategy is for improving volitional control. We investigated the differential effect of feedback, explicit instructions and monetary reward while training healthy individuals to up-regulate the blood-oxygen-level dependent (BOLD) signal in the supplementary motor area (SMA). Four groups were trained in a two-day rtfMRI-NF protocol: GF with NF only, GF,I with NF + explicit instructions (motor imagery), GF,R with NF + monetary reward, and GF,I,R with NF + explicit instructions (motor imagery) + monetary reward. Our results showed that GF increased significantly their BOLD self-regulation from day-1 to day-2 and GF,R showed the highest BOLD signal amplitude in SMA during the training. The two groups who were instructed to use motor imagery did not show a significant learning effect over the 2 days. The additional factors, namely motor imagery and reward, tended to increase the intersubject variability in the SMA during the course of training. Whole brain univariate and functional connectivity analyses showed common as well as distinct patterns in the four groups, representing the varied influences of feedback, reward, and instructions on the brain. Hum Brain Mapp 37:3153-3171, 2016. © 2016 Wiley Periodicals, Inc.

Explicit memory creation during sleep demonstrates a causal role of place cells in navigation

Hippocampal place cells assemblies are believed to support the cognitive map, and their reactivations during sleep are thought to be involved in spatial memory consolidation. By triggering intracranial rewarding stimulations by place cell spikes during sleep, we induced an explicit memory trace, leading to a goal-directed behavior toward the place field. This demonstrates that place cells' activity during sleep still conveys relevant spatial information and that this activity is functionally significant for navigation.

Bidirectional control of a one-dimensional robotic actuator by operant conditioning of a single unit in rat motor cortex

The design of efficient neuroprosthetic devices has become a major challenge for the long-term goal of restoring autonomy to motor-impaired patients. One approach for brain control of actuators consists in decoding the activity pattern obtained by simultaneously recording large neuronal ensembles in order to predict in real-time the subject's intention, and move the prosthesis accordingly. An alternative way is to assign the output of one or a few neurons by operant conditioning to control the prosthesis with rules defined by the experimenter, and rely on the functional adaptation of these neurons during learning to reach the desired behavioral outcome. Here, several motor cortex neurons were recorded simultaneously in head-fixed awake rats and were conditioned, one at a time, to modulate their firing rate up and down in order to control the speed and direction of a one-dimensional actuator carrying a water bottle. The goal was to maintain the bottle in front of the rat's mouth, allowing it to drink. After learning, all conditioned neurons modulated their firing rate, effectively controlling the bottle position so that the drinking time was increased relative to chance. The mean firing rate averaged over all bottle trajectories depended non-linearly on position, so that the mouth position operated as an attractor. Some modifications of mean firing rate were observed in the surrounding neurons, but to a lesser extent. Notably, the conditioned neuron reacted faster and led to a better control than surrounding neurons, as calculated by using the activity of those neurons to generate simulated bottle trajectories. Our study demonstrates the feasibility, even in the rodent, of using a motor cortex neuron to control a prosthesis in real-time bidirectionally. The learning process includes modifications of the activity of neighboring cortical neurons, while the conditioned neuron selectively leads the activity patterns associated with the prosthesis control.

Activity-dependent regulation of 'on' and 'off' responses in cat visual cortical receptive fields

1. A supervised learning procedure was applied to individual cat area 17 neurons to test the possible role of neuronal co-activity in controlling the plasticity of the spatial 'on-off' organization of visual cortical receptive fields (RFs). 2. Differential pairing between visual input evoked in a fixed position of the RF and preset levels of postsynaptic firing (imposed iontophoretically) were used alternately to boost the 'on' (or 'off') response to a 'high' level of firing (S+ pairing), and to reduce the opponent response (respectively 'off' or 'on') in the same position to a 'low' level (S- pairing). This associative procedure was repeated 50-100 times at a low temporal frequency (0.1-0.15 s-1). 3. Long-lasting modifications of the ratio of 'on-off' responses, measured in the paired position or integrated across the whole RF, were found in 44 % of the conditioned neurons (17/39), and in most cases this favoured the S+ paired characteristic. The amplitude change was on average half of that imposed during pairing. Comparable proportions of modified cells were obtained in 'simple' (13/27) and 'complex' (4/12) RFs, both in adult cats (4/11) and in kittens within the critical period (13/28). 4. The spatial selectivity of the pairing effects was studied by pseudorandomly stimulating both paired and spatially distinct unpaired positions within the RF. Most modifications were observed in the paired position (for 88 % of successful pairings). 5. In some cells (n = 13), a fixed delay pairing procedure was applied, in which the temporal phase of the onset of the current pulse was shifted by a few hundred milliseconds from the presentation or offset of the visual stimulus. Consecutive effects were observed in 4/13 cells, which retained the temporal pattern of activity imposed during pairing for 5-40 min. They were expressed in the paired region only. 6. The demonstration of long-lasting adaptive changes in the ratio of 'on' and 'off' responses, expressed in localized subregions of the RF, leads us to suggest that simple and complex RF organizations might be two stable functional states derived from a common connectivity scheme.

Evidence for the Involvement of Rat Olfactory Bulb in Processes Supporting Long-Term Olfactory Memory

Current advances in the neurobiology of learning and memory suggest the existence of experience-induced plasticity in sensorial pathways conveying relevant information to higher integrative brain structures. For instance, olfactory learning is known to induce long-lasting modifications of neural activity at the level of the first relay structure of the olfactory system, the olfactory bulb. The observed forms of plasticity depend on the action exerted during learning by ascending neuromodulatory systems, such as the noradrenergic (NA) system originating from the locus ceruleus. This study was aimed at investigating the importance of olfactory bulb plasticity in learning and retention of an olfactory task. In a daily training schedule animals had to learn to use multi-site electrical stimulation patterns of the olfactory bulb as discriminative cues for choosing between a palatable and a nonpalatable solution. We first examined the effects of a continuous intrabulbar infusion of propranolol (a beta-NA receptor antagonist) carried out during the learning period. We found that this treatment neither impaired the retention of a previously learned task nor the learning of a new task. However, the animals presented a severe deficit in long-term retention (>5 days) of the task learned under perfusion. Unexpectedly, this effect cannot be ascribed to a selective blockade of beta-NA receptors since infusion of the drug vehicle (saline-ascorbate) produced exactly the same deficit while a saline solution remained without effect. A final experiment showed that the selective deficit in long-term retention was not observed when the infusion of the saline-ascorbate solution started on the day following completion of learning. Taken together, these results suggest that ascorbate-sensitive neural processes occurring within the olfactory bulb during learning are of functional importance for long-term storage of olfactory information.

Field potential response changes in the rabbit olfactory bulb accompany behavioral habituation during the repeated presentation of unreinforced odors

Experiments were performed on waking rabbits to investigate the changes in both sniffing behavior and local field potential responses in the olfactory bulb during repeated exposure to unreinforced odors. Six rabbits were each implanted with 2 pairs of electrodes for differential recording of the bulbar extracellular field potential. Each animal was given 3 sequential sessions to each of 2 separate odors on 6 consecutive days, while monitoring the bulbar field potential activity and sniffing behavior. Behavioral sniffing responses exhibited rapid within-session decrement in amplitude and long term decrement across sessions. The within-session decrement showed spontaneous recovery between sessions. Both decremental changes in sniffing behavior were accompanied by changes in the bulbar field potential responses. The responses to novel odors were characterized by a reduction in amplitude of high frequency activity (40-80 Hz) and a corresponding increase in amplitude of low frequency activity (15-25 Hz). The high frequency component of the responses showed an initial increase in frequency to a novel odor on the first 3 presentations followed by a rapid decrease in frequency on subsequent trials in the first session which stabilized thereafter. No change in frequency or relative amplitude was observed for the low frequency component. The absolute difference between the odor evoked activity and the preceding control activity measured on each trial showed a significant decrement across sessions with no evidence for spontaneous recovery. The results demonstrate that olfactory bulb responses to novel unreinforced odors show both rapid and long-term changes which parallel changes in sniffing behavior.(ABSTRACT TRUNCATED AT 250 WORDS)

Operant conditioning in motor and neural integration

Empirical and theoretical reasons were given to investigate operant conditioning in a new, integrative approach within motor control physiology. Elements of inborn and learned behavior were presented in a framework specifying their stimuli and responses. The operant was redefined as a controlling discriminative stimulus, Sd, together with the response, R, it produces, on the basis of a previous literature of operant and instrumental research. Complex motor and neural activity were reviewed in accordance with partitioning of: responses, controlling stimulation, reinforcement, and functions of movement-produced stimulation. Schematics portrayed reinforcement principles through analysis of a fast pathway from Ia muscle spindle afferents to motor outflow. Methods were suggested to minimize operant units through selective reinforcement and establish them to defined end points of learning within composite, ongoing behavior. It was argued that operant neural mechanisms can be investigated efficiently only by starting with individual operants that are thoroughly characterized.

Operant conditioning of trigeminal nuclear evoked potentials

The purpose of this study was to demonstrate operant conditioning of potentials in the rat's trigeminal complex evoked by stimulation of the primary descending trigeminal tract. The amplitude of the prominent component of the potential, with a peak at 10 to 22 msec after the stimulus, was conditioned. Reinforcement was electrical stimulation of the medial forebrain bundle. A bidirectional conditioning paradigm was used to control for noncontingent effects of the reinforcer. Eight of the thirteen animals demonstrated control of the evoked potential. Application of the neural conditioning paradigm to the investigation of loci of neuronal plasticity is discussed.

Operant conditioning of vertical eye movements without visual feedback in the midpontine pretrigeminal cat

An operant conditioning of vertical eye movements was achieved in the midpontine pretrigeminal cat in total darkness by contingent reinforcement of spontaneous eye movements with lateral hypothalamic (LHT) reward stimulation, when each movement (upward direction was chosen in this experiment) exceeded a preset amplitude. However, the response rates in the dark were lower than those in the light and the time to reach the peak response rate was much longer. Recording of evoked potentials to optic chiasma (OC) stimulation revealed enhancement of late components of the visual cortex (VC) and superior colliculus (SC) responses in relation to eye movements. Sequential records of the averaged evoked responses associated with eye movements indicated that the amplitudes of the late components of the VC and SC waves gradually increased in the course of establishment of the operant conditioning, and decreased gradually during extinction. In a yoked control test, increase in amplitudes of the late components was much less significant during non-contingent reinforcement given independently of the eye movements. These results suggest that 'corollary discharge' may play a critical role as a cue in acquisition of the operant conditioning of vertical eye movements when visual feedback is absent in total darkness.

Instrumental cardiovascular conditioning: a review

This paper reviews experiments, hypotheses, and current controversies about instrumental cardiovascular conditioning. Demonstrations of such conditioning in curarized animals challenged a differentiation between instrumental and classical learning on the basis of their respective effector systems but did not prove direct operant learning by the autonomic nervous system. In humans, ethical prohibition of curarization and lack of adequate controls for respiration and muscle tension have resulted in incomplete understanding of the roles of voluntary, somatic mediators. Despite a variety of potential clinical applications of biofeedback, the available literature lacks studies of its efficacy compared to more standard modes of therapy. The physiological mechanisms and central neural pathways involved in instrumental cardiovascular conditioning remain almost totally unknown.

Epileptic and normal neurons in monkey neocortex: a quantitative study of degree of operant control

The object of this study was to quantify and compare the degree of control monkeys may assert over firing patterns of normal and epileptic neurons. Thirty-seven epileptic and 70 normal neurons were studied in detail. The operant task was for the monkey to generate specified consecutive interspike intervals (ISI). The monkeys demonstrated far greater accuracy in controlling consecutive ISIs of normal neurons and were only able to control the intervals between bursts from epileptic neurons. The data implies that high frequency bursts of action potentials from epileptic neurons are all-or-nothing events initiated by synaptic mechanisms. In addition, some data are from a monkey with epilepsia partialis continua; in comparison to less active foci, this focus was comprised of a higher percentage of 'pacemaker' epileptic neurons.

Operant conditioning of tonic firing patterns from precentral neurons in monkey neocortex

This report presents a single neurons operant conditioning paradigm which allows the quantification of operant control between neurons and monkeys. The operant task is for the monkey to change firing patterns of the neuron from phasic to tonic. In 60 neurons conditioned after the protocol had been standardized the following results were obtained. (1) For a fixed interspike interval target range, the time off target may be considered error, and this may be used as a parameter with which to judge operant neuronal control. (2) The degree to which the monkey could control a neuron was not correlative with the neuron's initial firing rate, firing rate variance, or pattern. (3) Neurons coactivated by distal arm muscle groups were, as a group, more highly controlled in comparison to neurons coactivated by proximal muscle groups. (4) Pyramidal tract neurons, as a group, appear more accurately controlled than non-pyramidal tract neurons. (5) The role of proprioception in response acquisition is discussed.

Binocular differences in cortical receptive fields of kittens after rotationally disparate binocular experience

Kittens were afforded visual experience only while wearing goggles fitted with prisms that rotated the inputs to the two eyes equally but in opposite directions about the visual axes (16 degrees for experimental subjects, 0 degrees for control subjects). Subsequently, receptive-field organization of the visual cortex was studied, special attention being given to the preferred orientation centered about the prism rotation experienced during early development. Thus, for moderate amounts of relative rotation, the development of interocular matching of orientation specificity in binocular cells of the visual cortex reflects the correspondence of early visual input between the two eyes.

Long-lasting facilitation of a synaptic potential following tetanization in the in vitro hippocampal slice

Field potentials evoked by stimulation of afferent fibers in stratum radiatum were recorded in the CA1 region of the hippocampal slice maintained in vitro. Stimulation rates of 3-50/sec produced a large increase in amplitude of the population spike in CA1. This increase was maintained for several hours after the tetanization. The facilitation phenomenon appeared to be specific to the synapse of stratum radiatum afferents onto CA1 pyramidal cells since: (1) stimulation outside the radiatum layer did not produce the effect, (2) antidromic field potentials recorded in CA3 were unchanged, (3) EPSP threshold in CA1 was unchanged, and (4) alveus tetanization did not produce a facilitatory effect.