Operant conditioning of rat navigation using electrical stimulation for directional cues and rewards

Amygdala electrical stimulation for operant conditioning in rat navigation

There have been several attempts to navigate the locomotion of animals by neuromodulation. The most common method is animal training with electrical brain stimulation for directional cues and rewards; the basic principle is to activate dopamine-mediated neural reward pathways such as the medial forebrain bundle (MFB) when the animal correctly follows the external commands. In this study, the amygdala, which is the brain region responsible for fear modulation, was targeted for punishment training. The brain regions of MFB, amygdala, and barrel cortex were electrically stimulated for reward, punishment, and directional cues, respectively. Electrical stimulation was applied to the amygdala of rats when they failed to follow directional commands. First, two different amygdala regions, i.e., basolateral amygdala (BLA) and central amygdala (CeA), were stimulated and compared in terms of behavior responses, success and correction rates for training, and gene expression for learning and memory. Then, the training was performed in three groups: group R (MFB stimulation for reward), group P (BLA stimulation for punishment), and group RP (both MFB and BLA stimulation for reward and punishment). In group P, after the training, RNA sequencing was conducted to detect gene expression and demonstrate the effect of punishment learning. Group P showed higher success rates than group R, and group RP exhibited the most effective locomotion control among the three groups. Gene expression results imply that BLA stimulation can be more effective as a punishment in the learning process than CeA stimulation. We developed a new method to navigate rat locomotion behaviors by applying amygdala stimulation.

Spatial preference behavior of robo-pigeons induced by electrical stimulus targeting fear nuclei

BACKGROUND

Numerous studies have confirmed that stimulating the mid-brain motor nuclei can regulate movement forcibly for robo-pigeons, but research on behavior modulation using non-motor nuclei is scarce.

OBJECTIVE

In this study, we constructed a spatial preference behavior by stimulating the stratum griseum periventriculare (SGP), a nucleus correlated with fear and escape, for robo-pigeons.

METHODS

The study was carried out in a square-enclosed experimental field, with a designated box serving as the 'safe' area for the robo-pigeons. If the robo-pigeon exits this area, the SGP will be stimulated. After a brief training period, the robo-pigeons will have a clear spatial preference for the box.

RESULTS

The result from five pigeons has shown that, after simple training, the animals develop a spatial preference for the box. They can quickly return to the box in any situation when the SGP is stimulated, with a success rate exceeding 80% (89.0 ± 6.5%). Moreover, this behavior is highly stable and remains consistent, unaffected by changes in the location of the box or the interference box.

CONCLUSION

The results prove that using the electrical stimulus could enable animals to accomplish more complex tasks. It may offer a novel approach to regulating pigeon behavior and further advance the study of cyborg animals.

Rattractor-Instant guidance of a rat into a virtual cage using the deep brain stimulation

We developed "Rattractor" (rat attractor), a system to apply electrical stimuli to the deep brain of a rat as it stays in a specified region or a virtual cage to demonstrate an instant electrophysiological feedback guidance for animals. Two wire electrodes were implanted in the brains of nine rats. The electrodes targeted the medial forebrain bundle (MFB), which is a part of the reward system in the deep brain. Following the recovery period, the rats were placed in a plain field where they could move freely, but wired to a stimulation circuit. An image sensor installed over the field detected the subject's position, which triggered the stimulator such that the rat remained within the virtual cage. We conducted a behavioral experiment to evaluate the sojourn ratio of rats residing in the region. Thereafter, a histological analysis of the rat brain was performed to confirm the position of the stimulation sites in the brain. Seven rats survived the surgery and the recovery period without technical failures such as connector breaks. We observed that three of them tended to stay in the virtual cage during stimulation, and this effect was maintained for two weeks. Histological analysis revealed that the electrode tips were correctly placed in the MFB region of the rats. The other four subjects showed no apparent preference for the virtual cage. In these rats, we did not find electrode tips in the MFB, or could not determine their positions. Almost half of the rats tended to remain inside the virtual cage when position-related reward stimuli were triggered in the MFB region. Notably, our system did not require previous training or sequential interventions to affect the behavioral preferences of subjects. This process is similar to the situation in which sheep are chased by a shepherd dog in the desired direction.

Somatosensory ECoG-based brain-machine interface with electrical stimulation on medial forebrain bundle

Brain-machine interface (BMI) provides an alternative route for controlling an external device with one's intention. For individuals with motor-related disability, the BMI technologies can be used to replace or restore motor functions. Therefore, BMIs for movement restoration generally decode the neural activity from the motor-related brain regions. In this study, however, we designed a BMI system that uses sensory-related neural signals for BMI combined with electrical stimulation for reward. Four-channel electrocorticographic (ECoG) signals were recorded from the whisker-related somatosensory cortex of rats and converted to extract the BMI signals to control the one-dimensional movement of a dot on the screen. At the same time, we used operant conditioning with electrical stimulation on medial forebrain bundle (MFB), which provides a virtual reward to motivate the rat to move the dot towards the desired center region. The BMI task training was performed for 7 days with ECoG recording and MFB stimulation. Animals successfully learned to move the dot location to the desired position using S1BF neural activity. This study successfully demonstrated that it is feasible to utilize the neural signals from the whisker somatosensory cortex for BMI system. In addition, the MFB electrical stimulation is effective for rats to learn the behavioral task for BMI.

A novel rat robot controlled by electrical stimulation of the nigrostriatal pathway

OBJECTIVE

Artificial manipulation of animal movement could offer interesting advantages and potential applications using the animal's inherited superior sensation and mobility. Although several behavior control models have been introduced, they generally epitomize virtual reward-based training models. In this model, rats are trained multiple times so they can recall the relationship between cues and rewards. It is well known that activation of one side of the nigrostriatal pathway (NSP) in the rat induces immediate turning toward the contralateral side. However, this NSP stimulation-induced directional movement has not been used for the purpose of animal-robot navigation. In this study, the authors aimed to electrically stimulate the NSP of conscious rats to build a command-prompt rat robot.

METHODS

Repetitive NSP stimulation at 1-second intervals was applied via implanted electrodes to induce immediate contraversive turning movements in 7 rats in open field tests in the absence of any sensory cues or rewards. The rats were manipulated to navigate from the start arm to a target zone in either the left or right arm of a T-maze. A leftward trial was followed by a rightward trial, and each rat completed a total of 10 trials. In the control group, 7 rats were tested in the same way without NSP stimulation. The time taken to navigate the maze was compared between experimental and control groups.

RESULTS

All rats in the experimental group successfully reached the target area for all 70 trials in a short period of time with a short interstimulus interval (< 0.7 seconds), but only 41% of rats in the control group reached the target area and required a longer period of time to do so. The experimental group made correct directional turning movements at the intersection zone of the T-maze, taking significantly less time than the control group. No significant difference in navigation duration for the forward movements on the start and goal arms was observed between the two groups. However, the experimental group showed quick and accurate movement at the intersection zone, which made the difference in the success rate and elapsed time of tasks.

CONCLUSIONS

The results of this study clearly indicate that a rat-robot model based on NSP stimulation can be a practical alternative to previously reported models controlled by virtual sensory cues and rewards.

NLM-HS: Navigation Learning Model Based on a Hippocampal-Striatal Circuit for Explaining Navigation Mechanisms in Animal Brains

Neurophysiological studies have shown that the hippocampus, striatum, and prefrontal cortex play different roles in animal navigation, but it is still less clear how these structures work together. In this paper, we establish a navigation learning model based on the hippocampal-striatal circuit (NLM-HS), which provides a possible explanation for the navigation mechanism in the animal brain. The hippocampal model generates a cognitive map of the environment and performs goal-directed navigation by using a place cell sequence planning algorithm. The striatal model performs reward-related habitual navigation by using the classic temporal difference learning algorithm. Since the two models may produce inconsistent behavioral decisions, the prefrontal cortex model chooses the most appropriate strategies by using a strategy arbitration mechanism. The cognitive and learning mechanism of the NLM-HS works in two stages of exploration and navigation. First, the agent uses a hippocampal model to construct the cognitive map of the unknown environment. Then, the agent uses the strategy arbitration mechanism in the prefrontal cortex model to directly decide which strategy to choose. To test the validity of the NLM-HS, the classical Tolman detour experiment was reproduced. The results show that the NLM-HS not only makes agents show environmental cognition and navigation behavior similar to animals, but also makes behavioral decisions faster and achieves better adaptivity than hippocampal or striatal models alone.

Toward Studying Cognition in a Dish

Bioengineered neural tissues help advance our understanding of neurodevelopment, regeneration, and neural disease; however, it remains unclear whether they can replicate higher-order functions including cognition. Building upon technical achievements in the fields of biomaterials, tissue engineering, and cell biology, investigators have generated an assortment of artificial brain structures and cocultured circuits. Though they have displayed basic electrochemical signaling, their capacities to generate minimal patterns of information processing suggestive of high-order cognitive analogues have not yet been explored. Here, we review the current state of neural tissue engineering and consider the possibility of a study of cognition in vitro. We adopt a practical definition of minimal cognition, anticipate problems of measurement, and discuss solutions toward a study of cognition in a dish.

Bilateral responses of rat ventral striatum tonically active neurons to unilateral medial forebrain bundle stimulation

Unilateral medial forebrain bundle (MFB) stimulation is an extremely effective promoter of reinforcement learning irrespective of the conditioned cue's laterality. The effectiveness of unilateral MFB stimulation, which activates the mesolimbic pathway connecting the ventral tegmental area to the ventral striatum (vStr), is surprising considering that these fibers rarely cross to the contralateral hemisphere. Specifically, this type of biased fiber distribution entails the activation of brain structures that are primarily ipsilateral to the stimulated MFB, along with weak to negligible activation of the contralateral structures, thus impeding the formation of a cue-outcome association. To better understand the spread of activation of MFB stimulation across hemispheres, we studied whether unilateral MFB stimulation primarily activates the ipsilateral vStr or the vStr of both hemispheres. We simultaneously recorded neuronal activity in the vStr of both hemispheres in response to several sets of unilateral MFB stimulation in anesthetized and freely moving rats. Unilateral MFB stimulation evoked strong stimulus-dependent activation of vStr tonically active neurons (TANs), presumably the cholinergic interneurons, in both hemispheres. However, the TANs' activation patterns and responsiveness depended on whether the stimulus was delivered ipsilaterally or contralaterally to the recorded neuron. These findings indicate that unilateral MFB stimulation effectively activates the vStr in both hemispheres in a stimulus-dependent manner which may serve as neuronal substrate for the formation of cue-outcome associations during reinforcement learning.

Adaptation effects of medial forebrain bundle micro-electrical stimulation

Brain micro-electrical stimulation and its applications are among the most important issues in the field of brain science and neurophysiology. Deep brain stimulation techniques have been used in different theraputic or alternative medicine applications including chronic pain control, tremor control, Parkinson’s disease control and depression control. Recently, brain electrical stimulation has been used for tele-control and navigation of small animals such as rodents and birds. Electrical stimulation of the medial forebrain bundle (MFB) area has been reported to induce a pleasure sensation in rat which can be used as a virtual reward for rat navigation. In all cases of electrical stimulation, the temporal adaptation may deteriorate the instantaneous effects of the stimulation. Here, we study the adaptation effects of the MFB electrical stimulation in rats. The animals are taught to press a key in an operant conditioning chamber to self-stimulate the MFB region and receive a virtual reward for each key press. Based on the number of key presses, and statistical analyses the effects of adaptation on MFB stimulation is evaluated. The stimulation frequency were changed from 100 to 400 Hz, the amplitude were changed from 50 to 170 µA and the pulse-width were changed from 180 to 2000 µs. In the frequency of 250 Hz the adaptation effect were observed. The amplitude did not show a significant effect on MFB adaptation. For all values of pulse-widths, the adaptation occurred over two consecutive days, meaning that the number of key presses on the second day was less than the first day.

Remote-Controlled Fully Implantable Neural Stimulator for Freely Moving Small Animal

The application of a neural stimulator to small animals is highly desired for the investigation of electrophysiological studies and development of neuroprosthetic devices. For this purpose, it is essential for the device to be implemented with the capabilities of full implantation and wireless control. Here, we present a fully implantable stimulator with remote controllability, compact size, and minimal power consumption. Our stimulator consists of modular units of (1) a surface-type cortical array for inducing directional change of a rat, (2) a depth-type array for providing rewards, and (3) a package for accommodating the stimulating electronics, a battery and ZigBee telemetry, all of which are assembled after independent fabrication and implantation using customized flat cables and connectors. All three modules were packaged using liquid crystal polymer (LCP) to avoid any chemical reaction after implantation. After bench-top evaluation of device functionality, the stimulator was implanted into rats to train the animals to turn to the left (or right) following a directional cue applied to the barrel cortex. Functionality of the device was also demonstrated in a three-dimensional (3D) maze structure, by guiding the rats to better navigate in the maze. The movement of the rat could be wirelessly controlled by a combination of artificial sensation evoked by the surface electrode array and reward stimulation. We could induce rats to turn left or right in free space and help their navigation through the maze. The polymeric packaging and modular design could encapsulate the devices with strict size limitations, which made it possible to fully implant the device into rats. Power consumption was minimized by a dual-mode power-saving scheme with duty cycling. The present study demonstrated feasibility of the proposed neural stimulator to be applied to neuroprosthesis research.

Spectral Modification by Operant Conditioning of Cortical Theta Suppression in Rats

Objective

Brain activity is known to be voluntarily controllable by neurofeedback, a kind of electroencephalographic (EEG) operant conditioning. Although its efficacy in clinical effects has been reported, it is yet to be uncovered whether or how a specific band activity is controllable. Here, we examined EEG spectral profiles along with conditioning training of a specific brain activity, theta band (4–8 Hz) amplitude, in rats.

Methods

During training, the experimental group received electrical stimulation to the medial forebrain bundle contingent to suppression of theta activity, while the control group received stimulation non-contingent to its own band activity.

Results

In the experimental group, theta activity gradually decreased within the training session, while there was an increase of theta activity in the control group. There was a significant difference in theta activity during the sessions between the two groups. The spectral theta peak, originally located at 7 Hz, shifted further towards higher frequencies in the experimental group.

Conclusion

Our results showed that an operant conditioning technique could train rats to control their specific EEG activity indirectly, and it may be used as an animal model for studying how neuronal systems work in human neurofeedback.

A wearable system for olfactory electrophysiological recording and animal motion control

BACKGROUND

Bran-computer interface (BCI) is an important technique used in brain science. However, the large size of equipment and wires severely limit its practical applications.

NEW METHODS

This study presents a wearable system with bidirectional brain-computer interface based on Wi-Fi technology, which can be used for olfactory electrophysiological recording and animal motion control.

RESULTS

On the "brain-to-computer" side, the results show that the wireless system can record high-quality olfactory electrophysiological signals for over a month. By analyzing the recorded data, we find that the same mitral/tufted (M/T) cells can be activated by many odorants and different M/T cells can be activated by a single odorant. Further, we find neurons in dorsal lateral OB are highly sensitive to isoamyl acetate. On the "computer-to-brain" side, the results show that we can efficiently control rats' motions by applying electrical stimulations to electrodes implanted in specific brain regions.

COMPARISON WITH EXISTING METHODS

Most existing wireless BCI systems are designed for either recording or stimulating while our system is a bidirectional BCI featured with both functions. Taking advantage of our years of experience in olfactory decoding, we developed the first wireless system for olfactory electrophysiological recording and animal motion control. It provides high-quality recording and efficient motion control for a long time.

CONCLUSIONS

The system provides possibility of practical BCI applications, such as in vivo bioelectronic nose and "rat-robot".

Manipulation of Rat Movement via Nigrostriatal Stimulation Controlled by Human Visually Evoked Potentials

Here, we report that the development of a brain-to-brain interface (BBI) system that enables a human user to manipulate rat movement without any previous training. In our model, the remotely-guided rats (known as ratbots) successfully navigated a T-maze via contralateral turning behaviour induced by electrical stimulation of the nigrostriatal (NS) pathway by a brain- computer interface (BCI) based on the human controller's steady-state visually evoked potentials (SSVEPs). The system allowed human participants to manipulate rat movement with an average success rate of 82.2% and at an average rat speed of approximately 1.9 m/min. The ratbots had no directional preference, showing average success rates of 81.1% and 83.3% for the left- and right-turning task, respectively. This is the first study to demonstrate the use of NS stimulation for developing a highly stable ratbot that does not require previous training, and is the first instance of a training-free BBI for rat navigation. The results of this study will facilitate the development of borderless communication between human and untrained animals, which could not only improve the understanding of animals in humans, but also allow untrained animals to more effectively provide humans with information obtained with their superior perception.

Maze learning by a hybrid brain-computer system

The combination of biological and artificial intelligence is particularly driven by two major strands of research: one involves the control of mechanical, usually prosthetic, devices by conscious biological subjects, whereas the other involves the control of animal behaviour by stimulating nervous systems electrically or optically. However, to our knowledge, no study has demonstrated that spatial learning in a computer-based system can affect the learning and decision making behaviour of the biological component, namely a rat, when these two types of intelligence are wired together to form a new intelligent entity. Here, we show how rule operations conducted by computing components contribute to a novel hybrid brain-computer system, i.e., ratbots, exhibit superior learning abilities in a maze learning task, even when their vision and whisker sensation were blocked. We anticipate that our study will encourage other researchers to investigate combinations of various rule operations and other artificial intelligence algorithms with the learning and memory processes of organic brains to develop more powerful cyborg intelligence systems. Our results potentially have profound implications for a variety of applications in intelligent systems and neural rehabilitation.

Automatic Training of Rat Cyborgs for Navigation

A rat cyborg system refers to a biological rat implanted with microelectrodes in its brain, via which the outer electrical stimuli can be delivered into the brain in vivo to control its behaviors. Rat cyborgs have various applications in emergency, such as search and rescue in disasters. Prior to a rat cyborg becoming controllable, a lot of effort is required to train it to adapt to the electrical stimuli. In this paper, we build a vision-based automatic training system for rat cyborgs to replace the time-consuming manual training procedure. A hierarchical framework is proposed to facilitate the colearning between rats and machines. In the framework, the behavioral states of a rat cyborg are visually sensed by a camera, a parameterized state machine is employed to model the training action transitions triggered by rat's behavioral states, and an adaptive adjustment policy is developed to adaptively adjust the stimulation intensity. The experimental results of three rat cyborgs prove the effectiveness of our system. To the best of our knowledge, this study is the first to tackle automatic training of animal cyborgs.

Intelligence-Augmented Rat Cyborgs in Maze Solving

Cyborg intelligence is an emerging kind of intelligence paradigm. It aims to deeply integrate machine intelligence with biological intelligence by connecting machines and living beings via neural interfaces, enhancing strength by combining the biological cognition capability with the machine computational capability. Cyborg intelligence is considered to be a new way to augment living beings with machine intelligence. In this paper, we build rat cyborgs to demonstrate how they can expedite the maze escape task with integration of machine intelligence. We compare the performance of maze solving by computer, by individual rats, and by computer-aided rats (i.e. rat cyborgs). They were asked to find their way from a constant entrance to a constant exit in fourteen diverse mazes. Performance of maze solving was measured by steps, coverage rates, and time spent. The experimental results with six rats and their intelligence-augmented rat cyborgs show that rat cyborgs have the best performance in escaping from mazes. These results provide a proof-of-principle demonstration for cyborg intelligence. In addition, our novel cyborg intelligent system (rat cyborg) has great potential in various applications, such as search and rescue in complex terrains.

A remote constant current stimulator designed for rat-robot navigation

In this paper, a remote stimulator is developed for rat-robot navigation based on the technique of Brain-Computer-Interface (BCI). The stimulator can output constant current from 0 to 1000 µA, which overcome several shortages of our previous constant voltage stimulator. The constant current stimulator consists of four major components, including power supply, micro control unit (MCU), constant current source and bluetooth transceiver for downloading stimulation commands. The stimulator has a weight of about 20 g and size of 32 mm*25 mm*6mm. It has five channels of stimulation, which are connected with implanted microelectrodes in rat brain. The electrical parameters were characterized on three rats with different recovery time after brain surgery. Increasing current stimulations were applied on the dorsolateral periaqueductal gray (dlPAG) area to prove the effect of current stimulation on rat behavior.