Decision making under uncertainty in a spiking neural network model of the basal ganglia

On computational models of theory of mind and the imitative reinforcement learning in spiking neural networks

Theory of Mind is referred to the ability of inferring other's mental states, and it plays a crucial role in social cognition and learning. Biological evidences indicate that complex circuits are involved in this ability, including the mirror neuron system. The mirror neuron system influences imitation abilities and action understanding, leading to learn through observing others. To simulate this imitative learning behavior, a Theory-of-Mind-based Imitative Reinforcement Learning (ToM-based ImRL) framework is proposed. Employing the bio-inspired spiking neural networks and the mechanisms of the mirror neuron system, ToM-based ImRL is a bio-inspired computational model which enables an agent to effectively learn how to act in an interactive environment through observing an expert, inferring its goals, and imitating its behaviors. The aim of this paper is to review some computational attempts in modeling ToM and to explain the proposed ToM-based ImRL framework which is tested in the environment of River Raid game from Atari 2600 series.

Bio-inspired neural networks for decision-making mechanisms and neuromodulation for motor control in a differential robot

The aim of this work is to propose bio-inspired neural networks for decision-making mechanisms and modulation of motor control of an automaton. In this work, we have adapted and applied cortical synaptic circuits, such as short-term memory circuits, winner-take-all (WTA) class competitive neural networks, modulation neural networks, and nonlinear oscillation circuits, in order to make the automaton able to avoid obstacles and explore simulated and real environments. The performance achieved by using biologically inspired neural networks to solve the task at hand is similar to that of several works mentioned in the specialized literature. Furthermore, this work contributed to bridging the fields of computational neuroscience and robotics.

Brain-Inspired Affective Empathy Computational Model and Its Application on Altruistic Rescue Task

Affective empathy is an indispensable ability for humans and other species' harmonious social lives, motivating altruistic behavior, such as consolation and aid-giving. How to build an affective empathy computational model has attracted extensive attention in recent years. Most affective empathy models focus on the recognition and simulation of facial expressions or emotional speech of humans, namely Affective Computing. However, these studies lack the guidance of neural mechanisms of affective empathy. From a neuroscience perspective, affective empathy is formed gradually during the individual development process: experiencing own emotion—forming the corresponding Mirror Neuron System (MNS)—understanding the emotions of others through the mirror mechanism. Inspired by this neural mechanism, we constructed a brain-inspired affective empathy computational model, this model contains two submodels: (1) We designed an Artificial Pain Model inspired by the Free Energy Principle (FEP) to the simulate pain generation process in living organisms. (2) We build an affective empathy spiking neural network (AE-SNN) that simulates the mirror mechanism of MNS and has self-other differentiation ability. We apply the brain-inspired affective empathy computational model to the pain empathy and altruistic rescue task to achieve the rescue of companions by intelligent agents. To the best of our knowledge, our study is the first one to reproduce the emergence process of mirror neurons and anti-mirror neurons in the SNN field. Compared with traditional affective empathy computational models, our model is more biologically plausible, and it provides a new perspective for achieving artificial affective empathy, which has special potential for the social robots field in the future.

A neurocomputational model of creative processes

Creativity is associated with finding novel, surprising, and useful solutions. We argue that creative cognitive processes, divergent thinking, abstraction, and improvisation are constructed on different novelty-based processes. The prefrontal cortex plays a role in creative ideation by providing a control mechanism. Moreover, thinking about novel solutions activates the distant or loosely connected neurons of a semantic network that involves the hippocampus. Novelty can also be interpreted as different combinations of earlier learned processes, such as the motor sequencing mechanism of the basal ganglia. In addition, the cerebellum is responsible for the precise control of movements, which is particularly important in improvisation. Our neurocomputational perspective is based on three creative processes centered on novelty seeking, subserved by the prefrontal cortex, hippocampus, cerebellum, basal ganglia, and dopamine. The algorithmic implementation of our model would enable us to describe commonalities and differences between these creative processes based on the proposed neural circuitry. Given that most previous studies have mainly provided theoretical and conceptual models of creativity, this article presents the first brain-inspired neural network model of creative cognition.

A Brain-Inspired Theory of Mind Spiking Neural Network for Reducing Safety Risks of Other Agents

Artificial Intelligence (AI) systems are increasingly applied to complex tasks that involve interaction with multiple agents. Such interaction-based systems can lead to safety risks. Due to limited perception and prior knowledge, agents acting in the real world may unconsciously hold false beliefs and strategies about their environment, leading to safety risks in their future decisions. For humans, we can usually rely on the high-level theory of mind (ToM) capability to perceive the mental states of others, identify risk-inducing errors, and offer our timely help to keep others away from dangerous situations. Inspired by the biological information processing mechanism of ToM, we propose a brain-inspired theory of mind spiking neural network (ToM-SNN) model to enable agents to perceive such risk-inducing errors inside others' mental states and make decisions to help others when necessary. The ToM-SNN model incorporates the multiple brain areas coordination mechanisms and biologically realistic spiking neural networks (SNNs) trained with Reward-modulated Spike-Timing-Dependent Plasticity (R-STDP). To verify the effectiveness of the ToM-SNN model, we conducted various experiments in the gridworld environments with random agents' starting positions and random blocking walls. Experimental results demonstrate that the agent with the ToM-SNN model selects rescue behavior to help others avoid safety risks based on self-experience and prior knowledge. To the best of our knowledge, this study provides a new perspective to explore how agents help others avoid potential risks based on bio-inspired ToM mechanisms and may contribute more inspiration toward better research on safety risks.

Quantum superposition inspired spiking neural network

Despite advances in artificial intelligence models, neural networks still cannot achieve human performance, partly due to differences in how information is encoded and processed compared with human brain. Information in an artificial neural network (ANN) is represented using a statistical method and processed as a fitting function, enabling handling of structural patterns in image, text, and speech processing. However, substantial changes to the statistical characteristics of the data, for example, reversing the background of an image, dramatically reduce the performance. Here, we propose a quantum superposition spiking neural network (QS-SNN) inspired by quantum mechanisms and phenomena in the brain, which can handle reversal of image background color. The QS-SNN incorporates quantum theory with brain-inspired spiking neural network models from a computational perspective, resulting in more robust performance compared with traditional ANN models, especially when processing noisy inputs. The results presented here will inform future efforts to develop brain-inspired artificial intelligence.

GLSNN: A Multi-Layer Spiking Neural Network Based on Global Feedback Alignment and Local STDP Plasticity

Spiking Neural Networks (SNNs) are considered as the third generation of artificial neural networks, which are more closely with information processing in biological brains. However, it is still a challenge for how to train the non-differential SNN efficiently and robustly with the form of spikes. Here we give an alternative method to train SNNs by biologically-plausible structural and functional inspirations from the brain. Firstly, inspired by the significant top-down structural connections, a global random feedback alignment is designed to help the SNN propagate the error target from the output layer directly to the previous few layers. Then inspired by the local plasticity of the biological system in which the synapses are more tuned by the neighborhood neurons, a differential STDP is used to optimize local plasticity. Extensive experimental results on the benchmark MNIST (98.62%) and Fashion MNIST (89.05%) have shown that the proposed algorithm performs favorably against several state-of-the-art SNNs trained with backpropagation.

A neural algorithm for Drosophila linear and nonlinear decision-making

It has been evidenced that vision-based decision-making in Drosophila consists of both simple perceptual (linear) decision and value-based (non-linear) decision. This paper proposes a general computational spiking neural network (SNN) model to explore how different brain areas are connected contributing to Drosophila linear and nonlinear decision-making behavior. First, our SNN model could successfully describe all the experimental findings in fly visual reinforcement learning and action selection among multiple conflicting choices as well. Second, our computational modeling shows that dopaminergic neuron-GABAergic neuron-mushroom body (DA-GABA-MB) works in a recurrent loop providing a key circuit for gain and gating mechanism of nonlinear decision making. Compared with existing models, our model shows more biologically plausible on the network design and working mechanism, and could amplify the small differences between two conflicting cues more clearly. Finally, based on the proposed model, the UAV could quickly learn to make clear-cut decisions among multiple visual choices and flexible reversal learning resembling to real fly. Compared with linear and uniform decision-making methods, the DA-GABA-MB mechanism helps UAV complete the decision-making task with fewer steps.

Conscious Multisensory Integration: Introducing a Universal Contextual Field in Biological and Deep Artificial Neural Networks

Conscious awareness plays a major role in human cognition and adaptive behavior, though its function in multisensory integration is not yet fully understood, hence, questions remain: How does the brain integrate the incoming multisensory signals with respect to different external environments? How are the roles of these multisensory signals defined to adhere to the anticipated behavioral-constraint of the environment? This work seeks to articulate a novel theory on conscious multisensory integration (CMI) that addresses the aforementioned research challenges. Specifically, the well-established contextual field (CF) in pyramidal cells and coherent infomax theory (Kay et al., 1998; Kay and Phillips, 2011) is split into two functionally distinctive integrated input fields: local contextual field (LCF) and universal contextual field (UCF). LCF defines the modulatory sensory signal coming from some other parts of the brain (in principle from anywhere in space-time) and UCF defines the outside environment and anticipated behavior (based on past learning and reasoning). Both LCF and UCF are integrated with the receptive field (RF) to develop a new class of contextually-adaptive neuron (CAN), which adapts to changing environments. The proposed theory is evaluated using human contextual audio-visual (AV) speech modeling. Simulation results provide new insights into contextual modulation and selective multisensory information amplification/suppression. The central hypothesis reviewed here suggests that the pyramidal cell, in addition to the classical excitatory and inhibitory signals, receives LCF and UCF inputs. The UCF (as a steering force or tuner) plays a decisive role in precisely selecting whether to amplify/suppress the transmission of relevant/irrelevant feedforward signals, without changing the content e.g., which information is worth paying more attention to? This, as opposed to, unconditional excitatory and inhibitory activity in existing deep neural networks (DNNs), is called conditional amplification/suppression.

A Curiosity-Based Learning Method for Spiking Neural Networks

Spiking Neural Networks (SNNs) have shown favorable performance recently. Nonetheless, the time-consuming computation on neuron level and complex optimization limit their real-time application. Curiosity has shown great performance in brain learning, which helps biological brains grasp new knowledge efficiently and actively. Inspired by this leaning mechanism, we propose a curiosity-based SNN (CBSNN) model, which contains four main learning processes. Firstly, the network is trained with biologically plausible plasticity principles to get the novelty estimations of all samples in only one epoch; secondly, the CBSNN begins to repeatedly learn the samples whose novelty estimations exceed the novelty threshold and dynamically update the novelty estimations of samples according to the learning results in five epochs; thirdly, in order to avoid the overfitting of the novel samples and forgetting of the learned samples, CBSNN retrains all samples in one epoch; finally, step two and step three are periodically taken until network convergence. Compared with the state-of-the-art Voltage-driven Plasticity-centric SNN (VPSNN) under standard architecture, our model achieves a higher accuracy of 98.55% with only 54.95% of its computation cost on the MNIST hand-written digit recognition dataset. Similar conclusion can also be found out in other datasets, i.e., Iris, NETtalk, Fashion-MNIST, and CIFAR-10, respectively. More experiments and analysis further prove that such curiosity-based learning theory is helpful in improving the efficiency of SNNs. As far as we know, this is the first practical combination of the curiosity mechanism and SNN, and these improvements will make the realistic application of SNNs possible on more specific tasks within the von Neumann framework.

MiR‑500a‑5p promotes glioblastoma cell proliferation, migration and invasion by targeting chromodomain helicase DNA binding protein 5

Glioblastoma is one of the most common malignant primary tumors and develops in brain. The molecular mechanism that regulates glioblastoma occurrence still remains unknown. MicroRNA (miR)-500a-5p has been reported to be involved in hepatocellular carcinoma and breast cancer. Whether miR-500a-5p regulates glioblastoma progression requires further investigation. In the present study, miR-500a-5p was highly expressed in malignant glioblastoma tissues and cell lines. Overexpression of miR-500a-5p promoted glioblastoma cell proliferation, migration and invasion in vitro. In addition, knockdown of miR-500a-5p accelerated cell apoptosis. Furthermore, miR-500a-5p inhibition significantly impaired tumor growth in vivo. The present study further explored the downstream mechanism. The luciferase reporter assay revealed that miR-500a-5p directly binds the 3′-untranslated region of chromodomain helicase DNA binding protein 5 (CHD5) mRNA. MiR-500a-5p markedly inhibited CHD5 expression in glioblastoma cells. Furthermore, CHD5 was downregulated in glioblastoma tissues, and the expression levels of miR-500a-5p and CHD5 were inversely correlated. In addition, knockdown of CHD5 restored the inhibition of cell proliferation and migration triggered by miR-500a-5p silence. Finally, it was demonstrated that miR-500a-5p can serve as a novel biomarker for the diagnosis and prognosis of glioblastoma patients. Taken together, the results of the present study indicated that miR-500a-5p may have promoted glioblastoma development and progression by targeting CHD5.