1
|
Capone C, Paolucci PS. Towards biologically plausible model-based reinforcement learning in recurrent spiking networks by dreaming new experiences. Sci Rep 2024; 14:14656. [PMID: 38918553 PMCID: PMC11199658 DOI: 10.1038/s41598-024-65631-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Accepted: 06/21/2024] [Indexed: 06/27/2024] Open
Abstract
Humans and animals can learn new skills after practicing for a few hours, while current reinforcement learning algorithms require a large amount of data to achieve good performances. Recent model-based approaches show promising results by reducing the number of necessary interactions with the environment to learn a desirable policy. However, these methods require biological implausible ingredients, such as the detailed storage of older experiences, and long periods of offline learning. The optimal way to learn and exploit world-models is still an open question. Taking inspiration from biology, we suggest that dreaming might be an efficient expedient to use an inner model. We propose a two-module (agent and model) spiking neural network in which "dreaming" (living new experiences in a model-based simulated environment) significantly boosts learning. Importantly, our model does not require the detailed storage of experiences, and learns online the world-model and the policy. Moreover, we stress that our network is composed of spiking neurons, further increasing the biological plausibility and implementability in neuromorphic hardware.
Collapse
Affiliation(s)
- Cristiano Capone
- INFN, Sezione di Roma, Rome, RM, 00185, Italy.
- Natl. Center for Radiation Protection and Computational Physics, Istituto Superiore di Sanitá, Rome, 00161, Italy.
| | | |
Collapse
|
2
|
Liu G, Deng W, Xie X, Huang L, Tang H. Human-Level Control Through Directly Trained Deep Spiking Q-Networks. IEEE TRANSACTIONS ON CYBERNETICS 2023; 53:7187-7198. [PMID: 36063509 DOI: 10.1109/tcyb.2022.3198259] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
As the third-generation neural networks, spiking neural networks (SNNs) have great potential on neuromorphic hardware because of their high energy efficiency. However, deep spiking reinforcement learning (DSRL), that is, the reinforcement learning (RL) based on SNNs, is still in its preliminary stage due to the binary output and the nondifferentiable property of the spiking function. To address these issues, we propose a deep spiking Q -network (DSQN) in this article. Specifically, we propose a directly trained DSRL architecture based on the leaky integrate-and-fire (LIF) neurons and deep Q -network (DQN). Then, we adapt a direct spiking learning algorithm for the DSQN. We further demonstrate the advantages of using LIF neurons in DSQN theoretically. Comprehensive experiments have been conducted on 17 top-performing Atari games to compare our method with the state-of-the-art conversion method. The experimental results demonstrate the superiority of our method in terms of performance, stability, generalization and energy efficiency. To the best of our knowledge, our work is the first one to achieve state-of-the-art performance on multiple Atari games with the directly trained SNN.
Collapse
|
3
|
Gupta S, Singal G, Garg D, Jagannathan S. QC_SANE: Robust Control in DRL Using Quantile Critic With Spiking Actor and Normalized Ensemble. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023; 34:6656-6662. [PMID: 34874871 DOI: 10.1109/tnnls.2021.3129525] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Recently introduced deep reinforcement learning (DRL) techniques in discrete-time have resulted in significant advances in online games, robotics, and so on. Inspired from recent developments, we have proposed an approach referred to as Quantile Critic with Spiking Actor and Normalized Ensemble (QC_SANE) for continuous control problems, which uses quantile loss to train critic and a spiking neural network (NN) to train an ensemble of actors. The NN does an internal normalization using a scaled exponential linear unit (SELU) activation function and ensures robustness. The empirical study on multijoint dynamics with contact (MuJoCo)-based environments shows improved training and test results than the state-of-the-art approach: population coded spiking actor network (PopSAN).
Collapse
|
4
|
Zhao Z, Zhao F, Zhao Y, Zeng Y, Sun Y. A brain-inspired theory of mind spiking neural network improves multi-agent cooperation and competition. PATTERNS (NEW YORK, N.Y.) 2023; 4:100775. [PMID: 37602221 PMCID: PMC10435963 DOI: 10.1016/j.patter.2023.100775] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 01/17/2023] [Accepted: 05/19/2023] [Indexed: 08/22/2023]
Abstract
During dynamic social interaction, inferring and predicting others' behaviors through theory of mind (ToM) is crucial for obtaining benefits in cooperative and competitive tasks. Current multi-agent reinforcement learning (MARL) methods primarily rely on agent observations to select behaviors, but they lack inspiration from ToM, which limits performance. In this article, we propose a multi-agent ToM decision-making (MAToM-DM) model, which consists of a MAToM spiking neural network (MAToM-SNN) module and a decision-making module. We design two brain-inspired ToM modules (Self-MAToM and Other-MAToM) to predict others' behaviors based on self-experience and observations of others, respectively. Each agent can adjust its behavior according to the predicted actions of others. The effectiveness of the proposed model has been demonstrated through experiments conducted in cooperative and competitive tasks. The results indicate that integrating the ToM mechanism can enhance cooperation and competition efficiency and lead to higher rewards compared with traditional MARL models.
Collapse
Affiliation(s)
- Zhuoya Zhao
- Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
- School of Future Technology, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Feifei Zhao
- Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
| | - Yuxuan Zhao
- Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
| | - Yi Zeng
- Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
- School of Future Technology, University of Chinese Academy of Sciences, Beijing 100049, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
- Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai 200031, China
- State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
| | - Yinqian Sun
- Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
- School of Future Technology, University of Chinese Academy of Sciences, Beijing 100049, China
| |
Collapse
|
5
|
Li X, Ni Z, Ruan J, Meng L, Shi J, Zhang T, Xu B. Mixture of personality improved spiking actor network for efficient multi-agent cooperation. Front Neurosci 2023; 17:1219405. [PMID: 37483340 PMCID: PMC10361619 DOI: 10.3389/fnins.2023.1219405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2023] [Accepted: 06/07/2023] [Indexed: 07/25/2023] Open
Abstract
Adaptive multi-agent cooperation with especially unseen partners is becoming more challenging in multi-agent reinforcement learning (MARL) research, whereby conventional deep-learning-based algorithms suffer from the poor new-player-generalization problem, possibly caused by not considering theory-of-mind theory (ToM). Inspired by the ToM personality in cognitive psychology, where a human can easily resolve this problem by predicting others' intuitive personality first before complex actions, we propose a biologically-plausible algorithm named the mixture of personality (MoP) improved spiking actor network (SAN). The MoP module contains a determinantal point process to simulate the formation and integration of different personality types, and the SAN module contains spiking neurons for efficient reinforcement learning. The experimental results on the benchmark cooperative overcooked task showed that the proposed MoP-SAN algorithm could achieve higher performance for the paradigms with (learning) and without (generalization) unseen partners. Furthermore, ablation experiments highlighted the contribution of MoP in SAN learning, and some visualization analysis explained why the proposed algorithm is superior to some counterpart deep actor networks.
Collapse
Affiliation(s)
- Xiyun Li
- Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Future Technology, University of Chinese Academy of Sciences, Beijing, China
| | - Ziyi Ni
- Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Jingqing Ruan
- Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Future Technology, University of Chinese Academy of Sciences, Beijing, China
| | - Linghui Meng
- Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Jing Shi
- Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Tielin Zhang
- Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Bo Xu
- Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Future Technology, University of Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
- Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
| |
Collapse
|
6
|
Ming F, Gao F, Liu K, Zhao C. Cooperative modular reinforcement learning for large discrete action space problem. Neural Netw 2023; 161:281-296. [PMID: 36774866 DOI: 10.1016/j.neunet.2023.01.046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2022] [Revised: 12/04/2022] [Accepted: 01/29/2023] [Indexed: 02/05/2023]
Abstract
Deep reinforcement learning (DRL) has achieved remarkable results on high-dimension state tasks. However, it suffers in hard convergence and low sample efficiency when solving large discrete action space problems. To meet these challenges, we develop a cooperative modular reinforcement learning (CMRL) method to distributedly solve the problems with a large discrete action space. A general yet effective task decomposition method is proposed to decompose the complex decision task in a large action space into multiple decision sub-tasks in small action subsets, using a rule-based action division method. The CMRL method consisting of multiple Critic networks is proposed to settle the multiple sub-tasks, where each Critic network learns a decomposed value function to obtain the local optimal action in a sub-task. The global optimal action is cooperatively chosen by all local optimal actions. Moreover, we propose a new parallel training mechanism, which trains multiple Critic networks with different models and multi-data in parallel. Mathematical properties are proposed to analyze the rationality and superiority of CMRL. Four different simulation experiments are conducted to verify the generality and effectiveness of CMRL for large action space problems. The results show that CMRL has superior performance on training efficiency compared with classical and latest DRL methods while maintaining the accuracy of the solution.
Collapse
Affiliation(s)
- Fangzhu Ming
- The State Key Laboratory for Manufacturing System Engineering, Xi'an Jiaotong University, Xi'an, Shannxi, China
| | - Feng Gao
- The State Key Laboratory for Manufacturing System Engineering, Xi'an Jiaotong University, Xi'an, Shannxi, China
| | - Kun Liu
- The MOE KLINNS Lab of Xi'an Jiaotong University, Xi'an, Shaanxi, China.
| | - Chengmei Zhao
- The State Key Laboratory for Manufacturing System Engineering, Xi'an Jiaotong University, Xi'an, Shannxi, China
| |
Collapse
|
7
|
Akl M, Ergene D, Walter F, Knoll A. Toward robust and scalable deep spiking reinforcement learning. Front Neurorobot 2023; 16:1075647. [PMID: 36742191 PMCID: PMC9894879 DOI: 10.3389/fnbot.2022.1075647] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Accepted: 12/23/2022] [Indexed: 01/21/2023] Open
Abstract
Deep reinforcement learning (DRL) combines reinforcement learning algorithms with deep neural networks (DNNs). Spiking neural networks (SNNs) have been shown to be a biologically plausible and energy efficient alternative to DNNs. Since the introduction of surrogate gradient approaches that allowed to overcome the discontinuity in the spike function, SNNs can now be trained with the backpropagation through time (BPTT) algorithm. While largely explored on supervised learning problems, little work has been done on investigating the use of SNNs as function approximators in DRL. Here we show how SNNs can be applied to different DRL algorithms like Deep Q-Network (DQN) and Twin-Delayed Deep Deteministic Policy Gradient (TD3) for discrete and continuous action space environments, respectively. We found that SNNs are sensitive to the additional hyperparameters introduced by spiking neuron models like current and voltage decay factors, firing thresholds, and that extensive hyperparameter tuning is inevitable. However, we show that increasing the simulation time of SNNs, as well as applying a two-neuron encoding to the input observations helps reduce the sensitivity to the membrane parameters. Furthermore, we show that randomizing the membrane parameters, instead of selecting uniform values for all neurons, has stabilizing effects on the training. We conclude that SNNs can be utilized for learning complex continuous control problems with state-of-the-art DRL algorithms. While the training complexity increases, the resulting SNNs can be directly executed on neuromorphic processors and potentially benefit from their high energy efficiency.
Collapse
|
8
|
Haşegan D, Deible M, Earl C, D’Onofrio D, Hazan H, Anwar H, Neymotin SA. Training spiking neuronal networks to perform motor control using reinforcement and evolutionary learning. Front Comput Neurosci 2022; 16:1017284. [PMID: 36249482 PMCID: PMC9563231 DOI: 10.3389/fncom.2022.1017284] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2022] [Accepted: 08/31/2022] [Indexed: 11/13/2022] Open
Abstract
Artificial neural networks (ANNs) have been successfully trained to perform a wide range of sensory-motor behaviors. In contrast, the performance of spiking neuronal network (SNN) models trained to perform similar behaviors remains relatively suboptimal. In this work, we aimed to push the field of SNNs forward by exploring the potential of different learning mechanisms to achieve optimal performance. We trained SNNs to solve the CartPole reinforcement learning (RL) control problem using two learning mechanisms operating at different timescales: (1) spike-timing-dependent reinforcement learning (STDP-RL) and (2) evolutionary strategy (EVOL). Though the role of STDP-RL in biological systems is well established, several other mechanisms, though not fully understood, work in concert during learning in vivo. Recreating accurate models that capture the interaction of STDP-RL with these diverse learning mechanisms is extremely difficult. EVOL is an alternative method and has been successfully used in many studies to fit model neural responsiveness to electrophysiological recordings and, in some cases, for classification problems. One advantage of EVOL is that it may not need to capture all interacting components of synaptic plasticity and thus provides a better alternative to STDP-RL. Here, we compared the performance of each algorithm after training, which revealed EVOL as a powerful method for training SNNs to perform sensory-motor behaviors. Our modeling opens up new capabilities for SNNs in RL and could serve as a testbed for neurobiologists aiming to understand multi-timescale learning mechanisms and dynamics in neuronal circuits.
Collapse
Affiliation(s)
- Daniel Haşegan
- Vilcek Institute of Graduate Biomedical Sciences, NYU Grossman School of Medicine, New York, NY, United States
| | - Matt Deible
- Department of Computer Science, University of Pittsburgh, Pittsburgh, PA, United States
| | - Christopher Earl
- Department of Computer Science, University of Massachusetts Amherst, Amherst, MA, United States
| | - David D’Onofrio
- Center for Biomedical Imaging and Neuromodulation, Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY, United States
| | - Hananel Hazan
- Allen Discovery Center, Tufts University, Boston, MA, United States
| | - Haroon Anwar
- Center for Biomedical Imaging and Neuromodulation, Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY, United States
| | - Samuel A. Neymotin
- Center for Biomedical Imaging and Neuromodulation, Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY, United States
- Department of Psychiatry, NYU Grossman School of Medicine, New York, NY, United States
| |
Collapse
|
9
|
Sun Y, Zeng Y, Li Y. Solving the spike feature information vanishing problem in spiking deep Q network with potential based normalization. Front Neurosci 2022; 16:953368. [PMID: 36090282 PMCID: PMC9453154 DOI: 10.3389/fnins.2022.953368] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Accepted: 07/29/2022] [Indexed: 11/18/2022] Open
Abstract
Brain-inspired spiking neural networks (SNNs) are successfully applied to many pattern recognition domains. The SNNs-based deep structure has achieved considerable results in perceptual tasks, such as image classification and target detection. However, applying deep SNNs in reinforcement learning (RL) tasks is still a problem to be explored. Although there have been previous studies on the combination of SNNs and RL, most focus on robotic control problems with shallow networks or using the ANN-SNN conversion method to implement spiking deep Q networks (SDQN). In this study, we mathematically analyzed the problem of the disappearance of spiking signal features in SDQN and proposed a potential-based layer normalization (pbLN) method to train spiking deep Q networks directly. Experiment shows that compared with state-of-art ANN-SNN conversion method and other SDQN works, the proposed pbLN spiking deep Q networks (PL-SDQN) achieved better performance on Atari game tasks.
Collapse
Affiliation(s)
- Yinqian Sun
- Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Future Technology, University of Chinese Academy of Sciences, Beijing, China
| | - Yi Zeng
- Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Future Technology, University of Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
- National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
- *Correspondence: Yi Zeng
| | - Yang Li
- Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
10
|
Wu G, Liang D, Luan S, Wang J. Training Spiking Neural Networks for Reinforcement Learning Tasks With Temporal Coding Method. Front Neurosci 2022; 16:877701. [PMID: 36061595 PMCID: PMC9428400 DOI: 10.3389/fnins.2022.877701] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2022] [Accepted: 06/24/2022] [Indexed: 11/13/2022] Open
Abstract
Recent years witness an increasing demand for using spiking neural networks (SNNs) to implement artificial intelligent systems. There is a demand of combining SNNs with reinforcement learning architectures to find an effective training method. Recently, temporal coding method has been proposed to train spiking neural networks while preserving the asynchronous nature of spiking neurons to preserve the asynchronous nature of SNNs. We propose a training method that enables temporal coding method in RL tasks. To tackle the problem of high sparsity of spikes, we introduce a self-incremental variable to push each spiking neuron to fire, which makes SNNs fully differentiable. In addition, an encoding method is proposed to solve the problem of information loss of temporal-coded inputs. The experimental results show that the SNNs trained by our proposed method can achieve comparable performance of the state-of-the-art artificial neural networks in benchmark tasks of reinforcement learning.
Collapse
Affiliation(s)
- Guanlin Wu
- Academy of Military Science, Beijing, China
| | - Dongchen Liang
- Academy of Military Science, Beijing, China
- *Correspondence: Dongchen Liang
| | | | - Ji Wang
- College of Systems Engineering, National University of Defense Technology, Changsha, China
| |
Collapse
|
11
|
Jang S, Kim HI. Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning. SENSORS (BASEL, SWITZERLAND) 2022; 22:5845. [PMID: 35957399 PMCID: PMC9371101 DOI: 10.3390/s22155845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Revised: 07/25/2022] [Accepted: 08/02/2022] [Indexed: 06/15/2023]
Abstract
Effective exploration is one of the critical factors affecting performance in deep reinforcement learning. Agents acquire data to learn the optimal policy through exploration, and if it is not guaranteed, the data quality deteriorates, which leads to performance degradation. This study investigates the effect of initial entropy, which significantly influences exploration, especially in the early learning stage. The results of this study on tasks with discrete action space show that (1) low initial entropy increases the probability of learning failure, (2) the distributions of initial entropy for various tasks are biased towards low values that inhibit exploration, and (3) the initial entropy for discrete action space varies with both the initial weight and task, making it hard to control. We then devise a simple yet powerful learning strategy to deal with these limitations, namely, entropy-aware model initialization. The proposed algorithm aims to provide a model with high initial entropy to a deep reinforcement learning algorithm for effective exploration. Our experiments showed that the devised learning strategy significantly reduces learning failures and enhances performance, stability, and learning speed.
Collapse
Affiliation(s)
- Sooyoung Jang
- Intelligence Convergence Research Laboratory, Electronics and Telecommunications Research Institute (ETRI), Daejeon 34129, Korea
| | - Hyung-Il Kim
- Artificial Intelligence Research Laboratory, Electronics and Telecommunications Research Institute (ETRI), Daejeon 34129, Korea
| |
Collapse
|
12
|
Anti-interference of a small-world spiking neural network against pulse noise. APPL INTELL 2022. [DOI: 10.1007/s10489-022-03804-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
13
|
Spiking Memory Policy with Population-encoding for Partially Observable Markov Decision Process Problems. Cognit Comput 2022. [DOI: 10.1007/s12559-022-10030-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
|
14
|
Tan W, Kozma R, Patel D. Optimization methods for improved efficiency and performance of Deep Q-Networks upon conversion to neuromorphic population platforms. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.108257] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
15
|
Wunsch Ii DC. Admiring the Great Mountain: A Celebration Special Issue in Honor of Stephen Grossberg's 80th Birthday. Neural Netw 2019; 120:1-4. [PMID: 31587821 DOI: 10.1016/j.neunet.2019.09.015] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
This editorial summarizes selected key contributions of Prof. Stephen Grossberg and describes the papers in this 80th birthday special issue in his honor. His productivity, creativity, and vision would each be enough to mark a scientist of the first caliber. In combination, they have resulted in contributions that have changed the entire discipline of neural networks. Grossberg has been tremendously influential in engineering, dynamical systems, and artificial intelligence as well. Indeed, he has been one of the most important mentors and role models in my career, and has done so with extraordinary generosity and encouragement. All authors in this special issue have taken great pleasure in hereby commemorating his extraordinary career and contributions.
Collapse
|