1
|
周 茜, 郑 鹏, 李 小. [A bio-inspired hierarchical spiking neural network with biological synaptic plasticity for event camera object recognition]. SHENG WU YI XUE GONG CHENG XUE ZA ZHI = JOURNAL OF BIOMEDICAL ENGINEERING = SHENGWU YIXUE GONGCHENGXUE ZAZHI 2023; 40:692-699. [PMID: 37666759 PMCID: PMC10477392 DOI: 10.7507/1001-5515.202207040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Revised: 07/14/2023] [Indexed: 09/06/2023]
Abstract
With inherent sparse spike-based coding and asynchronous event-driven computation, spiking neural network (SNN) is naturally suitable for processing event stream data of event cameras. In order to improve the feature extraction and classification performance of bio-inspired hierarchical SNNs, in this paper an event camera object recognition system based on biological synaptic plasticity is proposed. In our system input event streams were firstly segmented adaptively using spiking neuron potential to improve computational efficiency of the system. Multi-layer feature learning and classification are implemented by our bio-inspired hierarchical SNN with synaptic plasticity. After Gabor filter-based event-driven convolution layer which extracted primary visual features of event streams, we used a feature learning layer with unsupervised spiking timing dependent plasticity (STDP) rule to help the network extract frequent salient features, and a feature learning layer with reward-modulated STDP rule to help the network learn diagnostic features. The classification accuracies of the network proposed in this paper on the four benchmark event stream datasets were better than the existing bio-inspired hierarchical SNNs. Moreover, our method showed good classification ability for short event stream input data, and was robust to input event stream noise. The results show that our method can improve the feature extraction and classification performance of this kind of SNNs for event camera object recognition.
Collapse
Affiliation(s)
- 茜 周
- 河北工业大学 省部共建电工装备可靠性与智能化国家重点实验室(天津 300130)State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300130, P. R. China
- 河北工业大学 生命科学与健康工程学院 河北省生物电磁与神经工程重点实验室(天津 300130)Hebei Key Laboratory of Bioelectromagnetics and Neural Engineering, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300130, P. R. China
- 河北工业大学 生命科学与健康工程学院 天津市生物电工与智能健康重点实验室(天津 300130)Tianjin Key Laboratory of Bioelectricity and Intelligent Health, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300130, P. R. China
| | - 鹏 郑
- 河北工业大学 省部共建电工装备可靠性与智能化国家重点实验室(天津 300130)State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300130, P. R. China
- 河北工业大学 生命科学与健康工程学院 河北省生物电磁与神经工程重点实验室(天津 300130)Hebei Key Laboratory of Bioelectromagnetics and Neural Engineering, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300130, P. R. China
- 河北工业大学 生命科学与健康工程学院 天津市生物电工与智能健康重点实验室(天津 300130)Tianjin Key Laboratory of Bioelectricity and Intelligent Health, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300130, P. R. China
| | - 小虎 李
- 河北工业大学 省部共建电工装备可靠性与智能化国家重点实验室(天津 300130)State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin 300130, P. R. China
- 河北工业大学 生命科学与健康工程学院 河北省生物电磁与神经工程重点实验室(天津 300130)Hebei Key Laboratory of Bioelectromagnetics and Neural Engineering, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300130, P. R. China
- 河北工业大学 生命科学与健康工程学院 天津市生物电工与智能健康重点实验室(天津 300130)Tianjin Key Laboratory of Bioelectricity and Intelligent Health, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin 300130, P. R. China
| |
Collapse
|
2
|
Fernandez A, Acharya M, Lee HG, Schimpf J, Jiang Y, Lou D, Tian Z, Martin LW. Thin-Film Ferroelectrics. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2022; 34:e2108841. [PMID: 35353395 DOI: 10.1002/adma.202108841] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Revised: 03/03/2022] [Indexed: 06/14/2023]
Abstract
Over the last 30 years, the study of ferroelectric oxides has been revolutionized by the implementation of epitaxial-thin-film-based studies, which have driven many advances in the understanding of ferroelectric physics and the realization of novel polar structures and functionalities. New questions have motivated the development of advanced synthesis, characterization, and simulations of epitaxial thin films and, in turn, have provided new insights and applications across the micro-, meso-, and macroscopic length scales. This review traces the evolution of ferroelectric thin-film research through the early days developing understanding of the roles of size and strain on ferroelectrics to the present day, where such understanding is used to create complex hierarchical domain structures, novel polar topologies, and controlled chemical and defect profiles. The extension of epitaxial techniques, coupled with advances in high-throughput simulations, now stands to accelerate the discovery and study of new ferroelectric materials. Coming hand-in-hand with these new materials is new understanding and control of ferroelectric functionalities. Today, researchers are actively working to apply these lessons in a number of applications, including novel memory and logic architectures, as well as a host of energy conversion devices.
Collapse
Affiliation(s)
- Abel Fernandez
- Department of Materials Science and Engineering, University of California, Berkeley, Berkeley, CA, 94720, USA
- Materials Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, USA
| | - Megha Acharya
- Department of Materials Science and Engineering, University of California, Berkeley, Berkeley, CA, 94720, USA
- Materials Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, USA
| | - Han-Gyeol Lee
- Department of Materials Science and Engineering, University of California, Berkeley, Berkeley, CA, 94720, USA
- Materials Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, USA
| | - Jesse Schimpf
- Department of Materials Science and Engineering, University of California, Berkeley, Berkeley, CA, 94720, USA
- Materials Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, USA
| | - Yizhe Jiang
- Department of Materials Science and Engineering, University of California, Berkeley, Berkeley, CA, 94720, USA
- Materials Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, USA
| | - Djamila Lou
- Department of Materials Science and Engineering, University of California, Berkeley, Berkeley, CA, 94720, USA
- Materials Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, USA
| | - Zishen Tian
- Department of Materials Science and Engineering, University of California, Berkeley, Berkeley, CA, 94720, USA
- Materials Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, USA
| | - Lane W Martin
- Department of Materials Science and Engineering, University of California, Berkeley, Berkeley, CA, 94720, USA
- Materials Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, CA, 94720, USA
| |
Collapse
|
4
|
Berggren K, Xia Q, Likharev KK, Strukov DB, Jiang H, Mikolajick T, Querlioz D, Salinga M, Erickson JR, Pi S, Xiong F, Lin P, Li C, Chen Y, Xiong S, Hoskins BD, Daniels MW, Madhavan A, Liddle JA, McClelland JJ, Yang Y, Rupp J, Nonnenmann SS, Cheng KT, Gong N, Lastras-Montaño MA, Talin AA, Salleo A, Shastri BJ, de Lima TF, Prucnal P, Tait AN, Shen Y, Meng H, Roques-Carmes C, Cheng Z, Bhaskaran H, Jariwala D, Wang H, Shainline JM, Segall K, Yang JJ, Roy K, Datta S, Raychowdhury A. Roadmap on emerging hardware and technology for machine learning. NANOTECHNOLOGY 2021; 32:012002. [PMID: 32679577 DOI: 10.1088/1361-6528/aba70f] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Recent progress in artificial intelligence is largely attributed to the rapid development of machine learning, especially in the algorithm and neural network models. However, it is the performance of the hardware, in particular the energy efficiency of a computing system that sets the fundamental limit of the capability of machine learning. Data-centric computing requires a revolution in hardware systems, since traditional digital computers based on transistors and the von Neumann architecture were not purposely designed for neuromorphic computing. A hardware platform based on emerging devices and new architecture is the hope for future computing with dramatically improved throughput and energy efficiency. Building such a system, nevertheless, faces a number of challenges, ranging from materials selection, device optimization, circuit fabrication and system integration, to name a few. The aim of this Roadmap is to present a snapshot of emerging hardware technologies that are potentially beneficial for machine learning, providing the Nanotechnology readers with a perspective of challenges and opportunities in this burgeoning field.
Collapse
Affiliation(s)
- Karl Berggren
- Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, MA 02139, United States of America
| | - Qiangfei Xia
- Department of Electrical and Computer Engineering, University of Massachusetts, Amherst, MA, United States of America
| | | | - Dmitri B Strukov
- Department of Electrical and Computer Engineering, University of California at Santa Barbara, Santa Barbara, CA 93106, United States of America
| | - Hao Jiang
- School of Engineering & Applied Science Yale University, CT, United States of America
| | | | | | - Martin Salinga
- Institut für Materialphysik, Westfälische Wilhelms-Universität Münster, Germany
| | - John R Erickson
- Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA 15261, United States of America
| | - Shuang Pi
- Lam Research, Fremont, CA, United States of America
| | - Feng Xiong
- Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA 15261, United States of America
| | - Peng Lin
- Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, MA 02139, United States of America
| | - Can Li
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| | - Yu Chen
- School of information science and technology, Fudan University, Shanghai, People's Republic of China
| | - Shisheng Xiong
- School of information science and technology, Fudan University, Shanghai, People's Republic of China
| | - Brian D Hoskins
- Physical Measurements Laboratory, National Institute of Standards and Technology, Gaithersburg, MD 20899, United States of America
| | - Matthew W Daniels
- Physical Measurements Laboratory, National Institute of Standards and Technology, Gaithersburg, MD 20899, United States of America
| | - Advait Madhavan
- Physical Measurements Laboratory, National Institute of Standards and Technology, Gaithersburg, MD 20899, United States of America
- Institute for Research in Electronics and Applied Physics, University of Maryland, College Park, MD, United States of America
| | - James A Liddle
- Physical Measurements Laboratory, National Institute of Standards and Technology, Gaithersburg, MD 20899, United States of America
| | - Jabez J McClelland
- Physical Measurements Laboratory, National Institute of Standards and Technology, Gaithersburg, MD 20899, United States of America
| | - Yuchao Yang
- School of Electronics Engineering and Computer Science, Peking University, Beijing, People's Republic of China
| | - Jennifer Rupp
- Department of Materials Science and Engineering and Department of Electrical Engineering & Computer Science, Massachusetts Institute of Technology, Cambridge, MA 02139, United States of America
- Electrochemical Materials, ETHZ Department of Materials, Hönggerbergring 64, Zürich 8093, Switzerland
| | - Stephen S Nonnenmann
- Department of Mechanical & Industrial Engineering, University of Massachusetts-Amherst, MA, United States of America
| | - Kwang-Ting Cheng
- School of Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, People's Republic of China
| | - Nanbo Gong
- IBM T J Watson Research Center, Yorktown Heights, NY 10598, United States of America
| | - Miguel Angel Lastras-Montaño
- Instituto de Investigación en Comunicación Óptica, Facultad de Ciencias, Universidad Autónoma de San Luis Potosí, México
| | - A Alec Talin
- Sandia National Laboratories, Livermore, CA 94551, United States of America
| | - Alberto Salleo
- Department of Materials Science and Engineering, Stanford University, Stanford, California, United States of America
| | - Bhavin J Shastri
- Department of Physics, Engineering Physics & Astronomy, Queen's University, Kingston ON KL7 3N6, Canada
| | - Thomas Ferreira de Lima
- Department of Electrical Engineering, Princeton University, Princeton, NJ 08544, United States of America
| | - Paul Prucnal
- Department of Electrical Engineering, Princeton University, Princeton, NJ 08544, United States of America
| | - Alexander N Tait
- Physical Measurement Laboratory, National Institute of Standards and Technology (NIST), Boulder, CO 80305, United States of America
| | - Yichen Shen
- Lightelligence, 268 Summer Street, Boston, MA 02210, United States of America
| | - Huaiyu Meng
- Lightelligence, 268 Summer Street, Boston, MA 02210, United States of America
| | - Charles Roques-Carmes
- Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, MA 02139, United States of America
| | - Zengguang Cheng
- Department of Materials, University of Oxford, Oxford OX1 3PH, United Kingdom
- State Key Laboratory of ASIC and System, School of Microelectronics, Fudan University, Shanghai 200433, People's Republic of China
| | - Harish Bhaskaran
- Department of Materials, University of Oxford, Oxford OX1 3PH, United Kingdom
| | - Deep Jariwala
- Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia PA 19104, United States of America
| | - Han Wang
- University of Southern California, Los Angeles, CA 90089, United States of America
| | - Jeffrey M Shainline
- Physical Measurement Laboratory, National Institute of Standards and Technology (NIST), Boulder, CO 80305, United States of America
| | - Kenneth Segall
- Department of Physics and Astronomy, Colgate University, NY 13346, United States of America
| | - J Joshua Yang
- Department of Electrical and Computer Engineering, University of Massachusetts, Amherst, MA, United States of America
| | - Kaushik Roy
- School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, United States of America
| | - Suman Datta
- University of Notre Dame, Notre Dame, IN 46556, United States of America
| | - Arijit Raychowdhury
- Georgia Institute of Technology, Atlanta, GA 30332, United States of America
| |
Collapse
|