1
|
Chen X, Yang D, Hwang G, Dong Y, Cui B, Wang D, Chen H, Lin N, Zhang W, Li H, Shao R, Lin P, Hong H, Yao Y, Sun L, Wang Z, Yang H. Oscillatory Neural Network-Based Ising Machine Using 2D Memristors. ACS NANO 2024; 18:10758-10767. [PMID: 38598699 DOI: 10.1021/acsnano.3c10559] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/12/2024]
Abstract
Neural networks are increasingly used to solve optimization problems in various fields, including operations research, design automation, and gene sequencing. However, these networks face challenges due to the nondeterministic polynomial time (NP)-hard issue, which results in exponentially increasing computational complexity as the problem size grows. Conventional digital hardware struggles with the von Neumann bottleneck, the slowdown of Moore's law, and the complexity arising from heterogeneous system design. Two-dimensional (2D) memristors offer a potential solution to these hardware challenges, with their in-memory computing, decent scalability, and rich dynamic behaviors. In this study, we explore the use of nonvolatile 2D memristors to emulate synapses in a discrete-time Hopfield neural network, enabling the network to solve continuous optimization problems, like finding the minimum value of a quadratic polynomial, and tackle combinatorial optimization problems like Max-Cut. Additionally, we coupled volatile memristor-based oscillators with nonvolatile memristor synapses to create an oscillatory neural network-based Ising machine, a continuous-time analog dynamic system capable of solving combinatorial optimization problems including Max-Cut and map coloring through phase synchronization. Our findings demonstrate that 2D memristors have the potential to significantly enhance the efficiency, compactness, and homogeneity of integrated Ising machines, which is useful for future advances in neural networks for optimization problems.
Collapse
Affiliation(s)
- Xi Chen
- Centre for Quantum Physics, Key Laboratory of Advanced Optoelectronic Quantum Architecture and Measurement (MOE), School of Physics, Beijing Institute of Technology, Beijing 100081, China
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong, China
| | - Dongliang Yang
- Centre for Quantum Physics, Key Laboratory of Advanced Optoelectronic Quantum Architecture and Measurement (MOE), School of Physics, Beijing Institute of Technology, Beijing 100081, China
| | - Geunwoo Hwang
- Division of Chemical Engineering and Materials Science, Graduate Program in System Health Science and Engineering, Ewha Womans University, Seoul 03760, Korea
| | - Yujiao Dong
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong, China
- Institute of Modern Circuit and Intelligent Information, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Binbin Cui
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong, China
| | - Dingchen Wang
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong, China
| | - Hegan Chen
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong, China
| | - Ning Lin
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong, China
| | - Wenqi Zhang
- Department of Biomedical Engineering, City University of Hong Kong, Kowloon Tong, Hong Kong, China
| | - Huihan Li
- Centre for Quantum Physics, Key Laboratory of Advanced Optoelectronic Quantum Architecture and Measurement (MOE), School of Physics, Beijing Institute of Technology, Beijing 100081, China
| | - Ruiwen Shao
- Beijing Advanced Innovation Center for Intelligent Robots and Systems and Institute of Engineering Medicine, Beijing Institute of Technology, Beijing 100081, China
| | - Peng Lin
- College of Computer Science and Technology, Zhejiang University, Hang Zhou 310013, China
| | - Heemyoung Hong
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Korea
| | - Yugui Yao
- Centre for Quantum Physics, Key Laboratory of Advanced Optoelectronic Quantum Architecture and Measurement (MOE), School of Physics, Beijing Institute of Technology, Beijing 100081, China
| | - Linfeng Sun
- Centre for Quantum Physics, Key Laboratory of Advanced Optoelectronic Quantum Architecture and Measurement (MOE), School of Physics, Beijing Institute of Technology, Beijing 100081, China
| | - Zhongrui Wang
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong, China
| | - Heejun Yang
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Korea
| |
Collapse
|
2
|
Guo Y, Duan W, Liu X, Wang X, Wang L, Duan S, Ma C, Li H. Generative complex networks within a dynamic memristor with intrinsic variability. Nat Commun 2023; 14:6134. [PMID: 37783711 PMCID: PMC10545788 DOI: 10.1038/s41467-023-41921-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 09/21/2023] [Indexed: 10/04/2023] Open
Abstract
Artificial neural networks (ANNs) have gained considerable momentum in the past decade. Although at first the main task of the ANN paradigm was to tune the connection weights in fixed-architecture networks, there has recently been growing interest in evolving network architectures toward the goal of creating artificial general intelligence. Lagging behind this trend, current ANN hardware struggles for a balance between flexibility and efficiency but cannot achieve both. Here, we report on a novel approach for the on-demand generation of complex networks within a single memristor where multiple virtual nodes are created by time multiplexing and the non-trivial topological features, such as small-worldness, are generated by exploiting device dynamics with intrinsic cycle-to-cycle variability. When used for reservoir computing, memristive complex networks can achieve a noticeable increase in memory capacity a and respectable performance boost compared to conventional reservoirs trivially implemented as fully connected networks. This work expands the functionality of memristors for ANN computing.
Collapse
Affiliation(s)
- Yunpeng Guo
- Department of Precision Instrument, Center for Brain Inspired Computing Research, Tsinghua University, Beijing, 100084, China
| | - Wenrui Duan
- School of Instrument Science and Opto Electronics Engineering, Laboratory of Intelligent Microsystems, Beijing Information Science & Technology University, Beijing, 100101, China.
| | - Xue Liu
- Department of Precision Instrument, Center for Brain Inspired Computing Research, Tsinghua University, Beijing, 100084, China.
- School of Integrated Circuits, Tsinghua University, Beijing, 100084, China.
| | - Xinxin Wang
- Department of Precision Instrument, Center for Brain Inspired Computing Research, Tsinghua University, Beijing, 100084, China
| | - Lidan Wang
- School of Artificial Intelligence, Southwest University, Chongqing, 400715, China
| | - Shukai Duan
- School of Artificial Intelligence, Southwest University, Chongqing, 400715, China
| | - Cheng Ma
- Department of Precision Instrument, Center for Brain Inspired Computing Research, Tsinghua University, Beijing, 100084, China.
| | - Huanglong Li
- Department of Precision Instrument, Center for Brain Inspired Computing Research, Tsinghua University, Beijing, 100084, China.
- Chinese Institute for Brain Research, Beijing, 102206, China.
| |
Collapse
|
3
|
Jiang M, Shan K, He C, Li C. Efficient combinatorial optimization by quantum-inspired parallel annealing in analogue memristor crossbar. Nat Commun 2023; 14:5927. [PMID: 37739944 PMCID: PMC10516914 DOI: 10.1038/s41467-023-41647-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2023] [Accepted: 09/11/2023] [Indexed: 09/24/2023] Open
Abstract
Combinatorial optimization problems are prevalent in various fields, but obtaining exact solutions remains challenging due to the combinatorial explosion with increasing problem size. Special-purpose hardware such as Ising machines, particularly memristor-based analog Ising machines, have emerged as promising solutions. However, existing simulate-annealing-based implementations have not fully exploited the inherent parallelism and analog storage/processing features of memristor crossbar arrays. This work proposes a quantum-inspired parallel annealing method that enables full parallelism and improves solution quality, resulting in significant speed and energy improvement when implemented in analog memristor crossbars. We experimentally solved tasks, including unweighted and weighted Max-Cut and traveling salesman problem, using our integrated memristor chip. The quantum-inspired parallel annealing method implemented in memristor-based hardware has demonstrated significant improvements in time- and energy-efficiency compared to previously reported simulated annealing and Ising machine implemented on other technologies. This is because our approach effectively exploits the natural parallelism, analog conductance states, and all-to-all connection provided by memristor technology, promising its potential for solving complex optimization problems with greater efficiency.
Collapse
Affiliation(s)
- Mingrui Jiang
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| | - Keyi Shan
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| | - Chengping He
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China
| | - Can Li
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China.
| |
Collapse
|
4
|
Langenegger J, Karunaratne G, Hersche M, Benini L, Sebastian A, Rahimi A. In-memory factorization of holographic perceptual representations. NATURE NANOTECHNOLOGY 2023; 18:479-485. [PMID: 36997756 DOI: 10.1038/s41565-023-01357-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Accepted: 02/21/2023] [Indexed: 05/21/2023]
Abstract
Disentangling the attributes of a sensory signal is central to sensory perception and cognition and hence is a critical task for future artificial intelligence systems. Here we present a compute engine capable of efficiently factorizing high-dimensional holographic representations of combinations of such attributes, by exploiting the computation-in-superposition capability of brain-inspired hyperdimensional computing, and the intrinsic stochasticity associated with analogue in-memory computing based on nanoscale memristive devices. Such an iterative in-memory factorizer is shown to solve at least five orders of magnitude larger problems that cannot be solved otherwise, as well as substantially lowering the computational time and space complexity. We present a large-scale experimental demonstration of the factorizer by employing two in-memory compute chips based on phase-change memristive devices. The dominant matrix-vector multiplication operations take a constant time, irrespective of the size of the matrix, thus reducing the computational time complexity to merely the number of iterations. Moreover, we experimentally demonstrate the ability to reliably and efficiently factorize visual perceptual representations.
Collapse
Affiliation(s)
- Jovin Langenegger
- IBM Research-Zurich, Rüschlikon, Switzerland
- Department of Information Technology and Electrical Engineering, ETH Zürich, Zürich, Switzerland
| | - Geethan Karunaratne
- IBM Research-Zurich, Rüschlikon, Switzerland
- Department of Information Technology and Electrical Engineering, ETH Zürich, Zürich, Switzerland
| | - Michael Hersche
- IBM Research-Zurich, Rüschlikon, Switzerland
- Department of Information Technology and Electrical Engineering, ETH Zürich, Zürich, Switzerland
| | - Luca Benini
- Department of Information Technology and Electrical Engineering, ETH Zürich, Zürich, Switzerland
| | | | | |
Collapse
|
5
|
Bianchi S, Muñoz-Martin I, Covi E, Bricalli A, Piccolboni G, Regev A, Molas G, Nodin JF, Andrieu F, Ielmini D. A self-adaptive hardware with resistive switching synapses for experience-based neurocomputing. Nat Commun 2023; 14:1565. [PMID: 36944647 PMCID: PMC10030830 DOI: 10.1038/s41467-023-37097-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2021] [Accepted: 03/02/2023] [Indexed: 03/23/2023] Open
Abstract
Neurobiological systems continually interact with the surrounding environment to refine their behaviour toward the best possible reward. Achieving such learning by experience is one of the main challenges of artificial intelligence, but currently it is hindered by the lack of hardware capable of plastic adaptation. Here, we propose a bio-inspired recurrent neural network, mastered by a digital system on chip with resistive-switching synaptic arrays of memory devices, which exploits homeostatic Hebbian learning for improved efficiency. All the results are discussed experimentally and theoretically, proposing a conceptual framework for benchmarking the main outcomes in terms of accuracy and resilience. To test the proposed architecture for reinforcement learning tasks, we study the autonomous exploration of continually evolving environments and verify the results for the Mars rover navigation. We also show that, compared to conventional deep learning techniques, our in-memory hardware has the potential to achieve a significant boost in speed and power-saving.
Collapse
Affiliation(s)
- S Bianchi
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano and IUNET, Milano, 20133, Italy
- Infineon Technologies, Villach, Austria
| | - I Muñoz-Martin
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano and IUNET, Milano, 20133, Italy
- Infineon Technologies, Villach, Austria
| | - E Covi
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano and IUNET, Milano, 20133, Italy
- NaMLab gGmbH, Dresden, Germany
| | | | | | - A Regev
- Weebit Nano, Hod Hasharon, Israel
| | - G Molas
- Weebit Nano, Hod Hasharon, Israel
| | - J F Nodin
- Univ. Grenoble Alpes, CEA, Leti, F-38000, Grenoble, France
| | - F Andrieu
- Univ. Grenoble Alpes, CEA, Leti, F-38000, Grenoble, France
| | - D Ielmini
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano and IUNET, Milano, 20133, Italy.
| |
Collapse
|
6
|
Seok H, Son S, Jathar SB, Lee J, Kim T. Synapse-Mimetic Hardware-Implemented Resistive Random-Access Memory for Artificial Neural Network. SENSORS (BASEL, SWITZERLAND) 2023; 23:3118. [PMID: 36991829 PMCID: PMC10058286 DOI: 10.3390/s23063118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 03/11/2023] [Accepted: 03/13/2023] [Indexed: 06/19/2023]
Abstract
Memristors mimic synaptic functions in advanced electronics and image sensors, thereby enabling brain-inspired neuromorphic computing to overcome the limitations of the von Neumann architecture. As computing operations based on von Neumann hardware rely on continuous memory transport between processing units and memory, fundamental limitations arise in terms of power consumption and integration density. In biological synapses, chemical stimulation induces information transfer from the pre- to the post-neuron. The memristor operates as resistive random-access memory (RRAM) and is incorporated into the hardware for neuromorphic computing. Hardware composed of synaptic memristor arrays is expected to lead to further breakthroughs owing to their biomimetic in-memory processing capabilities, low power consumption, and amenability to integration; these aspects satisfy the upcoming demands of artificial intelligence for higher computational loads. Among the tremendous efforts toward achieving human-brain-like electronics, layered 2D materials have demonstrated significant potential owing to their outstanding electronic and physical properties, facile integration with other materials, and low-power computing. This review discusses the memristive characteristics of various 2D materials (heterostructures, defect-engineered materials, and alloy materials) used in neuromorphic computing for image segregation or pattern recognition. Neuromorphic computing, the most powerful artificial networks for complicated image processing and recognition, represent a breakthrough in artificial intelligence owing to their enhanced performance and lower power consumption compared with von Neumann architectures. A hardware-implemented CNN with weight control based on synaptic memristor arrays is expected to be a promising candidate for future electronics in society, offering a solution based on non-von Neumann hardware. This emerging paradigm changes the computing algorithm using entirely hardware-connected edge computing and deep neural networks.
Collapse
Affiliation(s)
- Hyunho Seok
- SKKU Advanced Institute of Nanotechnology (SAINT), Sungkyunkwan University, Suwon 16419, Republic of Korea
- Department of Nano Science and Technology, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Shihoon Son
- SKKU Advanced Institute of Nanotechnology (SAINT), Sungkyunkwan University, Suwon 16419, Republic of Korea
- Department of Nano Science and Technology, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Sagar Bhaurao Jathar
- SKKU Advanced Institute of Nanotechnology (SAINT), Sungkyunkwan University, Suwon 16419, Republic of Korea
- Department of Nano Science and Technology, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Jaewon Lee
- School of Mechanical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Taesung Kim
- SKKU Advanced Institute of Nanotechnology (SAINT), Sungkyunkwan University, Suwon 16419, Republic of Korea
- Department of Nano Science and Technology, Sungkyunkwan University, Suwon 16419, Republic of Korea
- School of Mechanical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea
| |
Collapse
|
7
|
Wang C, Shi G, Qiao F, Lin R, Wu S, Hu Z. Research progress in architecture and application of RRAM with computing-in-memory. NANOSCALE ADVANCES 2023; 5:1559-1573. [PMID: 36926563 PMCID: PMC10012847 DOI: 10.1039/d3na00025g] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Accepted: 02/04/2023] [Indexed: 06/18/2023]
Abstract
The development of new technologies has led to an explosion of data, while the computation ability of traditional computers is approaching its upper limit. The dominant system architecture is the von Neumann architecture, with the processing and storage units working independently. The data migrate between them via buses, reducing computing speed and increasing energy loss. Research is underway to increase computing power, such as developing new chips and adopting new system architectures. Computing-in-memory (CIM) technology allows data to be computed directly on the memory, changing the current computation-centric architecture and designing a new storage-centric architecture. Resistive random access memory (RRAM) is one of the advanced memories which has appeared in recent years. RRAM can change its resistance with electrical signals at both ends, and the state will be preserved after power-down. It has potential in logic computing, neural networks, brain-like computing, and fused technology of sense-storage-computing. These advanced technologies promise to break the performance bottleneck of traditional architectures and dramatically increase computing power. This paper introduces the basic concepts of computing-in-memory technology and the principle and applications of RRAM and finally gives a conclusion about these new technologies.
Collapse
Affiliation(s)
- Chenyu Wang
- College of Mechanical and Electrical Engineering, China Jiliang University Hangzhou China
| | - Ge Shi
- College of Mechanical and Electrical Engineering, China Jiliang University Hangzhou China
| | - Fei Qiao
- Dept of Electronic Engineering, Tsinghua University Beijing 310018 People's Republic of China
| | - Rubin Lin
- College of Mechanical and Electrical Engineering, China Jiliang University Hangzhou China
| | - Shien Wu
- College of Mechanical and Electrical Engineering, China Jiliang University Hangzhou China
| | - Zenan Hu
- College of Mechanical and Electrical Engineering, China Jiliang University Hangzhou China
| |
Collapse
|
8
|
Cen Q, Ding H, Hao T, Guan S, Qin Z, Lyu J, Li W, Zhu N, Xu K, Dai Y, Li M. Large-scale coherent Ising machine based on optoelectronic parametric oscillator. LIGHT, SCIENCE & APPLICATIONS 2022; 11:333. [PMID: 36433949 PMCID: PMC9700853 DOI: 10.1038/s41377-022-01013-1] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/07/2022] [Revised: 10/08/2022] [Accepted: 10/11/2022] [Indexed: 06/16/2023]
Abstract
Ising machines based on analog systems have the potential to accelerate the solution of ubiquitous combinatorial optimization problems. Although some artificial spins to support large-scale Ising machines have been reported, e.g., superconducting qubits in quantum annealers and short optical pulses in coherent Ising machines, the spin stability is fragile due to the ultra-low equivalent temperature or optical phase sensitivity. In this paper, we propose to use short microwave pulses generated from an optoelectronic parametric oscillator as the spins to implement a large-scale Ising machine with high stability. The proposed machine supports 25,600 spins and can operate continuously and stably for hours. Moreover, the proposed Ising machine is highly compatible with high-speed electronic devices for programmability, paving a low-cost, accurate, and easy-to-implement way toward solving real-world optimization problems.
Collapse
Affiliation(s)
- Qizhuang Cen
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
- School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, China
- Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China
| | - Hao Ding
- State Key Laboratory of Information Photonics and Optical Communications, Beijing University of Posts and Telecommunications, Beijing, China
| | - Tengfei Hao
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
- School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, China
- Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China
| | - Shanhong Guan
- State Key Laboratory of Information Photonics and Optical Communications, Beijing University of Posts and Telecommunications, Beijing, China
| | - Zhiqiang Qin
- State Key Laboratory of Information Photonics and Optical Communications, Beijing University of Posts and Telecommunications, Beijing, China
| | - Jiaming Lyu
- School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, China
| | - Wei Li
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
- School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, China
- Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China
| | - Ninghua Zhu
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
- School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, China
- Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China
| | - Kun Xu
- State Key Laboratory of Information Photonics and Optical Communications, Beijing University of Posts and Telecommunications, Beijing, China
| | - Yitang Dai
- State Key Laboratory of Information Photonics and Optical Communications, Beijing University of Posts and Telecommunications, Beijing, China.
- Peng Cheng Laboratory, Shenzhen, China.
| | - Ming Li
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China.
- School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, China.
- Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
9
|
Woo KS, Kim J, Han J, Kim W, Jang YH, Hwang CS. Probabilistic computing using Cu 0.1Te 0.9/HfO 2/Pt diffusive memristors. Nat Commun 2022; 13:5762. [PMID: 36180426 PMCID: PMC9525628 DOI: 10.1038/s41467-022-33455-x] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2022] [Accepted: 09/19/2022] [Indexed: 11/17/2022] Open
Abstract
A computing scheme that can solve complex tasks is necessary as the big data field proliferates. Probabilistic computing (p-computing) paves the way to efficiently handle problems based on stochastic units called probabilistic bits (p-bits). This study proposes p-computing based on the threshold switching (TS) behavior of a Cu0.1Te0.9/HfO2/Pt (CTHP) diffusive memristor. The theoretical background of the p-computing resembling the Hopfield network structure is introduced to explain the p-computing system. P-bits are realized by the stochastic TS behavior of CTHP diffusive memristors, and they are connected to form the p-computing network. The memristor-based p-bit is likely to be ‘0’ and ‘1’, of which probability is controlled by an input voltage. The memristor-based p-computing enables all 16 Boolean logic operations in both forward and inverted operations, showing the possibility of expanding its uses for complex operations, such as full adder and factorization. Designing a computing scheme to solve complex tasks as the big data field proliferates remains a challenge. Here, the authors present a probabilistic bit generation hardware built using the random nature of CuxTe1−x/HfO2/Pt memristors capable of performing logic gates with invertible mode, showing the expandability to complex logic circuits.
Collapse
Affiliation(s)
- Kyung Seok Woo
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Daehag-dong, Gwanak-gu, Seoul, 08826, Republic of Korea
| | - Jaehyun Kim
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Daehag-dong, Gwanak-gu, Seoul, 08826, Republic of Korea
| | - Janguk Han
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Daehag-dong, Gwanak-gu, Seoul, 08826, Republic of Korea
| | - Woohyun Kim
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Daehag-dong, Gwanak-gu, Seoul, 08826, Republic of Korea
| | - Yoon Ho Jang
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Daehag-dong, Gwanak-gu, Seoul, 08826, Republic of Korea
| | - Cheol Seong Hwang
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Daehag-dong, Gwanak-gu, Seoul, 08826, Republic of Korea.
| |
Collapse
|
10
|
Lanza M, Sebastian A, Lu WD, Le Gallo M, Chang MF, Akinwande D, Puglisi FM, Alshareef HN, Liu M, Roldan JB. Memristive technologies for data storage, computation, encryption, and radio-frequency communication. Science 2022; 376:eabj9979. [PMID: 35653464 DOI: 10.1126/science.abj9979] [Citation(s) in RCA: 78] [Impact Index Per Article: 39.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
Memristive devices, which combine a resistor with memory functions such that voltage pulses can change their resistance (and hence their memory state) in a nonvolatile manner, are beginning to be implemented in integrated circuits for memory applications. However, memristive devices could have applications in many other technologies, such as non-von Neumann in-memory computing in crossbar arrays, random number generation for data security, and radio-frequency switches for mobile communications. Progress toward the integration of memristive devices in commercial solid-state electronic circuits and other potential applications will depend on performance and reliability challenges that still need to be addressed, as described here.
Collapse
Affiliation(s)
- Mario Lanza
- Materials Science and Engineering Program, Physical Science and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Saudi Arabia
| | | | - Wei D Lu
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USA
| | | | - Meng-Fan Chang
- Taiwan Semiconductor Manufacturing Company (TSMC), Hsinchu, Taiwan.,Department of Electrical Engineering, National Tsing Hua University, Hsinchu 30013, Taiwan
| | - Deji Akinwande
- Microelectronics Research Center, University of Texas, Austin, TX, USA
| | - Francesco M Puglisi
- Dipartimento di Ingegneria "Enzo Ferrari," Università di Modena e Reggio Emilia, 41125 Modena, Italy
| | - Husam N Alshareef
- Materials Science and Engineering Program, Physical Science and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Saudi Arabia
| | - Ming Liu
- Key Laboratory of Microelectronic Devices and Integrated Technology, Institute of Microelectronics, Chinese Academy of Sciences, Beijing 100029, China
| | - Juan B Roldan
- Departamento de Electrónica y Tecnología de Computadores, Facultad de Ciencias, Universidad de Granada, 18071 Granada, Spain
| |
Collapse
|
11
|
Bao H, Zhou H, Li J, Pei H, Tian J, Yang L, Ren S, Tong S, Li Y, He Y, Chen J, Cai Y, Wu H, Liu Q, Wan Q, Miao X. Toward memristive in-memory computing: principles and applications. FRONTIERS OF OPTOELECTRONICS 2022; 15:23. [PMID: 36637566 PMCID: PMC9756267 DOI: 10.1007/s12200-022-00025-4] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Accepted: 03/07/2022] [Indexed: 05/08/2023]
Abstract
With the rapid growth of computer science and big data, the traditional von Neumann architecture suffers the aggravating data communication costs due to the separated structure of the processing units and memories. Memristive in-memory computing paradigm is considered as a prominent candidate to address these issues, and plentiful applications have been demonstrated and verified. These applications can be broadly categorized into two major types: soft computing that can tolerant uncertain and imprecise results, and hard computing that emphasizes explicit and precise numerical results for each task, leading to different requirements on the computational accuracies and the corresponding hardware solutions. In this review, we conduct a thorough survey of the recent advances of memristive in-memory computing applications, both on the soft computing type that focuses on artificial neural networks and other machine learning algorithms, and the hard computing type that includes scientific computing and digital image processing. At the end of the review, we discuss the remaining challenges and future opportunities of memristive in-memory computing in the incoming Artificial Intelligence of Things era.
Collapse
Affiliation(s)
- Han Bao
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
| | - Houji Zhou
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
| | - Jiancong Li
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
| | - Huaizhi Pei
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
| | - Jing Tian
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
| | - Ling Yang
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
| | - Shengguang Ren
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
| | - Shaoqin Tong
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
| | - Yi Li
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
- Hubei Yangtze Memory Laboratories, Wuhan, 430205 China
| | - Yuhui He
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
- Hubei Yangtze Memory Laboratories, Wuhan, 430205 China
| | - Jia Chen
- AI Chip Center for Emerging Smart Systems, InnoHK Centers, Hong Kong Science Park, Hong Kong, China
| | - Yimao Cai
- School of Integrated Circuits, Peking University, Beijing, 100871 China
| | - Huaqiang Wu
- School of Integrated Circuits, Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing, 100084 China
| | - Qi Liu
- Frontier Institute of Chip and System, Fudan University, Shanghai, 200433 China
| | - Qing Wan
- School of Electronic Science and Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093 China
| | - Xiangshui Miao
- School of Integrated Circuits, School of Optical and Electronic Information, Wuhan National Laboratory for Optoelectronics, Optics Valley Laboratory, Huazhong University of Science and Technology, Wuhan, 430074 China
- Hubei Yangtze Memory Laboratories, Wuhan, 430205 China
| |
Collapse
|
12
|
Reconfigurable halide perovskite nanocrystal memristors for neuromorphic computing. Nat Commun 2022; 13:2074. [PMID: 35440122 PMCID: PMC9018677 DOI: 10.1038/s41467-022-29727-1] [Citation(s) in RCA: 33] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Accepted: 03/28/2022] [Indexed: 12/18/2022] Open
Abstract
Many in-memory computing frameworks demand electronic devices with specific switching characteristics to achieve the desired level of computational complexity. Existing memristive devices cannot be reconfigured to meet the diverse volatile and non-volatile switching requirements, and hence rely on tailored material designs specific to the targeted application, limiting their universality. “Reconfigurable memristors” that combine both ionic diffusive and drift mechanisms could address these limitations, but they remain elusive. Here we present a reconfigurable halide perovskite nanocrystal memristor that achieves on-demand switching between diffusive/volatile and drift/non-volatile modes by controllable electrochemical reactions. Judicious selection of the perovskite nanocrystals and organic capping ligands enable state-of-the-art endurance performances in both modes – volatile (2 × 106 cycles) and non-volatile (5.6 × 103 cycles). We demonstrate the relevance of such proof-of-concept perovskite devices on a benchmark reservoir network with volatile recurrent and non-volatile readout layers based on 19,900 measurements across 25 dynamically-configured devices. Existing memristors cannot be reconfigured to meet the diverse switching requirements of various computing frameworks, limiting their universality. Here, the authors present a nanocrystal memristor that can be reconfigured on-demand to address these limitations
Collapse
|
13
|
Deiana AM, Tran N, Agar J, Blott M, Di Guglielmo G, Duarte J, Harris P, Hauck S, Liu M, Neubauer MS, Ngadiuba J, Ogrenci-Memik S, Pierini M, Aarrestad T, Bähr S, Becker J, Berthold AS, Bonventre RJ, Müller Bravo TE, Diefenthaler M, Dong Z, Fritzsche N, Gholami A, Govorkova E, Guo D, Hazelwood KJ, Herwig C, Khan B, Kim S, Klijnsma T, Liu Y, Lo KH, Nguyen T, Pezzullo G, Rasoulinezhad S, Rivera RA, Scholberg K, Selig J, Sen S, Strukov D, Tang W, Thais S, Unger KL, Vilalta R, von Krosigk B, Wang S, Warburton TK. Applications and Techniques for Fast Machine Learning in Science. Front Big Data 2022; 5:787421. [PMID: 35496379 PMCID: PMC9041419 DOI: 10.3389/fdata.2022.787421] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 01/31/2020] [Indexed: 01/10/2023] Open
Abstract
In this community review report, we discuss applications and techniques for fast machine learning (ML) in science-the concept of integrating powerful ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML across a number of scientific domains; techniques for training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms. We also present overlapping challenges across the multiple scientific domains where common solutions can be found. This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions. This is followed by a high-level overview and organization of technical advances, including an abundance of pointers to source material, which can enable these breakthroughs.
Collapse
Affiliation(s)
| | - Nhan Tran
- Fermi National Accelerator Laboratory, Batavia, IL, United States
- Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, United States
| | - Joshua Agar
- Department of Materials Science and Engineering, Lehigh University, Bethlehem, PA, United States
| | | | | | - Javier Duarte
- Department of Physics, University of California, San Diego, San Diego, CA, United States
| | - Philip Harris
- Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Scott Hauck
- Department of Electrical and Computer Engineering, University of Washington, Seattle, WA, United States
| | - Mia Liu
- Department of Physics and Astronomy, Purdue University, West Lafayette, IN, United States
| | - Mark S. Neubauer
- Department of Physics, University of Illinois Urbana-Champaign, Champaign, IL, United States
| | | | - Seda Ogrenci-Memik
- Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, United States
| | - Maurizio Pierini
- European Organization for Nuclear Research (CERN), Meyrin, Switzerland
| | - Thea Aarrestad
- European Organization for Nuclear Research (CERN), Meyrin, Switzerland
| | - Steffen Bähr
- Karlsruhe Institute of Technology, Karlsruhe, Germany
| | - Jürgen Becker
- Karlsruhe Institute of Technology, Karlsruhe, Germany
| | - Anne-Sophie Berthold
- Institute of Nuclear and Particle Physics, Technische Universität Dresden, Dresden, Germany
| | | | - Tomás E. Müller Bravo
- Department of Physics and Astronomy, University of Southampton, Southampton, United Kingdom
| | - Markus Diefenthaler
- Thomas Jefferson National Accelerator Facility, Newport News, VA, United States
| | - Zhen Dong
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA, United States
| | - Nick Fritzsche
- Institute of Nuclear and Particle Physics, Technische Universität Dresden, Dresden, Germany
| | - Amir Gholami
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA, United States
| | | | - Dongning Guo
- Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, United States
| | | | - Christian Herwig
- Fermi National Accelerator Laboratory, Batavia, IL, United States
| | - Babar Khan
- Department of Computer Science, Technical University Darmstadt, Darmstadt, Germany
| | - Sehoon Kim
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA, United States
| | - Thomas Klijnsma
- Fermi National Accelerator Laboratory, Batavia, IL, United States
| | - Yaling Liu
- Department of Bioengineering, Lehigh University, Bethlehem, PA, United States
| | - Kin Ho Lo
- Department of Physics, University of Florida, Gainesville, FL, United States
| | - Tri Nguyen
- Massachusetts Institute of Technology, Cambridge, MA, United States
| | | | | | - Ryan A. Rivera
- Fermi National Accelerator Laboratory, Batavia, IL, United States
| | - Kate Scholberg
- Department of Physics, Duke University, Durham, NC, United States
| | | | - Sougata Sen
- Birla Institute of Technology and Science, Pilani, India
| | - Dmitri Strukov
- Department of Electrical and Computer Engineering, University of California, Santa Barbara, Santa Barbara, CA, United States
| | - William Tang
- Department of Physics, Princeton University, Princeton, NJ, United States
| | - Savannah Thais
- Department of Physics, Princeton University, Princeton, NJ, United States
| | | | - Ricardo Vilalta
- Department of Computer Science, University of Houston, Houston, TX, United States
| | - Belina von Krosigk
- Karlsruhe Institute of Technology, Karlsruhe, Germany
- Department of Physics, Universität Hamburg, Hamburg, Germany
| | - Shen Wang
- Department of Physics, University of Florida, Gainesville, FL, United States
| | - Thomas K. Warburton
- Department of Physics and Astronomy, Iowa State University, Ames, IA, United States
| |
Collapse
|
14
|
Yon V, Amirsoleimani A, Alibart F, Melko RG, Drouin D, Beilliard Y. Exploiting Non-idealities of Resistive Switching Memories for Efficient Machine Learning. FRONTIERS IN ELECTRONICS 2022. [DOI: 10.3389/felec.2022.825077] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Novel computing architectures based on resistive switching memories (also known as memristors or RRAMs) have been shown to be promising approaches for tackling the energy inefficiency of deep learning and spiking neural networks. However, resistive switch technology is immature and suffers from numerous imperfections, which are often considered limitations on implementations of artificial neural networks. Nevertheless, a reasonable amount of variability can be harnessed to implement efficient probabilistic or approximate computing. This approach turns out to improve robustness, decrease overfitting and reduce energy consumption for specific applications, such as Bayesian and spiking neural networks. Thus, certain non-idealities could become opportunities if we adapt machine learning methods to the intrinsic characteristics of resistive switching memories. In this short review, we introduce some key considerations for circuit design and the most common non-idealities. We illustrate the possible benefits of stochasticity and compression with examples of well-established software methods. We then present an overview of recent neural network implementations that exploit the imperfections of resistive switching memory, and discuss the potential and limitations of these approaches.
Collapse
|
15
|
Acharya SK, Galli E, Mallinson JB, Bose SK, Wagner F, Heywood ZE, Bones PJ, Arnold MD, Brown SA. Stochastic Spiking Behavior in Neuromorphic Networks Enables True Random Number Generation. ACS APPLIED MATERIALS & INTERFACES 2021; 13:52861-52870. [PMID: 34719914 DOI: 10.1021/acsami.1c13668] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
There is currently a great deal of interest in the use of nanoscale devices to emulate the behaviors of neurons and synapses and to facilitate brain-inspired computation. Here, it is shown that percolating networks of nanoparticles exhibit stochastic spiking behavior that is strikingly similar to that observed in biological neurons. The spiking rate can be controlled by the input stimulus, similar to "rate coding" in biology, and the distributions of times between events are log-normal, providing insights into the atomic-scale spiking mechanism. The stochasticity of the spiking behavior is then used for true random number generation, and the high quality of the generated random bit-streams is demonstrated, opening up promising routes toward integration of neuromorphic computing with secure information processing.
Collapse
Affiliation(s)
- Susant K Acharya
- The MacDiarmid Institute for Advanced Materials and Nanotechnology, School of Physical and Chemical Sciences, Te Kura Matu, University of Canterbury, Private Bag 4800, Christchurch 8140, New Zealand
| | - Edoardo Galli
- The MacDiarmid Institute for Advanced Materials and Nanotechnology, School of Physical and Chemical Sciences, Te Kura Matu, University of Canterbury, Private Bag 4800, Christchurch 8140, New Zealand
| | - Joshua B Mallinson
- The MacDiarmid Institute for Advanced Materials and Nanotechnology, School of Physical and Chemical Sciences, Te Kura Matu, University of Canterbury, Private Bag 4800, Christchurch 8140, New Zealand
| | - Saurabh K Bose
- The MacDiarmid Institute for Advanced Materials and Nanotechnology, School of Physical and Chemical Sciences, Te Kura Matu, University of Canterbury, Private Bag 4800, Christchurch 8140, New Zealand
| | - Ford Wagner
- The MacDiarmid Institute for Advanced Materials and Nanotechnology, School of Physical and Chemical Sciences, Te Kura Matu, University of Canterbury, Private Bag 4800, Christchurch 8140, New Zealand
| | - Zachary E Heywood
- Electrical and Computer Engineering, University of Canterbury, Private Bag 4800, Christchurch 8140, New Zealand
| | - Philip J Bones
- Electrical and Computer Engineering, University of Canterbury, Private Bag 4800, Christchurch 8140, New Zealand
| | - Matthew D Arnold
- School of Mathematical and Physical Sciences, University of Technology Sydney, P.O. Box 123, Broadway, Sydney, New South Wales 2007, Australia
| | - Simon A Brown
- The MacDiarmid Institute for Advanced Materials and Nanotechnology, School of Physical and Chemical Sciences, Te Kura Matu, University of Canterbury, Private Bag 4800, Christchurch 8140, New Zealand
| |
Collapse
|
16
|
Yang K, Joshua Yang J, Huang R, Yang Y. Nonlinearity in Memristors for Neuromorphic Dynamic Systems. SMALL SCIENCE 2021. [DOI: 10.1002/smsc.202100049] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Affiliation(s)
- Ke Yang
- Department of Micro/nanoelectronics Peking University Beijing 100871 China
| | - J. Joshua Yang
- Electrical and Computer Engineering Department University of Southern California Los Angeles CA 90089 USA
| | - Ru Huang
- Department of Micro/nanoelectronics Peking University Beijing 100871 China
- Center for Brain Inspired Chips Institute for Artificial Intelligence Peking University Beijing 100871 China
- Center for Brain Inspired Intelligence Chinese Institute for Brain Research (CIBR) Beijing 102206 China
| | - Yuchao Yang
- Department of Micro/nanoelectronics Peking University Beijing 100871 China
- Center for Brain Inspired Chips Institute for Artificial Intelligence Peking University Beijing 100871 China
- Center for Brain Inspired Intelligence Chinese Institute for Brain Research (CIBR) Beijing 102206 China
| |
Collapse
|
17
|
Cheng S, Lee MH, Tran R, Shi Y, Li X, Navarro H, Adda C, Meng Q, Chen LQ, Dynes RC, Ong SP, Schuller IK, Zhu Y. Inherent stochasticity during insulator-metal transition in VO 2. Proc Natl Acad Sci U S A 2021; 118:e2105895118. [PMID: 34493666 PMCID: PMC8449351 DOI: 10.1073/pnas.2105895118] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Accepted: 08/04/2021] [Indexed: 11/18/2022] Open
Abstract
Vanadium dioxide (VO2), which exhibits a near-room-temperature insulator-metal transition, has great potential in applications of neuromorphic computing devices. Although its volatile switching property, which could emulate neuron spiking, has been studied widely, nanoscale studies of the structural stochasticity across the phase transition are still lacking. In this study, using in situ transmission electron microscopy and ex situ resistive switching measurement, we successfully characterized the structural phase transition between monoclinic and rutile VO2 at local areas in planar VO2/TiO2 device configuration under external biasing. After each resistive switching, different VO2 monoclinic crystal orientations are observed, forming different equilibrium states. We have evaluated a statistical cycle-to-cycle variation, demonstrated a stochastic nature of the volatile resistive switching, and presented an approach to study in-plane structural anisotropy. Our microscopic studies move a big step forward toward understanding the volatile switching mechanisms and the related applications of VO2 as the key material of neuromorphic computing.
Collapse
Affiliation(s)
- Shaobo Cheng
- Department of Condensed Matter Physics and Materials Science, Brookhaven National Laboratory, Upton, NY 11973
| | - Min-Han Lee
- Materials Science and Engineering Program, University of California San Diego, La Jolla, CA 92093
- Department of Physics, Center for Advanced Nanoscience, University of California San Diego, La Jolla, CA 92093
| | - Richard Tran
- Department of NanoEngineering, University of California San Diego, La Jolla, CA 92093
| | - Yin Shi
- Department of Materials Science and Engineering, Pennsylvania State University, University Park, PA 16802
| | - Xing Li
- Key Laboratory of Material Physics, Ministry of Education, School of Physics and Microelectronics, Zhengzhou University, Zhengzhou 450052, People's Republic of China
| | - Henry Navarro
- Department of Physics, Center for Advanced Nanoscience, University of California San Diego, La Jolla, CA 92093
| | - Coline Adda
- Department of Physics, Center for Advanced Nanoscience, University of California San Diego, La Jolla, CA 92093
| | - Qingping Meng
- Department of Condensed Matter Physics and Materials Science, Brookhaven National Laboratory, Upton, NY 11973
| | - Long-Qing Chen
- Department of Materials Science and Engineering, Pennsylvania State University, University Park, PA 16802
| | - R C Dynes
- Department of Physics, Center for Advanced Nanoscience, University of California San Diego, La Jolla, CA 92093;
| | - Shyue Ping Ong
- Department of NanoEngineering, University of California San Diego, La Jolla, CA 92093
| | - Ivan K Schuller
- Materials Science and Engineering Program, University of California San Diego, La Jolla, CA 92093
- Department of Physics, Center for Advanced Nanoscience, University of California San Diego, La Jolla, CA 92093
| | - Yimei Zhu
- Department of Condensed Matter Physics and Materials Science, Brookhaven National Laboratory, Upton, NY 11973;
| |
Collapse
|
18
|
Fahimi Z, Mahmoodi MR, Nili H, Polishchuk V, Strukov DB. Combinatorial optimization by weight annealing in memristive hopfield networks. Sci Rep 2021; 11:16383. [PMID: 34385475 PMCID: PMC8361025 DOI: 10.1038/s41598-020-78944-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2020] [Accepted: 11/17/2020] [Indexed: 11/16/2022] Open
Abstract
The increasing utility of specialized circuits and growing applications of optimization call for the development of efficient hardware accelerator for solving optimization problems. Hopfield neural network is a promising approach for solving combinatorial optimization problems due to the recent demonstrations of efficient mixed-signal implementation based on emerging non-volatile memory devices. Such mixed-signal accelerators also enable very efficient implementation of various annealing techniques, which are essential for finding optimal solutions. Here we propose a “weight annealing” approach, whose main idea is to ease convergence to the global minima by keeping the network close to its ground state. This is achieved by initially setting all synaptic weights to zero, thus ensuring a quick transition of the Hopfield network to its trivial global minima state and then gradually introducing weights during the annealing process. The extensive numerical simulations show that our approach leads to a better, on average, solutions for several representative combinatorial problems compared to prior Hopfield neural network solvers with chaotic or stochastic annealing. As a proof of concept, a 13-node graph partitioning problem and a 7-node maximum-weight independent set problem are solved experimentally using mixed-signal circuits based on, correspondingly, a 20 × 20 analog-grade TiO2 memristive crossbar and a 12 × 10 eFlash memory array.
Collapse
Affiliation(s)
- Z Fahimi
- UC Santa Barbara, Santa Barbara, CA, 93106-9560, USA.
| | - M R Mahmoodi
- UC Santa Barbara, Santa Barbara, CA, 93106-9560, USA.
| | - H Nili
- UC Santa Barbara, Santa Barbara, CA, 93106-9560, USA
| | | | - D B Strukov
- UC Santa Barbara, Santa Barbara, CA, 93106-9560, USA
| |
Collapse
|
19
|
Yi SI, Kim J. Novel Program Scheme of Vertical NAND Flash Memory for Reduction of Z-Interference. MICROMACHINES 2021; 12:mi12050584. [PMID: 34065435 PMCID: PMC8160891 DOI: 10.3390/mi12050584] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/15/2021] [Revised: 05/15/2021] [Accepted: 05/16/2021] [Indexed: 12/02/2022]
Abstract
Minimizing the variation in threshold voltage (Vt) of programmed cells is required to the extreme level for realizing multi-level-cells; as many as even 5 bits per cell recently. In this work, a recent program scheme to write the cells from the top, for instance the 170th layer, to the bottom, the 1st layer, (T-B scheme) in vertical NAND (VNAND) Flash Memory, is investigated to minimize Vt variation by reducing Z-interference. With the aid of Technology Computer Aided Design (TCAD) the Z-Interference for T-B (84 mV) is found to be better than B-T (105 mV). Moreover, under scaled cell dimensions (e.g., Lg: 31→24 nm), the improvement becomes protruding (T-B: 126 mV and B-T: 162 mV), emphasizing the significance of the T-B program scheme for the next generation VNAND products with the higher bit density.
Collapse
Affiliation(s)
- Su-in Yi
- Department of Electrical and Computer Engineering, Texas A&M University, College Station, TX 77843, USA;
- Samsung Electronics, Hwasung 18448, Kyeonggi, Korea
| | - Jungsik Kim
- Department of Electrical Engineering, Gyeongsang National University, Jinju 52828, Gyeongnam, Korea
- Engineering Research Institute (ERI), Gyeongsang National University, Jinju 52828, Gyeongnam, Korea
- Correspondence: ; Tel.: +82-55-772-1718
| |
Collapse
|
20
|
Covi E, Donati E, Liang X, Kappel D, Heidari H, Payvand M, Wang W. Adaptive Extreme Edge Computing for Wearable Devices. Front Neurosci 2021; 15:611300. [PMID: 34045939 PMCID: PMC8144334 DOI: 10.3389/fnins.2021.611300] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2020] [Accepted: 03/24/2021] [Indexed: 11/13/2022] Open
Abstract
Wearable devices are a fast-growing technology with impact on personal healthcare for both society and economy. Due to the widespread of sensors in pervasive and distributed networks, power consumption, processing speed, and system adaptation are vital in future smart wearable devices. The visioning and forecasting of how to bring computation to the edge in smart sensors have already begun, with an aspiration to provide adaptive extreme edge computing. Here, we provide a holistic view of hardware and theoretical solutions toward smart wearable devices that can provide guidance to research in this pervasive computing era. We propose various solutions for biologically plausible models for continual learning in neuromorphic computing technologies for wearable sensors. To envision this concept, we provide a systematic outline in which prospective low power and low latency scenarios of wearable sensors in neuromorphic platforms are expected. We successively describe vital potential landscapes of neuromorphic processors exploiting complementary metal-oxide semiconductors (CMOS) and emerging memory technologies (e.g., memristive devices). Furthermore, we evaluate the requirements for edge computing within wearable devices in terms of footprint, power consumption, latency, and data size. We additionally investigate the challenges beyond neuromorphic computing hardware, algorithms and devices that could impede enhancement of adaptive edge computing in smart wearable devices.
Collapse
Affiliation(s)
| | - Elisa Donati
- Institute of Neuroinformatics, University of Zurich, Eidgenössische Technische Hochschule Zürich (ETHZ), Zurich, Switzerland
| | - Xiangpeng Liang
- Microelectronics Lab, James Watt School of Engineering, University of Glasgow, Glasgow, United Kingdom
| | - David Kappel
- Bernstein Center for Computational Neuroscience, III Physikalisches Institut–Biophysik, Georg-August Universität, Göttingen, Germany
| | - Hadi Heidari
- Microelectronics Lab, James Watt School of Engineering, University of Glasgow, Glasgow, United Kingdom
| | - Melika Payvand
- Institute of Neuroinformatics, University of Zurich, Eidgenössische Technische Hochschule Zürich (ETHZ), Zurich, Switzerland
| | - Wei Wang
- The Andrew and Erna Viterbi Department of Electrical Engineering, Technion–Israel Institute of Technology, Haifa, Israel
| |
Collapse
|
21
|
Abstract
In-memory computing (IMC) refers to non-von Neumann architectures where data are processed in situ within the memory by taking advantage of physical laws. Among the memory devices that have been considered for IMC, the resistive switching memory (RRAM), also known as memristor, is one of the most promising technologies due to its relatively easy integration and scaling. RRAM devices have been explored for both memory and IMC applications, such as neural network accelerators and neuromorphic processors. This work presents the status and outlook on the RRAM for analog computing, where the precision of the encoded coefficients, such as the synaptic weights of a neural network, is one of the key requirements. We show the experimental study of the cycle-to-cycle variation of set and reset processes for HfO2-based RRAM, which indicate that gate-controlled pulses present the least variation in conductance. Assuming a constant variation of conductance σG, we then evaluate and compare various mapping schemes, including multilevel, binary, unary, redundant and slicing techniques. We present analytical formulas for the standard deviation of the conductance and the maximum number of bits that still satisfies a given maximum error. Finally, we discuss RRAM performance for various analog computing tasks compared to other computational memory devices. RRAM appears as one of the most promising devices in terms of scaling, accuracy and low-current operation.
Collapse
|
22
|
Faria R, Kaiser J, Camsari KY, Datta S. Hardware Design for Autonomous Bayesian Networks. Front Comput Neurosci 2021; 15:584797. [PMID: 33762919 PMCID: PMC7982658 DOI: 10.3389/fncom.2021.584797] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2020] [Accepted: 01/26/2021] [Indexed: 11/13/2022] Open
Abstract
Directed acyclic graphs or Bayesian networks that are popular in many AI-related sectors for probabilistic inference and causal reasoning can be mapped to probabilistic circuits built out of probabilistic bits (p-bits), analogous to binary stochastic neurons of stochastic artificial neural networks. In order to satisfy standard statistical results, individual p-bits not only need to be updated sequentially but also in order from the parent to the child nodes, necessitating the use of sequencers in software implementations. In this article, we first use SPICE simulations to show that an autonomous hardware Bayesian network can operate correctly without any clocks or sequencers, but only if the individual p-bits are appropriately designed. We then present a simple behavioral model of the autonomous hardware illustrating the essential characteristics needed for correct sequencer-free operation. This model is also benchmarked against SPICE simulations and can be used to simulate large-scale networks. Our results could be useful in the design of hardware accelerators that use energy-efficient building blocks suited for low-level implementations of Bayesian networks. The autonomous massively parallel operation of our proposed stochastic hardware has biological relevance since neural dynamics in brain is also stochastic and autonomous by nature.
Collapse
Affiliation(s)
- Rafatul Faria
- Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, United States
| | - Jan Kaiser
- Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, United States
| | - Kerem Y. Camsari
- Department of Electrical and Computer Engineering, University of California, Santa Barbara, Santa Barbara, CA, United States
| | - Supriyo Datta
- Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, United States
| |
Collapse
|
23
|
Wang W, Song W, Yao P, Li Y, Van Nostrand J, Qiu Q, Ielmini D, Yang JJ. Integration and Co-design of Memristive Devices and Algorithms for Artificial Intelligence. iScience 2020; 23:101809. [PMID: 33305176 PMCID: PMC7718163 DOI: 10.1016/j.isci.2020.101809] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
Memristive devices share remarkable similarities to biological synapses, dendrites, and neurons at both the physical mechanism level and unit functionality level, making the memristive approach to neuromorphic computing a promising technology for future artificial intelligence. However, these similarities do not directly transfer to the success of efficient computation without device and algorithm co-designs and optimizations. Contemporary deep learning algorithms demand the memristive artificial synapses to ideally possess analog weighting and linear weight-update behavior, requiring substantial device-level and circuit-level optimization. Such co-design and optimization have been the main focus of memristive neuromorphic engineering, which often abandons the “non-ideal” behaviors of memristive devices, although many of them resemble what have been observed in biological components. Novel brain-inspired algorithms are being proposed to utilize such behaviors as unique features to further enhance the efficiency and intelligence of neuromorphic computing, which calls for collaborations among electrical engineers, computing scientists, and neuroscientists.
Collapse
Affiliation(s)
- Wei Wang
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano and IU.NET, Piazza L. da Vinci 32, Milano 20133, Italy
| | - Wenhao Song
- Electrical and Computer Engineering Department, University of Southern California, Los Angeles, CA, USA
| | - Peng Yao
- Electrical and Computer Engineering Department, University of Southern California, Los Angeles, CA, USA
| | - Yang Li
- The Andrew and Erna Viterbi Department of Electrical Engineering, Technion-Israel Institute of Technology, Haifa 32000, Israel
| | | | - Qinru Qiu
- Electrical Engineering and Computer Science Department, Syracuse University, NY, USA
| | - Daniele Ielmini
- Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano and IU.NET, Piazza L. da Vinci 32, Milano 20133, Italy
| | - J Joshua Yang
- Electrical and Computer Engineering Department, University of Southern California, Los Angeles, CA, USA
| |
Collapse
|
24
|
Kumar S, Williams RS, Wang Z. Third-order nanocircuit elements for neuromorphic engineering. Nature 2020; 585:518-523. [PMID: 32968256 DOI: 10.1038/s41586-020-2735-5] [Citation(s) in RCA: 49] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2020] [Accepted: 08/03/2020] [Indexed: 11/09/2022]
Abstract
Current hardware approaches to biomimetic or neuromorphic artificial intelligence rely on elaborate transistor circuits to simulate biological functions. However, these can instead be more faithfully emulated by higher-order circuit elements that naturally express neuromorphic nonlinear dynamics1-4. Generating neuromorphic action potentials in a circuit element theoretically requires a minimum of third-order complexity (for example, three dynamical electrophysical processes)5, but there have been few examples of second-order neuromorphic elements, and no previous demonstration of any isolated third-order element6-8. Using both experiments and modelling, here we show how multiple electrophysical processes-including Mott transition dynamics-form a nanoscale third-order circuit element. We demonstrate simple transistorless networks of third-order elements that perform Boolean operations and find analogue solutions to a computationally hard graph-partitioning problem. This work paves a way towards very compact and densely functional neuromorphic computing primitives, and energy-efficient validation of neuroscientific models.
Collapse
|
25
|
Okawachi Y, Yu M, Jang JK, Ji X, Zhao Y, Kim BY, Lipson M, Gaeta AL. Demonstration of chip-based coupled degenerate optical parametric oscillators for realizing a nanophotonic spin-glass. Nat Commun 2020; 11:4119. [PMID: 32807796 PMCID: PMC7431591 DOI: 10.1038/s41467-020-17919-6] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2020] [Accepted: 07/22/2020] [Indexed: 12/04/2022] Open
Abstract
The need for solving optimization problems is prevalent in various physical applications, including neuroscience, network design, biological systems, socio-economics, and chemical reactions. Many of these are classified as non-deterministic polynomial-time hard and thus become intractable to solve as the system scales to a large number of elements. Recent research advances in photonics have sparked interest in using a network of coupled degenerate optical parametric oscillators (DOPOs) to effectively find the ground state of the Ising Hamiltonian, which can be used to solve other combinatorial optimization problems through polynomial-time mapping. Here, using the nanophotonic silicon-nitride platform, we demonstrate a spatial-multiplexed DOPO system using continuous-wave pumping. We experimentally demonstrate the generation and coupling of two microresonator-based DOPOs on a single chip. Through a reconfigurable phase link, we achieve both in-phase and out-of-phase operation, which can be deterministically achieved at a fast regeneration speed of 400 kHz with a large phase tolerance.
Collapse
Affiliation(s)
- Yoshitomo Okawachi
- Department of Applied Physics and Applied Mathematics, Columbia University, New York, NY, 10027, USA
| | - Mengjie Yu
- Department of Applied Physics and Applied Mathematics, Columbia University, New York, NY, 10027, USA
- School of Electrical and Computer Engineering, Cornell University, Ithaca, NY, 14853, USA
| | - Jae K Jang
- Department of Applied Physics and Applied Mathematics, Columbia University, New York, NY, 10027, USA
| | - Xingchen Ji
- Department of Electrical Engineering, Columbia University, New York, NY, 10027, USA
| | - Yun Zhao
- Department of Electrical Engineering, Columbia University, New York, NY, 10027, USA
| | - Bok Young Kim
- Department of Applied Physics and Applied Mathematics, Columbia University, New York, NY, 10027, USA
| | - Michal Lipson
- Department of Applied Physics and Applied Mathematics, Columbia University, New York, NY, 10027, USA
- Department of Electrical Engineering, Columbia University, New York, NY, 10027, USA
| | - Alexander L Gaeta
- Department of Applied Physics and Applied Mathematics, Columbia University, New York, NY, 10027, USA.
- Department of Electrical Engineering, Columbia University, New York, NY, 10027, USA.
| |
Collapse
|
26
|
Yang K, Duan Q, Wang Y, Zhang T, Yang Y, Huang R. Transiently chaotic simulated annealing based on intrinsic nonlinearity of memristors for efficient solution of optimization problems. SCIENCE ADVANCES 2020; 6:eaba9901. [PMID: 32851168 PMCID: PMC7428342 DOI: 10.1126/sciadv.aba9901] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/21/2020] [Accepted: 07/01/2020] [Indexed: 05/04/2023]
Abstract
Optimization problems are ubiquitous in scientific research, engineering, and daily lives. However, solving a complex optimization problem often requires excessive computing resource and time and faces challenges in easily getting trapped into local optima. Here, we propose a memristive optimizer hardware based on a Hopfield network, which introduces transient chaos to simulated annealing in aid of jumping out of the local optima while ensuring convergence. A single memristor crossbar is used to store the weight parameters of a fully connected Hopfield network and adjust the network dynamics in situ. Furthermore, we harness the intrinsic nonlinearity of memristors within the crossbar to implement an efficient and simplified annealing process for the optimization. Solutions of continuous function optimizations on sphere function and Matyas function as well as combinatorial optimization on Max-cut problem are experimentally demonstrated, indicating great potential of the transiently chaotic memristive network in solving optimization problems in general.
Collapse
Affiliation(s)
- Ke Yang
- Key Laboratory of Microelectronic Devices and Circuits (MOE), Department of Micro/nanoelectronics, Peking University, Beijing 100871, China
| | - Qingxi Duan
- Key Laboratory of Microelectronic Devices and Circuits (MOE), Department of Micro/nanoelectronics, Peking University, Beijing 100871, China
| | - Yanghao Wang
- Key Laboratory of Microelectronic Devices and Circuits (MOE), Department of Micro/nanoelectronics, Peking University, Beijing 100871, China
| | - Teng Zhang
- Key Laboratory of Microelectronic Devices and Circuits (MOE), Department of Micro/nanoelectronics, Peking University, Beijing 100871, China
| | - Yuchao Yang
- Key Laboratory of Microelectronic Devices and Circuits (MOE), Department of Micro/nanoelectronics, Peking University, Beijing 100871, China
- Center for Brain Inspired Chips, Institute for Artificial Intelligence, Peking University, Beijing 100871, China
- Frontiers Science Center for Nano-optoelectronics, Peking University, Beijing 100871, China
- Corresponding author. (Y.Y.); (R.H.)
| | - Ru Huang
- Key Laboratory of Microelectronic Devices and Circuits (MOE), Department of Micro/nanoelectronics, Peking University, Beijing 100871, China
- Center for Brain Inspired Chips, Institute for Artificial Intelligence, Peking University, Beijing 100871, China
- Frontiers Science Center for Nano-optoelectronics, Peking University, Beijing 100871, China
- Corresponding author. (Y.Y.); (R.H.)
| |
Collapse
|