1
|
Zheng H, Liu Q, Kravchenko II, Zhang X, Huo Y, Valentine JG. Multichannel meta-imagers for accelerating machine vision. NATURE NANOTECHNOLOGY 2024; 19:471-478. [PMID: 38177276 PMCID: PMC11031328 DOI: 10.1038/s41565-023-01557-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 10/27/2023] [Indexed: 01/06/2024]
Abstract
Rapid developments in machine vision technology have impacted a variety of applications, such as medical devices and autonomous driving systems. These achievements, however, typically necessitate digital neural networks with the downside of heavy computational requirements and consequent high energy consumption. As a result, real-time decision-making is hindered when computational resources are not readily accessible. Here we report a meta-imager designed to work together with a digital back end to offload computationally expensive convolution operations into high-speed, low-power optics. In this architecture, metasurfaces enable both angle and polarization multiplexing to create multiple information channels that perform positively and negatively valued convolution operations in a single shot. We use our meta-imager for object classification, achieving 98.6% accuracy in handwritten digits and 88.8% accuracy in fashion images. Owing to its compactness, high speed and low power consumption, our approach could find a wide range of applications in artificial intelligence and machine vision applications.
Collapse
Affiliation(s)
- Hanyu Zheng
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA
| | - Quan Liu
- Department of Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Ivan I Kravchenko
- Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, TN, USA
| | - Xiaomeng Zhang
- Department of Mechanical Engineering, Vanderbilt University, Nashville, TN, USA
| | - Yuankai Huo
- Department of Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Jason G Valentine
- Department of Mechanical Engineering, Vanderbilt University, Nashville, TN, USA.
| |
Collapse
|
2
|
Ma SY, Wang T, Laydevant J, Wright LG, McMahon PL. Quantum-noise-limited optical neural networks operating at a few quanta per activation. RESEARCH SQUARE 2023:rs.3.rs-3318262. [PMID: 37961369 PMCID: PMC10635295 DOI: 10.21203/rs.3.rs-3318262/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2023]
Abstract
A practical limit to energy efficiency in computation is ultimately from noise, with quantum noise [1] as the fundamental floor. Analog physical neural networks [2], which hold promise for improved energy efficiency and speed compared to digital electronic neural networks, are nevertheless typically operated in a relatively high-power regime so that the signal-to-noise ratio (SNR) is large (>10). We study optical neural networks [3] operated in the limit where all layers except the last use only a single photon to cause a neuron activation. In this regime, activations are dominated by quantum noise from the fundamentally probabilistic nature of single-photon detection. We show that it is possible to perform accurate machine-learning inference in spite of the extremely high noise (signal-to-noise ratio ~ 1). We experimentally demonstrated MNIST handwritten-digit classification with a test accuracy of 98% using an optical neural network with a hidden layer operating in the single-photon regime; the optical energy used to perform the classification corresponds to 0.008 photons per multiply-accumulate (MAC) operation, which is equivalent to 0.003 attojoules of optical energy per MAC. Our experiment also used >40× fewer photons per inference than previous state-of-the-art low-optical-energy demonstrations [4, 5] to achieve the same accuracy of >90%. Our training approach, which directly models the system's stochastic behavior, might also prove useful with non-optical ultra-low-power hardware.
Collapse
Affiliation(s)
- Shi-Yuan Ma
- School of Applied and Engineering Physics, Cornell University, Ithaca, 14853, NY, USA
| | - Tianyu Wang
- School of Applied and Engineering Physics, Cornell University, Ithaca, 14853, NY, USA
| | - Jérémie Laydevant
- School of Applied and Engineering Physics, Cornell University, Ithaca, 14853, NY, USA
- USRA Research Institute for Advanced Computer Science, Mountain View, 94035, CA, USA
| | - Logan G. Wright
- School of Applied and Engineering Physics, Cornell University, Ithaca, 14853, NY, USA
- NTT Physics and Informatics Laboratories, NTT Research, Inc., Sunnyvale, 94085, CA, USA
| | - Peter L. McMahon
- School of Applied and Engineering Physics, Cornell University, Ithaca, 14853, NY, USA
- Kavli Institute at Cornell for Nanoscale Science, Cornell University, Ithaca, 14853, NY, USA
| |
Collapse
|