Frequency-multiplexed and pipelined iterative optical systolic array processors

Freely scalable and reconfigurable optical hardware for deep learning

As deep neural network (DNN) models grow ever-larger, they can achieve higher accuracy and solve more complex problems. This trend has been enabled by an increase in available compute power; however, efforts to continue to scale electronic processors are impeded by the costs of communication, thermal management, power delivery and clocking. To improve scalability, we propose a digital optical neural network (DONN) with intralayer optical interconnects and reconfigurable input values. The path-length-independence of optical energy consumption enables information locality between a transmitter and a large number of arbitrarily arranged receivers, which allows greater flexibility in architecture design to circumvent scaling limitations. In a proof-of-concept experiment, we demonstrate optical multicast in the classification of 500 MNIST images with a 3-layer, fully-connected network. We also analyze the energy consumption of the DONN and find that digital optical data transfer is beneficial over electronics when the spacing of computational units is on the order of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$>10\,\upmu $$\end{document}>10μm.

Ho-Kashyap optical associative processors

A Ho-Kashyap (H-K) associative processor (AP) is shown to have a larger storage capacity than the pseudoinverse and correlation APs and to accurately store linearly dependent key vectors. Prior APs have not demonstrated good performance on linearly dependent key vectors. The AP is attractive for optical implementation. A new robust H-K AP is proposed to improve noise performance. These results are demonstrated both theoretically and by Monte Carlo simulation. The H-K AP is also shown to outperform the pseudoinverse AP in an aircraft recognition case study. A technique is developed to indicate the least reliable output vector elements and a new AP error correcting synthesis technique is advanced.

Key and recollection vector effects on heteroassociative memory performance

Most associative memory work has concentrated on autoassociative memories (AAMs). These associative processors provide reduced noise and error correction in their output data. We will consider heteroassociative memories (HAMs), which are needed to provide decisions on the class of the input data and inferences for subsequent processing. We derive new equations for the storage capacity and noise performance of HAMs, emphasize how they differ from those derived for AAMs, suggest new performance measures to be used, and show how different recollection vector encodings can improve HAM performance.

Acousto-optic matrix-matrix multiplier

A new architecture for an optical matrix-matrix multiplier is presented. It is based on the beam-modulation and Its parallel structure makes it very fast. Some physical limitations are discussed.

Implementation of a fast digital optical matrix-vector multiplier using a holographic look-up table and residue arithmetic

The design and implementation of a digital (numerical) optical matrix-vector multiplier are presented. A Hughes liquid crystal light valve, the residue arithmetic representation, and a holographic optical memory are used to construct position coded optical look-up tables. All operations are performed in effectively one light valve response time with a potential for a high information density.

Optical linear algebra processors: noise and error-source modeling

The modeling of system and component noise and error sources in optical linear algebra processors (OLAP's) are considered, with attention to the frequency-multiplexed OLAP. General expressions are obtained for the output produced as a function of various component errors and noise. A digital simulator for this model is discussed.

Direct and implicit optical matrix-vector algorithms

New direct and implicit algorithms for optical matrix-vector and systolic array processors are considered. Direct rather than indirect algorithms to solve linear systems and implicit rather than explicit solutions to solve second-order partial differential equations are discussed. In many cases, such approaches more properly utilize the advantageous features of optical systolic array processors. The matrix-decomposition operation (rather than solution of the simplified matrix-vector equation that results) is recognized as the computationally burdensome aspect of such problems that should be computed on an optical system. The Householder QR matrix-decomposition algorithm is considered as a specific example of a direct solution. Extensions to eigenvalue computation and formation of matrices of special structure are also noted.

Feedback methods for optical systolic and engagement matrix processors

The matching of the feedback circuitry to the optical systolic or engagement processor permits simple pipelining of stationary iterative algorithms as well as on-the-fly scale adjustment similar in effect to floating-point calculation.