Multiple-view D2NNs array: realizing robust 3D object recognition

Wavefront-aberration-tolerant diffractive deep neural networks using volume holographic optical elements

As the demand for computational performance in artificial intelligence (AI) continues to increase, diffractive deep neural networks (D2NNs), which can perform AI computing at the speed of light by repeated optical modulation with diffractive optical elements (DOEs), are attracting attention. DOEs are varied in terms of fabrication methods and materials, and among them, volume holographic optical elements (vHOEs) have unique features such as high selectivity and multiplex recordability for wavelength and angle. However, when those are used for D2NNs, they suffer from unknown wavefront aberrations compounded by multiple fabrication errors. Here, we propose a training method to adapt the model to be unknown wavefront aberrations and demonstrate a D2NN using vHOEs. As a result, the proposed method improved the classification accuracy by approximately 58 percentage points in the optical experiment, with the model trained to classify handwritten digits. The achievement of this study can be extended to the D2NN that enables the independent modulation of multiple wavelengths owing to their wavelength selectivity and wavelength division multiplex recordability. Therefore, it might be promising for various applications that require multiple wavelengths in parallel optical computing, bioimaging, and optical communication.

Multi-wavelength diffractive neural network with the weighting method

Recently, the diffractive deep neural network (D2NN) has demonstrated the advantages to achieve large-scale computational tasks in terms of high speed, low power consumption, parallelism, and scalability. A typical D2NN with cascaded diffractive elements is designed for monochromatic illumination. Here, we propose a framework to achieve the multi-wavelength D2NN (MW-D2NN) based on the method of weight coefficients. In training, each wavelength is assigned a specific weighting and their output planes construct the wavelength weighting loss function. The trained MW-D2NN can implement the classification of images of handwritten digits at multi-wavelength incident beams. The designed 3-layers MW-D2NN achieves a simulation classification accuracy of 83.3%. We designed a 1-layer MW-D2NN. The simulation and experiment classification accuracy are 71.4% and 67.5% at RGB wavelengths. Furthermore, the proposed MW-D2NN can be extended to intelligent machine vision systems for multi-wavelength and incoherent illumination.

The mr-MDA: An Invariant to Shifting, Scaling, and Rotating Variance for 3D Object Recognition Using Diffractive Deep Neural Network

The diffractive deep neural network (D2NN) can efficiently accomplish 2D object recognition based on rapid optical manipulation. Moreover, the multiple-view D2NN array (MDA) possesses the obvious advantage of being able to effectively achieve 3D object classification. At present, 3D target recognition should be performed in a high-speed and dynamic way. It should be invariant to the typical shifting, scaling, and rotating variance of targets in relatively complicated circumstances, which remains a shortcoming of optical neural network architectures. In order to efficiently recognize 3D targets based on the developed D2NN, a more robust MDA (mr-MDA) is proposed in this paper. Through utilizing a new training strategy to tackle several random disturbances introduced into the optical neural network system, a trained mr-MDA model constructed by us was numerically verified, demonstrating that the training strategy is able to dynamically recognize 3D objects in a relatively stable way.

Multiscale diffractive U-Net: a robust all-optical deep learning framework modeled with sampling and skip connections

As an all-optical learning framework, diffractive deep neural networks (D²NNs) have great potential in running speed, data throughput, and energy consumption. The depth of networks and the misalignment of layers are two problems to limit its further development. In this work, a robust all-optical network framework (multiscale diffractive U-Net, MDUNet) based on multi-scale features fusion has been proposed. The depth expansion and alignment robustness of the network can be significantly improved by introducing sampling and skip connections. Compared with common all-optical learning frameworks, MDUNet achieves the highest accuracy of 98.81% and 89.11% on MNIST and Fashion-MNIST respectively. The testing accuracy of MNIST and Fashion-MNIST can be further improved to 99.06% and 89.86% respectively by using the ensemble learning method to construct the optoelectronic hybrid neural network.

Classification and reconstruction of spatially overlapping phase images using diffractive optical networks

Diffractive optical networks unify wave optics and deep learning to all-optically compute a given machine learning or computational imaging task as the light propagates from the input to the output plane. Here, we report the design of diffractive optical networks for the classification and reconstruction of spatially overlapping, phase-encoded objects. When two different phase-only objects spatially overlap, the individual object functions are perturbed since their phase patterns are summed up. The retrieval of the underlying phase images from solely the overlapping phase distribution presents a challenging problem, the solution of which is generally not unique. We show that through a task-specific training process, passive diffractive optical networks composed of successive transmissive layers can all-optically and simultaneously classify two different randomly-selected, spatially overlapping phase images at the input. After trained with ~ 550 million unique combinations of phase-encoded handwritten digits from the MNIST dataset, our blind testing results reveal that the diffractive optical network achieves an accuracy of > 85.8% for all-optical classification of two overlapping phase images of new handwritten digits. In addition to all-optical classification of overlapping phase objects, we also demonstrate the reconstruction of these phase images based on a shallow electronic neural network that uses the highly compressed output of the diffractive optical network as its input (with e.g., ~ 20-65 times less number of pixels) to rapidly reconstruct both of the phase images, despite their spatial overlap and related phase ambiguity. The presented phase image classification and reconstruction framework might find applications in e.g., computational imaging, microscopy and quantitative phase imaging fields.

Broad-spectrum diffractive network via ensemble learning

We propose a broad-spectrum diffractive deep neural network (BS-D²NN) framework, which incorporates multiwavelength channels of input lightfields and performs a parallel phase-only modulation using a layered passive mask architecture. A complementary multichannel base learner cluster is formed in a homogeneous ensemble framework based on the diffractive dispersion during lightwave modulation. In addition, both the optical sum operation and the hybrid (optical-electronic) maxout operation are performed for motivating the BS-D²NN to learn and construct a mapping between input lightfields and truth labels under heterochromatic ambient lighting. The BS-D²NN can be trained using deep learning algorithms to perform a kind of wavelength-insensitive high-accuracy object classification.