1
|
Lin X, Fu Y, Zhang K, Zhang X, Feng S, Hu X. Polarization and wavelength routers based on diffractive neural network. FRONTIERS OF OPTOELECTRONICS 2024; 17:22. [PMID: 39009949 PMCID: PMC11250754 DOI: 10.1007/s12200-024-00126-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Accepted: 06/18/2024] [Indexed: 07/17/2024]
Abstract
In the field of information processing, all-optical routers are significant for achieving high-speed, high-capacity signal processing and transmission. In this study, we developed three types of structurally simple and flexible routers using the deep diffractive neural network (D2NN), capable of routing incident light based on wavelength and polarization. First, we implemented a polarization router for routing two orthogonally polarized light beams. The second type is the wavelength router that can route light with wavelengths of 1550, 1300, and 1100 nm, demonstrating outstanding performance with insertion loss as low as 0.013 dB and an extinction ratio of up to 18.96 dB, while also maintaining excellent polarization preservation. The final router is the polarization-wavelength composite router, capable of routing six types of input light formed by pairwise combinations of three wavelengths (1550, 1300, and 1100 nm) and two orthogonal linearly polarized lights, thereby enhancing the information processing capability of the device. These devices feature compact structures, maintaining high contrast while exhibiting low loss and passive characteristics, making them suitable for integration into future optical components. This study introduces new avenues and methodologies to enhance performance and broaden the applications of future optical information processing systems.
Collapse
Affiliation(s)
- Xiaohong Lin
- School of Physics and Optoelectronic Engineering, Beijing University of Technology, Beijing, 100124, China
| | - Yulan Fu
- School of Physics and Optoelectronic Engineering, Beijing University of Technology, Beijing, 100124, China.
| | - Kuo Zhang
- School of Science, Minzu University of China, Beijing, 100081, China
| | - Xinping Zhang
- School of Physics and Optoelectronic Engineering, Beijing University of Technology, Beijing, 100124, China
| | - Shuai Feng
- School of Science, Minzu University of China, Beijing, 100081, China
| | - Xiaoyong Hu
- State Key Laboratory for Mesoscopic Physics and Department of Physics, Collaborative Innovation Center of Quantum Matter, Beijing Academy of Quantum Information Sciences, Nano-Optoelectronics Frontier Center of Ministry of Education, Peking University, Beijing, 100871, China.
- Collaborative Innovation Center of Extreme Optics, Shanxi University, Taiyuan, 030006, China.
- Peking University Yangtze Delta Institute of Optoelectronics, Nantong, 226010, China.
| |
Collapse
|
2
|
Li J, Li Y, Gan T, Shen CY, Jarrahi M, Ozcan A. All-optical complex field imaging using diffractive processors. LIGHT, SCIENCE & APPLICATIONS 2024; 13:120. [PMID: 38802376 PMCID: PMC11130282 DOI: 10.1038/s41377-024-01482-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 05/11/2024] [Accepted: 05/13/2024] [Indexed: 05/29/2024]
Abstract
Complex field imaging, which captures both the amplitude and phase information of input optical fields or objects, can offer rich structural insights into samples, such as their absorption and refractive index distributions. However, conventional image sensors are intensity-based and inherently lack the capability to directly measure the phase distribution of a field. This limitation can be overcome using interferometric or holographic methods, often supplemented by iterative phase retrieval algorithms, leading to a considerable increase in hardware complexity and computational demand. Here, we present a complex field imager design that enables snapshot imaging of both the amplitude and quantitative phase information of input fields using an intensity-based sensor array without any digital processing. Our design utilizes successive deep learning-optimized diffractive surfaces that are structured to collectively modulate the input complex field, forming two independent imaging channels that perform amplitude-to-amplitude and phase-to-intensity transformations between the input and output planes within a compact optical design, axially spanning ~100 wavelengths. The intensity distributions of the output fields at these two channels on the sensor plane directly correspond to the amplitude and quantitative phase profiles of the input complex field, eliminating the need for any digital image reconstruction algorithms. We experimentally validated the efficacy of our complex field diffractive imager designs through 3D-printed prototypes operating at the terahertz spectrum, with the output amplitude and phase channel images closely aligning with our numerical simulations. We envision that this complex field imager will have various applications in security, biomedical imaging, sensing and material science, among others.
Collapse
Affiliation(s)
- Jingxi Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Yuhang Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Tianyi Gan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Che-Yung Shen
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Mona Jarrahi
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA.
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA.
| |
Collapse
|
3
|
Hu J, Mengu D, Tzarouchis DC, Edwards B, Engheta N, Ozcan A. Diffractive optical computing in free space. Nat Commun 2024; 15:1525. [PMID: 38378715 PMCID: PMC10879514 DOI: 10.1038/s41467-024-45982-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Accepted: 02/09/2024] [Indexed: 02/22/2024] Open
Abstract
Structured optical materials create new computing paradigms using photons, with transformative impact on various fields, including machine learning, computer vision, imaging, telecommunications, and sensing. This Perspective sheds light on the potential of free-space optical systems based on engineered surfaces for advancing optical computing. Manipulating light in unprecedented ways, emerging structured surfaces enable all-optical implementation of various mathematical functions and machine learning tasks. Diffractive networks, in particular, bring deep-learning principles into the design and operation of free-space optical systems to create new functionalities. Metasurfaces consisting of deeply subwavelength units are achieving exotic optical responses that provide independent control over different properties of light and can bring major advances in computational throughput and data-transfer bandwidth of free-space optical processors. Unlike integrated photonics-based optoelectronic systems that demand preprocessed inputs, free-space optical processors have direct access to all the optical degrees of freedom that carry information about an input scene/object without needing digital recovery or preprocessing of information. To realize the full potential of free-space optical computing architectures, diffractive surfaces and metasurfaces need to advance symbiotically and co-evolve in their designs, 3D fabrication/integration, cascadability, and computing accuracy to serve the needs of next-generation machine vision, computational imaging, mathematical computing, and telecommunication technologies.
Collapse
Affiliation(s)
- Jingtian Hu
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Deniz Mengu
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Dimitrios C Tzarouchis
- Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, 19104, USA
- Meta Materials Inc., Athens, 15123, Greece
| | - Brian Edwards
- Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, 19104, USA
| | - Nader Engheta
- Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, 19104, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA.
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA.
| |
Collapse
|
4
|
Shao J, Zhou L, Yeung SYF, Lei T, Zhang W, Yuan X. Pulmonary Nodule Detection and Classification Using All-Optical Deep Diffractive Neural Network. Life (Basel) 2023; 13:life13051148. [PMID: 37240793 DOI: 10.3390/life13051148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Revised: 04/29/2023] [Accepted: 05/07/2023] [Indexed: 05/28/2023] Open
Abstract
A deep diffractive neural network (D2NN) is a fast optical computing structure that has been widely used in image classification, logical operations, and other fields. Computed tomography (CT) imaging is a reliable method for detecting and analyzing pulmonary nodules. In this paper, we propose using an all-optical D2NN for pulmonary nodule detection and classification based on CT imaging for lung cancer. The network was trained based on the LIDC-IDRI dataset, and the performance was evaluated on a test set. For pulmonary nodule detection, the existence of nodules scanned from CT images were estimated with two-class classification based on the network, achieving a recall rate of 91.08% from the test set. For pulmonary nodule classification, benign and malignant nodules were also classified with two-class classification with an accuracy of 76.77% and an area under the curve (AUC) value of 0.8292. Our numerical simulations show the possibility of using optical neural networks for fast medical image processing and aided diagnosis.
Collapse
Affiliation(s)
- Junjie Shao
- Nanophotonics Research Center, Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen 518060, China
| | - Lingxiao Zhou
- Nanophotonics Research Center, Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen 518060, China
| | - Sze Yan Fion Yeung
- State Key Laboratory on Advanced Displays and Optoelectronics Technologies, Department of Electronic & Computer Engineering, Hong Kong University of Science and Technology, Hong Kong SAR, China
| | - Ting Lei
- Nanophotonics Research Center, Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen 518060, China
| | - Wanlong Zhang
- Nanophotonics Research Center, Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen 518060, China
| | - Xiaocong Yuan
- Nanophotonics Research Center, Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen 518060, China
- Research Center for Humanoid Sensing, Research Institute of Intelligent Sensing, Zhejiang Lab, Hangzhou 311100, China
| |
Collapse
|
5
|
Bai B, Li Y, Luo Y, Li X, Çetintaş E, Jarrahi M, Ozcan A. All-optical image classification through unknown random diffusers using a single-pixel diffractive network. LIGHT, SCIENCE & APPLICATIONS 2023; 12:69. [PMID: 36894546 PMCID: PMC9998891 DOI: 10.1038/s41377-023-01116-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/20/2022] [Revised: 02/22/2023] [Accepted: 02/22/2023] [Indexed: 06/01/2023]
Abstract
Classification of an object behind a random and unknown scattering medium sets a challenging task for computational imaging and machine vision fields. Recent deep learning-based approaches demonstrated the classification of objects using diffuser-distorted patterns collected by an image sensor. These methods demand relatively large-scale computing using deep neural networks running on digital computers. Here, we present an all-optical processor to directly classify unknown objects through unknown, random phase diffusers using broadband illumination detected with a single pixel. A set of transmissive diffractive layers, optimized using deep learning, forms a physical network that all-optically maps the spatial information of an input object behind a random diffuser into the power spectrum of the output light detected through a single pixel at the output plane of the diffractive network. We numerically demonstrated the accuracy of this framework using broadband radiation to classify unknown handwritten digits through random new diffusers, never used during the training phase, and achieved a blind testing accuracy of 87.74 ± 1.12%. We also experimentally validated our single-pixel broadband diffractive network by classifying handwritten digits "0" and "1" through a random diffuser using terahertz waves and a 3D-printed diffractive network. This single-pixel all-optical object classification system through random diffusers is based on passive diffractive layers that process broadband input light and can operate at any part of the electromagnetic spectrum by simply scaling the diffractive features proportional to the wavelength range of interest. These results have various potential applications in, e.g., biomedical imaging, security, robotics, and autonomous driving.
Collapse
Affiliation(s)
- Bijie Bai
- Electrical and Computer Engineering Department, University of California, Los Angeles, California, 90095, USA
- Bioengineering Department, University of California, Los Angeles, California, 90095, USA
- California Nano Systems Institute (CNSI), University of California, Los Angeles, California, 90095, USA
| | - Yuhang Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, California, 90095, USA
- Bioengineering Department, University of California, Los Angeles, California, 90095, USA
- California Nano Systems Institute (CNSI), University of California, Los Angeles, California, 90095, USA
| | - Yi Luo
- Electrical and Computer Engineering Department, University of California, Los Angeles, California, 90095, USA
- Bioengineering Department, University of California, Los Angeles, California, 90095, USA
- California Nano Systems Institute (CNSI), University of California, Los Angeles, California, 90095, USA
| | - Xurong Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, California, 90095, USA
- California Nano Systems Institute (CNSI), University of California, Los Angeles, California, 90095, USA
| | - Ege Çetintaş
- Electrical and Computer Engineering Department, University of California, Los Angeles, California, 90095, USA
- Bioengineering Department, University of California, Los Angeles, California, 90095, USA
- California Nano Systems Institute (CNSI), University of California, Los Angeles, California, 90095, USA
| | - Mona Jarrahi
- Electrical and Computer Engineering Department, University of California, Los Angeles, California, 90095, USA
- California Nano Systems Institute (CNSI), University of California, Los Angeles, California, 90095, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, California, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, California, 90095, USA.
- California Nano Systems Institute (CNSI), University of California, Los Angeles, California, 90095, USA.
| |
Collapse
|
6
|
Sadeghzadeh H, Koohi S. Translation-invariant optical neural network for image classification. Sci Rep 2022; 12:17232. [PMID: 36241863 PMCID: PMC9568607 DOI: 10.1038/s41598-022-22291-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 10/12/2022] [Indexed: 01/06/2023] Open
Abstract
The classification performance of all-optical Convolutional Neural Networks (CNNs) is greatly influenced by components' misalignment and translation of input images in the practical applications. In this paper, we propose a free-space all-optical CNN (named Trans-ONN) which accurately classifies translated images in the horizontal, vertical, or diagonal directions. Trans-ONN takes advantages of an optical motion pooling layer which provides the translation invariance property by implementing different optical masks in the Fourier plane for classifying translated test images. Moreover, to enhance the translation invariance property, global average pooling (GAP) is utilized in the Trans-ONN structure, rather than fully connected layers. The comparative studies confirm that taking advantage of vertical and horizontal masks along GAP operation provide the best translation invariance property, compared to the alternative network models, for classifying horizontally and vertically shifted test images up to 50 pixel shifts of Kaggle Cats and Dogs, CIFAR-10, and MNIST datasets, respectively. Also, adopting the diagonal mask along GAP operation achieves the best classification accuracy for classifying translated test images in the diagonal direction for large number of pixel shifts (i.e. more than 30 pixel shifts). It is worth mentioning that the proposed translation invariant networks are capable of classifying the translated test images not included in the training procedure.
Collapse
Affiliation(s)
- Hoda Sadeghzadeh
- grid.412553.40000 0001 0740 9747Department of Computer Engineering, Sharif University of Technology, Tehran, Iran
| | - Somayyeh Koohi
- grid.412553.40000 0001 0740 9747Department of Computer Engineering, Sharif University of Technology, Tehran, Iran
| |
Collapse
|
7
|
Li J, Hung YC, Kulce O, Mengu D, Ozcan A. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network. LIGHT, SCIENCE & APPLICATIONS 2022; 11:153. [PMID: 35614046 PMCID: PMC9133014 DOI: 10.1038/s41377-022-00849-x] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Revised: 05/10/2022] [Accepted: 05/11/2022] [Indexed: 05/15/2023]
Abstract
Research on optical computing has recently attracted significant attention due to the transformative advances in machine learning. Among different approaches, diffractive optical networks composed of spatially-engineered transmissive surfaces have been demonstrated for all-optical statistical inference and performing arbitrary linear transformations using passive, free-space optical layers. Here, we introduce a polarization-multiplexed diffractive processor to all-optically perform multiple, arbitrarily-selected linear transformations through a single diffractive network trained using deep learning. In this framework, an array of pre-selected linear polarizers is positioned between trainable transmissive diffractive materials that are isotropic, and different target linear transformations (complex-valued) are uniquely assigned to different combinations of input/output polarization states. The transmission layers of this polarization-multiplexed diffractive network are trained and optimized via deep learning and error-backpropagation by using thousands of examples of the input/output fields corresponding to each one of the complex-valued linear transformations assigned to different input/output polarization combinations. Our results and analysis reveal that a single diffractive network can successfully approximate and all-optically implement a group of arbitrarily-selected target transformations with a negligible error when the number of trainable diffractive features/neurons (N) approaches [Formula: see text], where Ni and No represent the number of pixels at the input and output fields-of-view, respectively, and Np refers to the number of unique linear transformations assigned to different input/output polarization combinations. This polarization-multiplexed all-optical diffractive processor can find various applications in optical computing and polarization-based machine vision tasks.
Collapse
Affiliation(s)
- Jingxi Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Yi-Chun Hung
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
| | - Onur Kulce
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Deniz Mengu
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA.
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA.
| |
Collapse
|