Computational inverse design for cascaded systems of metasurface optics

Hybrid meta/refractive lens design with an inverse design using physical optics

Hybrid lenses are created by combining metasurface optics with refractive optics, where refractive elements contribute optical power, while metasurfaces correct optical aberrations. We present an algorithm for optimizing metasurface nanostructures within a hybrid lens, allowing flexible interleaving of metasurface and refractive optics in the optical train. To efficiently optimize metasurface nanostructures, we develop a scalar field, ray-wave hybrid propagation method. This method facilitates the propagation of incident and derived adjoint fields through optical elements, enabling effective metasurface optimization within the framework of adjoint gradient optimization. Numerical examples of various lens configurations are presented to illustrate the versatility of the algorithm and showcase the benefits offered by the proposed approach, allowing metasurfaces to be positioned beyond the image space of a lens. Taking a F/2, 40° field-of-view, midwave infrared lens as an example, the lens exhibits an average focusing efficiency of 38% before the integration of metasurfaces. Utilizing the new algorithm to design two metasurfaces-one in the object space and one in the image space-results in significant enhancement of the average focusing efficiency to over 90%. In contrast, a counterpart design with both metasurfaces limited to the image space yields a lower average focusing efficiency of 73%.

Monolayer directional metasurface for all-optical image classifier doublet

Diffractive deep neural networks, known for their passivity, high scalability, and high efficiency, offer great potential in holographic imaging, target recognition, and object classification. However, previous endeavors have been hampered by spatial size and alignment. To address these issues, this study introduces a monolayer directional metasurface, aimed at reducing spatial constraints and mitigating alignment issues. Utilizing this methodology, we use MNIST datasets to train diffractive deep neural networks and realize digital classification, revealing that the metasurface can achieve excellent digital image classification results, and the classification accuracy of ideal phase mask plates and metasurface for phase-only modulation can reach 84.73% and 84.85%, respectively. Despite a certain loss of degrees of freedom compared to multi-layer phase mask plates, the single-layer metasurface is easier to fabricate and align, thereby improving spatial utilization efficiency.

Multi-Dimensional Multiplexed Metasurface Holography by Inverse Design

Multi-dimensional multiplexed metasurface holography extends holographic information capacity and promises revolutionary advancements for vivid imaging, information storage, and encryption. However, achieving multifunctional metasurface holography by forward design method is still difficult because it relies heavily on Jones matrix engineering, which places high demands on physical knowledge and processing technology. To break these limitations and simplify the design process, here, an end-to-end inverse design framework is proposed. By directly linking the metasurface to the reconstructed images and employing a loss function to guide the update of metasurface, the calculation of hologram can be omitted; thus, greatly simplifying the design process. In addition, the requirements on the completeness of meta-library can also be significantly reduced, allowing multi-channel hologram to be achieved using meta-atoms with only two degrees of freedom, which is very friendly to processing. By exploiting the proposed method, metasurface hologram containing up to 12 channels of multi-wavelength, multi-plane, and multi-polarization is designed and experimentally demonstrated, which exhibits the state-of-the-art information multiplexing capacity of the metasurface composed of simple meta-atoms. This method is conducive to promoting the intelligent design of multifunctional meta-devices, and it is expected to eventually accelerate the application of meta-devices in colorful display, imaging, storage and other fields.

Broadband and large-aperture metasurface edge encoders for incoherent infrared radiation

The prevalence of computer vision systems necessitates hardware-based approaches to relieve the high computational demand of deep neural networks in resource-limited applications. One solution would be to off-load low-level image feature extraction, such as edge detection, from the digital network to the analog imaging system. To that end, this work demonstrates incoherent, broadband, low-noise optical edge detection of real-world scenes by combining the wavefront shaping of a 24-mm aperture metasurface with a refractive lens. An inverse design approach is used to optimize the metasurface for Laplacian-based edge detection across the 7.5- to 13.5-μm LWIR imaging band, allowing for facile integration with uncooled microbolometer-based LWIR imagers to encode edge information. A polarization multiplexed approach leveraging a birefringent metasurface is also demonstrated as a single-aperture implementation. This work could be applied to improve computer vision capabilities of resource-constrained systems by leveraging optical preprocessing to alleviate the computational requirements for high-accuracy image segmentation and classification.

Cascaded metasurfaces for high-purity vortex generation

We introduce a new paradigm for generating high-purity vortex beams with metasurfaces. By applying optical neural networks to a system of cascaded phase-only metasurfaces, we demonstrate the efficient generation of high-quality Laguerre-Gaussian (LG) vortex modes. Our approach is based on two metasurfaces where one metasurface redistributes the intensity profile of light in accord with Rayleigh-Sommerfeld diffraction rules, and then the second metasurface matches the required phases for the vortex beams. Consequently, we generate high-purity LGp,l optical modes with record-high Laguerre polynomial orders p = 10 and l = 200, and with the purity in p, l and relative conversion efficiency as 96.71%, 85.47%, and 70.48%, respectively. Our engineered cascaded metasurfaces suppress greatly the backward reflection with a ratio exceeding -17 dB. Such higher-order optical vortices with multiple orthogonal states can revolutionize next-generation optical information processing.

Revisiting the Design Strategies for Metasurfaces: Fundamental Physics, Optimization, and Beyond

Over the last two decades, the capabilities of metasurfaces in light modulation with subwavelength thickness have been proven, and metasurfaces are expected to miniaturize conventional optical components and add various functionalities. Herein, various metasurface design strategies are reviewed thoroughly. First, the scalar diffraction theory is revisited to provide the basic principle of light propagation. Then, widely used design methods based on the unit-cell approach are discussed. The methods include a set of simplified steps, including the phase-map retrieval and meta-atom unit-cell design. Then, recently emerging metasurfaces that may not be accurately designed using unit-cell approach are introduced. Unconventional metasurfaces are examined where the conventional design methods fail and finally potential design methods for such metasurfaces are discussed.

Computational inverse design for cascaded systems of metasurface optics: comment

In a recently published article by Backer [Opt. Express27(21), 30308 (2019).10.1364/OE.27.030308], a computational inverse design method is developed for designing optical systems composed of multiple metasurfaces. The forward propagation model used in this method was a discretized version of the angular spectrum propagator described by Goodman [Introduction to Fourier Optics, 1996]. However, slight modifications are necessary to increase the accuracy of this inverse design method. This comment examines the accuracy of the results obtained by the above-mentioned method by a full-wave electromagnetic solver and explains the reason of their difference. Thereafter, slight modifications to the method proposed by Backer are suggested, and the accuracy of final formulation is verified by a full-wave electromagnetic solver.

Realization of optical logic gates using on-chip diffractive optical neural networks

Optical computing is highly desired as a potential strategy for circumventing the performance limitations of semiconductor-based electronic devices and circuits. Optical logic gates are considered as fundamental building blocks for optical computation and they enable logic functions to be performed extremely quickly without the generation of heat and crosstalk. Here, we discuss the design of a multi-functional optical logic gate based on an on-chip diffractive optical neural network that can perform AND, NOT and OR logic operations at the wavelength of 1.55 µm. The wavelength-independent operation of the multi-functional logic gate at seven wavelengths (over a bandwidth of 60 nm) is also studied which paves the way for wavelength division multiplexed parallel computation. This simple, highly-integrable, low-loss, energy-efficient and broadband optical logic gate provides a path for the development of high-speed on-chip nanophotonic processors for future optical computing applications.

Doublet Metalens with Simultaneous Chromatic and Monochromatic Correction in the Mid-Infrared

Metalenses provide a powerful paradigm for mid-infrared (MIR) imaging and detection while keeping the optical system compact. However, the design of MIR metalenses simultaneously correcting chromatic aberration and off-axis monochromatic aberration remains challenging. Here, we propose an MIR doublet metalens composed of a silicon aperture metalens and a silicon focusing metalens separated by a fused silica substrate. By performing ray-tracing optimization and particle-swarm optimization, we optimized the required phase profiles as well as the sizes and spatial distributions of silicon nanopillars of the doublet metalens. Simulation results showed that the MIR doublet metalens simultaneously achieved chromatic and off-axis monochromatic aberration reduction, realizing a continuous 400 nm bandwidth and 20° field-of-view (FOV). Thanks to its planar configuration, this metalens is suitable for integration with CMOS image sensor to achieve MIR imaging and detection, which has potential application in troubleshooting and intelligent inspection of power grids. This work may facilitate the practical application of metalens-integrated micro/nanosensors in intelligent energy.

Broadband metasurface aberration correctors for hybrid meta/refractive MWIR lenses

A method for designing multi-metasurface layouts for optical aberration correction is presented. All-dielectric metasurfaces are combined with conventional refractive optics to form a hybrid lens. The optical power of a hybrid lens is primarily provided by refractive optics, and metasurfaces are optimized to control optical aberrations. This approach greatly reduces the magnitude of phase gradient required for a largescale metasurface and hence its diffraction loss. An inverse design technique is incorporated to optimize all physical parameters on a metasurface to minimize image spots across all sampling field angles and wavelengths. This approach is put to test by designing a hybrid lens composed of a midwave infrared refractive lens followed by a pair of metasurfaces. Moreover, we demonstrate the working bandwidth of the hybrid lens can be further extended by reducing phase dispersion introduced by a metasurface using holey meta-atoms instead of pillar meta-atoms.

Artificial Intelligence in Meta-optics

Recent years have witnessed promising artificial intelligence (AI) applications in many disciplines, including optics, engineering, medicine, economics, and education. In particular, the synergy of AI and meta-optics has greatly benefited both fields. Meta-optics are advanced flat optics with novel functions and light-manipulation abilities. The optical properties can be engineered with a unique design to meet various optical demands. This review offers comprehensive coverage of meta-optics and artificial intelligence in synergy. After providing an overview of AI and meta-optics, we categorize and discuss the recent developments integrated by these two topics, namely AI for meta-optics and meta-optics for AI. The former describes how to apply AI to the research of meta-optics for design, simulation, optical information analysis, and application. The latter reports the development of the optical Al system and computation via meta-optics. This review will also provide an in-depth discussion of the challenges of this interdisciplinary field and indicate future directions. We expect that this review will inspire researchers in these fields and benefit the next generation of intelligent optical device design.

Neural Schrödinger Equation: Physical Law as Deep Neural Network

We show a new family of neural networks based on the Schrödinger equation (SE-NET). In this analogy, the trainable weights of the neural networks correspond to the physical quantities of the Schrödinger equation. These physical quantities can be trained using the complex-valued adjoint method. Since the propagation of the SE-NET can be described by the evolution of physical systems, its outputs can be computed by using a physical solver. The trained network is transferable to actual optical systems. As a demonstration, we implemented the SE-NET with the Crank-Nicolson finite difference method on Pytorch. From the results of numerical simulations, we found that the performance of the SE-NET becomes better when the SE-NET becomes wider and deeper. However, the training of the SE-NET was unstable due to gradient explosions when SE-NET becomes deeper. Therefore, we also introduced phase-only training, which only updates the phase of the potential field (refractive index) in the Schrödinger equation. This enables stable training even for the deep SE-NET model because the unitarity of the system is kept under the training. In addition, the SE-NET enables a joint optimization of physical structures and digital neural networks. As a demonstration, we performed a numerical demonstration of end-to-end machine learning (ML) with an optical frontend toward a compact spectrometer. Our results extend the application field of ML to hybrid physical-digital optimizations.

Inverse design enables large-scale high-performance meta-optics reshaping virtual reality

Meta-optics has achieved major breakthroughs in the past decade; however, conventional forward design faces challenges as functionality complexity and device size scale up. Inverse design aims at optimizing meta-optics design but has been currently limited by expensive brute-force numerical solvers to small devices, which are also difficult to realize experimentally. Here, we present a general inverse-design framework for aperiodic large-scale (20k × 20k λ2) complex meta-optics in three dimensions, which alleviates computational cost for both simulation and optimization via a fast approximate solver and an adjoint method, respectively. Our framework naturally accounts for fabrication constraints via a surrogate model. In experiments, we demonstrate aberration-corrected metalenses working in the visible with high numerical aperture, poly-chromatic focusing, and large diameter up to the centimeter scale. Such large-scale meta-optics opens a new paradigm for applications, and we demonstrate its potential for future virtual-reality platforms by using a meta-eyepiece and a laser back-illuminated micro-Liquid Crystal Display.

Integrated photonic metasystem for image classifications at telecommunication wavelength

Miniaturized image classifiers are potential for revolutionizing their applications in optical communication, autonomous vehicles, and healthcare. With subwavelength structure enabled directional diffraction and dispersion engineering, the light propagation through multi-layer metasurfaces achieves wavelength-selective image recognitions on a silicon photonic platform at telecommunication wavelength. The metasystems implement high-throughput vector-by-matrix multiplications, enabled by near 10³ nanoscale phase shifters as weight elements within 0.135 mm² footprints. The diffraction manifested computing capability incorporates the fabrication and measurement related phase fluctuations, and thus the pre-trained metasystem can handle uncertainties in inputs without post-tuning. Here we demonstrate three functional metasystems: a 15-pixel spatial pattern classifier that reaches near 90% accuracy with femtosecond inputs, a multi-channel wavelength demultiplexer, and a hyperspectral image classifier. The diffractive metasystem provides an alternative machine learning architecture for photonic integrated circuits, with densely integrated phase shifters, spatially multiplexed throughput, and data processing capabilities.

Metasystem architectures are attractive alternatives to waveguide-based integrated photonic processors due to the subwavelength structures. Here, the authors report a 1D passive silicon photonic metasystem with near 90% spatial pattern classification accuracy at telecommunication wavelength.

Inverse design of gradient-index volume multimode converters

Graded-index optical elements are capable of shaping light precisely and in very specific ways. While classical freeform optics uses only a two-dimensional domain such as the surface of a lens, recent technological advances in laser manufacturing offer promising prospects for the realization of arbitrary three-dimensional graded-index volumes, i.e. transparent dielectric substrates with voxel-wise modified refractive index distributions. Such elements would be able to perform complex light transformations on compact scales. Here we present an algorithmic approach for computing 3D graded-index devices, which utilizes numerical beam propagation and error reduction based on gradient descent. We present solutions for millimeter-sized elements addressing important tasks in photonics: a mode sorter, a photonic lantern and a multimode intensity beam shaper. We further discuss suitable cost functions for all designs to be used in the algorithm. The 3D graded-index designs are spatially smooth and require a relatively small refractive index range in the order of 10^-2, which is within the reach of direct laser writing manufacturing processes such as two-photon polymerization.

Adjoint-optimized metasurfaces for compact mode-division multiplexing

Optical fiber communications rely on multiplexing techniques that encode information onto various degrees of freedom of light to increase the transmission capacity of a fiber. However, the rising demand for larger data capacity is driving the need for a multiplexer for the spatial dimension of light. We introduce a mode-division multiplexer and demultiplexer design based on a metasurface cavity. This device performs, on a single surface, mode conversion and coupling to fibers without any additional optics. Converted modes have high fidelity due to the repeated interaction of light with the metasurface's phase profile that was optimized using an inverse design technique known as adjoint analysis. We experimentally demonstrate a compact and highly integrated metasurface-based mode multiplexer that takes three single-mode fiber inputs and converts them into the first three linearly polarized spatial modes of a few-mode fiber with fidelities of up to 72% in the C-band (1530-1565 nm).

Volumetric imaging efficiency: the fundamental limit to compactness of imaging systems

A new metric for imaging systems, the volumetric imaging efficiency (VIE), is introduced. It compares the compactness and capacity of an imager against fundamental limits imposed by diffraction. Two models are proposed for this fundamental limit based on an idealized thin-lens and the optical volume required to form diffraction-limited images. The VIE is computed for 2,871 lens designs and plotted as a function of FOV; this quantifies the challenge of creating compact, wide FOV lenses. We identify an empirical limit to the VIE given by VIE < 0.920 × 10^-0.582×FOV when using conventional bulk optics imaging onto a flat sensor. We evaluate VIE for lenses employing curved image surfaces and planar, monochromatic metasurfaces to show that these new optical technologies can surpass the limit of conventional lenses and yield >100x increase in VIE.

Differentiable scattering matrix for optimization of photonic structures

The scattering matrix, which quantifies the optical reflection and transmission of a photonic structure, is pivotal for understanding the performance of the structure. In many photonic design tasks, it is also desired to know how the structure's optical performance changes with respect to design parameters, that is, the scattering matrix's derivatives (or gradient). Here we address this need. We present a new algorithm for computing scattering matrix derivatives accurately and robustly. In particular, we focus on the computation in semi-analytical methods (such as rigorous coupled-wave analysis). To compute the scattering matrix of a structure, these methods must solve an eigen-decomposition problem. However, when it comes to computing scattering matrix derivatives, differentiating the eigen-decomposition poses significant numerical difficulties. We show that the differentiation of the eigen-decomposition problem can be completely sidestepped, and thereby propose a robust algorithm. To demonstrate its efficacy, we use our algorithm to optimize metasurface structures and reach various optical design goals.

Integrated photonic neural network based on silicon metalines

An integrated photonic neural network is proposed based on on-chip cascaded one-dimensional (1D) metasurfaces. High-contrast transmitarray metasurfaces, termed as metalines in this paper, are defined sequentially in the silicon-on-insulator substrate with a distance much larger than the operation wavelength. Matrix-vector multiplications can be accomplished in parallel and with low energy consumption due to intrinsic parallelism and low-loss of silicon metalines. The proposed on-chip whole-passive fully-optical meta-neural-network is very compact and works at the speed of light, with very low energy consumption. Various complex functions that are performed by digital neural networks can be implemented by our proposal at the wavelength of 1.55 µm. As an example, the performance of our optical neural network is benchmarked on the prototypical machine learning task of classification of handwritten digits images from the Modified National Institute of Standards and Technology (MNIST) dataset, and an accuracy comparable to the state of the art is achieved.

VIPR: vectorial implementation of phase retrieval for fast and accurate microscopic pixel-wise pupil estimation

In microscopy, proper modeling of the image formation has a substantial effect on the precision and accuracy in localization experiments and facilitates the correction of aberrations in adaptive optics experiments. The observed images are subject to polarization effects, refractive index variations, and system specific constraints. Previously reported techniques have addressed these challenges by using complicated calibration samples, computationally heavy numerical algorithms, and various mathematical simplifications. In this work, we present a phase retrieval approach based on an analytical derivation of the vectorial diffraction model. Our method produces an accurate estimate of the system's phase information, without any prior knowledge about the aberrations, in under a minute.