Ultracompact meta-imagers for arbitrary all-optical convolution

Compact meta-differentiator for achieving isotropically high-contrast ultrasonic imaging

Ultrasonic imaging is crucial in the fields of biomedical engineering for its deep penetration capabilities and non-ionizing nature. However, traditional techniques heavily rely on impedance differences within objects, resulting in poor contrast when imaging acoustically transparent targets. Here, we propose a compact spatial differentiator for underwater isotropic edge-enhanced imaging, which enhances the imaging contrast without the need for contrast agents or external physical fields. This design incorporates an amplitude meta-grating for linear transmission along the radial direction, combined with a phase meta-grating that utilizes focus and spiral phases with a first-order topological charge. Through theoretical analysis, numerical simulations, and experimental validation, we substantiate the effectiveness of our technique in distinguishing amplitude objects with isotropic edge enhancements. Importantly, this method also enables the accurate detection of both phase objects and artificial biological models. This breakthrough creates new opportunities for applications in medical diagnosis and nondestructive testing.

Multichannel meta-imagers for accelerating machine vision

Rapid developments in machine vision technology have impacted a variety of applications, such as medical devices and autonomous driving systems. These achievements, however, typically necessitate digital neural networks with the downside of heavy computational requirements and consequent high energy consumption. As a result, real-time decision-making is hindered when computational resources are not readily accessible. Here we report a meta-imager designed to work together with a digital back end to offload computationally expensive convolution operations into high-speed, low-power optics. In this architecture, metasurfaces enable both angle and polarization multiplexing to create multiple information channels that perform positively and negatively valued convolution operations in a single shot. We use our meta-imager for object classification, achieving 98.6% accuracy in handwritten digits and 88.8% accuracy in fashion images. Owing to its compactness, high speed and low power consumption, our approach could find a wide range of applications in artificial intelligence and machine vision applications.

Reconfigurable transmissive metasurface with a combination of scissor and rotation actuators for independently controlling beam scanning and polarization conversion

Polarization conversion and beam scanning metasurfaces are commonly used to reduce polarization mismatch and direct electromagnetic waves in a specific direction to improve the strength of a wireless signal. However, identifying suitable active and mechanically reconfigurable metasurfaces for polarization conversion and beam scanning is a considerable challenge, and the reported metasurfaces have narrow scanning ranges, are expensive, and cannot be independently controlled. In this paper, we propose a reconfigurable transmissive metasurface combined with a scissor and rotation actuator for independently controlling beam scanning and polarization conversion functions. The metasurface is constructed with rotatable unit cells (UCs) that can switch the polarization state between right-handed (RHCP) and left-handed circular polarization (LHCP) by flipping the UCs to reverse their phase variation. Moreover, independent beam scanning is achieved using the scissor actuator to linearly change the distance between the UCs. Numerical and experimental results confirm that the proposed metasurface can perform beam scanning in the range of 28° for both the positive and negative regions of a radiation pattern (RHCP and LHCP beams) at an operational frequency of 10.5 GHz.

Broadband angular spectrum differentiation using dielectric metasurfaces

Signal processing is of critical importance for various science and technology fields. Analog optical processing can provide an effective solution to perform large-scale and real-time data processing, superior to its digital counterparts, which have the disadvantages of low operation speed and large energy consumption. As an important branch of modern optics, Fourier optics exhibits great potential for analog optical image processing, for instance for edge detection. While these operations have been commonly explored to manipulate the spatial content of an image, mathematical operations that act directly over the angular spectrum of an image have not been pursued. Here, we demonstrate manipulation of the angular spectrum of an image, and in particular its differentiation, using dielectric metasurfaces operating across the whole visible spectrum. We experimentally show that this technique can be used to enhance desired portions of the angular spectrum of an image. Our approach can be extended to develop more general angular spectrum analog meta-processors, and may open opportunities for optical analog data processing and biological imaging.

Broadband and large-aperture metasurface edge encoders for incoherent infrared radiation

The prevalence of computer vision systems necessitates hardware-based approaches to relieve the high computational demand of deep neural networks in resource-limited applications. One solution would be to off-load low-level image feature extraction, such as edge detection, from the digital network to the analog imaging system. To that end, this work demonstrates incoherent, broadband, low-noise optical edge detection of real-world scenes by combining the wavefront shaping of a 24-mm aperture metasurface with a refractive lens. An inverse design approach is used to optimize the metasurface for Laplacian-based edge detection across the 7.5- to 13.5-μm LWIR imaging band, allowing for facile integration with uncooled microbolometer-based LWIR imagers to encode edge information. A polarization multiplexed approach leveraging a birefringent metasurface is also demonstrated as a single-aperture implementation. This work could be applied to improve computer vision capabilities of resource-constrained systems by leveraging optical preprocessing to alleviate the computational requirements for high-accuracy image segmentation and classification.

Switchable Two-Dimensional AND and Exclusive OR Operation Based on Dual-Wavelength Metasurfaces

Logic operation serves as the foundation and core element of computing networks; it will bring huge vitality to advanced information processing with its adaptation in the optical domain. As fundamental logic operations, AND and exclusive OR (XOR) operations serve a multitude of purposes, such as their ability to cooperate in enabling image processing and interpretation. Here, we propose and experimentally demonstrate a wavelength multiplexed AND and XOR function based on metasurfaces. By combining two cosine gratings with distinct frequencies and an initial phase difference of π/2, we extract the similarities and differences between two input images simultaneously by illuminating them with 445 and 633 nm wavelengths. Additionally, we explore its potential in information encryption, where overall security is enhanced by distributing distinct parts of initial information and encoded keys to different receivers. This design possesses the benefits of convenient mode switching and high-quality imaging, facilitating advanced applications in pattern recognition, machine vision, medical diagnosis, etc.

Correlated optical convolutional neural network with "quantum speedup"

Compared with electrical neural networks, optical neural networks (ONNs) have the potentials to break the limit of the bandwidth and reduce the consumption of energy, and therefore draw much attention in recent years. By far, several types of ONNs have been implemented. However, the current ONNs cannot realize the acceleration as powerful as that indicated by the models like quantum neural networks. How to construct and realize an ONN with the quantum speedup is a huge challenge. Here, we propose theoretically and demonstrate experimentally a new type of optical convolutional neural network by introducing the optical correlation. It is called the correlated optical convolutional neural network (COCNN). We show that the COCNN can exhibit "quantum speedup" in the training process. The character is verified from the two aspects. One is the direct illustration of the faster convergence by comparing the loss function curves of the COCNN with that of the traditional convolutional neural network (CNN). Such a result is compatible with the training performance of the recently proposed quantum convolutional neural network (QCNN). The other is the demonstration of the COCNN's capability to perform the QCNN phase recognition circuit, validating the connection between the COCNN and the QCNN. Furthermore, we take the COCNN analog to the 3-qubit QCNN phase recognition circuit as an example and perform an experiment to show the soundness and the feasibility of it. The results perfectly match the theoretical calculations. Our proposal opens up a new avenue for realizing the ONNs with the quantum speedup, which will benefit the information processing in the era of big data.

Metalens for Accelerated Optoelectronic Edge Detection under Ambient Illumination

Analog systems may allow image processing, such as edge detection, with low computational power. However, most demonstrated analog systems, based on either conventional 4-f imaging systems or nanophotonic structures, rely on coherent laser sources for illumination, which significantly restricts their use in routine imaging tasks with ambient, incoherent illumination. Here, we demonstrated a metalens-assisted imaging system that can allow optoelectronic edge detection under ambient illumination conditions. The metalens was designed to generate polarization-dependent optical transfer functions (OTFs), resulting in a synthetic OTF with an isotropic high-pass frequency response after digital subtraction. We integrated the polarization-multiplexed metalens with a polarization camera and experimentally demonstrated single-shot edge detection of indoor and outdoor scenes, including a flying airplane, under ambient sunlight illumination. The proposed system showcased the potential of using polarization multiplexing for the construction of complex optical convolution kernels toward accelerated machine vision tasks such as object detection and classification under ambient illumination.

Surface topography detection based on an optical differential metasurface

Surface topography detection can extract critical characteristics from objects, playing an important role in target identification and precision measurement. Here, an optical method with the advantages of low power consumption, high speed, and simple devices is proposed to realize the surface topography detection of low-contrast phase objects. By constructing reflected light paths, a metasurface can perform spatial differential operation via receiving the light directly reflected from a target. Therefore, our scheme is experimentally demonstrated as having remarkable universality, which can be used not only for opaque objects, but also for transparent pure phase objects. It provides a new, to the best of our knowledge, application for optical differential metasurfaces in precise detection of microscale surface topography.

Trichannel Spin-Selective Metalenses

Metalenses have the potential to revolutionize optical devices into the next generation of consumer devices. Through new inventive strategies, metalenses with advanced functionalities have been released to integrate multiple responses into a single flat device. Here, we design metalenses that are sensitive to the incident spin angular momentum to provide three distinct modes based on the handedness of the incident and transmitted light. Propagation phase is employed to encode a hyperbolic lens phase to the metalens, while geometric phase is exploited for additional spin-selective properties. We experimentally demonstrate two different metalenses: the co-polarized channels function as a standard metalens, while the cross-polarized channels (1) deflect and (2) introduce orbital angular momentum to the transmitted light. We experimentally characterize the metalenses and prove their use for spin-selective imaging of visible light. We envision that such trichannel metalenses could be employed in chiral bioimaging, optical computing, and computer vision.

Bright-Field and Edge-Enhanced Imaging Using an Electrically Tunable Dual-Mode Metalens

The imaging of microscopic biological samples faces numerous difficulties due to their small feature sizes and low-amplitude contrast. Metalenses have shown great promise in bioimaging as they have access to the complete complex information, which, alongside their extremely small and compact footprint and potential to integrate multiple functionalities into a single device, allow for miniaturized microscopy with exceptional features. Here, we design and experimentally realize a dual-mode metalens integrated with a liquid crystal cell that can be electrically switched between bright-field and edge-enhanced imaging on the millisecond scale. We combine the concepts of geometric and propagation phase to design the dual-mode metalens and physically encode the required phase profiles using hydrogenated amorphous silicon for operation at visible wavelengths. The two distinct metalens phase profiles include (1) a conventional hyperbolic metalens for bright-field imaging and (2) a spiral metalens with a topological charge of +1 for edge-enhanced imaging. We demonstrate the focusing and vortex generation ability of the metalens under different states of circular polarization and prove its use for biological imaging. This work proves a method for in vivo observation and monitoring of the cell response and drug screening within a compact form factor.

Integrated metasurfaces for re-envisioning a near-future disruptive optical platform

Metasurfaces have been continuously garnering attention in both scientific and industrial fields, owing to their unprecedented wavefront manipulation capabilities using arranged subwavelength artificial structures. To date, research has mainly focused on the full control of electromagnetic characteristics, including polarization, phase, amplitude, and even frequencies. Consequently, versatile possibilities of electromagnetic wave control have been achieved, yielding practical optical components such as metalenses, beam-steerers, metaholograms, and sensors. Current research is now focused on integrating the aforementioned metasurfaces with other standard optical components (e.g., light-emitting diodes, charged-coupled devices, micro-electro-mechanical systems, liquid crystals, heaters, refractive optical elements, planar waveguides, optical fibers, etc.) for commercialization with miniaturization trends of optical devices. Herein, this review describes and classifies metasurface-integrated optical components, and subsequently discusses their promising applications with metasurface-integrated optical platforms including those of augmented/virtual reality, light detection and ranging, and sensors. In conclusion, this review presents several challenges and prospects that are prevalent in the field in order to accelerate the commercialization of metasurfaces-integrated optical platforms.

Single-shot isotropic differential interference contrast microscopy

Differential interference contrast (DIC) microscopy allows high-contrast, low-phototoxicity, and label-free imaging of transparent biological objects, and has been applied in the field of cellular morphology, cell segmentation, particle tracking, optical measurement and others. Commercial DIC microscopy based on Nomarski or Wollaston prism resorts to the interference of two polarized waves with a lateral differential offset (shear) and axial phase shift (bias). However, the shear generated by these prisms is limited to the rectilinear direction, unfortunately resulting in anisotropic contrast imaging. Here we propose an ultracompact metasurface-assisted isotropic DIC (i-DIC) microscopy based on a grand original pattern of radial shear interferometry, that converts the rectilinear shear into rotationally symmetric along radial direction, enabling single-shot isotropic imaging capabilities. The i-DIC presents a complementary fusion of typical meta-optics, traditional microscopes and integrated optical system, and showcases the promising and synergetic advancements in edge detection, particle motion tracking, and label-free cellular imaging.

Dielectric Metasurface for Synchronously Spiral Phase Contrast and Bright-Field Imaging

Spiral phase contrast imaging and bright-field imaging are two widely used modes in microscopy, providing distinct morphological information about objects. However, conventional microscopes are always unable to operate with these two modes at the same time and need additional optical elements to switch between them. Here, we present a microscopy setup that incorporates a dielectric metasurface capable of achieving spiral phase contrast imaging and bright-field imaging synchronously. The metasurface not only can focus the light for diffraction-limited imaging but also can perform a two-dimensional spatial differentiation operation by imparting an orbital angular momentum to the incident light field. This allows two spatially separated images to be simultaneously obtained, one containing high-frequency edge information and the other showing the entirety of the object. Combined with the advantages of planar architecture and ultrathin thickness of the metasurface, this approach is expected to provide support in the fields of microscopy, biomedicine, and materials science.

Spin-decoupled excitation and wavefront shaping of structured surface waves via on-chip terahertz metasurfaces

Surface waves (SWs) are of great importance in terahertz (THz) photonics applications due to their subwavelength properties. Hence, it is crucial to develop surface wavefront shaping techniques, which is urgent in modern information technologies. In this paper, a new scheme is proposed to realize SW excitation and spin-decoupled wavefront shaping with an ultracompact planar meta-device working in the THz range. The meta-device is composed of two parts: meta-atoms (in the center) and plasmonic metals (on the left and right sides). By carefully setting the geometry size and rotation angle of each meta-atom, the encoded spin-decoupled phase distributions for both left circularly polarized (LCP) and right circularly polarized (RCP) incident THz waves are determined. In this way, circularly polarized (CP) incident THz waves can be converted to SWs propagating along plasmonic metals with unique wavefront profiles, i.e., Bessel and focusing profiles. Full-wave simulations and THz near-field scanning experiments were performed to verify the functionalities of the meta-device, both of which are in great agreement with theoretical predictions. Our findings may provide more solutions to design THz integrated photonic devices and systems.

Selectively reflective edge detection system based on cholesteric liquid crystal

Optical analog computing has attracted extensive interest in image processing and optical engineering in recent decades. Here, we propose a reflective optical analog computing system based on a cholesteric liquid crystal (CLC), which simplifies the traditional optical analog computing system by taking advantage of the CLC reflecting the light with specified circular polarization and provides a new, to the best of our knowledge, idea for the integration of optical analog computing systems. Meanwhile, we present results in which a section of an insect foot is observed using the reflective optical analog computing system, which may develop valuable applications in biomedical imaging.

Visualization of transparent particles based on optical spatial differentiation

Optical analog computing operates on the amplitude, phase, polarization, and frequency distributions of the electromagnetic field through the interaction of light and matter. The differentiation operation is widely used in all-optical image processing technology, such as edge detection. Here, we propose a concise way to observe transparent particles, incorporating the optical differential operation that occurs on a single particle. The particle's scattering and cross-polarization components combine into our differentiator. We achieve high-contrast optical images of transparent liquid crystal molecules. The visualization of aleurone grains (the structures that store protein particles in plant cells) in maize seed was experimentally demonstrated with a broadband incoherent light source. Avoiding the interference of stains, our designed method provides the possibility to observe protein particles directly in complex biological tissues.

Realization of all-optical higher-order spatial differentiators based on cascaded operations

Cascaded operations play an important role in traditional electronic computing systems for the realization of advanced strategies. Here, we introduce the idea of cascaded operations into all-optical spatial analog computing. The single function of the first-order operation has difficulty meeting the requirements of practical applications in image recognition. The all-optical second-order spatial differentiators are implemented by cascading two first-order differential operation units, and the image edge detection of amplitude and phase objects are demonstrated. Our scheme provides a possible pathway toward the development of compact multifunctional differentiators and advanced optical analog computing networks.

Dual-channel anticounterfeiting color-nanoprinting with a single-size nanostructured metasurface

Metasurface-based structural-colors are usually implemented by changing the dimensions of nanostructures to produce different spectral responses. Therefore, a single-size nanostructured metasurface usually cannot display structural-colors since it has only one design degree of freedom (DOF), i.e., the orientation angles of nanostructures. Here, we show structural-color nanoprinting images can be generated with a single-size nanostructured metasurface, enabled by designing the anisotropic nanostructure with different spectral responses along its long- and short-axis directions, respectively. More interestingly, the concept of orientation degeneracy of nanostructures can be applied in the metasurface design, which shows two spectral modulations can be implemented under different polarization directions of output light, thus extending the color-nanoprinting from single-channel to dual-channel. The proposed dual-channel metasurface used for anticounterfeiting color-nanoprinting has presented the advantages of ultra-compactness, high information capacity, and vivid colors, which can develop broad applications in fields such as high-end anticounterfeiting, high-density information storage, optical encryption, etc.

Computing metasurfaces enabled chiral edge image sensing

Computing metasurfaces have shown the extraordinary ability to precisely perform optical analog operations to the input light wave, and therefore exhibit greater potentials toward sensing applications. Here, we propose a unique application of computing metasurface for chiral edge sensing by incorporating a weak-value amplification technique. The computing metasurface performs the spatial differentiation operations of phase objects and extracts the edge-enhanced images, because the phase gradient generally occurs at the edge. The chirality-induced polarization rotation acts as the preselection state and the spatial differentiation operations in the metasurface provide weak coupling. The amplified pointer shift related to the tiny polarization rotation will eventually lead to an asymmetric edge-enhanced image. Owing to the high sensitivity of the weak-value amplification, we experimentally demonstrate a high-contrast recognition of chirality by edge detection, which may have potential applications in real-time measurement and separation of chiral enantiomers.

•

Computing metasurfaces perform the spatial differentiation operations of phase object

•

Weak-value amplification technique has been proposed for the chiral sensing

•

The chiral edge image sensing has been demonstrated

Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible

Replacing electrons with photons is a compelling route toward high-speed, massively parallel, and low-power artificial intelligence computing. Recently, diffractive networks composed of phase surfaces were trained to perform machine learning tasks through linear optical transformations. However, the existing architectures often comprise bulky components and, most critically, they cannot mimic the human brain for multitasking. Here, we demonstrate a multi-skilled diffractive neural network based on a metasurface device, which can perform on-chip multi-channel sensing and multitasking in the visible. The polarization multiplexing scheme of the subwavelength nanostructures is applied to construct a multi-channel classifier framework for simultaneous recognition of digital and fashionable items. The areal density of the artificial neurons can reach up to 6.25 × 10⁶ mm^-2 multiplied by the number of channels. The metasurface is integrated with the mature complementary metal-oxide semiconductor imaging sensor, providing a chip-scale architecture to process information directly at physical layers for energy-efficient and ultra-fast image processing in machine vision, autonomous driving, and precision medicine.

Metasurface-Based Imagers Enabled Arbitrary Optical Convolution Processing

Using meta-imagers composed of a meta-lens and a complex-amplitude meta-modulator, all-optical convolutional processing that arbitrarily reshapes the point spread function of an optical system can now be implemented.