Snapshot spectral imaging with parallel metasystems

Imaging Spectropolarimeter Using a Multifunctional Metasurface

Spectral polarization imaging is critical for broad applications ranging from remote sensing to biomedicine. Here, we propose and experimentally demonstrate an imaging spectropolarimeter based on a single multifunctional metasurface. The designed metasurface accurately maps spectral and polarization information onto focal points and vortex beams, enabling simultaneous detection through intensity distributions. More specifically, spectral detection is achieved by determining the azimuthal angle of the strongest focal point, while polarization detection is accomplished by synthesizing the intensity of focal points and the interference pattern of output vortex beams. Experimental results indicate the successful reconstruction for six discrete wavelengths, with the average relative polarization error ranging from 7.85% to 13%. Additionally, the metasurface exhibits excellent imaging and edge detection capabilities owing to the focusing properties and the generation of vortex beams, achieving an imaging resolution of up to 1.4-fold wavelength and offering a new solution for a wide range of applications.

Miniaturized Hyperspectral Imager Utilizing a Reconfigurable Filter Array for Both High Spatial and Spectral Resolutions

Miniaturized hyperspectral imaging based on filter arrays has attracted much attention in consumer applications, such as food safety and biomedical applications. In this Letter, we demonstrate a miniaturized hyperspectral imager using a reconfigurable filter array to tackle the existing trade-off issue between the spectral and spatial resolutions. Utilizing tens of intermediate states of a vanadium dioxide cavity, we increase the total number of physical spectral channels by tens of times from a 2 × 2 mosaic filter unit, providing both high spatial and spectral resolutions for spectral imaging. The reconfigurable filter has a good spectral resolvability of 10 nm in the visible range with a wavelength inaccuracy of less than 2.1 nm. Hyperspectral imaging is demonstrated with a frame rate of 4.5 Hz.

Electrically tunable planar liquid-crystal singlets for simultaneous spectrometry and imaging

Conventional hyperspectral cameras cascade lenses and spectrometers to acquire the spectral datacube, which forms the fundamental framework for hyperspectral imaging. However, this cascading framework involves tradeoffs among spectral and imaging performances when the system is driven toward miniaturization. Here, we propose a spectral singlet lens that unifies optical imaging and computational spectrometry functions, enabling the creation of minimalist, miniaturized and high-performance hyperspectral cameras. As a paradigm, we capitalize on planar liquid crystal optics to implement the proposed framework, with each liquid-crystal unit cell acting as both phase modulator and electrically tunable spectral filter. Experiments with various targets show that the resulting millimeter-scale hyperspectral camera exhibits both high spectral fidelity ( > 95%) and high spatial resolutions ( ~1.7 times the diffraction limit). The proposed "two-in-one" framework can resolve the conflicts between spectral and imaging resolutions, which paves a practical pathway for advancing hyperspectral imaging systems toward miniaturization and portable applications.

Infrared color-sorting metasurfaces

The process of sorting light based on colors (photon energy) is a prerequisite in broadband optical systems, typically achieved in the form of guiding incoming signals through a sequence of spectral filters. The assembly of filters often leads to lengthy optical trains and consequently, large system footprints. In this work, we address this issue by proposing a flat color-sorting device comprising a diffraction grating and a dielectric Huygens' metasurface. Upon the incidence of a broadband beam, the grating disperses wavelengths to a continuous range of angles in accordance with the law of diffraction. The following metasurface with multiple paired Huygens' resonances corrects the dispersion and binds wavelengths to the corresponding waveband with a designated output angle. We demonstrate the sorting efficacy by designing a device with a color-sorting metasurface with two discrete dispersion-compensated outputs (10.8 ± 0.3 μm and 11.9 ± 0.3 μm), based on the proposed approach. The optimized metasurface possesses an overall transmittance exceeding 57% and reduces lateral dispersion by 90% at the output. The proposed color-sorting mechanism provides a solution that benefits the designing of metasurfaces for miniature multi-band systems.

Exploring the efficacy of subwavelength gratings as short-wavelength infrared filters

Advancements in nanofabrication technology have greatly facilitated research on nanostructures and their associated properties. Among these structures, subwavelength components have emerged as promising candidates for ultra-compact optical elements, can potentially supplant conventional optical components and enable the realization of compact and efficient optical devices. Spectral analysis within the infrared spectrum offers a wealth of information for monitoring crop health, industrial processes, and target identification. However, conventional spectrometers are typically bulky and expensive, driving an increasing demand for cost-effective spectral sensors. Here we investigate three distinct subwavelength grating structures designed to function as narrowband filters within the short-wavelength infrared (SWIR) range. Through simple adjustments to the period of grating strips, these filters selectively transmit light across a wide wavelength range from 1100 to 1700 nm with transmission exceeding 70% and full width at half maximum (FWHM) down to 6 nm. Based on a simple design, the results present great potential of subwavelength grating filters for multiband integration and developing ultra-compact spectral sensors.

Spectrometer-Less Remote Sensing Image Classification Based on Gate-Tunable van der Waals Heterostructures

Remote sensing technology, which conventionally employs spectrometers to capture hyperspectral images, allowing for the classification and unmixing based on the reflectance spectrum, has been extensively applied in diverse fields, including environmental monitoring, land resource management, and agriculture. However, miniaturization of remote sensing systems remains a challenge due to the complicated and dispersive optical components of spectrometers. Here, m-phase GaTe0.5Se0.5 with wide-spectral photoresponses (250-1064 nm) and stack it with WSe2 are utilizes to construct a two-dimensional van der Waals heterojunction (2D-vdWH), enabling the design of a gate-tunable wide-spectral photodetector. By utilizing the multi-photoresponses under varying gate voltages, high accuracy recognition can be achieved aided by deep learning algorithms without the original hyperspectral reflectance data. The proof-of-concept device, featuring dozens of tunable gate voltages, achieves an average classification accuracy of 87.00% on 6 prevalent hyperspectral datasets, which is competitive with the accuracy of 250-1000 nm hyperspectral data (88.72%) and far superior to the accuracy of non-tunable photoresponse (71.17%). Artificially designed gate-tunable wide-spectral 2D-vdWHs GaTe0.5Se0.5/WSe2-based photodetector present a promising pathway for the development of miniaturized and cost-effective remote sensing classification technology.

Multichannel meta-imagers for accelerating machine vision

Rapid developments in machine vision technology have impacted a variety of applications, such as medical devices and autonomous driving systems. These achievements, however, typically necessitate digital neural networks with the downside of heavy computational requirements and consequent high energy consumption. As a result, real-time decision-making is hindered when computational resources are not readily accessible. Here we report a meta-imager designed to work together with a digital back end to offload computationally expensive convolution operations into high-speed, low-power optics. In this architecture, metasurfaces enable both angle and polarization multiplexing to create multiple information channels that perform positively and negatively valued convolution operations in a single shot. We use our meta-imager for object classification, achieving 98.6% accuracy in handwritten digits and 88.8% accuracy in fashion images. Owing to its compactness, high speed and low power consumption, our approach could find a wide range of applications in artificial intelligence and machine vision applications.

Metasurface-empowered snapshot hyperspectral imaging with convex/deep (CODE) small-data learning theory

Hyperspectral imaging is vital for material identification but traditional systems are bulky, hindering the development of compact systems. While previous metasurfaces address volume issues, the requirements of complicated fabrication processes and significant footprint still limit their applications. This work reports a compact snapshot hyperspectral imager by incorporating the meta-optics with a small-data convex/deep (CODE) deep learning theory. Our snapshot hyperspectral imager comprises only one single multi-wavelength metasurface chip working in the visible window (500-650 nm), significantly reducing the device area. To demonstrate the high performance of our hyperspectral imager, a 4-band multispectral imaging dataset is used as the input. Through the CODE-driven imaging system, it efficiently generates an 18-band hyperspectral data cube with high fidelity using only 18 training data points. We expect the elegant integration of multi-resonant metasurfaces with small-data learning theory will enable low-profile advanced instruments for fundamental science studies and real-world applications.

Pixelated Filter Array for On-Chip Polarized Spectral Detection

On-chip multi-dimensional detection systems integrating pixelated polarization and spectral filter arrays are the latest trend in optical detection instruments, showing broad application potential for diagnostic medical imaging and remote sensing. However, thin-film or microstructure-based filter arrays typically have a trade-off between the detection dimension, optical efficiency, and spectral resolution. Here, we demonstrate novel on-chip integrated polarization spectral detection filter arrays consisting of metasurfaces and multilayer films. The metasurfaces with two nanopillars in one supercell are designed to modulate the Jones matrix for polarization selection. The angle of diffraction of the metasurfaces and the optical Fabry-Perot (FP) cavities determine the spectrum's center wavelength. The polarization spectral filter arrays are placed on top of the CMOS sensor; each array corresponds to one pixel, resulting in high spectral resolution and optical efficiency in the selected polarization state. To verify the methodology, we designed nine-channel polarized spectral filter arrays in a wavelength range of 1350 nm to 1550 nm for transverse electric (TE) linear polarization. The array has a 10 nm balanced spectral resolution and average peak transmission efficiency of over 75%, which is maintained by utilizing lossless dielectric material. The proposed array can be fabricated using overlay e-beam lithography, and the process is CMOS-compatible. The proposed array enables broader applications of in situ on-chip polarization spectral detection with high efficiency and spectral resolution, as well as in vivo imaging systems.

Handheld snapshot multi-spectral camera at tens-of-megapixel resolution

Multi-spectral imaging is a fundamental tool characterizing the constituent energy of scene radiation. However, current multi-spectral video cameras cannot scale up beyond megapixel resolution due to optical constraints and the complexity of the reconstruction algorithms. To circumvent the above issues, we propose a tens-of-megapixel handheld multi-spectral videography approach (THETA), with a proof-of-concept camera achieving 65-megapixel videography of 12 wavebands within visible light range. The high performance is brought by multiple designs: We propose an imaging scheme to fabricate a thin mask for encoding spatio-spectral data using a conventional film camera. Afterwards, a fiber optic plate is introduced for building a compact prototype supporting pixel-wise encoding with a large space-bandwidth product. Finally, a deep-network-based algorithm is adopted for large-scale multi-spectral data decoding, with the coding pattern specially designed to facilitate efficient coarse-to-fine model training. Experimentally, we demonstrate THETA's advantageous and wide applications in outdoor imaging of large macroscopic scenes.

Flexible long-wave infrared snapshot multispectral imaging with a pixel-level spectral filter array

This paper proposes and demonstrates a flexible long-wave infrared snapshot multispectral imaging system consisting of a simple re-imaging system and a pixel-level spectral filter array. A six-band multispectral image in the spectral range of 8-12 µm with full width at half maximum of about 0.7 µm each band is acquired in the experiment. The pixel-level multispectral filter array is placed at the primary imaging plane of the re-imaging system instead of directly encapsulated on the detector chip, which diminishes the complexity of pixel-level chip packaging. Furthermore, the proposed method possesses the merit of flexible functions switching between multispectral imaging and intensity imaging by plugging and unplugging the pixel-level spectral filter array. Our approach could be viable for various practical long-wave infrared detection applications.

Polychromatic full-polarization control in mid-infrared light

Objects with different shapes, materials and temperatures can emit distinct polarizations and spectral information in mid-infrared band, which provides a unique signature in the transparent window for object identification. However, the crosstalk among various polarization and wavelength channels prevents from accurate mid-infrared detections at high signal-to-noise ratio. Here, we report full-polarization metasurfaces to break the inherent eigen-polarization constraint over the wavelengths in mid-infrared. This recipe enables to select arbitrary orthogonal polarization basis at individual wavelength independently, therefore alleviating the crosstalk and efficiency degradation. A six-channel all-silicon metasurface is specifically presented to project focused mid-infrared light to distinct positions at three wavelengths, each with a pair of arbitrarily chosen orthogonal polarizations. An isolation ratio of 117 between neighboring polarization channels is experimentally recorded, exhibiting detection sensitivity one order of magnitude higher than existing infrared detectors. Remarkably, the high aspect ratio ~30 of our meta-structures manufactured by deep silicon etching technology at temperature -150 °C guarantees the large and precise phase dispersion control over a broadband from 3 to 4.5 μm. We believe our results would benefit the noise-immune mid-infrared detections in remote sensing and space-to-ground communications.

Grayscale-patterned integrated multilayer-metal-dielectric microcavities for on-chip multi/hyperspectral imaging in the extended visible bandwidth

Pixelated filter arrays of Fabry-Perot (FP) cavities are widely integrated with photodetectors to achieve a WYSIWYG ("what you see is what you get") on-chip spectral measurements. However, FP-filter-based spectral sensors typically have a trade-off between their spectral resolution and working bandwidth due to design limitations of conventional metal or dielectric multilayer microcavities. Here, we propose a new idea of integrated color filter arrays (CFAs) consisting of multilayer metal-dielectric-mirror FP microcavities that, enable a hyperspectral resolution over an extended visible bandwidth (∼300 nm). By introducing another two dielectric layers on the metallic film, the broadband reflectance of the FP-cavity mirror was greatly enhanced, accompanied by as-flat-as-possible reflection-phase dispersion. This resulted in balanced spectral resolution (∼10 nm) and spectral bandwidth from 450 nm to 750 nm. In the experiment, we used a one-step rapid manufacturing process by using grayscale e-beam lithography. A 16-channel (4 × 4) CFA was fabricated and demonstrated on-chip spectral imaging with a CMOS sensor and an impressive identification capability. Our results provide an attractive method for developing high-performance spectral sensors and have potential commercial applications by extending the utility of low-cost manufacturing process.

mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instruments, slow data acquisition rate, and spatiospectral trade-off. Here we introduce hyperspectral learning for snapshot hyperspectral imaging in which sampled hyperspectral data in a small subarea are incorporated into a learning algorithm to recover the hypercube. Hyperspectral learning exploits the idea that a photograph is more than merely a picture and contains detailed spectral information. A small sampling of hyperspectral data enables spectrally informed learning to recover a hypercube from a red-green-blue (RGB) image without complete hyperspectral measurements. Hyperspectral learning is capable of recovering full spectroscopic resolution in the hypercube, comparable to high spectral resolutions of scientific spectrometers. Hyperspectral learning also enables ultrafast dynamic imaging, leveraging ultraslow video recording in an off-the-shelf smartphone, given that a video comprises a time series of multiple RGB images. To demonstrate its versatility, an experimental model of vascular development is used to extract hemodynamic parameters via statistical and deep learning approaches. Subsequently, the hemodynamics of peripheral microcirculation is assessed at an ultrafast temporal resolution up to a millisecond, using a conventional smartphone camera. This spectrally informed learning method is analogous to compressed sensing; however, it further allows for reliable hypercube recovery and key feature extractions with a transparent learning algorithm. This learning-powered snapshot hyperspectral imaging method yields high spectral and temporal resolutions and eliminates the spatiospectral trade-off, offering simple hardware requirements and potential applications of various machine learning techniques.

High-efficiency metalens-based compact multispectral variable spectrometer

Conventional spectrometers are bulky, and researchers have continuously made efforts in their miniaturization and integration in recent years. Among these studies, metalenses have attracted immense interest because of their merits of a flat shape and flexible regulation. Herein, we introduce a design of a polarization-insensitive metalens-based spectrometer that utilizes an off-axis high-efficiency broadband metalens in the wavelength range of 500-1000 nm. The demonstrated metalens consisting of nanopillars employs propagation phase and phase function optimization methods and can achieve spectral resolutions of 0.6 nm with efficiency as high as 77%. By stitching metalenses with different focal lengths, the functionality of the spectrometer can be expanded. Hence, a compact variable design with favorable focusing and dispersive properties can be achieved by one single component instead of traditional cascading optics, thus shrinking the volume to the millimeter scale and reducing cost. This research proves the potential for applications of metalenses in spectrometers as well as other consumer and industry products.

Silicon metasurface embedded Fabry-Perot cavity enables the high-quality transmission structural color

While nanoscale color generations have been studied for years, the high-performance transmission structural color, simultaneously equipped with large gamut, high resolution, and optical multiplexing abilities, still remains as a hanging issue. Here, a silicon metasurface embedded Fabry-Perot cavity is demonstrated to address this problem. By changing the planar geometries of meta-atoms, the cavities provide transmission colors with 194% sRGB gamut coverage and 141,111 DPI resolution, along with more than 300% enhanced angular tolerance. Such high density allows two-dimensional color mixing at the diffraction limit scale. Benefitting from the polarization manipulation capacity of the metasurface, arbitrary color arrangements between cyan and red for two orthogonal linear polarizations are also realized. Our proposed cavities can be used in filters, printings, optical storage, and many other applications in need of high quality and density colors.

Meta-optic accelerators for object classifiers

Rapid advances in deep learning have led to paradigm shifts in a number of fields, from medical image analysis to autonomous systems. These advances, however, have resulted in digital neural networks with large computational requirements, resulting in high energy consumption and limitations in real-time decision-making when computation resources are limited. Here, we demonstrate a meta-optic-based neural network accelerator that can off-load computationally expensive convolution operations into high-speed and low-power optics. In this architecture, metasurfaces enable both spatial multiplexing and additional information channels, such as polarization, in object classification. End-to-end design is used to co-optimize the optical and digital systems, resulting in a robust classifier that achieves 93.1% accurate classification of handwriting digits and 93.8% accuracy in classifying both the digit and its polarization state. This approach could enable compact, high-speed, and low-power image and information processing systems for a wide range of applications in machine vision and artificial intelligence.

Compact multispectral photodetectors based on nanodisk arrays atop optical cavity substrates

It is challenging for the multi-spectral photodetector to have a compact structure, high spectral resolution, and high detection efficiency. This paper reports on a new approach for compact multi-spectral visible light detecting based on the hexagonal lattice silver nanodisk arrays atop optical cavity substrates. Through numerical calculations and optimizations of experiments, we verified that the narrow band responsivity of the photodetector was caused by coupling the surface plasmonic resonances and cavity mode. The multi-spectral photodetector exhibited that the minimum FWHM and the maximum responsivity of was achieved to be 80 nm and 91.5 mA·W^-1, respectively. Besides, we also analyzed the influence of the proposed structure on the energy wastage by numerical comparison. The proposed way for multi-spectral photodetector is promising to be an excellent design for the narrow band spectral detection. The design can also be easily integrated with CMOS devices and applied to other spectral regimes for different applications.

Inverse design enables large-scale high-performance meta-optics reshaping virtual reality

Meta-optics has achieved major breakthroughs in the past decade; however, conventional forward design faces challenges as functionality complexity and device size scale up. Inverse design aims at optimizing meta-optics design but has been currently limited by expensive brute-force numerical solvers to small devices, which are also difficult to realize experimentally. Here, we present a general inverse-design framework for aperiodic large-scale (20k × 20k λ2) complex meta-optics in three dimensions, which alleviates computational cost for both simulation and optimization via a fast approximate solver and an adjoint method, respectively. Our framework naturally accounts for fabrication constraints via a surrogate model. In experiments, we demonstrate aberration-corrected metalenses working in the visible with high numerical aperture, poly-chromatic focusing, and large diameter up to the centimeter scale. Such large-scale meta-optics opens a new paradigm for applications, and we demonstrate its potential for future virtual-reality platforms by using a meta-eyepiece and a laser back-illuminated micro-Liquid Crystal Display.

Integrated photonic metasystem for image classifications at telecommunication wavelength

Miniaturized image classifiers are potential for revolutionizing their applications in optical communication, autonomous vehicles, and healthcare. With subwavelength structure enabled directional diffraction and dispersion engineering, the light propagation through multi-layer metasurfaces achieves wavelength-selective image recognitions on a silicon photonic platform at telecommunication wavelength. The metasystems implement high-throughput vector-by-matrix multiplications, enabled by near 10³ nanoscale phase shifters as weight elements within 0.135 mm² footprints. The diffraction manifested computing capability incorporates the fabrication and measurement related phase fluctuations, and thus the pre-trained metasystem can handle uncertainties in inputs without post-tuning. Here we demonstrate three functional metasystems: a 15-pixel spatial pattern classifier that reaches near 90% accuracy with femtosecond inputs, a multi-channel wavelength demultiplexer, and a hyperspectral image classifier. The diffractive metasystem provides an alternative machine learning architecture for photonic integrated circuits, with densely integrated phase shifters, spatially multiplexed throughput, and data processing capabilities.

Metasystem architectures are attractive alternatives to waveguide-based integrated photonic processors due to the subwavelength structures. Here, the authors report a 1D passive silicon photonic metasystem with near 90% spatial pattern classification accuracy at telecommunication wavelength.

Synthetic coded aperture snapshot spectral imaging based on coprime sub-aperture sampling

The imaging of a large area scene is difficult to achieve for a single camera. Alternatively, a virtual large aperture can be synthesized by sub-aperture cooperation. We propose a solution through the combination of a coded aperture snapshot spectral imager and coprime array. This method reduces the amount of data by using a smaller sub-aperture for sampling. The position of the sub-aperture is determined according to the element distribution of the coprime array, so that the data obtained from each sampling are about the target object and its adjacent area, which can ensure high reconstruction accuracy. The feasibility of coprime sub-aperture sampling is verified by numerical simulation.