Ultrafast dynamic machine vision with spatiotemporal photonic computing

Reconfigurable metamaterial processing units that solve arbitrary linear calculus equations

Calculus equations serve as fundamental frameworks in mathematics, enabling describing an extensive range of natural phenomena and scientific principles, such as thermodynamics and electromagnetics. Analog computing with electromagnetic waves presents an intriguing opportunity to solve calculus equations with unparalleled speed, while facing an inevitable tradeoff in computing density and equation reconfigurability. Here, we propose a reconfigurable metamaterial processing unit (MPU) that solves arbitrary linear calculus equations at a very fast speed. Subwavelength kernels based on inverse-designed pixel metamaterials are used to perform calculus operations on time-domain signals. In addition, feedback mechanisms and reconfigurable components are used to formulate and solve calculus equations with different orders and coefficients. A prototype of this MPU with a compact planar size of 0.93λ0×0.93λ0 (λ0 is the free-space wavelength) is constructed and evaluated in microwave frequencies. Experimental results demonstrate the MPU's ability to successfully solve arbitrary linear calculus equations. With the merits of compactness, easy integration, reconfigurability, and reusability, the proposed MPU provides a potential route for integrated analog computing with high speed of signal processing.

Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence

The pursuit of artificial general intelligence (AGI) continuously demands higher computing performance. Despite the superior processing speed and efficiency of integrated photonic circuits, their capacity and scalability are restricted by unavoidable errors, such that only simple tasks and shallow models are realized. To support modern AGIs, we designed Taichi-large-scale photonic chiplets based on an integrated diffractive-interference hybrid design and a general distributed computing architecture that has millions-of-neurons capability with 160-tera-operations per second per watt (TOPS/W) energy efficiency. Taichi experimentally achieved on-chip 1000-category-level classification (testing at 91.89% accuracy in the 1623-category Omniglot dataset) and high-fidelity artificial intelligence-generated content with up to two orders of magnitude of improvement in efficiency. Taichi paves the way for large-scale photonic computing and advanced tasks, further exploiting the flexibility and potential of photonics for modern AGI.

Photonic neuromorphic architecture for tens-of-task lifelong learning

Scalable, high-capacity, and low-power computing architecture is the primary assurance for increasingly manifold and large-scale machine learning tasks. Traditional electronic artificial agents by conventional power-hungry processors have faced the issues of energy and scaling walls, hindering them from the sustainable performance improvement and iterative multi-task learning. Referring to another modality of light, photonic computing has been progressively applied in high-efficient neuromorphic systems. Here, we innovate a reconfigurable lifelong-learning optical neural network (L2ONN), for highly-integrated tens-of-task machine intelligence with elaborated algorithm-hardware co-design. Benefiting from the inherent sparsity and parallelism in massive photonic connections, L2ONN learns each single task by adaptively activating sparse photonic neuron connections in the coherent light field, while incrementally acquiring expertise on various tasks by gradually enlarging the activation. The multi-task optical features are parallelly processed by multi-spectrum representations allocated with different wavelengths. Extensive evaluations on free-space and on-chip architectures confirm that for the first time, L2ONN avoided the catastrophic forgetting issue of photonic computing, owning versatile skills on challenging tens-of-tasks (vision classification, voice recognition, medical diagnosis, etc.) with a single model. Particularly, L2ONN achieves more than an order of magnitude higher efficiency than the representative electronic artificial neural networks, and 14× larger capacity than existing optical neural networks while maintaining competitive performance on each individual task. The proposed photonic neuromorphic architecture points out a new form of lifelong learning scheme, permitting terminal/edge AI systems with light-speed efficiency and unprecedented scalability.

Spatiotemporal diffractive deep neural networks

A spatiotemporal diffractive deep neural network (STD2NN) is proposed for spatiotemporal signal processing. The STD2NN is formed by gratings, which convert the signal from the frequency domain to the spatial domain, and multiple layers consisting of spatial lenses and space light modulators (SLMs), which conduct spatiotemporal phase modulation. An all-optical backpropagation (BP) algorithm for SLM phase tuning is proposed, with the gradient of the loss function computed by the inner product of the forward propagating optical field and the backward propagating conjugated error field. As a proof of concept, a spatiotemporal word "OPTICA" is generated by the STD2NN. Afterwards, a spatiotemporal optical vortex (STOV) beam multiplexer based on the STD2NN is demonstrated, which converts the spatially separated Gaussian beams into the STOV wave-packets with different topological charges. Both cases illustrate the capability of the proposed STD2NN to generate and process the spatiotemporal signals.

Reproduction of mode-locked pulses by spectrotemporal domain-informed deep learning

The accurate reproduction of unique pulse states in a mode-locked fiber laser is an important scientific issue and has wide applications in the laser industry. We present what we believe to be a novel method for automatically and precisely reproducing targeted soliton states in a mode-locked fiber laser by spectrotemporal domain-informed deep learning. Targeted solitons are experimentally reproduced via a superior matching process with a spectrotemporal mean square error (MSE) of 3.99 × 10-5. The outstanding feature of our reproduction algorithm is that the pulse information in both the spectral and temporal domains is jointly adopted for reconstructing targeted soliton states from white noise, rather than establishing arbitrary mode-locked pulse states, as described in previous studies. Additionally, a single-layer perceptron model is proposed to retrieve the phase distribution of a mode-locked pulse, validating the physical completeness of our reproduction approach. Our approach advances ultrafast laser technology, enabling the precise control of pulse dynamics in applications such as optical communication and nonlinear optics.