An optical neural chip for implementing complex-valued neural network

Cascadable optical nonlinear activation function based on Ge-Si

To augment the capabilities of optical computing, specialized nonlinear devices as optical activation functions are crucial for enhancing the complexity of optical neural networks. However, existing optical nonlinear activation function devices often encounter challenges in preparation, compatibility, and multi-layer cascading. Here, we propose a cascadable optical nonlinear activation function architecture based on Ge-Si structured devices. Leveraging dual-source modulation, this architecture achieves cascading and wavelength switching by compensating for loss. Experimental comparisons with traditional Ge-Si devices validate the cascading capability of the new architecture. We first verified the versatility of this activation function in a MNIST task, and then in a multi-layer optical dense neural network designed for complex gesture recognition classification, the proposed architecture improves accuracy by an average of 23% compared to a linear network and 15% compared to a network with a traditional activation function architecture. With its advantages of cascadability and high compatibility, this work underscores the potential of all-optical activation functions for large-scale optical neural network scaling and complex task handling.

Multimodal In-Sensor Computing System Using Integrated Silicon Photonic Convolutional Processor

Photonic integrated circuits offer miniaturized solutions for multimodal spectroscopic sensory systems by leveraging the simultaneous interaction of light with temperature, chemicals, and biomolecules, among others. The multimodal spectroscopic sensory data is complex and has huge data volume with high redundancy, thus requiring high communication bandwidth associated with high communication power consumption to transfer the sensory data. To circumvent this high communication cost, the photonic sensor and processor are brought into intimacy and propose a photonic multimodal in-sensor computing system using an integrated silicon photonic convolutional processor. A microring resonator crossbar array is used as the photonic processor to implement convolutional operation with 5-bit accuracy, validated through image edge detection tasks. Further integrating the processor with a photonic spectroscopic sensor, the in situ processing of multimodal spectroscopic sensory data is demonstrated, achieving the classification of protein species of different types and concentrations at various temperatures. A classification accuracy of 97.58% across 45 different classes is achieved. The multimodal in-sensor computing system demonstrates the feasibility of integrating photonic processors and photonic sensors to enhance the data processing capability of photonic devices at the edge.

120 GOPS Photonic tensor core in thin-film lithium niobate for inference and in situ training

Photonics offers a transformative approach to artificial intelligence (AI) and neuromorphic computing by enabling low-latency, high-speed, and energy-efficient computations. However, conventional photonic tensor cores face significant challenges in constructing large-scale photonic neuromorphic networks. Here, we propose a fully integrated photonic tensor core, consisting of only two thin-film lithium niobate (TFLN) modulators, a III-V laser, and a charge-integration photoreceiver. Despite its simple architecture, it is capable of implementing an entire layer of a neural network with a computational speed of 120 GOPS, while also allowing flexible adjustment of the number of inputs (fan-in) and outputs (fan-out). Our tensor core supports rapid in-situ training with a weight update speed of 60 GHz. Furthermore, it successfully classifies (supervised learning) and clusters (unsupervised learning) 112  × 112-pixel images through in-situ training. To enable in-situ training for clustering AI tasks, we offer a solution for performing multiplications between two negative numbers.

Comparative Transcriptome Analysis of Bovine, Porcine, and Sheep Muscle Using Interpretable Machine Learning Models

The growth and development of muscle tissue play a pivotal role in the economic value and quality of meat in agricultural animals, garnering close attention from breeders and researchers. The quality and palatability of muscle tissue directly determine the market competitiveness of meat products and the satisfaction of consumers. Therefore, a profound understanding and management of muscle growth is essential for enhancing the overall economic efficiency and product quality of the meat industry. Despite this, systematic research on muscle development-related genes across different species still needs to be improved. This study addresses this gap through extensive cross-species muscle transcriptome analysis, combined with interpretable machine learning models. Utilizing a comprehensive dataset of 275 publicly available transcriptomes derived from porcine, bovine, and ovine muscle tissues, encompassing samples from ten distinct muscle types such as the semimembranosus and longissimus dorsi, this study analyzes 113 porcine (n = 113), 94 bovine (n = 94), and 68 ovine (n = 68) specimens. We employed nine machine learning models, such as Support Vector Classifier (SVC) and Support Vector Machine (SVM). Applying the SHapley Additive exPlanations (SHAP) method, we analyzed the muscle transcriptome data of cattle, pigs, and sheep. The optimal model, adaptive boosting (AdaBoost), identified key genes potentially influencing muscle growth and development across the three species, termed SHAP genes. Among these, 41 genes (including NANOG, ADAMTS8, LHX3, and TLR9) were consistently expressed in all three species, designated as homologous genes. Specific candidate genes for cattle included SLC47A1, IGSF1, IRF4, EIF3F, CGAS, ZSWIM9, RROB1, and ABHD18; for pigs, DRP2 and COL12A1; and for sheep, only COL10A1. Through the analysis of SHAP genes utilizing Kyoto Encyclopedia of Genes and Genomes (KEGG) pathways, relevant pathways such as ether lipid metabolism, cortisol synthesis and secretion, and calcium signaling pathways have been identified, revealing their pivotal roles in muscle growth and development.

Optical neural networks: progress and challenges

Artificial intelligence has prevailed in all trades and professions due to the assistance of big data resources, advanced algorithms, and high-performance electronic hardware. However, conventional computing hardware is inefficient at implementing complex tasks, in large part because the memory and processor in its computing architecture are separated, performing insufficiently in computing speed and energy consumption. In recent years, optical neural networks (ONNs) have made a range of research progress in optical computing due to advantages such as sub-nanosecond latency, low heat dissipation, and high parallelism. ONNs are in prospect to provide support regarding computing speed and energy consumption for the further development of artificial intelligence with a novel computing paradigm. Herein, we first introduce the design method and principle of ONNs based on various optical elements. Then, we successively review the non-integrated ONNs consisting of volume optical components and the integrated ONNs composed of on-chip components. Finally, we summarize and discuss the computational density, nonlinearity, scalability, and practical applications of ONNs, and comment on the challenges and perspectives of the ONNs in the future development trends.

Photonic probabilistic machine learning using quantum vacuum noise

Probabilistic machine learning utilizes controllable sources of randomness to encode uncertainty and enable statistical modeling. Harnessing the pure randomness of quantum vacuum noise, which stems from fluctuating electromagnetic fields, has shown promise for high speed and energy-efficient stochastic photonic elements. Nevertheless, photonic computing hardware which can control these stochastic elements to program probabilistic machine learning algorithms has been limited. Here, we implement a photonic probabilistic computer consisting of a controllable stochastic photonic element - a photonic probabilistic neuron (PPN). Our PPN is implemented in a bistable optical parametric oscillator (OPO) with vacuum-level injected bias fields. We then program a measurement-and-feedback loop for time-multiplexed PPNs with electronic processors (FPGA or GPU) to solve certain probabilistic machine learning tasks. We showcase probabilistic inference and image generation of MNIST-handwritten digits, which are representative examples of discriminative and generative models. In both implementations, quantum vacuum noise is used as a random seed to encode classification uncertainty or probabilistic generation of samples. In addition, we propose a path towards an all-optical probabilistic computing platform, with an estimated sampling rate of  ~1 Gbps and energy consumption of  ~5 fJ/MAC. Our work paves the way for scalable, ultrafast, and energy-efficient probabilistic machine learning hardware.

Tunable optical matrix convolution of 20-Gbit/s QPSK 2-D data with a kernel using optical wave mixing

Compared to its electronic counterpart, optically performed matrix convolution can accommodate phase-encoded data at high rates while avoiding optical-to-electronic-to-optical (OEO) conversions. We experimentally demonstrate a reconfigurable matrix convolution of quadrature phase-shift keying (QPSK)-encoded input data. The two-dimensional (2-D) input data is serialized, and its time-shifted replicas are generated. This 2-D data is convolved with a 1-D kernel with coefficients, which are applied by adjusting the relative phase and amplitude of the kernel pumps. Time-shifted data replicas (TSDRs) and kernel pumps are coherently mixed using nonlinear wave mixing in a periodically poled lithium niobate (PPLN) waveguide. To show the tunability and reconfigurability of this approach, we vary the kernel coefficients, kernel sizes (e.g., 2 × 1 or 3 × 1), and input data rates (e.g., 6-20 Gbit/s). The convolution results are verified to be error-free under an applied: (a) 2 × 1 kernel, resulting in a 16-quadrature amplitude modulation (QAM) output with an error vector magnitude (EVM) of ∼5.1-8.5%; and (b) 3 × 1 kernel, resulting in a 64-QAM output with an EVM of ∼4.9-5.5%.

Thin film ferroelectric photonic-electronic memory

To reduce system complexity and bridge the interface between electronic and photonic circuits, there is a high demand for a non-volatile memory that can be accessed both electrically and optically. However, practical solutions are still lacking when considering the potential for large-scale complementary metal-oxide semiconductor compatible integration. Here, we present an experimental demonstration of a non-volatile photonic-electronic memory based on a 3-dimensional monolithic integrated ferroelectric-silicon ring resonator. We successfully demonstrate programming and erasing the memory using both electrical and optical methods, assisted by optical-to-electrical-to-optical conversion. The memory cell exhibits a high optical extinction ratio of 6.6 dB at a low working voltage of 5 V and an endurance of 4 × 104 cycles. Furthermore, the multi-level storage capability is analyzed in detail, revealing stable performance with a raw bit-error-rate smaller than 5.9 × 10-2. This ground-breaking work could be a key technology enabler for future hybrid electronic-photonic systems, targeting a wide range of applications such as photonic interconnect, high-speed data communication, and neuromorphic computing.

Partial coherence enhances parallelized photonic computing

Advancements in optical coherence control1-5 have unlocked many cutting-edge applications, including long-haul communication, light detection and ranging (LiDAR) and optical coherence tomography6-8. Prevailing wisdom suggests that using more coherent light sources leads to enhanced system performance and device functionalities9-11. Our study introduces a photonic convolutional processing system that takes advantage of partially coherent light to boost computing parallelism without substantially sacrificing accuracy, potentially enabling larger-size photonic tensor cores. The reduction of the degree of coherence optimizes bandwidth use in the photonic convolutional processing system. This breakthrough challenges the traditional belief that coherence is essential or even advantageous in integrated photonic accelerators, thereby enabling the use of light sources with less rigorous feedback control and thermal-management requirements for high-throughput photonic computing. Here we demonstrate such a system in two photonic platforms for computing applications: a photonic tensor core using phase-change-material photonic memories that delivers parallel convolution operations to classify the gaits of ten patients with Parkinson's disease with 92.2% accuracy (92.7% theoretically) and a silicon photonic tensor core with embedded electro-absorption modulators (EAMs) to facilitate 0.108 tera operations per second (TOPS) convolutional processing for classifying the Modified National Institute of Standards and Technology (MNIST) handwritten digits dataset with 92.4% accuracy (95.0% theoretically).

Multimodal deep learning using on-chip diffractive optics with in situ training capability

Multimodal deep learning plays a pivotal role in supporting the processing and learning of diverse data types within the realm of artificial intelligence generated content (AIGC). However, most photonic neuromorphic processors for deep learning can only handle a single data modality (either vision or audio) due to the lack of abundant parameter training in optical domain. Here, we propose and demonstrate a trainable diffractive optical neural network (TDONN) chip based on on-chip diffractive optics with massive tunable elements to address these constraints. The TDONN chip includes one input layer, five hidden layers, and one output layer, and only one forward propagation is required to obtain the inference results without frequent optical-electrical conversion. The customized stochastic gradient descent algorithm and the drop-out mechanism are developed for photonic neurons to realize in situ training and fast convergence in the optical domain. The TDONN chip achieves a potential throughput of 217.6 tera-operations per second (TOPS) with high computing density (447.7 TOPS/mm2), high system-level energy efficiency (7.28 TOPS/W), and low optical latency (30.2 ps). The TDONN chip has successfully implemented four-class classification in different modalities (vision, audio, and touch) and achieve 85.7% accuracy on multimodal test sets. Our work opens up a new avenue for multimodal deep learning with integrated photonic processors, providing a potential solution for low-power AI large models using photonic technology.

I/O-efficient iterative matrix inversion with photonic integrated circuits

Photonic integrated circuits have been extensively explored for optical processing with the aim of breaking the speed and energy efficiency bottlenecks of digital electronics. However, the input/output (IO) bottleneck remains one of the key barriers. Here we report a photonic iterative processor (PIP) for matrix-inversion-intensive applications. The direct reuse of inputted data in the optical domain unlocks the potential to break the IO bottleneck. We demonstrate notable IO advantages with a lossless PIP for real-valued matrix inversion and integral-differential equation solving, as well as a coherent PIP with optical loops integrated on-chip, enabling complex-valued computation and a net inversion time of 1.2 ns. Furthermore, we estimate at least an order of magnitude enhancement in IO efficiency of a PIP over photonic single-pass processors and the state-of-the-art electronic processors for reservoir training tasks and multiple-input and multiple-output (MIMO) precoding tasks, indicating the huge potential of PIP technology in practical applications.

Perfect linear optics using silicon photonics

Recently there has been growing interest in using photonics to perform the linear algebra operations of neuromorphic and quantum computing applications, aiming at harnessing silicon photonics' (SiPho) high-speed and energy-efficiency credentials. Accurately mapping, however, a matrix into optics remains challenging, since state-of-the-art optical architectures are sensitive to fabrication imperfections. This leads to reduced fidelity that degrades as the insertion losses of the optical matrix nodes or the matrix dimensions increase. In this work, we present the experimental deployment of a 4 × 4 coherent crossbar (Xbar) as a silicon chip and validate experimentally its theoretically predicted fidelity restoration credentials. We demonstrate the experimental implementation of 10,000 arbitrary linear transformations achieving a record-high fidelity of 99.997% ± 0.002, limited mainly by the measurement equipment. Our work represents an integrated optical circuit providing almost unity and loss-independent fidelity in the realization of arbitrary matrices, highlighting light's credentials in resolving complex computations.

Interdisciplinary analysis and optimization of digital photonic devices for meta-photonics

With the continuous integration and development of AI and natural sciences, we have been diligently exploring a computational analysis framework for digital photonic devices. Here, We have overcome the challenge of limited datasets through the use of Generative Adversarial Network networks and transfer learning, providing AI feedback that aligns with human knowledge systems. Furthermore, we have introduced knowledge from disciplines such as image denoising, multi-agent modeling of Physarum polycephalum, percolation theory, wave function collapse algorithms, and others to analyze this new design system. It represents an accomplishment unattainable within the framework of classical photonics theory and significantly improves the performance of the designed devices. Notably, we present theoretical analyses for the drastic changes in device performance and the enhancement of device robustness, which have not been reported in previous research. The proposed concept of meta-photonics transcends the conventional boundaries of disciplinary silos, demonstrating the transformative potential of interdisciplinary fusion.

Multiplexable all-optical nonlinear activator for optical computing

As an alternative solution to surpass electronic neural networks, optical neural networks (ONNs) offer significant advantages in terms of energy consumption and computing speed. Despite the optical hardware platform could provide an efficient approach to realizing neural network algorithms than traditional hardware, the lack of optical nonlinearity limits the development of ONNs. Here, we proposed and experimentally demonstrated an all-optical nonlinear activator based on the stimulated Brillouin scattering (SBS). Utilizing the exceptional carrier dynamics of SBS, our activator supports two types of nonlinear functions, saturable absorption and rectified linear unit (Relu) models. Moreover, the proposed activator exhibits large dynamic response bandwidth (∼11.24 GHz), low nonlinear threshold (∼2.29 mW), high stability, and wavelength division multiplexing identities. These features have potential advantages for the physical realization of optical nonlinearities. As a proof of concept, we verify the performance of the proposed activator as an ONN nonlinear mapping unit via numerical simulations. Simulation shows that our approach achieves comparable performance to the activation functions commonly used in computers. The proposed approach provides support for the realization of all-optical neural networks.

Empowering high-dimensional optical fiber communications with integrated photonic processors

Mode-division multiplexing (MDM) in optical fibers enables multichannel capabilities for various applications, including data transmission, quantum networks, imaging, and sensing. However, high-dimensional optical fiber systems, usually necessity bulk-optics approaches for launching different orthogonal fiber modes into the optical fiber, and multiple-input multiple-output digital electronic signal processing at the receiver to undo the arbitrary mode scrambling introduced by coupling and transmission in a multi-mode fiber. Here we show that a high-dimensional optical fiber communication system can be implemented by a reconfigurable integrated photonic processor, featuring kernels of multichannel mode multiplexing transmitter and all-optical descrambling receiver. Effective mode management can be achieved through the configuration of the integrated optical mesh. Inter-chip MDM optical communications involving six spatial- and polarization modes was realized, despite the presence of unknown mode mixing and polarization rotation in the circular-core optical fiber. The proposed photonic integration approach holds promising prospects for future space-division multiplexing applications.

An optoacoustic field-programmable perceptron for recurrent neural networks

Recurrent neural networks (RNNs) can process contextual information such as time series signals and language. But their tracking of internal states is a limiting factor, motivating research on analog implementations in photonics. While photonic unidirectional feedforward neural networks (NNs) have demonstrated big leaps, bi-directional optical RNNs present a challenge: the need for a short-term memory that (i) programmable and coherently computes optical inputs, (ii) minimizes added noise, and (iii) allows scalability. Here, we experimentally demonstrate an optoacoustic recurrent operator (OREO) which meets (i, ii, iii). OREO contextualizes the information of an optical pulse sequence via acoustic waves. The acoustic waves link different optical pulses, capturing their information and using it to manipulate subsequent operations. OREO's all-optical control on a pulse-by-pulse basis offers simple reconfigurability and is used to implement a recurrent drop-out and pattern recognition of 27 optical pulse patterns. Finally, we introduce OREO as bi-directional perceptron for new classes of optical NNs.

Multichannel meta-imagers for accelerating machine vision

Rapid developments in machine vision technology have impacted a variety of applications, such as medical devices and autonomous driving systems. These achievements, however, typically necessitate digital neural networks with the downside of heavy computational requirements and consequent high energy consumption. As a result, real-time decision-making is hindered when computational resources are not readily accessible. Here we report a meta-imager designed to work together with a digital back end to offload computationally expensive convolution operations into high-speed, low-power optics. In this architecture, metasurfaces enable both angle and polarization multiplexing to create multiple information channels that perform positively and negatively valued convolution operations in a single shot. We use our meta-imager for object classification, achieving 98.6% accuracy in handwritten digits and 88.8% accuracy in fashion images. Owing to its compactness, high speed and low power consumption, our approach could find a wide range of applications in artificial intelligence and machine vision applications.

Programmable Photonic Time Circuits for Highly Scalable Universal Unitaries

Programmable photonic circuits (PPCs) have garnered substantial interest for their potential in facilitating deep learning accelerations and universal quantum computations. Although photonic computation using PPCs offers ultrafast operation, energy-efficient matrix calculations, and room-temperature quantum states, its poor scalability hinders integration. This challenge arises from the temporally one-shot operation of propagating light in conventional PPCs, resulting in a light-speed increase in device footprints. Here we propose the concept of programmable photonic time circuits, utilizing time-cycle-based computations analogous to gate cycling in the von Neumann architecture and quantum computation. Our building block is a reconfigurable SU(2) time gate, consisting of two resonators with tunable resonances, and coupled via time-coded dual-channel gauge fields. We demonstrate universal U(N) operations with high fidelity using an assembly of the SU(2) time gates, substantially improving scalability from O(N^{2}) to O(N) in terms of both the footprint and the number of gates. This result paves the way for PPC implementation in very large-scale integration.

Inorganic perovskite-based active multifunctional integrated photonic devices

The development of highly efficient active integrated photonic circuits is crucial for advancing information and computing science. Lead halide perovskite semiconductors, with their exceptional optoelectronic properties, offer a promising platform for such devices. In this study, active micro multifunctional photonic devices were fabricated on monocrystalline CsPbBr3 perovskite thin films using a top-down etching technique with focused ion beams. The etched microwire exhibited a high-quality micro laser that could serve as a light source for integrated devices, facilitating angle-dependent effective propagation between coupled perovskite-microwire waveguides. Employing this strategy, multiple perovskite-based active integrated photonic devices were realized for the first time. These devices included a micro beam splitter that coherently separated lasing signals, an X-coupler performing transfer matrix functions with two distinguishable light sources, and a Mach-Zehnder interferometer manipulating the splitting and coalescence of coherent light beams. These results provide a proof-of-concept for active integrated functionalized photonic devices based on perovskite semiconductors, representing a promising avenue for practical applications in integrated optical chips.

Analog spatiotemporal feature extraction for cognitive radio-frequency sensing with integrated photonics

Analog feature extraction (AFE) is an appealing strategy for low-latency and efficient cognitive sensing systems since key features are much sparser than the Nyquist-sampled data. However, applying AFE to broadband radio-frequency (RF) scenarios is challenging due to the bandwidth and programmability bottlenecks of analog electronic circuitry. Here, we introduce a photonics-based scheme that extracts spatiotemporal features from broadband RF signals in the analog domain. The feature extractor structure inspired by convolutional neural networks is implemented on integrated photonic circuits to process RF signals from multiple antennas, extracting valid features from both temporal and spatial dimensions. Because of the tunability of the photonic devices, the photonic spatiotemporal feature extractor is trainable, which enhances the validity of the extracted features. Moreover, a digital-analog-hybrid transfer learning method is proposed for the effective and low-cost training of the photonic feature extractor. To validate our scheme, we demonstrate a radar target recognition task with a 4-GHz instantaneous bandwidth. Experimental results indicate that the photonic analog feature extractor tackles broadband RF signals and reduces the sampling rate of analog-to-digital converters to 1/4 of the Nyquist sampling while maintaining a high target recognition accuracy of 97.5%. Our scheme offers a promising path for exploiting the AFE strategy in the realm of cognitive RF sensing, with the potential to contribute to the efficient signal processing involved in applications such as autonomous driving, robotics, and smart factories.

Complex networks with complex weights

In many studies, it is common to use binary (i.e., unweighted) edges to examine networks of entities that are either adjacent or not adjacent. Researchers have generalized such binary networks to incorporate edge weights, which allow one to encode node-node interactions with heterogeneous intensities or frequencies (e.g., in transportation networks, supply chains, and social networks). Most such studies have considered real-valued weights, despite the fact that networks with complex weights arise in fields as diverse as quantum information, quantum chemistry, electrodynamics, rheology, and machine learning. Many of the standard network-science approaches in the study of classical systems rely on the real-valued nature of edge weights, so it is necessary to generalize them if one seeks to use them to analyze networks with complex edge weights. In this paper, we examine how standard network-analysis methods fail to capture structural features of networks with complex edge weights. We then generalize several network measures to the complex domain and show that random-walk centralities provide a useful approach to examine node importances in networks with complex weights.

Correlated optical convolutional neural network with "quantum speedup"

Compared with electrical neural networks, optical neural networks (ONNs) have the potentials to break the limit of the bandwidth and reduce the consumption of energy, and therefore draw much attention in recent years. By far, several types of ONNs have been implemented. However, the current ONNs cannot realize the acceleration as powerful as that indicated by the models like quantum neural networks. How to construct and realize an ONN with the quantum speedup is a huge challenge. Here, we propose theoretically and demonstrate experimentally a new type of optical convolutional neural network by introducing the optical correlation. It is called the correlated optical convolutional neural network (COCNN). We show that the COCNN can exhibit "quantum speedup" in the training process. The character is verified from the two aspects. One is the direct illustration of the faster convergence by comparing the loss function curves of the COCNN with that of the traditional convolutional neural network (CNN). Such a result is compatible with the training performance of the recently proposed quantum convolutional neural network (QCNN). The other is the demonstration of the COCNN's capability to perform the QCNN phase recognition circuit, validating the connection between the COCNN and the QCNN. Furthermore, we take the COCNN analog to the 3-qubit QCNN phase recognition circuit as an example and perform an experiment to show the soundness and the feasibility of it. The results perfectly match the theoretical calculations. Our proposal opens up a new avenue for realizing the ONNs with the quantum speedup, which will benefit the information processing in the era of big data.

Neuromorphic Photonics Circuits: Contemporary Review

Neuromorphic photonics is a cutting-edge fusion of neuroscience-inspired computing and photonics technology to overcome the constraints of conventional computing architectures. Its significance lies in the potential to transform information processing by mimicking the parallelism and efficiency of the human brain. Using optics and photonics principles, neuromorphic devices can execute intricate computations swiftly and with impressive energy efficiency. This innovation holds promise for advancing artificial intelligence and machine learning while addressing the limitations of traditional silicon-based computing. Neuromorphic photonics could herald a new era of computing that is more potent and draws inspiration from cognitive processes, leading to advancements in robotics, pattern recognition, and advanced data processing. This paper reviews the recent developments in neuromorphic photonic integrated circuits, applications, and current challenges.

All-analog photoelectronic chip for high-speed vision tasks

Photonic computing enables faster and more energy-efficient processing of vision data1-5. However, experimental superiority of deployable systems remains a challenge because of complicated optical nonlinearities, considerable power consumption of analog-to-digital converters (ADCs) for downstream digital processing and vulnerability to noises and system errors1,6-8. Here we propose an all-analog chip combining electronic and light computing (ACCEL). It has a systemic energy efficiency of 74.8 peta-operations per second per watt and a computing speed of 4.6 peta-operations per second (more than 99% implemented by optics), corresponding to more than three and one order of magnitude higher than state-of-the-art computing processors, respectively. After applying diffractive optical computing as an optical encoder for feature extraction, the light-induced photocurrents are directly used for further calculation in an integrated analog computing chip without the requirement of analog-to-digital converters, leading to a low computing latency of 72 ns for each frame. With joint optimizations of optoelectronic computing and adaptive training, ACCEL achieves competitive classification accuracies of 85.5%, 82.0% and 92.6%, respectively, for Fashion-MNIST, 3-class ImageNet classification and time-lapse video recognition task experimentally, while showing superior system robustness in low-light conditions (0.14 fJ μm-2 each frame). ACCEL can be used across a broad range of applications such as wearable devices, autonomous driving and industrial inspections.

High-frame-rate reconfigurable diffractive neural network based on superpixels

The existing implementations of reconfigurable diffractive neural networks rely on both a liquid-crystal spatial light modulator and a digital micromirror device, which results in complexity in the alignment of the optical system and a constrained computational speed. Here, we propose a superpixel diffractive neural network that leverages solely a digital micromirror device to control the neuron bias and connection. This approach considerably simplifies the optical system and achieves a computational speed of 326 Hz per neural layer. We validate our method through experiments in digit classification, achieving an accuracy of 82.6%, and action recognition, attaining a perfect accuracy of 100%. Our findings demonstrate the effectiveness of the superpixel diffractive neural network in simplifying the optical system and enhancing computational speed, opening up new possibilities for real-time optical information processing applications.

Polymer and Hybrid Optical Devices Manipulated by the Thermo-Optic Effect

The thermo-optic effect is a crucial driving mechanism for optical devices. The application of the thermo-optic effect in integrated photonics has received extensive investigation, with continuous progress in the performance and fabrication processes of thermo-optic devices. Due to the high thermo-optic coefficient, polymers have become an excellent candidate for the preparation of high-performance thermo-optic devices. Firstly, this review briefly introduces the principle of the thermo-optic effect and the materials commonly used. In the third section, a brief introduction to the waveguide structure of thermo-optic devices is provided. In addition, three kinds of thermo-optic devices based on polymers, including an optical switch, a variable optical attenuator, and a temperature sensor, are reviewed. In the fourth section, the typical fabrication processes for waveguide devices based on polymers are introduced. Finally, thermo-optic devices play important roles in various applications. Nevertheless, the large-scale integrated applications of polymer-based thermo-optic devices are still worth investigating. Therefore, we propose a future direction for the development of polymers.

Electrical programmable multilevel nonvolatile photonic random-access memory

Photonic Random-Access Memories (P-RAM) are an essential component for the on-chip non-von Neumann photonic computing by eliminating optoelectronic conversion losses in data links. Emerging Phase-Change Materials (PCMs) have been showed multilevel memory capability, but demonstrations still yield relatively high optical loss and require cumbersome WRITE-ERASE approaches increasing power consumption and system package challenges. Here we demonstrate a multistate electrically programmed low-loss nonvolatile photonic memory based on a broadband transparent phase-change material (Ge2Sb2Se5, GSSe) with ultralow absorption in the amorphous state. A zero-static-power and electrically programmed multi-bit P-RAM is demonstrated on a silicon-on-insulator platform, featuring efficient amplitude modulation up to 0.2 dB/μm and an ultralow insertion loss of total 0.12 dB for a 4-bit memory showing a 100× improved signal to loss ratio compared to other phase-change-materials based photonic memories. We further optimize the positioning of dual microheaters validating performance tradeoffs. Experimentally we demonstrate a half-a-million cyclability test showcasing the robust approach of this material and device. Low-loss photonic retention-of-state adds a key feature for photonic functional and programmable circuits impacting many applications including neural networks, LiDAR, and sensors for example.

Compact lithium niobate microring resonators in the ultrahigh Q/V regime

Lithium niobate (LN) is a promising material for future complex photonic-electronic circuits, with wide applications in such fields as communications, sensing, quantum optics, and computation. LN took a great stride toward compact photonic integrated circuits (PICs) with the development of partially etched LN on insulator (LNOI) waveguides. However, integration density is still limited for future highly compact PICs, owing to the partial etching nature of their waveguides. Here, we demonstrate a fully etched LN PIC platform, which, for the first time to our knowledge, simultaneously achieves ultralow propagation loss and compact circuit size. The tightly confined fully etched LN waveguides with smooth sidewalls allow us to bring the bending radius down to 20 μm (corresponding to 1 THz free spectral range). We have achieved compact high Q microring resonators with Q/V of 8.7 × 10⁴ μm^-3, almost one order of magnitude larger than previous demonstrations. The statistical mean propagation losses of our LN waveguides is 8.5 dB/m (corresponding to a mean Q factor of 4.9 × 10⁶), even with a small bending radius of 40 μm. Our compact and ultralow-loss LN platform shows great potential in future miniaturized multifunctional integration systems. As complementary evidence to show the utility of our platform, we demonstrate soliton microcombs with an ultrahigh repetition rate of 500 GHz in LN.

Inverse design of an ultra-compact dual-band wavelength demultiplexing power splitter with detailed analysis of hyperparameters

Inverse design has been widely studied as an efficient method to reduce footprint and improve performance for integrated silicon photonic (SiP) devices. In this study, we have used inverse design to develop a series of ultra-compact dual-band wavelength demultiplexing power splitters (WDPSs) that can simultaneously perform both wavelength demultiplexing and 1:1 optical power splitting. These WDPSs could facilitate the potential coexistence of dual-band passive optical networks (PONs). The design is performed on a standard silicon-on-insulator (SOI) platform using, what we believe to be, a novel two-step direct binary search (TS-DBS) method and the impact of different hyperparameters related to the physical structure and the optimization algorithm is analyzed in detail. Our inverse-designed WDPS with a minimum feature size of 130 nm achieves a 12.77-times reduction in footprint and a slight increase in performance compared with the forward-designed WDPS. We utilize the optimal combination of hyperparameters to design another WDPS with a minimum feature size reduced to 65 nm, which achieves ultra-low insertion losses of 0.36 dB and 0.37 dB and crosstalk values of -19.91 dB and -17.02 dB at wavelength channels of 1310 nm and 1550 nm, respectively. To the best of our knowledge, the hyperparameters of optimization-based inverse design are systematically discussed for the first time. Our work demonstrates that appropriate setting of hyperparameters greatly improves device performance, throwing light on the manipulation of hyperparameters for future inverse design.

C-DONN: compact diffractive optical neural network with deep learning regression

A new method to improve the integration level of an on-chip diffractive optical neural network (DONN) is proposed based on a standard silicon-on-insulator (SOI) platform. The metaline, which represents a hidden layer in the integrated on-chip DONN, is composed of subwavelength silica slots, providing a large computation capacity. However, the physical propagation process of light in the subwavelength metalinses generally requires an approximate characterization using slot groups and extra length between adjacent layers, which limits further improvements of the integration of on-chip DONN. In this work, a deep mapping regression model (DMRM) is proposed to characterize the process of light propagation in the metalines. This method improves the integration level of on-chip DONN to over 60,000 and elimnates the need for approximate conditions. Based on this theory, a compact-DONN (C-DONN) is exploited and benchmarked on the Iris plants dataset to verify the performance, yielding a testing accuracy of 93.3%. This method provides a potential solution for future large-scale on-chip integration.

Photonic Hopfield neural network for the Ising problem

The Ising problem, a vital combinatorial optimization problem in various fields, is hard to solve by traditional Von Neumann computing architecture on a large scale. Thus, lots of application-specific physical architectures are reported, including quantum-based, electronics-based, and optical-based platforms. A Hopfield neural network combined with a simulated annealing algorithm is considered one of the effective approaches but is still limited by large resource consumption. Here, we propose to accelerate the Hopfield network on a photonic integrated circuit composed of the arrays of Mach-Zehnder interferometer. Our proposed Photonic Hopfield Neural Network (PHNN), utilizing the massively parallel operations and integrated circuit with ultrafast iteration rate, converges to a stable ground state solution with high probability. The average success probabilities for the MaxCut problem with a problem size of 100 and the Spin-glass problem with a problem size of 60 can both reach more than 80%. Moreover, our proposed architecture is inherently robust to the noise induced by the imperfect characteristics of components on chip.

Parallel photonic acceleration processor for matrix-matrix multiplication

We propose and experimentally demonstrate a highly parallel photonic acceleration processor based on a wavelength division multiplexing (WDM) system and a non-coherent Mach-Zehnder interferometer (MZI) array for matrix-matrix multiplication. The dimensional expansion is achieved by WDM devices, which play a crucial role in realizing matrix-matrix multiplication together with the broadband characteristics of an MZI. We implemented a 2 × 2 arbitrary nonnegative valued matrix using a reconfigurable 8 × 8 MZI array structure. Through experimentation, we verified that this structure could achieve 90.5% inference accuracy in a classification task for the Modified National Institute of Standards and Technology (MNIST) handwritten dataset. This provides a new effective solution for large-scale integrated optical computing systems based on convolution acceleration processors.

Microring-based programmable coherent optical neural networks

Coherent programmable integrated photonics circuits have shown great potential as specialized hardware accelerators for deep learning tasks, which usually involve the use of linear matrix multiplication and nonlinear activation components. We design, simulate and train an optical neural network fully based on microring resonators, which shows advantages in terms of device footprint and energy efficiency. We use tunable coupled double ring structures as the interferometer components for the linear multiplication layers and modulated microring resonators as the reconfigurable nonlinear activation components. We then develop optimization algorithms to train the direct tuning parameters such as applied voltages based on the transfer matrix method and using automatic differentiation for all optical components.

Compact optical convolution processing unit based on multimode interference

Convolutional neural networks are an important category of deep learning, currently facing the limitations of electrical frequency and memory access time in massive data processing. Optical computing has been demonstrated to enable significant improvements in terms of processing speeds and energy efficiency. However, most present optical computing schemes are hardly scalable since the number of optical elements typically increases quadratically with the computational matrix size. Here, a compact on-chip optical convolutional processing unit is fabricated on a low-loss silicon nitride platform to demonstrate its capability for large-scale integration. Three 2 × 2 correlated real-valued kernels are made of two multimode interference cells and four phase shifters to perform parallel convolution operations. Although the convolution kernels are interrelated, ten-class classification of handwritten digits from the MNIST database is experimentally demonstrated. The linear scalability of the proposed design with respect to computational size translates into a solid potential for large-scale integration.

In-memory photonic dot-product engine with electrically programmable weight banks

Electronically reprogrammable photonic circuits based on phase-change chalcogenides present an avenue to resolve the von-Neumann bottleneck; however, implementation of such hybrid photonic-electronic processing has not achieved computational success. Here, we achieve this milestone by demonstrating an in-memory photonic-electronic dot-product engine, one that decouples electronic programming of phase-change materials (PCMs) and photonic computation. Specifically, we develop non-volatile electronically reprogrammable PCM memory cells with a record-high 4-bit weight encoding, the lowest energy consumption per unit modulation depth (1.7 nJ/dB) for Erase operation (crystallization), and a high switching contrast (158.5%) using non-resonant silicon-on-insulator waveguide microheater devices. This enables us to perform parallel multiplications for image processing with a superior contrast-to-noise ratio (≥87.36) that leads to an enhanced computing accuracy (standard deviation σ ≤ 0.007). An in-memory hybrid computing system is developed in hardware for convolutional processing for recognizing images from the MNIST database with inferencing accuracies of 86% and 87%.

Why the power of diversity does not always produce better groups and societies

Diversity is supposed to create better groups and societies but sometimes fails. It is explained why the power of diversity may not create better groups in the current diversity prediction theory. Diversity may hurt civic life and introduce distrust. This is because the current diversity prediction theory is based on real numbers that ignore individual abilities. Its diversity prediction theory maximizes performance with infinite population size. Contrary to this, collective intelligence or swarm intelligence is not maximized by infinite population size, but by population size. The extended diversity prediction theory using the complex number allows us to express individual abilities or qualities. The diversity of complex numbers always produces better groups and societies. The wisdom of crowds, collective intelligence, swarm intelligence or nature-inspired intelligence is implemented in the current machine learning or artificial intelligence, called Random Forest. The problem of the current diversity prediction theory is detailed in this paper.

Experimentally realized in situ backpropagation for deep learning in photonic neural networks

Integrated photonic neural networks provide a promising platform for energy-efficient, high-throughput machine learning with extensive scientific and commercial applications. Photonic neural networks efficiently transform optically encoded inputs using Mach-Zehnder interferometer mesh networks interleaved with nonlinearities. We experimentally trained a three-layer, four-port silicon photonic neural network with programmable phase shifters and optical power monitoring to solve classification tasks using "in situ backpropagation," a photonic analog of the most popular method to train conventional neural networks. We measured backpropagated gradients for phase-shifter voltages by interfering forward- and backward-propagating light and simulated in situ backpropagation for 64-port photonic neural networks trained on MNIST image recognition given errors. All experiments performed comparably to digital simulations ([Formula: see text]94% test accuracy), and energy scaling analysis indicated a route to scalable machine learning.

Cyber Attack Detection for Self-Driving Vehicle Networks Using Deep Autoencoder Algorithms

Connected and autonomous vehicles (CAVs) present exciting opportunities for the improvement of both the mobility of people and the efficiency of transportation systems. The small computers in autonomous vehicles (CAVs) are referred to as electronic control units (ECUs) and are often perceived as being a component of a broader cyber-physical system. Subsystems of ECUs are often networked together via a variety of in-vehicle networks (IVNs) so that data may be exchanged, and the vehicle can operate more efficiently. The purpose of this work is to explore the use of machine learning and deep learning methods in defence against cyber threats to autonomous cars. Our primary emphasis is on identifying erroneous information implanted in the data buses of various automobiles. In order to categorise this type of erroneous data, the gradient boosting method is used, providing a productive illustration of machine learning. To examine the performance of the proposed model, two real datasets, namely the Car-Hacking and UNSE-NB15 datasets, were used. Real automated vehicle network datasets were used in the verification process of the proposed security solution. These datasets included spoofing, flooding and replay attacks, as well as benign packets. The categorical data were transformed into numerical form via pre-processing. Machine learning and deep learning algorithms, namely k-nearest neighbour (KNN) and decision trees, long short-term memory (LSTM), and deep autoencoders, were employed to detect CAN attacks. According to the findings of the experiments, using the decision tree and KNN algorithms as machine learning approaches resulted in accuracy levels of 98.80% and 99%, respectively. On the other hand, the use of LSTM and deep autoencoder algorithms as deep learning approaches resulted in accuracy levels of 96% and 99.98%, respectively. The maximum accuracy was achieved when using the decision tree and deep autoencoder algorithms. Statistical analysis methods were used to analyse the results of the classification algorithms, and the determination coefficient measurement for the deep autoencoder was found to reach a value of R² = 95%. The performance of all of the models that were built in this way surpassed that of those already in use, with almost perfect levels of accuracy being achieved. The system developed is able to overcome security issues in IVNs.

Heavy tails and pruning in programmable photonic circuits for universal unitaries

Developing hardware for high-dimensional unitary operators plays a vital role in implementing quantum computations and deep learning accelerations. Programmable photonic circuits are singularly promising candidates for universal unitaries owing to intrinsic unitarity, ultrafast tunability and energy efficiency of photonic platforms. Nonetheless, when the scale of a photonic circuit increases, the effects of noise on the fidelity of quantum operators and deep learning weight matrices become more severe. Here we demonstrate a nontrivial stochastic nature of large-scale programmable photonic circuits-heavy-tailed distributions of rotation operators-that enables the development of high-fidelity universal unitaries through designed pruning of superfluous rotations. The power law and the Pareto principle for the conventional architecture of programmable photonic circuits are revealed with the presence of hub phase shifters, allowing for the application of network pruning to the design of photonic hardware. For the Clements design of programmable photonic circuits, we extract a universal architecture for pruning random unitary matrices and prove that "the bad is sometimes better to be removed" to achieve high fidelity and energy efficiency. This result lowers the hurdle for high fidelity in large-scale quantum computing and photonic deep learning accelerators.

Fully Complex-Valued Dendritic Neuron Model

A single dendritic neuron model (DNM) that owns the nonlinear information processing ability of dendrites has been widely used for classification and prediction. Complex-valued neural networks that consist of a number of multiple/deep-layer McCulloch-Pitts neurons have achieved great successes so far since neural computing was utilized for signal processing. Yet no complex value representations appear in single neuron architectures. In this article, we first extend DNM from a real-value domain to a complex-valued one. Performance of complex-valued DNM (CDNM) is evaluated through a complex XOR problem, a non-minimum phase equalization problem, and a real-world wind prediction task. Also, a comparative analysis on a set of elementary transcendental functions as an activation function is implemented and preparatory experiments are carried out for determining hyperparameters. The experimental results indicate that the proposed CDNM significantly outperforms real-valued DNM, complex-valued multi-layer perceptron, and other complex-valued neuron models.

Revealing the Formation Mechanism and Optimizing the Synthesis Conditions of Layered Double Hydroxides for the Oxygen Evolution Reaction

Layered double hydroxides (LDHs), whose formation is strongly related to OH^- concentration, have attracted significant interest in various fields. However, the effect of the real-time change of OH^- concentration on LDHs' formation has not been fully explored due to the unsuitability of the existing synthesis methods for in situ characterization. Here, the deliberately designed combination of NH₃ gas diffusion and in situ pH measurement provides a solution to the above problem. The obtained results revealed the formation mechanism and also guided us to synthesize a library of LDHs with the desired attributes in water at room temperature without using any additives. After evaluating their oxygen evolution reaction performance, we found that FeNi-LDH with a Fe/Ni ratio of 25/75 exhibits one of the best performances so far reported.

MXene-Nanoflakes-Enabled All-Optical Nonlinear Activation Function for On-Chip Photonic Deep Neural Networks

2D metal carbides and nitrides (MXene) are promising material platforms for on-chip neural networks owing to their nonlinear saturable absorption effect. The localized surface plasmon resonances in metallic MXene nanoflakes may play an important role in enhancing the electromagnetic absorption; however, their contribution is not determined due to the lack of a precise understanding of its localized surface plasmon behavior. Here, a saturable absorber made of MXene thin film and a silicon waveguide with MXene flakes overlayer are developed to perform neuromorphic tasks. The proposed configurations are reconfigurable and can therefore be adjusted for various applications without the need to modify the physical structure of the proposed MXene-based activator configurations via tuning the wavelength of operation. The capability and feasibility of the obtained results of machine-learning applications are confirmed via handwritten digit classification task, with near 99% accuracy. These findings can guide the design of advanced ultrathin saturable absorption materials on a chip for a broad range of applications.

Coherent optical neuron control based on reinforcement learning

Optical neural networks take optical neurons as the cornerstone to achieve complex functions. The coherent optical neuron has become one of the mainstream implementations because it can effectively perform natural and even complex number calculations. However, its state variability and requirement for reliability and effectiveness render traditional control methods no longer applicable. In this Letter, deep reinforcement coherent optical neuron control (DRCON) is proposed, and its effectiveness is experimentally demonstrated. Compared with the standard stochastic gradient descent, the average convergence rate of DRCON is 33% faster, while the effective number of bits increases from less than 2 bits to 5.5 bits. DRCON is a promising first step for large-scale optical neural network control.

Co-designed metaoptoelectronic deep learning

A metaoptical system is co-designed with electronic hardware to implement deep learning image recognition. The optical convolution block includes a reflective metasurface to perform one layer of a deep neural network. The optical and digital components are jointly optimized to perform an image classification task attaining 65% accuracy, which is close to the 66% accuracy of a fully-digital network where the optical block is replaced by a digital convolution layer.

Optically Readable Electrochromic-Based Microfiber Synaptic Device for Photonic Neuromorphic Systems

Optical synaptic devices possess great potential in both artificial intelligence and neuromorphic photonics. In this work, an optically readable electrochromic-based microfiber synaptic device was designed by the combination of a multimode fiber and an electrochromic device and using an external voltage to control the transmission of light in the fiber. The proposed synaptic device has the ability to imitate various basic functions of the biological synapses, such as synaptic plasticity, and paired-pulse facilitation (PPF), as well as the transition from short-term memory to long-term memory. Moreover, the proposed device decodes the output optical signal with the international Morse code to express the signal "HFUT" in two ways, and a 3 × 3 array composed of this device can simulate the perceptual learning process. The device can be easily prepared for a wide range of applications, and the incorporated microfibers can be replaced by planar optical waveguides, making it easy to be integrated into a more complex and versatile photonic neuromorphic system.

Evolving scattering networks for engineering disorder

Network science provides a powerful tool for unraveling the complexities of social, technological and biological systems. Constructing networks using wave phenomena is also of great interest in devising advanced hardware for machine learning, as shown in optical neural networks. Although most wave-based networks have employed static network models, the impact of evolving models in network science provides strong motivation to apply dynamical network modeling to wave physics. Here the concept of evolving scattering networks for scattering phenomena is developed. The network is defined by links, node degrees and their evolution processes modeling multi-particle interferences, which directly determine scattering from disordered materials. I demonstrate the concept by examining network-based material classification, microstructure screening and preferential attachment in evolutions, which are applied to stealthy hyperuniformity. The results enable independent control of scattering from different length scales, revealing superdense material phases in short-range order. The proposed concept provides a bridge between wave physics and network science to resolve multiscale material complexities and open-system material design.

Prospects and applications of on-chip lasers

Integrated silicon photonics has sparked a significant ramp-up of investment in both academia and industry as a scalable, power-efficient, and eco-friendly solution. At the heart of this platform is the light source, which in itself, has been the focus of research and development extensively. This paper sheds light and conveys our perspective on the current state-of-the-art in different aspects of application-driven on-chip silicon lasers. We tackle this from two perspectives: device-level and system-wide points of view. In the former, the different routes taken in integrating on-chip lasers are explored from different material systems to the chosen integration methodologies. Then, the discussion focus is shifted towards system-wide applications that show great prospects in incorporating photonic integrated circuits (PIC) with on-chip lasers and active devices, namely, optical communications and interconnects, optical phased array-based LiDAR, sensors for chemical and biological analysis, integrated quantum technologies, and finally, optical computing. By leveraging the myriad inherent attractive features of integrated silicon photonics, this paper aims to inspire further development in incorporating PICs with on-chip lasers in, but not limited to, these applications for substantial performance gains, green solutions, and mass production.

Mechanical neural networks: Architected materials that learn behaviors

Aside from some living tissues, few materials can autonomously learn to exhibit desired behaviors as a consequence of prolonged exposure to unanticipated ambient loading scenarios. Still fewer materials can continue to exhibit previously learned behaviors in the midst of changing conditions (e.g., rising levels of internal damage, varying fixturing scenarios, and fluctuating external loads) while also acquiring new behaviors best suited for the situation at hand. Here, we describe a class of architected materials, called mechanical neural networks (MNNs), that achieve such learning capabilities by tuning the stiffness of their constituent beams similar to how artificial neural networks (ANNs) tune their weights. An example lattice was fabricated to demonstrate its ability to learn multiple mechanical behaviors simultaneously, and a study was conducted to determine the effect of lattice size, packing configuration, algorithm type, behavior number, and linear-versus-nonlinear stiffness tunability on MNN learning as proposed. Thus, this work lays the foundation for artificial-intelligent (AI) materials that can learn behaviors and properties.

Resource-efficient high-dimensional subspace teleportation with a quantum autoencoder

Quantum autoencoders serve as efficient means for quantum data compression. Here, we propose and demonstrate their use to reduce resource costs for quantum teleportation of subspaces in high-dimensional systems. We use a quantum autoencoder in a compress-teleport-decompress manner and report the first demonstration with qutrits using an integrated photonic platform for future scalability. The key strategy is to compress the dimensionality of input states by erasing redundant information and recover the initial states after chip-to-chip teleportation. Unsupervised machine learning is applied to train the on-chip autoencoder, enabling the compression and teleportation of any state from a high-dimensional subspace. Unknown states are decompressed at a high fidelity (~0.971), obtaining a total teleportation fidelity of ~0.894. Subspace encodings hold great potential as they support enhanced noise robustness and increased coherence. Laying the groundwork for machine learning techniques in quantum systems, our scheme opens previously unidentified paths toward high-dimensional quantum computing and networking.