An optical neural chip for implementing complex-valued neural network

Asymmetrical estimator for training encapsulated deep photonic neural networks

Photonic neural networks (PNNs) are fast in-propagation and high bandwidth paradigms that aim to popularize reproducible NN acceleration with higher efficiency and lower cost. However, the training of PNN is known to be challenging, where the device-to-device and system-to-system variations create imperfect knowledge of the PNN. Despite backpropagation (BP)-based training algorithms being the industry standard for their robustness, generality, and fast gradient convergence for digital training, existing PNN-BP methods rely heavily on accurate intermediate state extraction or extensive computational resources for deep PNNs (DPNNs). The truncated photonic signal propagation and the computation overhead bottleneck DPNN's operation efficiency and increase system construction cost. Here, we introduce the asymmetrical training (AsyT) method, tailored for encapsulated DPNNs, where the signal is preserved in the analogue photonic domain for the entire structure. AsyT offers a lightweight solution for DPNNs with minimum readouts, fast and energy-efficient operation, and minimum system footprint. AsyT's ease of operation, error tolerance, and generality aim to promote PNN acceleration in a widened operational scenario despite the fabrication variations and imperfect controls. We demonstrated AsyT for encapsulated DPNN with integrated photonic chips, repeatably enhancing the performance from in-silico BP for different network structures and datasets.

Optical sorting: past, present and future

Optical sorting combines optical tweezers with diverse techniques, including optical spectrum, artificial intelligence (AI) and immunoassay, to endow unprecedented capabilities in particle sorting. In comparison to other methods such as microfluidics, acoustics and electrophoresis, optical sorting offers appreciable advantages in nanoscale precision, high resolution, non-invasiveness, and is becoming increasingly indispensable in fields of biophysics, chemistry, and materials science. This review aims to offer a comprehensive overview of the history, development, and perspectives of various optical sorting techniques, categorised as passive and active sorting methods. To begin, we elucidate the fundamental physics and attributes of both conventional and exotic optical forces. We then explore sorting capabilities of active optical sorting, which fuses optical tweezers with a diversity of techniques, including Raman spectroscopy and machine learning. Afterwards, we reveal the essential roles played by deterministic light fields, configured with lens systems or metasurfaces, in the passive sorting of particles based on their varying sizes and shapes, sorting resolutions and speeds. We conclude with our vision of the most promising and futuristic directions, including AI-facilitated ultrafast and bio-morphology-selective sorting. It can be envisioned that optical sorting will inevitably become a revolutionary tool in scientific research and practical biomedical applications.

Empowering nanophotonic applications via artificial intelligence: pathways, progress, and prospects

Empowering nanophotonic devices via artificial intelligence (AI) has revolutionized both scientific research methodologies and engineering practices, addressing critical challenges in the design and optimization of complex systems. Traditional methods for developing nanophotonic devices are often constrained by the high dimensionality of design spaces and computational inefficiencies. This review highlights how AI-driven techniques provide transformative solutions by enabling the efficient exploration of vast design spaces, optimizing intricate parameter systems, and predicting the performance of advanced nanophotonic materials and devices with high accuracy. By bridging the gap between computational complexity and practical implementation, AI accelerates the discovery of novel nanophotonic functionalities. Furthermore, we delve into emerging domains, such as diffractive neural networks and quantum machine learning, emphasizing their potential to exploit photonic properties for innovative strategies. The review also examines AI's applications in advanced engineering areas, e.g., optical image recognition, showcasing its role in addressing complex challenges in device integration. By facilitating the development of highly efficient, compact optical devices, these AI-powered methodologies are paving the way for next-generation nanophotonic systems with enhanced functionalities and broader applications.

Optimizing optical neural network design for enhanced compatibility with analog computation

This paper breaks away from traditional approaches that merely emulate digital neural networks. Using Mach-Zehnder interferometer (MZI) networks as a case study, we explore the impact of the inherent properties of analog computation on performance and identify the characteristics that optical neural networks (ONNs) components should possess to better adapt to these specific properties. Specifically, we examine the influence of analog computation on bias power and activation functions, as well as the impact of optical pruning on ONN's performance. The results show that a suitably larger bias power relative to normalized data and concave activation functions are more compatible with the characteristics of ONNs. These factors can significantly improve classification accuracy across different datasets and ξ values, with improvements reaching up to 35%. Additionally, optical pruning reduces the number of MZIs by two-thirds while maintaining performance. Moreover, these measures significantly enhance the robustness of ONNs against MZI losses and phase errors. Although this research primarily focuses on feedforward MZI-based networks, the proposed design principles are widely applicable to other types of ONNs.

Versatile parallel signal processing with a scalable silicon photonic chip

Silicon photonic signal processors promise a new generation of signal processing hardware with significant advancements in processing bandwidth, low power consumption, and minimal latency. Programmable silicon photonic signal processors, facilitated by tuning elements, can reduce hardware development cycles and costs. However, traditional programmable photonic signal processors based on optical switches face scalability and performance challenges due to control complexity and transmission losses. Here, we propose a scalable parallel signal processor on silicon for versatile applications by interleaving wavelength and temporal optical dimensions. Additionally, it incorporates ultra-low-loss waveguides and low-phase-error optical switch techniques, achieving an overall insertion loss of 10 dB. This design offers low loss, high scalability, and simplified control, enabling advanced functionalities such as accurate microwave reception, narrowband microwave photonic filtering, wide-bandwidth arbitrary waveform generation, and high-speed parallel optical computing without the need for tuning elements calibration. Our programmable parallel signal processor demonstrates advantages in both scale and performance, marking a significant advancement in large-scale, high-performance, multifunctional photonic systems.

TOPS-speed complex-valued convolutional accelerator for feature extraction and inference

Complex-valued neural networks process both amplitude and phase information, in contrast to conventional artificial neural networks, achieving additive capabilities in recognizing phase-sensitive data inherent in wave-related phenomena. The ever-increasing data capacity and network scale place substantial demands on underlying computing hardware. In parallel with the successes and extensive efforts made in electronics, optical neuromorphic hardware is promising to achieve ultra-high computing performances due to its inherent analog architecture and wide bandwidth. Here, we report a complex-valued optical convolution accelerator operating at over 2 Tera operations per second (TOPS). With appropriately designed phasors we demonstrate its performance in the recognition of synthetic aperture radar (SAR) images captured by the Sentinel-1 satellite, which are inherently complex-valued and more intricate than what optical neural networks have previously processed. Experimental tests with 500 images yield an 83.8% accuracy, close to in-silico results. This approach facilitates feature extraction of phase-sensitive information, and represents a pivotal advance in artificial intelligence towards real-time, high-dimensional data analysis of complex and dynamic environments.

Spectral convolutional neural network chip for in-sensor edge computing of incoherent natural light

Optical neural networks are considered next-generation physical implementations of artificial neural networks, but their capabilities are limited by on-chip integration scale and requirement for coherent light sources. This study proposes a spectral convolutional neural network (SCNN) with matter meta-imaging. The optical convolutional layer is implemented by integrating very large-scale and pixel-aligned spectral filters on CMOS image sensor. It facilitates highly parallel spectral vector-inner products of incident incoherent natural light i.e., the direct information carrier, which empowers in-sensor optical analog computing at extremely high energy efficiency. To the best of our knowledge, this is the first integrated optical computing utilizing natural light. We employ the same SCNN chip for completely different real-world complex tasks and achieve accuracies of over 96% for pathological diagnosis and almost 100% for face anti-spoofing at video rates. These results indicate a feasible and scalable in-sensor edge computing chip of natural light for various portable terminals.

Multimodal In-Sensor Computing System Using Integrated Silicon Photonic Convolutional Processor

Photonic integrated circuits offer miniaturized solutions for multimodal spectroscopic sensory systems by leveraging the simultaneous interaction of light with temperature, chemicals, and biomolecules, among others. The multimodal spectroscopic sensory data is complex and has huge data volume with high redundancy, thus requiring high communication bandwidth associated with high communication power consumption to transfer the sensory data. To circumvent this high communication cost, the photonic sensor and processor are brought into intimacy and propose a photonic multimodal in-sensor computing system using an integrated silicon photonic convolutional processor. A microring resonator crossbar array is used as the photonic processor to implement convolutional operation with 5-bit accuracy, validated through image edge detection tasks. Further integrating the processor with a photonic spectroscopic sensor, the in situ processing of multimodal spectroscopic sensory data is demonstrated, achieving the classification of protein species of different types and concentrations at various temperatures. A classification accuracy of 97.58% across 45 different classes is achieved. The multimodal in-sensor computing system demonstrates the feasibility of integrating photonic processors and photonic sensors to enhance the data processing capability of photonic devices at the edge.

Cascadable optical nonlinear activation function based on Ge-Si

To augment the capabilities of optical computing, specialized nonlinear devices as optical activation functions are crucial for enhancing the complexity of optical neural networks. However, existing optical nonlinear activation function devices often encounter challenges in preparation, compatibility, and multi-layer cascading. Here, we propose a cascadable optical nonlinear activation function architecture based on Ge-Si structured devices. Leveraging dual-source modulation, this architecture achieves cascading and wavelength switching by compensating for loss. Experimental comparisons with traditional Ge-Si devices validate the cascading capability of the new architecture. We first verified the versatility of this activation function in a MNIST task, and then in a multi-layer optical dense neural network designed for complex gesture recognition classification, the proposed architecture improves accuracy by an average of 23% compared to a linear network and 15% compared to a network with a traditional activation function architecture. With its advantages of cascadability and high compatibility, this work underscores the potential of all-optical activation functions for large-scale optical neural network scaling and complex task handling.

120 GOPS Photonic tensor core in thin-film lithium niobate for inference and in situ training

Photonics offers a transformative approach to artificial intelligence (AI) and neuromorphic computing by enabling low-latency, high-speed, and energy-efficient computations. However, conventional photonic tensor cores face significant challenges in constructing large-scale photonic neuromorphic networks. Here, we propose a fully integrated photonic tensor core, consisting of only two thin-film lithium niobate (TFLN) modulators, a III-V laser, and a charge-integration photoreceiver. Despite its simple architecture, it is capable of implementing an entire layer of a neural network with a computational speed of 120 GOPS, while also allowing flexible adjustment of the number of inputs (fan-in) and outputs (fan-out). Our tensor core supports rapid in-situ training with a weight update speed of 60 GHz. Furthermore, it successfully classifies (supervised learning) and clusters (unsupervised learning) 112  × 112-pixel images through in-situ training. To enable in-situ training for clustering AI tasks, we offer a solution for performing multiplications between two negative numbers.

Comparative Transcriptome Analysis of Bovine, Porcine, and Sheep Muscle Using Interpretable Machine Learning Models

The growth and development of muscle tissue play a pivotal role in the economic value and quality of meat in agricultural animals, garnering close attention from breeders and researchers. The quality and palatability of muscle tissue directly determine the market competitiveness of meat products and the satisfaction of consumers. Therefore, a profound understanding and management of muscle growth is essential for enhancing the overall economic efficiency and product quality of the meat industry. Despite this, systematic research on muscle development-related genes across different species still needs to be improved. This study addresses this gap through extensive cross-species muscle transcriptome analysis, combined with interpretable machine learning models. Utilizing a comprehensive dataset of 275 publicly available transcriptomes derived from porcine, bovine, and ovine muscle tissues, encompassing samples from ten distinct muscle types such as the semimembranosus and longissimus dorsi, this study analyzes 113 porcine (n = 113), 94 bovine (n = 94), and 68 ovine (n = 68) specimens. We employed nine machine learning models, such as Support Vector Classifier (SVC) and Support Vector Machine (SVM). Applying the SHapley Additive exPlanations (SHAP) method, we analyzed the muscle transcriptome data of cattle, pigs, and sheep. The optimal model, adaptive boosting (AdaBoost), identified key genes potentially influencing muscle growth and development across the three species, termed SHAP genes. Among these, 41 genes (including NANOG, ADAMTS8, LHX3, and TLR9) were consistently expressed in all three species, designated as homologous genes. Specific candidate genes for cattle included SLC47A1, IGSF1, IRF4, EIF3F, CGAS, ZSWIM9, RROB1, and ABHD18; for pigs, DRP2 and COL12A1; and for sheep, only COL10A1. Through the analysis of SHAP genes utilizing Kyoto Encyclopedia of Genes and Genomes (KEGG) pathways, relevant pathways such as ether lipid metabolism, cortisol synthesis and secretion, and calcium signaling pathways have been identified, revealing their pivotal roles in muscle growth and development.

Optical neural networks: progress and challenges

Artificial intelligence has prevailed in all trades and professions due to the assistance of big data resources, advanced algorithms, and high-performance electronic hardware. However, conventional computing hardware is inefficient at implementing complex tasks, in large part because the memory and processor in its computing architecture are separated, performing insufficiently in computing speed and energy consumption. In recent years, optical neural networks (ONNs) have made a range of research progress in optical computing due to advantages such as sub-nanosecond latency, low heat dissipation, and high parallelism. ONNs are in prospect to provide support regarding computing speed and energy consumption for the further development of artificial intelligence with a novel computing paradigm. Herein, we first introduce the design method and principle of ONNs based on various optical elements. Then, we successively review the non-integrated ONNs consisting of volume optical components and the integrated ONNs composed of on-chip components. Finally, we summarize and discuss the computational density, nonlinearity, scalability, and practical applications of ONNs, and comment on the challenges and perspectives of the ONNs in the future development trends.

Photonic probabilistic machine learning using quantum vacuum noise

Probabilistic machine learning utilizes controllable sources of randomness to encode uncertainty and enable statistical modeling. Harnessing the pure randomness of quantum vacuum noise, which stems from fluctuating electromagnetic fields, has shown promise for high speed and energy-efficient stochastic photonic elements. Nevertheless, photonic computing hardware which can control these stochastic elements to program probabilistic machine learning algorithms has been limited. Here, we implement a photonic probabilistic computer consisting of a controllable stochastic photonic element - a photonic probabilistic neuron (PPN). Our PPN is implemented in a bistable optical parametric oscillator (OPO) with vacuum-level injected bias fields. We then program a measurement-and-feedback loop for time-multiplexed PPNs with electronic processors (FPGA or GPU) to solve certain probabilistic machine learning tasks. We showcase probabilistic inference and image generation of MNIST-handwritten digits, which are representative examples of discriminative and generative models. In both implementations, quantum vacuum noise is used as a random seed to encode classification uncertainty or probabilistic generation of samples. In addition, we propose a path towards an all-optical probabilistic computing platform, with an estimated sampling rate of  ~1 Gbps and energy consumption of  ~5 fJ/MAC. Our work paves the way for scalable, ultrafast, and energy-efficient probabilistic machine learning hardware.

Tunable optical matrix convolution of 20-Gbit/s QPSK 2-D data with a kernel using optical wave mixing

Compared to its electronic counterpart, optically performed matrix convolution can accommodate phase-encoded data at high rates while avoiding optical-to-electronic-to-optical (OEO) conversions. We experimentally demonstrate a reconfigurable matrix convolution of quadrature phase-shift keying (QPSK)-encoded input data. The two-dimensional (2-D) input data is serialized, and its time-shifted replicas are generated. This 2-D data is convolved with a 1-D kernel with coefficients, which are applied by adjusting the relative phase and amplitude of the kernel pumps. Time-shifted data replicas (TSDRs) and kernel pumps are coherently mixed using nonlinear wave mixing in a periodically poled lithium niobate (PPLN) waveguide. To show the tunability and reconfigurability of this approach, we vary the kernel coefficients, kernel sizes (e.g., 2 × 1 or 3 × 1), and input data rates (e.g., 6-20 Gbit/s). The convolution results are verified to be error-free under an applied: (a) 2 × 1 kernel, resulting in a 16-quadrature amplitude modulation (QAM) output with an error vector magnitude (EVM) of ∼5.1-8.5%; and (b) 3 × 1 kernel, resulting in a 64-QAM output with an EVM of ∼4.9-5.5%.

Thin film ferroelectric photonic-electronic memory

To reduce system complexity and bridge the interface between electronic and photonic circuits, there is a high demand for a non-volatile memory that can be accessed both electrically and optically. However, practical solutions are still lacking when considering the potential for large-scale complementary metal-oxide semiconductor compatible integration. Here, we present an experimental demonstration of a non-volatile photonic-electronic memory based on a 3-dimensional monolithic integrated ferroelectric-silicon ring resonator. We successfully demonstrate programming and erasing the memory using both electrical and optical methods, assisted by optical-to-electrical-to-optical conversion. The memory cell exhibits a high optical extinction ratio of 6.6 dB at a low working voltage of 5 V and an endurance of 4 × 104 cycles. Furthermore, the multi-level storage capability is analyzed in detail, revealing stable performance with a raw bit-error-rate smaller than 5.9 × 10-2. This ground-breaking work could be a key technology enabler for future hybrid electronic-photonic systems, targeting a wide range of applications such as photonic interconnect, high-speed data communication, and neuromorphic computing.

Partial coherence enhances parallelized photonic computing

Advancements in optical coherence control1-5 have unlocked many cutting-edge applications, including long-haul communication, light detection and ranging (LiDAR) and optical coherence tomography6-8. Prevailing wisdom suggests that using more coherent light sources leads to enhanced system performance and device functionalities9-11. Our study introduces a photonic convolutional processing system that takes advantage of partially coherent light to boost computing parallelism without substantially sacrificing accuracy, potentially enabling larger-size photonic tensor cores. The reduction of the degree of coherence optimizes bandwidth use in the photonic convolutional processing system. This breakthrough challenges the traditional belief that coherence is essential or even advantageous in integrated photonic accelerators, thereby enabling the use of light sources with less rigorous feedback control and thermal-management requirements for high-throughput photonic computing. Here we demonstrate such a system in two photonic platforms for computing applications: a photonic tensor core using phase-change-material photonic memories that delivers parallel convolution operations to classify the gaits of ten patients with Parkinson's disease with 92.2% accuracy (92.7% theoretically) and a silicon photonic tensor core with embedded electro-absorption modulators (EAMs) to facilitate 0.108 tera operations per second (TOPS) convolutional processing for classifying the Modified National Institute of Standards and Technology (MNIST) handwritten digits dataset with 92.4% accuracy (95.0% theoretically).

Multimodal deep learning using on-chip diffractive optics with in situ training capability

Multimodal deep learning plays a pivotal role in supporting the processing and learning of diverse data types within the realm of artificial intelligence generated content (AIGC). However, most photonic neuromorphic processors for deep learning can only handle a single data modality (either vision or audio) due to the lack of abundant parameter training in optical domain. Here, we propose and demonstrate a trainable diffractive optical neural network (TDONN) chip based on on-chip diffractive optics with massive tunable elements to address these constraints. The TDONN chip includes one input layer, five hidden layers, and one output layer, and only one forward propagation is required to obtain the inference results without frequent optical-electrical conversion. The customized stochastic gradient descent algorithm and the drop-out mechanism are developed for photonic neurons to realize in situ training and fast convergence in the optical domain. The TDONN chip achieves a potential throughput of 217.6 tera-operations per second (TOPS) with high computing density (447.7 TOPS/mm2), high system-level energy efficiency (7.28 TOPS/W), and low optical latency (30.2 ps). The TDONN chip has successfully implemented four-class classification in different modalities (vision, audio, and touch) and achieve 85.7% accuracy on multimodal test sets. Our work opens up a new avenue for multimodal deep learning with integrated photonic processors, providing a potential solution for low-power AI large models using photonic technology.

I/O-efficient iterative matrix inversion with photonic integrated circuits

Photonic integrated circuits have been extensively explored for optical processing with the aim of breaking the speed and energy efficiency bottlenecks of digital electronics. However, the input/output (IO) bottleneck remains one of the key barriers. Here we report a photonic iterative processor (PIP) for matrix-inversion-intensive applications. The direct reuse of inputted data in the optical domain unlocks the potential to break the IO bottleneck. We demonstrate notable IO advantages with a lossless PIP for real-valued matrix inversion and integral-differential equation solving, as well as a coherent PIP with optical loops integrated on-chip, enabling complex-valued computation and a net inversion time of 1.2 ns. Furthermore, we estimate at least an order of magnitude enhancement in IO efficiency of a PIP over photonic single-pass processors and the state-of-the-art electronic processors for reservoir training tasks and multiple-input and multiple-output (MIMO) precoding tasks, indicating the huge potential of PIP technology in practical applications.

Perfect linear optics using silicon photonics

Recently there has been growing interest in using photonics to perform the linear algebra operations of neuromorphic and quantum computing applications, aiming at harnessing silicon photonics' (SiPho) high-speed and energy-efficiency credentials. Accurately mapping, however, a matrix into optics remains challenging, since state-of-the-art optical architectures are sensitive to fabrication imperfections. This leads to reduced fidelity that degrades as the insertion losses of the optical matrix nodes or the matrix dimensions increase. In this work, we present the experimental deployment of a 4 × 4 coherent crossbar (Xbar) as a silicon chip and validate experimentally its theoretically predicted fidelity restoration credentials. We demonstrate the experimental implementation of 10,000 arbitrary linear transformations achieving a record-high fidelity of 99.997% ± 0.002, limited mainly by the measurement equipment. Our work represents an integrated optical circuit providing almost unity and loss-independent fidelity in the realization of arbitrary matrices, highlighting light's credentials in resolving complex computations.

Interdisciplinary analysis and optimization of digital photonic devices for meta-photonics

With the continuous integration and development of AI and natural sciences, we have been diligently exploring a computational analysis framework for digital photonic devices. Here, We have overcome the challenge of limited datasets through the use of Generative Adversarial Network networks and transfer learning, providing AI feedback that aligns with human knowledge systems. Furthermore, we have introduced knowledge from disciplines such as image denoising, multi-agent modeling of Physarum polycephalum, percolation theory, wave function collapse algorithms, and others to analyze this new design system. It represents an accomplishment unattainable within the framework of classical photonics theory and significantly improves the performance of the designed devices. Notably, we present theoretical analyses for the drastic changes in device performance and the enhancement of device robustness, which have not been reported in previous research. The proposed concept of meta-photonics transcends the conventional boundaries of disciplinary silos, demonstrating the transformative potential of interdisciplinary fusion.

Multiplexable all-optical nonlinear activator for optical computing

As an alternative solution to surpass electronic neural networks, optical neural networks (ONNs) offer significant advantages in terms of energy consumption and computing speed. Despite the optical hardware platform could provide an efficient approach to realizing neural network algorithms than traditional hardware, the lack of optical nonlinearity limits the development of ONNs. Here, we proposed and experimentally demonstrated an all-optical nonlinear activator based on the stimulated Brillouin scattering (SBS). Utilizing the exceptional carrier dynamics of SBS, our activator supports two types of nonlinear functions, saturable absorption and rectified linear unit (Relu) models. Moreover, the proposed activator exhibits large dynamic response bandwidth (∼11.24 GHz), low nonlinear threshold (∼2.29 mW), high stability, and wavelength division multiplexing identities. These features have potential advantages for the physical realization of optical nonlinearities. As a proof of concept, we verify the performance of the proposed activator as an ONN nonlinear mapping unit via numerical simulations. Simulation shows that our approach achieves comparable performance to the activation functions commonly used in computers. The proposed approach provides support for the realization of all-optical neural networks.

Towards large-scale programmable silicon photonic chip for signal processing

Optical signal processing has been playing a crucial part as powerful engine for various information systems in the practical applications. In particular, achieving large-scale programmable chips for signal processing are highly desirable for high flexibility, low cost and powerful processing. Silicon photonics, which has been developed successfully in the past decade, provides a promising option due to its unique advantages. Here, recent progress of large-scale programmable silicon photonic chip for signal processing in microwave photonics, optical communications, optical computing, quantum photonics as well as dispersion controlling are reviewed. Particularly, we give a discussion about the realization of high-performance building-blocks, including ultra-low-loss silicon photonic waveguides, 2 × 2 Mach-Zehnder switches and microring resonator switches. The methods for configuring large-scale programmable silicon photonic chips are also discussed. The representative examples are summarized for the applications of beam steering, optical switching, optical computing, quantum photonic processing as well as optical dispersion controlling. Finally, we give an outlook for the challenges of further developing large-scale programmable silicon photonic chips.

Empowering high-dimensional optical fiber communications with integrated photonic processors

Mode-division multiplexing (MDM) in optical fibers enables multichannel capabilities for various applications, including data transmission, quantum networks, imaging, and sensing. However, high-dimensional optical fiber systems, usually necessity bulk-optics approaches for launching different orthogonal fiber modes into the optical fiber, and multiple-input multiple-output digital electronic signal processing at the receiver to undo the arbitrary mode scrambling introduced by coupling and transmission in a multi-mode fiber. Here we show that a high-dimensional optical fiber communication system can be implemented by a reconfigurable integrated photonic processor, featuring kernels of multichannel mode multiplexing transmitter and all-optical descrambling receiver. Effective mode management can be achieved through the configuration of the integrated optical mesh. Inter-chip MDM optical communications involving six spatial- and polarization modes was realized, despite the presence of unknown mode mixing and polarization rotation in the circular-core optical fiber. The proposed photonic integration approach holds promising prospects for future space-division multiplexing applications.

An optoacoustic field-programmable perceptron for recurrent neural networks

Recurrent neural networks (RNNs) can process contextual information such as time series signals and language. But their tracking of internal states is a limiting factor, motivating research on analog implementations in photonics. While photonic unidirectional feedforward neural networks (NNs) have demonstrated big leaps, bi-directional optical RNNs present a challenge: the need for a short-term memory that (i) programmable and coherently computes optical inputs, (ii) minimizes added noise, and (iii) allows scalability. Here, we experimentally demonstrate an optoacoustic recurrent operator (OREO) which meets (i, ii, iii). OREO contextualizes the information of an optical pulse sequence via acoustic waves. The acoustic waves link different optical pulses, capturing their information and using it to manipulate subsequent operations. OREO's all-optical control on a pulse-by-pulse basis offers simple reconfigurability and is used to implement a recurrent drop-out and pattern recognition of 27 optical pulse patterns. Finally, we introduce OREO as bi-directional perceptron for new classes of optical NNs.

Multichannel meta-imagers for accelerating machine vision

Rapid developments in machine vision technology have impacted a variety of applications, such as medical devices and autonomous driving systems. These achievements, however, typically necessitate digital neural networks with the downside of heavy computational requirements and consequent high energy consumption. As a result, real-time decision-making is hindered when computational resources are not readily accessible. Here we report a meta-imager designed to work together with a digital back end to offload computationally expensive convolution operations into high-speed, low-power optics. In this architecture, metasurfaces enable both angle and polarization multiplexing to create multiple information channels that perform positively and negatively valued convolution operations in a single shot. We use our meta-imager for object classification, achieving 98.6% accuracy in handwritten digits and 88.8% accuracy in fashion images. Owing to its compactness, high speed and low power consumption, our approach could find a wide range of applications in artificial intelligence and machine vision applications.

Programmable Photonic Time Circuits for Highly Scalable Universal Unitaries

Programmable photonic circuits (PPCs) have garnered substantial interest for their potential in facilitating deep learning accelerations and universal quantum computations. Although photonic computation using PPCs offers ultrafast operation, energy-efficient matrix calculations, and room-temperature quantum states, its poor scalability hinders integration. This challenge arises from the temporally one-shot operation of propagating light in conventional PPCs, resulting in a light-speed increase in device footprints. Here we propose the concept of programmable photonic time circuits, utilizing time-cycle-based computations analogous to gate cycling in the von Neumann architecture and quantum computation. Our building block is a reconfigurable SU(2) time gate, consisting of two resonators with tunable resonances, and coupled via time-coded dual-channel gauge fields. We demonstrate universal U(N) operations with high fidelity using an assembly of the SU(2) time gates, substantially improving scalability from O(N^{2}) to O(N) in terms of both the footprint and the number of gates. This result paves the way for PPC implementation in very large-scale integration.

Inorganic perovskite-based active multifunctional integrated photonic devices

The development of highly efficient active integrated photonic circuits is crucial for advancing information and computing science. Lead halide perovskite semiconductors, with their exceptional optoelectronic properties, offer a promising platform for such devices. In this study, active micro multifunctional photonic devices were fabricated on monocrystalline CsPbBr3 perovskite thin films using a top-down etching technique with focused ion beams. The etched microwire exhibited a high-quality micro laser that could serve as a light source for integrated devices, facilitating angle-dependent effective propagation between coupled perovskite-microwire waveguides. Employing this strategy, multiple perovskite-based active integrated photonic devices were realized for the first time. These devices included a micro beam splitter that coherently separated lasing signals, an X-coupler performing transfer matrix functions with two distinguishable light sources, and a Mach-Zehnder interferometer manipulating the splitting and coalescence of coherent light beams. These results provide a proof-of-concept for active integrated functionalized photonic devices based on perovskite semiconductors, representing a promising avenue for practical applications in integrated optical chips.

Analog spatiotemporal feature extraction for cognitive radio-frequency sensing with integrated photonics

Analog feature extraction (AFE) is an appealing strategy for low-latency and efficient cognitive sensing systems since key features are much sparser than the Nyquist-sampled data. However, applying AFE to broadband radio-frequency (RF) scenarios is challenging due to the bandwidth and programmability bottlenecks of analog electronic circuitry. Here, we introduce a photonics-based scheme that extracts spatiotemporal features from broadband RF signals in the analog domain. The feature extractor structure inspired by convolutional neural networks is implemented on integrated photonic circuits to process RF signals from multiple antennas, extracting valid features from both temporal and spatial dimensions. Because of the tunability of the photonic devices, the photonic spatiotemporal feature extractor is trainable, which enhances the validity of the extracted features. Moreover, a digital-analog-hybrid transfer learning method is proposed for the effective and low-cost training of the photonic feature extractor. To validate our scheme, we demonstrate a radar target recognition task with a 4-GHz instantaneous bandwidth. Experimental results indicate that the photonic analog feature extractor tackles broadband RF signals and reduces the sampling rate of analog-to-digital converters to 1/4 of the Nyquist sampling while maintaining a high target recognition accuracy of 97.5%. Our scheme offers a promising path for exploiting the AFE strategy in the realm of cognitive RF sensing, with the potential to contribute to the efficient signal processing involved in applications such as autonomous driving, robotics, and smart factories.

Complex networks with complex weights

In many studies, it is common to use binary (i.e., unweighted) edges to examine networks of entities that are either adjacent or not adjacent. Researchers have generalized such binary networks to incorporate edge weights, which allow one to encode node-node interactions with heterogeneous intensities or frequencies (e.g., in transportation networks, supply chains, and social networks). Most such studies have considered real-valued weights, despite the fact that networks with complex weights arise in fields as diverse as quantum information, quantum chemistry, electrodynamics, rheology, and machine learning. Many of the standard network-science approaches in the study of classical systems rely on the real-valued nature of edge weights, so it is necessary to generalize them if one seeks to use them to analyze networks with complex edge weights. In this paper, we examine how standard network-analysis methods fail to capture structural features of networks with complex edge weights. We then generalize several network measures to the complex domain and show that random-walk centralities provide a useful approach to examine node importances in networks with complex weights.

Correlated optical convolutional neural network with "quantum speedup"

Compared with electrical neural networks, optical neural networks (ONNs) have the potentials to break the limit of the bandwidth and reduce the consumption of energy, and therefore draw much attention in recent years. By far, several types of ONNs have been implemented. However, the current ONNs cannot realize the acceleration as powerful as that indicated by the models like quantum neural networks. How to construct and realize an ONN with the quantum speedup is a huge challenge. Here, we propose theoretically and demonstrate experimentally a new type of optical convolutional neural network by introducing the optical correlation. It is called the correlated optical convolutional neural network (COCNN). We show that the COCNN can exhibit "quantum speedup" in the training process. The character is verified from the two aspects. One is the direct illustration of the faster convergence by comparing the loss function curves of the COCNN with that of the traditional convolutional neural network (CNN). Such a result is compatible with the training performance of the recently proposed quantum convolutional neural network (QCNN). The other is the demonstration of the COCNN's capability to perform the QCNN phase recognition circuit, validating the connection between the COCNN and the QCNN. Furthermore, we take the COCNN analog to the 3-qubit QCNN phase recognition circuit as an example and perform an experiment to show the soundness and the feasibility of it. The results perfectly match the theoretical calculations. Our proposal opens up a new avenue for realizing the ONNs with the quantum speedup, which will benefit the information processing in the era of big data.

Neuromorphic Photonics Circuits: Contemporary Review

Neuromorphic photonics is a cutting-edge fusion of neuroscience-inspired computing and photonics technology to overcome the constraints of conventional computing architectures. Its significance lies in the potential to transform information processing by mimicking the parallelism and efficiency of the human brain. Using optics and photonics principles, neuromorphic devices can execute intricate computations swiftly and with impressive energy efficiency. This innovation holds promise for advancing artificial intelligence and machine learning while addressing the limitations of traditional silicon-based computing. Neuromorphic photonics could herald a new era of computing that is more potent and draws inspiration from cognitive processes, leading to advancements in robotics, pattern recognition, and advanced data processing. This paper reviews the recent developments in neuromorphic photonic integrated circuits, applications, and current challenges.

All-analog photoelectronic chip for high-speed vision tasks

Photonic computing enables faster and more energy-efficient processing of vision data1-5. However, experimental superiority of deployable systems remains a challenge because of complicated optical nonlinearities, considerable power consumption of analog-to-digital converters (ADCs) for downstream digital processing and vulnerability to noises and system errors1,6-8. Here we propose an all-analog chip combining electronic and light computing (ACCEL). It has a systemic energy efficiency of 74.8 peta-operations per second per watt and a computing speed of 4.6 peta-operations per second (more than 99% implemented by optics), corresponding to more than three and one order of magnitude higher than state-of-the-art computing processors, respectively. After applying diffractive optical computing as an optical encoder for feature extraction, the light-induced photocurrents are directly used for further calculation in an integrated analog computing chip without the requirement of analog-to-digital converters, leading to a low computing latency of 72 ns for each frame. With joint optimizations of optoelectronic computing and adaptive training, ACCEL achieves competitive classification accuracies of 85.5%, 82.0% and 92.6%, respectively, for Fashion-MNIST, 3-class ImageNet classification and time-lapse video recognition task experimentally, while showing superior system robustness in low-light conditions (0.14 fJ μm-2 each frame). ACCEL can be used across a broad range of applications such as wearable devices, autonomous driving and industrial inspections.

Human emotion recognition with a microcomb-enabled integrated optical neural network

State-of-the-art deep learning models can converse and interact with humans by understanding their emotions, but the exponential increase in model parameters has triggered an unprecedented demand for fast and low-power computing. Here, we propose a microcomb-enabled integrated optical neural network (MIONN) to perform the intelligent task of human emotion recognition at the speed of light and with low power consumption. Large-scale tensor data can be independently encoded in dozens of frequency channels generated by the on-chip microcomb and computed in parallel when flowing through the microring weight bank. To validate the proposed MIONN, we fabricated proof-of-concept chips and a prototype photonic-electronic artificial intelligence (AI) computing engine with a potential throughput up to 51.2 TOPS (tera-operations per second). We developed automatic feedback control procedures to ensure the stability and 8 bits weighting precision of the MIONN. The MIONN has successfully recognized six basic human emotions, and achieved 78.5 % accuracy on the blind test set. The proposed MIONN provides a high-speed and energy-efficient neuromorphic computing hardware for deep learning models with emotional interaction capabilities.

High-frame-rate reconfigurable diffractive neural network based on superpixels

The existing implementations of reconfigurable diffractive neural networks rely on both a liquid-crystal spatial light modulator and a digital micromirror device, which results in complexity in the alignment of the optical system and a constrained computational speed. Here, we propose a superpixel diffractive neural network that leverages solely a digital micromirror device to control the neuron bias and connection. This approach considerably simplifies the optical system and achieves a computational speed of 326 Hz per neural layer. We validate our method through experiments in digit classification, achieving an accuracy of 82.6%, and action recognition, attaining a perfect accuracy of 100%. Our findings demonstrate the effectiveness of the superpixel diffractive neural network in simplifying the optical system and enhancing computational speed, opening up new possibilities for real-time optical information processing applications.

Polymer and Hybrid Optical Devices Manipulated by the Thermo-Optic Effect

The thermo-optic effect is a crucial driving mechanism for optical devices. The application of the thermo-optic effect in integrated photonics has received extensive investigation, with continuous progress in the performance and fabrication processes of thermo-optic devices. Due to the high thermo-optic coefficient, polymers have become an excellent candidate for the preparation of high-performance thermo-optic devices. Firstly, this review briefly introduces the principle of the thermo-optic effect and the materials commonly used. In the third section, a brief introduction to the waveguide structure of thermo-optic devices is provided. In addition, three kinds of thermo-optic devices based on polymers, including an optical switch, a variable optical attenuator, and a temperature sensor, are reviewed. In the fourth section, the typical fabrication processes for waveguide devices based on polymers are introduced. Finally, thermo-optic devices play important roles in various applications. Nevertheless, the large-scale integrated applications of polymer-based thermo-optic devices are still worth investigating. Therefore, we propose a future direction for the development of polymers.

Electrical programmable multilevel nonvolatile photonic random-access memory

Photonic Random-Access Memories (P-RAM) are an essential component for the on-chip non-von Neumann photonic computing by eliminating optoelectronic conversion losses in data links. Emerging Phase-Change Materials (PCMs) have been showed multilevel memory capability, but demonstrations still yield relatively high optical loss and require cumbersome WRITE-ERASE approaches increasing power consumption and system package challenges. Here we demonstrate a multistate electrically programmed low-loss nonvolatile photonic memory based on a broadband transparent phase-change material (Ge2Sb2Se5, GSSe) with ultralow absorption in the amorphous state. A zero-static-power and electrically programmed multi-bit P-RAM is demonstrated on a silicon-on-insulator platform, featuring efficient amplitude modulation up to 0.2 dB/μm and an ultralow insertion loss of total 0.12 dB for a 4-bit memory showing a 100× improved signal to loss ratio compared to other phase-change-materials based photonic memories. We further optimize the positioning of dual microheaters validating performance tradeoffs. Experimentally we demonstrate a half-a-million cyclability test showcasing the robust approach of this material and device. Low-loss photonic retention-of-state adds a key feature for photonic functional and programmable circuits impacting many applications including neural networks, LiDAR, and sensors for example.

Compact lithium niobate microring resonators in the ultrahigh Q/V regime

Lithium niobate (LN) is a promising material for future complex photonic-electronic circuits, with wide applications in such fields as communications, sensing, quantum optics, and computation. LN took a great stride toward compact photonic integrated circuits (PICs) with the development of partially etched LN on insulator (LNOI) waveguides. However, integration density is still limited for future highly compact PICs, owing to the partial etching nature of their waveguides. Here, we demonstrate a fully etched LN PIC platform, which, for the first time to our knowledge, simultaneously achieves ultralow propagation loss and compact circuit size. The tightly confined fully etched LN waveguides with smooth sidewalls allow us to bring the bending radius down to 20 μm (corresponding to 1 THz free spectral range). We have achieved compact high Q microring resonators with Q/V of 8.7 × 10⁴ μm^-3, almost one order of magnitude larger than previous demonstrations. The statistical mean propagation losses of our LN waveguides is 8.5 dB/m (corresponding to a mean Q factor of 4.9 × 10⁶), even with a small bending radius of 40 μm. Our compact and ultralow-loss LN platform shows great potential in future miniaturized multifunctional integration systems. As complementary evidence to show the utility of our platform, we demonstrate soliton microcombs with an ultrahigh repetition rate of 500 GHz in LN.

Inverse design of an ultra-compact dual-band wavelength demultiplexing power splitter with detailed analysis of hyperparameters

Inverse design has been widely studied as an efficient method to reduce footprint and improve performance for integrated silicon photonic (SiP) devices. In this study, we have used inverse design to develop a series of ultra-compact dual-band wavelength demultiplexing power splitters (WDPSs) that can simultaneously perform both wavelength demultiplexing and 1:1 optical power splitting. These WDPSs could facilitate the potential coexistence of dual-band passive optical networks (PONs). The design is performed on a standard silicon-on-insulator (SOI) platform using, what we believe to be, a novel two-step direct binary search (TS-DBS) method and the impact of different hyperparameters related to the physical structure and the optimization algorithm is analyzed in detail. Our inverse-designed WDPS with a minimum feature size of 130 nm achieves a 12.77-times reduction in footprint and a slight increase in performance compared with the forward-designed WDPS. We utilize the optimal combination of hyperparameters to design another WDPS with a minimum feature size reduced to 65 nm, which achieves ultra-low insertion losses of 0.36 dB and 0.37 dB and crosstalk values of -19.91 dB and -17.02 dB at wavelength channels of 1310 nm and 1550 nm, respectively. To the best of our knowledge, the hyperparameters of optimization-based inverse design are systematically discussed for the first time. Our work demonstrates that appropriate setting of hyperparameters greatly improves device performance, throwing light on the manipulation of hyperparameters for future inverse design.

C-DONN: compact diffractive optical neural network with deep learning regression

A new method to improve the integration level of an on-chip diffractive optical neural network (DONN) is proposed based on a standard silicon-on-insulator (SOI) platform. The metaline, which represents a hidden layer in the integrated on-chip DONN, is composed of subwavelength silica slots, providing a large computation capacity. However, the physical propagation process of light in the subwavelength metalinses generally requires an approximate characterization using slot groups and extra length between adjacent layers, which limits further improvements of the integration of on-chip DONN. In this work, a deep mapping regression model (DMRM) is proposed to characterize the process of light propagation in the metalines. This method improves the integration level of on-chip DONN to over 60,000 and elimnates the need for approximate conditions. Based on this theory, a compact-DONN (C-DONN) is exploited and benchmarked on the Iris plants dataset to verify the performance, yielding a testing accuracy of 93.3%. This method provides a potential solution for future large-scale on-chip integration.

Photonic Hopfield neural network for the Ising problem

The Ising problem, a vital combinatorial optimization problem in various fields, is hard to solve by traditional Von Neumann computing architecture on a large scale. Thus, lots of application-specific physical architectures are reported, including quantum-based, electronics-based, and optical-based platforms. A Hopfield neural network combined with a simulated annealing algorithm is considered one of the effective approaches but is still limited by large resource consumption. Here, we propose to accelerate the Hopfield network on a photonic integrated circuit composed of the arrays of Mach-Zehnder interferometer. Our proposed Photonic Hopfield Neural Network (PHNN), utilizing the massively parallel operations and integrated circuit with ultrafast iteration rate, converges to a stable ground state solution with high probability. The average success probabilities for the MaxCut problem with a problem size of 100 and the Spin-glass problem with a problem size of 60 can both reach more than 80%. Moreover, our proposed architecture is inherently robust to the noise induced by the imperfect characteristics of components on chip.

Parallel photonic acceleration processor for matrix-matrix multiplication

We propose and experimentally demonstrate a highly parallel photonic acceleration processor based on a wavelength division multiplexing (WDM) system and a non-coherent Mach-Zehnder interferometer (MZI) array for matrix-matrix multiplication. The dimensional expansion is achieved by WDM devices, which play a crucial role in realizing matrix-matrix multiplication together with the broadband characteristics of an MZI. We implemented a 2 × 2 arbitrary nonnegative valued matrix using a reconfigurable 8 × 8 MZI array structure. Through experimentation, we verified that this structure could achieve 90.5% inference accuracy in a classification task for the Modified National Institute of Standards and Technology (MNIST) handwritten dataset. This provides a new effective solution for large-scale integrated optical computing systems based on convolution acceleration processors.

Microring-based programmable coherent optical neural networks

Coherent programmable integrated photonics circuits have shown great potential as specialized hardware accelerators for deep learning tasks, which usually involve the use of linear matrix multiplication and nonlinear activation components. We design, simulate and train an optical neural network fully based on microring resonators, which shows advantages in terms of device footprint and energy efficiency. We use tunable coupled double ring structures as the interferometer components for the linear multiplication layers and modulated microring resonators as the reconfigurable nonlinear activation components. We then develop optimization algorithms to train the direct tuning parameters such as applied voltages based on the transfer matrix method and using automatic differentiation for all optical components.

Compact optical convolution processing unit based on multimode interference

Convolutional neural networks are an important category of deep learning, currently facing the limitations of electrical frequency and memory access time in massive data processing. Optical computing has been demonstrated to enable significant improvements in terms of processing speeds and energy efficiency. However, most present optical computing schemes are hardly scalable since the number of optical elements typically increases quadratically with the computational matrix size. Here, a compact on-chip optical convolutional processing unit is fabricated on a low-loss silicon nitride platform to demonstrate its capability for large-scale integration. Three 2 × 2 correlated real-valued kernels are made of two multimode interference cells and four phase shifters to perform parallel convolution operations. Although the convolution kernels are interrelated, ten-class classification of handwritten digits from the MNIST database is experimentally demonstrated. The linear scalability of the proposed design with respect to computational size translates into a solid potential for large-scale integration.

In-memory photonic dot-product engine with electrically programmable weight banks

Electronically reprogrammable photonic circuits based on phase-change chalcogenides present an avenue to resolve the von-Neumann bottleneck; however, implementation of such hybrid photonic-electronic processing has not achieved computational success. Here, we achieve this milestone by demonstrating an in-memory photonic-electronic dot-product engine, one that decouples electronic programming of phase-change materials (PCMs) and photonic computation. Specifically, we develop non-volatile electronically reprogrammable PCM memory cells with a record-high 4-bit weight encoding, the lowest energy consumption per unit modulation depth (1.7 nJ/dB) for Erase operation (crystallization), and a high switching contrast (158.5%) using non-resonant silicon-on-insulator waveguide microheater devices. This enables us to perform parallel multiplications for image processing with a superior contrast-to-noise ratio (≥87.36) that leads to an enhanced computing accuracy (standard deviation σ ≤ 0.007). An in-memory hybrid computing system is developed in hardware for convolutional processing for recognizing images from the MNIST database with inferencing accuracies of 86% and 87%.

Why the power of diversity does not always produce better groups and societies

Diversity is supposed to create better groups and societies but sometimes fails. It is explained why the power of diversity may not create better groups in the current diversity prediction theory. Diversity may hurt civic life and introduce distrust. This is because the current diversity prediction theory is based on real numbers that ignore individual abilities. Its diversity prediction theory maximizes performance with infinite population size. Contrary to this, collective intelligence or swarm intelligence is not maximized by infinite population size, but by population size. The extended diversity prediction theory using the complex number allows us to express individual abilities or qualities. The diversity of complex numbers always produces better groups and societies. The wisdom of crowds, collective intelligence, swarm intelligence or nature-inspired intelligence is implemented in the current machine learning or artificial intelligence, called Random Forest. The problem of the current diversity prediction theory is detailed in this paper.

Experimentally realized in situ backpropagation for deep learning in photonic neural networks

Integrated photonic neural networks provide a promising platform for energy-efficient, high-throughput machine learning with extensive scientific and commercial applications. Photonic neural networks efficiently transform optically encoded inputs using Mach-Zehnder interferometer mesh networks interleaved with nonlinearities. We experimentally trained a three-layer, four-port silicon photonic neural network with programmable phase shifters and optical power monitoring to solve classification tasks using "in situ backpropagation," a photonic analog of the most popular method to train conventional neural networks. We measured backpropagated gradients for phase-shifter voltages by interfering forward- and backward-propagating light and simulated in situ backpropagation for 64-port photonic neural networks trained on MNIST image recognition given errors. All experiments performed comparably to digital simulations ([Formula: see text]94% test accuracy), and energy scaling analysis indicated a route to scalable machine learning.

Cyber Attack Detection for Self-Driving Vehicle Networks Using Deep Autoencoder Algorithms

Connected and autonomous vehicles (CAVs) present exciting opportunities for the improvement of both the mobility of people and the efficiency of transportation systems. The small computers in autonomous vehicles (CAVs) are referred to as electronic control units (ECUs) and are often perceived as being a component of a broader cyber-physical system. Subsystems of ECUs are often networked together via a variety of in-vehicle networks (IVNs) so that data may be exchanged, and the vehicle can operate more efficiently. The purpose of this work is to explore the use of machine learning and deep learning methods in defence against cyber threats to autonomous cars. Our primary emphasis is on identifying erroneous information implanted in the data buses of various automobiles. In order to categorise this type of erroneous data, the gradient boosting method is used, providing a productive illustration of machine learning. To examine the performance of the proposed model, two real datasets, namely the Car-Hacking and UNSE-NB15 datasets, were used. Real automated vehicle network datasets were used in the verification process of the proposed security solution. These datasets included spoofing, flooding and replay attacks, as well as benign packets. The categorical data were transformed into numerical form via pre-processing. Machine learning and deep learning algorithms, namely k-nearest neighbour (KNN) and decision trees, long short-term memory (LSTM), and deep autoencoders, were employed to detect CAN attacks. According to the findings of the experiments, using the decision tree and KNN algorithms as machine learning approaches resulted in accuracy levels of 98.80% and 99%, respectively. On the other hand, the use of LSTM and deep autoencoder algorithms as deep learning approaches resulted in accuracy levels of 96% and 99.98%, respectively. The maximum accuracy was achieved when using the decision tree and deep autoencoder algorithms. Statistical analysis methods were used to analyse the results of the classification algorithms, and the determination coefficient measurement for the deep autoencoder was found to reach a value of R² = 95%. The performance of all of the models that were built in this way surpassed that of those already in use, with almost perfect levels of accuracy being achieved. The system developed is able to overcome security issues in IVNs.

Heavy tails and pruning in programmable photonic circuits for universal unitaries

Developing hardware for high-dimensional unitary operators plays a vital role in implementing quantum computations and deep learning accelerations. Programmable photonic circuits are singularly promising candidates for universal unitaries owing to intrinsic unitarity, ultrafast tunability and energy efficiency of photonic platforms. Nonetheless, when the scale of a photonic circuit increases, the effects of noise on the fidelity of quantum operators and deep learning weight matrices become more severe. Here we demonstrate a nontrivial stochastic nature of large-scale programmable photonic circuits-heavy-tailed distributions of rotation operators-that enables the development of high-fidelity universal unitaries through designed pruning of superfluous rotations. The power law and the Pareto principle for the conventional architecture of programmable photonic circuits are revealed with the presence of hub phase shifters, allowing for the application of network pruning to the design of photonic hardware. For the Clements design of programmable photonic circuits, we extract a universal architecture for pruning random unitary matrices and prove that "the bad is sometimes better to be removed" to achieve high fidelity and energy efficiency. This result lowers the hurdle for high fidelity in large-scale quantum computing and photonic deep learning accelerators.

Fully Complex-Valued Dendritic Neuron Model

A single dendritic neuron model (DNM) that owns the nonlinear information processing ability of dendrites has been widely used for classification and prediction. Complex-valued neural networks that consist of a number of multiple/deep-layer McCulloch-Pitts neurons have achieved great successes so far since neural computing was utilized for signal processing. Yet no complex value representations appear in single neuron architectures. In this article, we first extend DNM from a real-value domain to a complex-valued one. Performance of complex-valued DNM (CDNM) is evaluated through a complex XOR problem, a non-minimum phase equalization problem, and a real-world wind prediction task. Also, a comparative analysis on a set of elementary transcendental functions as an activation function is implemented and preparatory experiments are carried out for determining hyperparameters. The experimental results indicate that the proposed CDNM significantly outperforms real-valued DNM, complex-valued multi-layer perceptron, and other complex-valued neuron models.

Analog nanophotonic computing going practical: silicon photonic deep learning engines for tiled optical matrix multiplication with dynamic precision

Analog photonic computing comprises a promising candidate for accelerating the linear operations of deep neural networks (DNNs), since it provides ultrahigh bandwidth, low footprint and low power consumption computing capabilities. However, the confined photonic hardware size, along with the limited bit precision of high-speed electro-optical components, impose stringent requirements towards surpassing the performance levels of current digital processors. Herein, we propose and experimentally demonstrate a speed-optimized dynamic precision neural network (NN) inference via tiled matrix multiplication (TMM) on a low-radix silicon photonic processor. We introduce a theoretical model that relates the noise figure of a photonic neuron with the bit precision requirements per neural layer. The inference evaluation of an NN trained for the classification of the IRIS dataset is, then, experimentally performed over a silicon coherent photonic neuron that can support optical TMM up to 50 GHz, allowing, simultaneously, for dynamic-precision calculations. Targeting on a high-accuracy and speed-optimized classification performance, we experimentally applied the model-extracted mixed-precision NN inference scheme via the respective alteration of the operational compute rates per neural layer. This dynamic-precision NN inference revealed a 55% decrease in the execution time of the linear operations compared to a fixed-precision scheme, without degrading its accuracy.