The inverse design of structural color using machine learning

Hybrid supervised and reinforcement learning for the design and optimization of nanophotonic structures

From higher computational efficiency to enabling the discovery of novel and complex structures, deep learning has emerged as a powerful framework for the design and optimization of nanophotonic circuits and components. However, both data-driven and exploration-based machine learning strategies have limitations in their effectiveness for nanophotonic inverse design. Supervised machine learning approaches require large quantities of training data to produce high-performance models and have difficulty generalizing beyond training data given the complexity of the design space. Unsupervised and reinforcement learning-based approaches on the other hand can have very lengthy training or optimization times associated with them. Here we demonstrate a hybrid supervised learning and reinforcement learning approach to the inverse design of nanophotonic structures and show this approach can reduce training data dependence, improve the generalizability of model predictions, and significantly shorten exploratory training times. The presented strategy thus addresses several contemporary deep learning-based challenges, while opening the door for new design methodologies that leverage multiple classes of machine learning algorithms to produce more effective and practical solutions for photonic design.

Rational Design of Full-Color Fluorescent C3N Quantum Dots

Carbon-based quantum dots (QDs) exhibit unique photoluminescence due to size-dependent quantum confinement, giving rise to fascinating full-color emission properties. Accurate emission calculations using time-dependent density functional theory are a time-costing and expensive process. Herein, we employed an artificial neural network (ANN) combined with statistical learning to establish the relationship between geometrical/electronic structures of ground states and emission wavelength for C3N QDs. The emission energy of these QDs can be doubly modulated by size and edge effects, which are governed by the number of C4N2 rings and the CH group, respectively. Moreover, these two structural characteristics also determine the phonon vibration mode of C3N QDs to harmonize the emission intensity and lifetime of hot electrons in the electron-hole recombination process, as indicated by nonadiabatic molecular dynamics simulation. These computational results provide a general approach to atomically precise design the full-color fluorescent carbon-based QDs with targeted functions and high performance.

Artificial Intelligence for Surface-Enhanced Raman Spectroscopy

Surface-enhanced Raman spectroscopy (SERS), well acknowledged as a fingerprinting and sensitive analytical technique, has exerted high applicational value in a broad range of fields including biomedicine, environmental protection, food safety among the others. In the endless pursuit of ever-sensitive, robust, and comprehensive sensing and imaging, advancements keep emerging in the whole pipeline of SERS, from the design of SERS substrates and reporter molecules, synthetic route planning, instrument refinement, to data preprocessing and analysis methods. Artificial intelligence (AI), which is created to imitate and eventually exceed human behaviors, has exhibited its power in learning high-level representations and recognizing complicated patterns with exceptional automaticity. Therefore, facing up with the intertwining influential factors and explosive data size, AI has been increasingly leveraged in all the above-mentioned aspects in SERS, presenting elite efficiency in accelerating systematic optimization and deepening understanding about the fundamental physics and spectral data, which far transcends human labors and conventional computations. In this review, the recent progresses in SERS are summarized through the integration of AI, and new insights of the challenges and perspectives are provided in aim to better gear SERS toward the fast track.

Tensor completion algorithm-aided structural color design

In recent years, structural color has developed rapidly due to its distinct advantages, such as low loss, high spatial resolution and environmental friendliness. Various inverse design methods have been extensively investigated to efficiently design optical structures. However, the optimization method for the inverse design of structural color remains a formidable challenge. Traditional optimization approaches, such as genetic algorithms require time-consuming repetitions of structural simulations. Deep learning-assisted design necessitates prior simulations and large amounts of data, making it less efficient for systems with a small number of features. This study proposes a tensor completion algorithm capable of swiftly and accurately predicting missing datasets based on partially obtained datasets to assist in structural color design. Transforming the complex physical problem of structural color design into a spatial structure relationship problem linking geometric parameters and spectral data. The method utilizes tensor multilinear data analysis to effectively capture the complex relationships associated with geometric parameters and spectral data in higher-order data. Numerical and experimental results demonstrate that the algorithm exhibits high reliability in terms of speed and accuracy for diverse structures, datasets of varying sizes, and different materials, significantly enhancing design efficiency. The proposed algorithm offers a viable solution for inverse design problems involving complex physical systems, thereby introducing a novel approach to the design of photonic devices. Additionally, numerical experiments illustrate that the structural color of cruciform resonators with diamond can overcome the high loss issues observed in traditional dielectric materials within the blue wavelength region and enhance the corrosion resistance of the structure. We achieve a wide color gamut and a high-narrow reflection spectrum nearing 1 by this structure, and the theoretical analysis further verifies that diamond holds great promise in the realm of optics.

Recent developments in Chalcogenide phase change material-based nanophotonics

There is now a deep interest in actively reconfigurable nanophotonics as they will enable the next generation of optical devices. Of the various alternatives being explored for reconfigurable nanophotonics, Chalcogenide phase change materials (PCMs) are considered highly promising owing to the nonvolatile nature of their phase change. Chalcogenide PCM nanophotonics can be broadly classified into integrated photonics (with guided wave light propagation) and Meta-optics (with free space light propagation). Despite some early comprehensive reviews, the pace of development in the last few years has shown the need for a topical review. Our comprehensive review covers recent progress on nanophotonic architectures, tuning mechanisms, and functionalities in tunable PCM Chalcogenides. In terms of integrated photonics, we identify novel PCM nanoantenna geometries, novel material utilization, the use of nanostructured waveguides, and sophisticated excitation pulsing schemes. On the meta-optics front, the breadth of functionalities has expanded, enabled by exploring design aspects for better performance. The review identifies immediate, and intermediate-term challenges and opportunities in (1) the development of novel chalcogenide PCM, (2) advance in tuning mechanism, and (3) formal inverse design methods, including machine learning augmented inverse design, and provides perspectives on these aspects. The topical review will interest researchers in further advancing this rapidly growing subfield of nanophotonics.

Precise chirp control with model-based reinforcement learning for broadband frequency-swept laser of LiDAR

Artificial intelligence (AI) has been widely used in various fields of physics and engineering in recent decades. In this work, we introduce model-based reinforcement learning (MBRL), which is an important branch of machine learning in the AI domain, to the broadband frequency-swept laser control for frequency modulated continuous wave (FMCW) light detection and ranging (LiDAR). With the concern of the direct interaction between the optical system and the MBRL agent, we establish the frequency measurement system model on the basis of the experimental data and the nonlinearity property of the system. In light of the difficulty of this challenging high-dimensional control task, we propose a twin critic network on the basis of the Actor-Critic structure to better learn the complex dynamic characteristics of the frequency-swept process. Furthermore, the proposed MBRL structure would stabilize the optimization process greatly. In the training process of the neural network, we apply a delaying strategy to the policy update and introduce a smoothing regularization strategy to the target policy to further enhance the network stability. With the well-trained control policy, the agent generates the excellent and regularly updated modulation signals to control the laser chirp precisely and an excellent detection resolution is obtained eventually. Our proposed work demonstrates that the integration of data-driven reinforcement learning (RL) and optical system control gives an opportunity to reduce the system complexity and accelerate the investigation and optimization of control systems.

Nanostructured surface plasmon resonance sensors: Toward narrow linewidths

Surface plasmon resonance sensors have found wide applications in optical sensing field due to their excellent sensitivity to the slight refractive index change of surrounding medium. However, the intrinsically high optical losses in metals make it nontrivial to obtain narrow resonance spectra, which greatly limits the performance of surface plasmon resonance sensors. This review first introduces the influence factors of plasmon linewidths of metallic nanostructures. Then, various approaches to achieve narrow resonance linewidths are summarized, including the fabrication of nanostructured surface plasmon resonance sensors supporting surface lattice resonance/plasmonic Fano resonance or coupling with a photonic cavity, the preparation of surface plasmon resonance sensors with ultra-narrow resonators, as well as strategies such as platform-induced modification, alternating different dielectric layers, and the coupling with whispering-gallery-modes. Lastly, the applications and some existing challenges of surface plasmon resonance sensors are discussed. This review aims to provide guidance for the further development of nanostructured surface plasmon resonance sensors.

Sb2S3-Based Dynamically Tuned Color Filter Array via Genetic Algorithm

Color displays have become increasingly attractive, with dielectric optical nanoantennas demonstrating especially promising applications due to the high refractive index of the material, enabling devices to support geometry-dependent Mie resonance in the visible band. Although many structural color designs based on dielectric nanoantennas employ the method of artificial positive adjustment, the design cycle is too lengthy and the approach is non-intelligent. The commonly used phase change material Ge₂Sb₂Te₅ (GST) is characterized by high absorption and a small contrast to the real part of the refractive index in the visible light band, thereby restricting its application in this range. The Sb₂S₃ phase change material is endowed with a wide band gap of 1.7 to 2 eV, demonstrating two orders of magnitude lower propagation loss compared to GST, when integrated onto a silicon waveguide, and exhibiting a maximum refractive index contrast close to 1 at 614 nm. Thus, Sb₂S₃ is a more suitable phase change material than GST for tuning visible light. In this paper, genetic algorithms and finite-difference time-domain (FDTD) solutions are combined and introduced as Sb₂S₃ phase change material to design nanoantennas. Structural color is generated in the reflection mode through the Mie resonance inside the structure, and the properties of Sb₂S₃ in different phase states are utilized to achieve tunability. Compared to traditional methods, genetic algorithms are superior-optimization algorithms that require low computational effort and a high population performance. Furthermore, Sb₂S₃ material can be laser-induced to switch the transitions of the crystallized and amorphous states, achieving reversible color. The large chromatic aberration ∆E modulation of 64.8, 28.1, and 44.1 was, respectively, achieved by the Sb₂S₃ phase transition in this paper. Moreover, based on the sensitivity of the structure to the incident angle, it can also be used in fields such as angle-sensitive detectors.

Quantitative and Real-Time Evaluation of Human Respiration Signals with a Shape-Conformal Wireless Sensing System

Respiration signals reflect many underlying health conditions, including cardiopulmonary functions, autonomic disorders and respiratory distress, therefore continuous measurement of respiration is needed in various cases. Unfortunately, there is still a lack of effective portable electronic devices that meet the demands for medical and daily respiration monitoring. This work showcases a soft, wireless, and non-invasive device for quantitative and real-time evaluation of human respiration. This device simultaneously captures respiration and temperature signatures using customized capacitive and resistive sensors, encapsulated by a breathable layer, and does not limit the user's daily life. Further a machine learning-based respiration classification algorithm with a set of carefully studied features as inputs is proposed and it is deployed into mobile clients. The body status of users, such as being quiet, active and coughing, can be accurately recognized by the algorithm and displayed on clients. Moreover, multiple devices can be linked to a server network to monitor a group of users and provide each user with the statistical duration of physiological activities, coughing alerts, and body health advice. With these devices, individual and group respiratory health status can be quantitatively collected, analyzed, and stored for daily physiological signal detections as well as medical assistance.

Tunable structural colors on display

Structural coloration takes inspiration from the bright hues found in nature to control the reflection and transmission of light from artificially structured materials. Combining them with active electrical tuning heralds breakthrough applications in optical displays.

NEUTRON: Neural particle swarm optimization for material-aware inverse design of structural color

Designing optical structures for generating structural colors is challenging because of the complex relationship between the optical structures and the color perceived by human eyes. Machine learning-based approaches have been developed to expedite this design process. However, existing methods solely focus on structural parameters of the optical design, which could lead to suboptimal color generation because of the inability to optimize the selection of materials. To address this issue, an approach known as Neural Particle Swarm Optimization is proposed in this paper. The proposed method achieves high design accuracy and efficiency on two structural color design tasks; the first task is designing environment-friendly alternatives to chrome coatings, and the second task concerns reconstructing pictures with multilayer optical thin films. Several designs that could replace chrome coatings have been discovered; pictures with more than 200,000 pixels and thousands of unique colors can be accurately reconstructed in a few hours.

•

NEUTRON combines machine learning and optimization for structural color designs

•

On two benchmark tasks, NEUTRON demonstrates high design accuracy and efficiency

Artificial intelligence to bring nanomedicine to life

The technology of drug delivery systems (DDSs) has demonstrated an outstanding performance and effectiveness in production of pharmaceuticals, as it is proved by many FDA-approved nanomedicines that have an enhanced selectivity, manageable drug release kinetics and synergistic therapeutic actions. Nonetheless, to date, the rational design and high-throughput development of nanomaterial-based DDSs for specific purposes is far from a routine practice and is still in its infancy, mainly due to the limitations in scientists' capabilities to effectively acquire, analyze, manage, and comprehend complex and ever-growing sets of experimental data, which is vital to develop DDSs with a set of desired functionalities. At the same time, this task is feasible for the data-driven approaches, high throughput experimentation techniques, process automatization, artificial intelligence (AI) technology, and machine learning (ML) approaches, which is referred to as The Fourth Paradigm of scientific research. Therefore, an integration of these approaches with nanomedicine and nanotechnology can potentially accelerate the rational design and high-throughput development of highly efficient nanoformulated drugs and smart materials with pre-defined functionalities. In this Review, we survey the important results and milestones achieved to date in the application of data science, high throughput, as well as automatization approaches, combined with AI and ML to design and optimize DDSs and related nanomaterials. This manuscript mission is not only to reflect the state-of-art in data-driven nanomedicine, but also show how recent findings in the related fields can transform the nanomedicine's image. We discuss how all these results can be used to boost nanomedicine translation to the clinic, as well as highlight the future directions for the development, data-driven, high throughput experimentation-, and AI-assisted design, as well as the production of nanoformulated drugs and smart materials with pre-defined properties and behavior. This Review will be of high interest to the chemists involved in materials science, nanotechnology, and DDSs development for biomedical applications, although the general nature of the presented approaches enables knowledge translation to many other fields of science.

Instantaneous Property Prediction and Inverse Design of Plasmonic Nanostructures Using Machine Learning: Current Applications and Future Directions

Advances in plasmonic materials and devices have given rise to a variety of applications in photocatalysis, microscopy, nanophotonics, and metastructures. With the advent of computing power and artificial neural networks, the characterization and design process of plasmonic nanostructures can be significantly accelerated using machine learning as opposed to conventional FDTD simulations. The machine learning (ML) based methods can not only perform with high accuracy and return optical spectra and optimal design parameters, but also maintain a stable high computing efficiency without being affected by the structural complexity. This work reviews the prominent ML methods involved in forward simulation and inverse design of plasmonic nanomaterials, such as Convolutional Neural Networks, Generative Adversarial Networks, Genetic Algorithms and Encoder–Decoder Networks. Moreover, we acknowledge the current limitations of ML methods in the context of plasmonics and provide perspectives on future research directions.

Prediction and Inverse Design of Structural Colors of Nanoparticle Systems via Deep Neural Network

Noniridescent and nonfading structural colors generated from metallic and dielectric nanoparticles with extraordinary optical properties hold great promise in applications such as image display, color printing, and information security. Yet, due to the strong wavelength dependence of optical constants and the radiation pattern, it is difficult and time-consuming to design nanoparticles with the desired hue, saturation, and brightness. Herein, we combined the Monte Carlo and Mie scattering simulations and a bidirectional neural network (BNN) to improve the design of gold nanoparticles' structural colors. The optical simulations provided a dataset including color properties and geometric parameters of gold nanoparticle systems, while the BNN was proposed to accurately predict the structural colors of gold nanoparticle systems and inversely design the geometric parameters for the desired colors. Taking the human chromatic discrimination ability as a criterion, our proposed approach achieved a high accuracy of 99.83% on the predicted colors and 98.5% on the designed geometric parameters. This work provides a general method to accurately and efficiently design the structural colors of nanoparticle systems, which can be exploited in a variety of applications and contribute to the development of advanced optical materials.

Asymmetric angular dependence for multicolor display based on plasmonic inclined-nanopillar array

Asymmetric multicolor displays have unique and fascinating applications in the field of artificial color engineering. However, the realization of such multicolor displays still faces challenges, due to limitations associated with nanofabrication techniques. In this work, asymmetric photonic structures were realized through inclined 2D aluminum nanopillar arrays, which demonstrate asymmetric angle-dependence as multicolor displays. It was numerically and experimentally demonstrated that the distinctive symmetry breaking leads to the plasmonic coupling effect with angle-dependence and reflection differences with the opposite observing angle. Based on this concept, several color printings were designed as prototypes, which prove the utility of the controlled asymmetric color display with varied observing angles. Our results demonstrate a simple and efficient platform for asymmetric plasmonic nanostructures, which paves the way for further study and designation in artificial color engineering.

All-Dielectric Silicon Nanoring Metasurface for Full-Color Printing

Structural color has been particularly attractive as it provides a highly promising approach for next-generation color printing. Plasmonic nanostructures have been intensively investigated for color printing, while suffering from intrinsic loss that degrades the quality of the coloration. Dielectric materials have emerged as an alternative because of their high refractive index that enables highly confined optical modes within the nanostructure at the diffraction limit. Here, we demonstrate an all-dielectric nanoring metasurface for coloration. By harnessing the intrinsic nanoring structure design, vivid structural color has been achieved in the visible range. The color gamut is expected to occupy 115% of the standard color space (sRGB) on the chromaticity diagram of the International Commission on Illumination (CIE) 1931 in theory. Our structure can be applied to various complex devices and materials and find potential applications such as displays, information, and art works.

Design of high-performance plasmonic nanosensors by particle swarm optimization algorithm combined with machine learning

Metallic plasmonic nanosensors that are ultra-sensitive, label-free, and operate in real time hold great promise in the field of chemical and biological research. Conventionally, the design of these nanostructures has strongly relied on time-consuming electromagnetic simulations that iteratively solve Maxwell's equations to scan multi-dimensional parameter space until the desired sensing performance is attained. Here, we propose an algorithm based on particle swarm optimization (PSO), which in combination with a machine learning (ML) model, is used to design plasmonic sensors. The ML model is trained with the geometric structure and sensing performance of the plasmonic sensor to accurately capture the geometry-sensing performance relationships, and the well-trained ML model is then applied to the PSO algorithm to obtain the plasmonic structure with the desired sensing performance. Using the trained ML model to predict the sensing performance instead of using complex electromagnetic calculation methods allows the PSO algorithm to optimize the solutions fours orders of magnitude faster. Implementation of this composite algorithm enabled us to quickly and accurately realize a nanoridge plasmonic sensor with sensitivity as high as 142,500 nm/RIU. We expect this efficient and accurate approach to pave the way for the design of nanophotonic devices in future.

Machine learning and evolutionary algorithm studies of graphene metamaterials for optimized plasmon-induced transparency

Machine learning and optimization algorithms have been widely applied in the design and optimization for photonics devices. We briefly review recent progress of this field of research and show data-driven applications, including spectrum prediction, inverse design and performance optimization, for novel graphene metamaterials (GMs). The structure of the GMs is well-designed to achieve the wideband plasmon induced transparency (PIT) effect, which can be theoretically demonstrated by using the transfer matrix method. Some traditional machine learning algorithms, including k nearest neighbour, decision tree, random forest and artificial neural networks, are utilized to equivalently substitute the numerical simulation in the forward spectrum prediction and complete the inverse design for the GMs. The calculated results demonstrate that all algorithms are effective and the random forest has advantages in terms of accuracy and training speed. Moreover, evolutionary algorithms, including single-objective (genetic algorithm) and multi-objective optimization (NSGA-II), are used to achieve the steep transmission characteristics of PIT effect by synthetically taking many different performance metrics into consideration. The maximum difference between the transmission peaks and dips in the optimized transmission spectrum reaches 0.97. In comparison to previous works, we provide a guidance for intelligent design of photonics devices based on machine learning and evolutionary algorithms and a reference for the selection of machine learning algorithms for simple inverse design problems.

Deep learning: a new tool for photonic nanostructure design

Early results have shown the potential of Deep Learning (DL) to disrupt the fields of optical inverse-design, particularly, the inverse design of nanostructures. In the last three years, the complexity of the optical nanostructure being designed and the sophistication of the employed DL methodology have steadily increased. This topical review comprehensively surveys DL based design examples from the nanophotonics literature. Notwithstanding the early success of this approach, its limitations, range of validity and its place among established design techniques remain to be assessed. The review also provides a perspective on the limitations of this approach and emerging research directions. It is hoped that this topical review may help readers to identify unaddressed problems, to choose an initial setup for a specific problem, and, to identify means to improve the performance of existing DL based workflows.