Biometrics-protected optical communication enabled by deep learning-enhanced triboelectric/photonic synergistic interface

Development of Photonic In-Sensor Computing Based on a Mid-Infrared Silicon Waveguide Platform

Neuromorphic in-sensor computing has provided an energy-efficient solution to smart sensor design and on-chip data processing. In recent years, various free-space-configured optoelectronic chips have been demonstrated for on-chip neuromorphic vision processing. However, on-chip waveguide-based in-sensor computing with different data modalities is still lacking. Here, by integrating a responsivity-tunable graphene photodetector onto the silicon waveguide, an on-chip waveguide-based in-sensor processing unit is realized in the mid-infrared wavelength range. The weighting operation is achieved by dynamically tuning the bias of the photodetector, which could reach 4 bit weighting precision. Three different neural network tasks are performed to demonstrate the capabilities of our device. First, image preprocessing is performed for handwritten digits and fashion product classification as a general task. Next, resistive-type glove sensor signals are reversed and applied to the photodetector as an input for gesture recognition. Finally, spectroscopic data processing for binary gas mixture classification is demonstrated by utilizing the broadband performance of the device from 3.65 to 3.8 μm. By extending the wavelength from near-infrared to mid-infrared, our work shows the capability of a waveguide-integrated tunable graphene photodetector as a viable weighting solution for photonic in-sensor computing. Furthermore, such a solution could be used for large-scale neuromorphic in-sensor computing in photonic integrated circuits at the edge.

Ultra-robust informational metasurfaces based on spatial coherence structures engineering

Optical information transmission is vital in modern optics and photonics due to its concurrent and multi-dimensional nature, leading to tremendous applications such as optical microscopy, holography, and optical sensing. Conventional optical information transmission technologies suffer from bulky optical setup and information loss/crosstalk when meeting scatterers or obstacles in the light path. Here, we theoretically propose and experimentally realize the simultaneous manipulation of the coherence lengths and coherence structures of the light beams with the disordered metasurfaces. The ultra-robust optical information transmission and self-reconstruction can be realized by the generated partially coherent beam with modulated coherence structure even 93% of light is recklessly obstructed during light transmission, which brings new light to robust optical information transmission with a single metasurface. Our method provides a generic principle for the generalized coherence manipulation on the photonic platform and displays a variety of functionalities advancing capabilities in optical information transmission such as meta-holography and imaging in disordered and perturbative media.

Dynamic Optical Encryption Fueled via Tunable Mechanical Composite Micrograting Systems

Optical grating devices based on micro/nanostructured functional surfaces are widely employed to precisely manipulate light propagation, which is significant for information technologies, optical data storage, and light sensors. However, the parameters of rigid periodic structures are difficult to tune after manufacturing, which seriously limits their capacity for in situ light manipulation. Here, a novel anti-eavesdropping, anti-damage, and anti-tamper dynamic optical encryption strategy are reported via tunable mechanical composite wrinkle micrograting encryption systems (MCWGES). By mechanically composing multiple in-situ tunable ordered wrinkle gratings, the dynamic keys with large space capacity are generated to obtain encrypted diffraction patterns, which can provide a higher level of security for the encrypted systems. Furthermore, a multiple grating cone diffraction model is proposed to reveal the dynamic optical encryption principle of MCWGES. Optical encryption communication using dynamic keys has the effect of preventing eavesdropping, damage, and tampering. This dynamic encryption method based on optical manipulation of wrinkle grating demonstrates the potential applications of micro/nanostructured functional surfaces in the field of information security.

Machine learning-assisted novel recyclable flexible triboelectric nanogenerators for intelligent motion

In the smart era, big data analysis based on sensor units is important in intelligent motion. In this study, a dance sports and injury monitoring system (DIMS) based on a recyclable flexible triboelectric nanogenerator (RF-TENG) sensor module, a data processing hardware module, and an upper computer intelligent analysis module are developed to promote intelligent motion. The resultant RF-TENG exhibits an ultra-fast response time of 17 ms, coupled with robust stability demonstrated over 4200 operational cycles, with 6% variation in output voltage. The DIMS enables immersive training by providing visual feedback on sports status and interacting with virtual games. Combined with machine learning (K-nearest neighbor), good classification results are achieved for ground-jumping techniques. In addition, it shows some potential in sports injury prediction (i.e., ankle sprains, knee hyperextension). Overall, the sensing system designed in this study has broad prospects for future applications in intelligent motion and healthcare.

Passive Internet of Events Enabled by Broadly Compatible Self-Powered Visualized Platform Toward Real-Time Surveillance

Surveillance is an intricate challenge worldwide especially in those complicated environments such as nuclear plants, banks, crowded areas, barns, etc. Deploying self-powered wireless sensor nodes can increase the system's event detection capabilities by collecting environmental changes, while the incompatibility among components (energy harvesters, sensors, and wireless modules) limits their application. Here, a broadly compatible self-powered visualized platform (SPVP) is reported to construct a passive internet of events (IoE) network for surveillance systems. By encoding electric signals into reference and working LEDs, SPVP can visualize resistance change generated by commercial resistive sensors with a broad working range (<107  Ω) and the transmission distance is up to 30 meters. Visible light signals are captured by surveillance cameras and processed by the cloud to achieve real-time event monitoring and identification, which forms the passive IoE network. It is demonstrated that the passive-IoE-based surveillance system can detect intrusion, theft, fire alarm, and distress signals quickly (30 ms) for 106 cycles. Moreover, the confidential information can be encrypted by SPVPs and accessed through a phone application. This universal scheme may have huge potential for the construction of safe and smart cities.

Recent development of piezoelectric biosensors for physiological signal detection and machine learning assisted cardiovascular disease diagnosis

As cardiovascular disease stands as a global primary cause of mortality, there has been an urgent need for continuous and real-time heart monitoring to effectively identify irregular heart rhythms and to offer timely patient alerts. However, conventional cardiac monitoring systems encounter challenges due to inflexible interfaces and discomfort during prolonged monitoring. In this review article, we address these issues by emphasizing the recent development of the flexible, wearable, and comfortable piezoelectric passive sensor assisted by machine learning technology for diagnosis. This innovative device not only harmonizes with the dynamic mechanical properties of human skin but also facilitates continuous and real-time collection of physiological signals. Addressing identified challenges and constraints, this review provides insights into recent advances in piezoelectric cardiac sensors, from devices to circuit systems. Furthermore, this review delves into the integration of machine learning technologies, showcasing their pivotal role in facilitating continuous and real-time assessment of cardiac status. The synergistic combination of flexible piezoelectric sensor design and machine learning holds substantial potential in automating the detection of cardiac irregularities with minimal human intervention. This transformative approach has the power to revolutionize patient care paradigms.

Research Progress in Surface-Enhanced Infrared Absorption Spectroscopy: From Performance Optimization, Sensing Applications, to System Integration

Infrared absorption spectroscopy is an effective tool for the detection and identification of molecules. However, its application is limited by the low infrared absorption cross-section of the molecule, resulting in low sensitivity and a poor signal-to-noise ratio. Surface-Enhanced Infrared Absorption (SEIRA) spectroscopy is a breakthrough technique that exploits the field-enhancing properties of periodic nanostructures to amplify the vibrational signals of trace molecules. The fascinating properties of SEIRA technology have aroused great interest, driving diverse sensing applications. In this review, we first discuss three ways for SEIRA performance optimization, including material selection, sensitivity enhancement, and bandwidth improvement. Subsequently, we discuss the potential applications of SEIRA technology in fields such as biomedicine and environmental monitoring. In recent years, we have ushered in a new era characterized by the Internet of Things, sensor networks, and wearable devices. These new demands spurred the pursuit of miniaturized and consolidated infrared spectroscopy systems and chips. In addition, the rise of machine learning has injected new vitality into SEIRA, bringing smart device design and data analysis to the foreground. The final section of this review explores the anticipated trajectory that SEIRA technology might take, highlighting future trends and possibilities.

A Wearable Multidimensional Motion Sensor for AI-Enhanced VR Sports

Regular exercise paves the way to a healthy life. However, conventional sports events are susceptible to weather conditions. Current motion sensors for home-based sports are mainly limited by operation power consumption, single-direction sensitivity, or inferior data analysis. Herein, by leveraging the 3-dimensional printing technique and triboelectric effect, a wearable self-powered multidimensional motion sensor has been developed to detect both the vertical and planar movement trajectory. By integrating with a belt, this sensor could be used to identify some low degree of freedom motions, e.g., waist or gait motion, with a high accuracy of 93.8%. Furthermore, when wearing the sensor at the ankle position, signals generated from shank motions that contain more abundant information could also be effectively collected. By means of a deep learning algorithm, the kicking direction and force could be precisely differentiated with an accuracy of 97.5%. Toward practical application, a virtual reality-enabled fitness game and a shooting game were successfully demonstrated. This work is believed to open up new insights for the development of future household sports or rehabilitation.

Advances in Humidity Nanosensors and Their Application: Review

As the technology revolution and industrialization have flourished in the last few decades, the development of humidity nanosensors has become more important for the detection and control of humidity in the industry production line, food preservation, chemistry, agriculture and environmental monitoring. The new nanostructured materials and fabrication in nanosensors are linked to better sensor performance, especially for superior humidity sensing, following the intensive research into the design and synthesis of nanomaterials in the last few years. Various nanomaterials, such as ceramics, polymers, semiconductor and sulfide, carbon-based, triboelectrical nanogenerator (TENG), and MXene, have been studied for their potential ability to sense humidity with structures of nanowires, nanotubes, nanopores, and monolayers. These nanosensors have been synthesized via a wide range of processes, including solution synthesis, anodization, physical vapor deposition (PVD), or chemical vapor deposition (CVD). The sensing mechanism, process improvement and nanostructure modulation of different types of materials are mostly inexhaustible, but they are all inseparable from the goals of the effective response, high sensitivity and low response-recovery time of humidity sensors. In this review, we focus on the sensing mechanism of direct and indirect sensing, various fabrication methods, nanomaterial geometry and recent advances in humidity nanosensors. Various types of capacitive, resistive and optical humidity nanosensors are introduced, alongside illustration of the properties and nanostructures of various materials. The similarities and differences of the humidity-sensitive mechanisms of different types of materials are summarized. Applications such as IoT, and the environmental and human-body monitoring of nanosensors are the development trends for futures advancements.

Machine learning-augmented surface-enhanced spectroscopy toward next-generation molecular diagnostics

The world today is witnessing the significant role and huge demand for molecular detection and screening in healthcare and medical diagnosis, especially during the outbreak of COVID-19. Surface-enhanced spectroscopy techniques, including Surface-Enhanced Raman Scattering (SERS) and Infrared Absorption (SEIRA), provide lattice and molecular vibrational fingerprint information which is directly linked to the molecular constituents, chemical bonds, and configuration. These properties make them an unambiguous, nondestructive, and label-free toolkit for molecular diagnostics and screening. However, new issues in molecular diagnostics, such as increasing molecular species, faster spread of viruses, and higher requirements for detection accuracy and sensitivity, have brought great challenges to detection technology. Advancements in artificial intelligence and machine learning (ML) techniques show promising potential in empowering SERS and SEIRA with rapid analysis and automatic data processing to jointly tackle the challenge. This review introduces the combination of ML and SERS/SEIRA by investigating how ML algorithms can be beneficial to SERS/SEIRA, discussing the general process of combining ML and SEIRA/SERS, highlighting the molecular diagnostics and screening applications based on ML-combined SEIRA/SERS, and providing perspectives on the future development of ML-integrated SEIRA/SERS. In general, this review offers comprehensive knowledge about the recent advances and the future outlook regarding ML-integrated SEIRA/SERS for molecular diagnostics and screening.

Midinfrared Spectroscopic Analysis of Aqueous Mixtures Using Artificial-Intelligence-Enhanced Metamaterial Waveguide Sensing Platform

As miniaturized solutions, mid-infrared (MIR) waveguide sensors are promising for label-free compositional detection of mixtures leveraging plentiful absorption fingerprints. However, the quantitative analysis of liquid mixtures is still challenging using MIR waveguide sensors, as the absorption spectrum overlaps for multiple organic components accompanied by strong water absorption background. Here, we present an artificial-intelligence-enhanced metamaterial waveguide sensing platform (AIMWSP) for aqueous mixture analysis in the MIR. With the sensitivity-improved metamaterial waveguide and assistance of machine learning, the MIR absorption spectra of a ternary mixture in water can be successfully distinguished and decomposed to single-component spectra for predicting concentration. A classification accuracy of 98.88% for 64 mixing ratios and 92.86% for four concentrations below the limit of detection (972 ppm, based on 3σ) with steps of 300 ppm are realized. Besides, the mixture concentration prediction with root-mean-squared error varying from 0.107 vol % to 1.436 vol % is also achieved. Our work indicates the potential of further extending this sensing platform to MIR spectrometer-on-chip aiming for the data analytics of multiple organic components in aqueous environments.

Soft Modular Glove with Multimodal Sensing and Augmented Haptic Feedback Enabled by Materials' Multifunctionalities

Immersive communications rely on smart perception based on diversified and augmented sensing and feedback technologies. However, the increasing of functional components also raises the issue of increased system complexity. Here, we propose a modular soft glove with multimodal sensing and feedback functions by exploring and utilizing the multiple properties of glove materials. With a single design of basic structure, the main functional unit possesses triboelectric-based sensing of static and dynamic contact, vibration, strain, and pneumatic actuation. Additionally, the same unit is also capable of offering pneumatic tactile haptic feedback and electroresistive thermal haptic feedback. Together with a machine learning algorithm, the proposed glove not only performs real-time detection of dexterous hand motion and direct feedback but also realizes intelligent object recognition and augmented feedback, which significantly enhance the communication and perception of more comprehensive information. In general, this glove utilizes a facile designed sensing and feedback device to achieve dual-way and multimodal communication among humans, machines, and the virtual world via smart perceptions.

Silicon Photonic Phase Shifters and Their Applications: A Review

With the development of silicon photonics, dense photonic integrated circuits play a significant role in applications such as light detection and ranging systems, photonic computing accelerators, miniaturized spectrometers, and so on. Recently, extensive research work has been carried out on the phase shifter, which acts as the fundamental building block in the photonic integrated circuit. In this review, we overview different types of silicon photonic phase shifters, including micro-electro-mechanical systems (MEMS), thermo-optics, and free-carrier depletion types, highlighting the MEMS-based ones. The major working principles of these phase shifters are introduced and analyzed. Additionally, the related works are summarized and compared. Moreover, some emerging applications utilizing phase shifters are introduced, such as neuromorphic computing systems, photonic accelerators, multi-purpose processing cores, etc. Finally, a discussion on each kind of phase shifter is given based on the figures of merit.

Deep Learning Enabled Neck Motion Detection Using a Triboelectric Nanogenerator

The state of neck motion reflects cervical health. To detect the motion state of the human neck is of important significance to healthcare intelligence. A practical neck motion detector should be wearable, flexible, power efficient, and low cost. Here, we report such a neck motion detector comprising a self-powered triboelectric sensor group and a deep learning block. Four flexible and stretchable silicon rubber based triboelectric sensors are integrated on a neck collar. With different neck motions, these four sensors lead-out voltage signals with different amplitudes and/or directions. Thus, the combination of these four signals can represent one motion state. Significantly, a carbon-doped silicon rubber layer is attached between the neck collar and the sensors to shield the external electric field (i.e., electrical changes at the skin surface) for a far more robust identification. Furthermore, a deep learning model based on the convolutional neural network is designed to recognize 11 classes of neck motion including eight directions of bending, two directions of twisting, and one resting state with an average recognition accuracy of 92.63%. This developed neck motion detector has promising applications in neck monitoring, rehabilitation, and control.

Deep Learning-Assisted Triboelectric Smart Mats for Personnel Comprehensive Monitoring toward Maritime Safety

Monitoring the crew of a ship can be performed by combining sensors and artificial intelligence methods to process sensing data. In this study, we developed a deep learning (DL)-assisted minimalist structure triboelectric smart mat system for obtaining abundant crew information without the privacy concerns of taking video. The smart mat system is fabricated using a conductive sponge with different filling rates and a fluorinated ethylene propylene membrane. The proposed dual-channel measurement method improves the stability of the generated signal. Comprehensive crew and cargo monitoring, including personnel and status identification, as well as positioning and counting functions are realized by the DL-assisted triboelectric smart mat system according to the analysis of instant sensory data. Real-time monitoring of crews through fixed and mobile devices improves the ability and efficiency of handling emergencies. The smart mat system provides privacy concerns and an effective way to build ship Internet of Things and ensure personnel safety.

Applying Deep Learning-Based Human Motion Recognition System in Sports Competition

The exploration here intends to compensate for the traditional human motion recognition (HMR) systems' poor performance on large-scale datasets and micromotions. To this end, improvement is designed for the HMR in sports competition based on the deep learning (DL) algorithm. First, the background and research status of HMR are introduced. Then, a new HMR algorithm is proposed based on kernel extreme learning machine (KELM) multidimensional feature fusion (MFF). Afterward, a simulation experiment is designed to evaluate the performance of the proposed KELM-MFF-based HMR algorithm. The results showed that the recognition rate of the proposed KELM-MFF-based HMR is higher than other algorithms. The recognition rate at 10 video frame sampling points is ranked from high to low: the proposed KELM-MFF-based HMR, support vector machine (SVM)-MFF-based HMR, convolutional neural network (CNN) + optical flow (CNN-T)-based HMR, improved dense trajectory (IDT)-based HMR, converse3D (C3D)-based HMR, and CNN-based HMR. Meanwhile, the feature recognition rate of the proposed KELM-MFF-based HMR for the color dimension is higher than the time dimension, by up to 24%. Besides, the proposed KELM-MFF-based HMR algorithm's recognition rate is 92.4% under early feature fusion and 92.1% under late feature fusion, higher than 91.8 and 90.5% of the SVM-MFF-based HMR. Finally, the proposed KELM-MFF-based HMR algorithm takes 30 and 15 s for training and testing. Therefore, the algorithm designed here can be used to deal with large-scale datasets and capture and recognize micromotions. The research content provides a reference for applying extreme learning machine algorithms in sports competitions.

Progress of Advanced Devices and Internet of Things Systems as Enabling Technologies for Smart Homes and Health Care

In the Internet of Things (IoT) era, various devices (e.g., sensors, actuators, energy harvesters, etc.) and systems have been developed toward the realization of smart homes/buildings and personal health care. These advanced devices can be categorized into ambient devices and wearable devices based on their usage scenarios, to enable motion tracking, health monitoring, daily care, home automation, fall detection, intelligent interaction, assistance, living convenience, and security in smart homes. With the rapidly increasing number of such advanced devices and IoT systems, achieving fully self-sustained and multimodal intelligent systems is becoming more and more important to realize a sustainable and all-in-one smart home platform. Hence, in this Review, we systematically present the recent progress of the development of advanced materials, fabrication techniques, devices, and systems for enabling smart home and health care applications. First, advanced polymer, fiber, and fabric materials as well as their respective fabrication techniques for large-scale manufacturing are discussed. After that, functional devices classified into ambient devices (at home ambiance such as door, floor, table, chair, bed, toilet, window, wall, etc.) and wearable devices (on body parts such as finger, wrist, arm, throat, face, back, etc.) are presented for diverse monitoring and auxiliary applications. Next, the current developments of self-sustained systems and intelligent systems are reviewed in detail, indicating two promising research directions in this field. Last, conclusions and outlook pinpointed on the existing challenges and opportunities are provided for the research community to consider.