1
|
Chen Y, Zhang H, Ma J, Cui TJ, Del Hougne P, Li L. Semantic-Electromagnetic Inversion With Pretrained Multimodal Generative Model. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2024:e2406793. [PMID: 39246254 DOI: 10.1002/advs.202406793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/19/2024] [Revised: 07/28/2024] [Indexed: 09/10/2024]
Abstract
Across diverse domains of science and technology, electromagnetic (EM) inversion problems benefit from the ability to account for multimodal prior information to regularize their inherent ill-posedness. Indeed, besides priors that are formulated mathematically or learned from quantitative data, valuable prior information may be available in the form of text or images. Besides handling semantic multimodality, it is furthermore important to minimize the cost of adapting to a new physical measurement operator and to limit the requirements for costly labeled data. Here, these challenges are tackled with a frugal and multimodal semantic-EM inversion technique. The key ingredient is a multimodal generator of reconstruction results that can be pretrained, being agnostic to the physical measurement operator. The generator is fed by a multimodal foundation model encoding the multimodal semantic prior and a physical adapter encoding the measured data. For a new physical setting, only the lightweight physical adapter is retrained. The authors' architecture also enables a flexible iterative step-by-step solution to the inverse problem where each step can be semantically controlled. The feasibility and benefits of this methodology are demonstrated for three EM inverse problems: a canonical two-dimensional inverse-scattering problem in numerics, as well as three-dimensional and four-dimensional compressive microwave meta-imaging experiments.
Collapse
Affiliation(s)
- Yanjin Chen
- State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, 100871, China
| | - Hongrui Zhang
- State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, 100871, China
| | - Jie Ma
- State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, 100871, China
| | - Tie Jun Cui
- State Key Laboratory of Millimeter Waves, Southeast University, Nanjing, 210096, China
- Pazhou Laboratory (Huangpu), Guangzhou, 510555, China
| | | | - Lianlin Li
- State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, 100871, China
- Pazhou Laboratory (Huangpu), Guangzhou, 510555, China
| |
Collapse
|
2
|
Zhang H, Chen Y, Wang Z, Cui TJ, Del Hougne P, Li L. Semantic regularization of electromagnetic inverse problems. Nat Commun 2024; 15:3869. [PMID: 38719933 PMCID: PMC11079068 DOI: 10.1038/s41467-024-48115-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 04/19/2024] [Indexed: 05/12/2024] Open
Abstract
Solving ill-posed inverse problems typically requires regularization based on prior knowledge. To date, only prior knowledge that is formulated mathematically (e.g., sparsity of the unknown) or implicitly learned from quantitative data can be used for regularization. Thereby, semantically formulated prior knowledge derived from human reasoning and recognition is excluded. Here, we introduce and demonstrate the concept of semantic regularization based on a pre-trained large language model to overcome this vexing limitation. We study the approach, first, numerically in a prototypical 2D inverse scattering problem, and, second, experimentally in 3D and 4D compressive microwave imaging problems based on programmable metasurfaces. We highlight that semantic regularization enables new forms of highly-sought privacy protection for applications like smart homes, touchless human-machine interaction and security screening: selected subjects in the scene can be concealed, or their actions and postures can be altered in the reconstruction by manipulating the semantic prior with suitable language-based control commands.
Collapse
Affiliation(s)
- Hongrui Zhang
- State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, 100871, China
| | - Yanjin Chen
- State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, 100871, China
| | - Zhuo Wang
- State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, 100871, China
| | - Tie Jun Cui
- State Key Laboratory of Millimeter Waves, Southeast University, Nanjing, 210096, China.
- Pazhou Laboratory (Huangpu), Guangzhou, Guangdong, 510555, China.
| | | | - Lianlin Li
- State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, 100871, China.
- Pazhou Laboratory (Huangpu), Guangzhou, Guangdong, 510555, China.
| |
Collapse
|
3
|
Zhang H, Ruan H, Zhao H, Wang Z, Hu S, Cui TJ, del Hougne P, Li L. Microwave Speech Recognizer Empowered by a Programmable Metasurface. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2024; 11:e2309826. [PMID: 38380552 PMCID: PMC11077686 DOI: 10.1002/advs.202309826] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Revised: 01/28/2024] [Indexed: 02/22/2024]
Abstract
Speech recognition becomes increasingly important in the modern society, especially for human-machine interactions, but its deployment is still severely thwarted by the struggle of machines to recognize voiced commands in challenging real-life settings: oftentimes, ambient noise drowns the acoustic sound signals, and walls, face masks or other obstacles hide the mouth motion from optical sensors. To address these formidable challenges, an experimental prototype of a microwave speech recognizer empowered by programmable metasurface is presented here that can remotely recognize human voice commands and speaker identities even in noisy environments and if the speaker's mouth is hidden behind a wall or face mask. The programmable metasurface is the pivotal hardware ingredient of the system because its large aperture and huge number of degrees of freedom allows the system to perform a complex sequence of sensing tasks, orchestrated by artificial-intelligence tools. Relying solely on microwave data, the system avoids visual privacy infringements. The developed microwave speech recognizer can enable privacy-respecting voice-commanded human-machine interactions is experimentally demonstrated in many important but to-date inaccessible application scenarios. The presented strategy will unlock new possibilities and have expectations for future smart homes, ambient-assisted health monitoring, as well as intelligent surveillance and security.
Collapse
Affiliation(s)
- Hongrui Zhang
- State Key Laboratory of Advanced Optical Communication Systems and NetworksSchool of ElectronicsPeking UniversityBeijing100871China
| | - Hengxin Ruan
- State Key Laboratory of Advanced Optical Communication Systems and NetworksSchool of ElectronicsPeking UniversityBeijing100871China
- Peng Cheng LaboratoryShenzhenGuangdong518000China
| | - Hanting Zhao
- State Key Laboratory of Advanced Optical Communication Systems and NetworksSchool of ElectronicsPeking UniversityBeijing100871China
| | - Zhuo Wang
- State Key Laboratory of Advanced Optical Communication Systems and NetworksSchool of ElectronicsPeking UniversityBeijing100871China
| | - Shengguo Hu
- State Key Laboratory of Advanced Optical Communication Systems and NetworksSchool of ElectronicsPeking UniversityBeijing100871China
| | - Tie Jun Cui
- State Key Laboratory of Millimeter WavesSoutheast UniversityNanjing210096China
- Pazhou Laboratory (Huangpu)GuangzhouGuangdong510555China
| | | | - Lianlin Li
- State Key Laboratory of Advanced Optical Communication Systems and NetworksSchool of ElectronicsPeking UniversityBeijing100871China
- Pazhou Laboratory (Huangpu)GuangzhouGuangdong510555China
| |
Collapse
|
4
|
Zhang Y, Wang X, Wen J, Zhu X. WiFi-based non-contact human presence detection technology. Sci Rep 2024; 14:3605. [PMID: 38351067 PMCID: PMC10864388 DOI: 10.1038/s41598-024-54077-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Accepted: 02/08/2024] [Indexed: 02/16/2024] Open
Abstract
In the swiftly evolving landscape of Internet of Things (IoT) technology, the demand for adaptive non-contact sensing has seen a considerable surge. Traditional human perception technologies, such as vision-based approaches, often grapple with problems including lack of sensor versatility and sub-optimal accuracy. To address these issues, this paper introduces a novel, non-contact method for human presence perception, relying on WiFi. This innovative approach involves a sequential process, beginning with the pre-processing of collected Channel State Information (CSI), followed by feature extraction, and finally, classification. By establishing signal models that correspond to varying states, this method enables the accurate perception and recognition of human presence. Remarkably, this technique exhibits a high level of precision, with sensing accuracy reaching up to 99[Formula: see text]. The potential applications of this approach are extensive, proving to be particularly beneficial in contexts such as smart homes and healthcare, amongst various other everyday scenarios. This underscores the significant role this novel method could play in enhancing the sophistication and effectiveness of human presence detection and recognition systems in the IoT era.
Collapse
Affiliation(s)
- Yang Zhang
- School of Economics and Management, Shanghai Polytechnic University, Shanghai, 201209, China.
| | - Xuechun Wang
- School of Electrical and Electronic Engineering, Hubei University of Technology, Wuhan, 430068, China
| | - Jinghao Wen
- School of Computer Science, Central China Normal University, Wuhan, 430079, China
| | - Xianxun Zhu
- School of Communication and Information Engineering, Shanghai University, Shanghai, 200444, China
| |
Collapse
|
5
|
Zhao H, Hu S, Zhang H, Wang Z, Dong H, del Hougne P, Cui TJ, Li L. Intelligent indoor metasurface robotics. Natl Sci Rev 2023; 10:nwac266. [PMID: 37396141 PMCID: PMC10309179 DOI: 10.1093/nsr/nwac266] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 11/09/2022] [Accepted: 11/15/2022] [Indexed: 07/29/2023] Open
Abstract
Intelligent indoor robotics is expected to rapidly gain importance in crucial areas of our modern society such as at-home health care and factories. Yet, existing mobile robots are limited in their ability to perceive and respond to dynamically evolving complex indoor environments because of their inherently limited sensing and computing resources that are, moreover, traded off against their cruise time and payload. To address these formidable challenges, here we propose intelligent indoor metasurface robotics (I2MR), where all sensing and computing are relegated to a centralized robotic brain endowed with microwave perception; and I2MR's limbs (motorized vehicles, airborne drones, etc.) merely execute the wirelessly received instructions from the brain. The key aspect of our concept is the centralized use of a computation-enabled programmable metasurface that can flexibly mold microwave propagation in the indoor wireless environment, including a sensing and localization modality based on configurational diversity and a communication modality to establish a preferential high-capacity wireless link between the I2MR's brain and limbs. The metasurface-enhanced microwave perception is capable of realizing low-latency and high-resolution three-dimensional imaging of humans, even around corners and behind thick concrete walls, which is the basis for action decisions of the I2MR's brain. I2MR is thus endowed with real-time and full-context awareness of its operating indoor environment. We implement, experimentally, a proof-of-principle demonstration at ∼2.4 GHz, in which I2MR provides health-care assistance to a human inhabitant. The presented strategy opens a new avenue for the conception of smart and wirelessly networked indoor robotics.
Collapse
Affiliation(s)
| | | | | | - Zhuo Wang
- State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing 100871, China
| | - Hao Dong
- Center on Frontiers of Computing Studies, School of Computer Science, Peking University, Beijing 100871, China
| | | | | | | |
Collapse
|
6
|
SluŸters A, Lambot S, Vanderdonckt J, Vatavu RD. RadarSense: Accurate Recognition of Mid-Air Hand Gestures with Radar Sensing and Few Training Examples. ACM T INTERACT INTEL 2023. [DOI: 10.1145/3589645] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/03/2023]
Abstract
Microwave radars bring many benefits to mid-air gesture sensing due to their large field of view and independence from environmental conditions, such as ambient light and occlusion. However, radar signals are highly dimensional and usually require complex deep learning approaches. To understand this landscape, we report results from a systematic literature review of (
N
= 118) scientific papers on radar sensing, unveiling a large variety of radar technology of different operating frequencies and bandwidths, antenna configurations, but also various gesture recognition techniques. Although highly accurate, these techniques require a large amount of training data that depend on the type of radar. Therefore, the training results cannot be easily transferred to other radars. To address this aspect, we introduce a new gesture recognition pipeline that implements advanced full-wave electromagnetic modeling and inversion to retrieve physical characteristics of gestures that are radar independent,
i.e.
, independent of the source, antennas, and radar-hand interactions. Inversion of radar signals further reduces the size of the dataset by several orders of magnitude, while preserving the essential information. This approach is compatible with conventional gesture recognizers, such as those based on template matching, which only need a few training examples to deliver high recognition accuracy rates. To evaluate our gesture recognition pipeline, we conducted user-dependent and user-independent evaluations on a dataset of 16 gesture types collected with the Walabot, a low-cost off-the-shelf array radar. We contrast these results with those obtained for the same gesture types collected with an ultra-wideband radar made of a vector network analyzer with a single horn antenna and with a computer vision sensor, respectively. Based on our findings, we suggest some design implications to support future development in radar-based gesture recognition.
Collapse
|
7
|
Zhu R, Liu D, Shen L, Zhuang Y, Bi G, Cai T. Huygens' metasurface-based surface plasmon coupler with near-unit efficiency. OPTICS LETTERS 2022; 47:5708-5711. [PMID: 37219309 DOI: 10.1364/ol.468696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Accepted: 10/06/2022] [Indexed: 05/24/2023]
Abstract
Surface plasmon polaritons (SPPs) and their counterparts at low frequency (i.e., spoof SPPs) have been attracting a lot of attention recently due to their potential application for routing information with high speeds and bandwidth. To further develop integrated plasmonics, a high-efficiency surface plasmon coupler is required for full elimination of the intrinsic scattering and reflection when exciting the highly confined plasmonic modes, but a solution to this challenge has remained elusive so far. To take on this challenge, here we propose a feasible spoof SPP coupler based on a transparent Huygens' metasurface, which is able to realize more than 90% efficiency in near- and far-field experiments. To be specific, electrical and magnetic resonators are designed separately on both sides of the metasurface to satisfy the impedance-matching condition everywhere, leading to full conversion of plane wave propagation into surface wave propagation. Moreover, a well-optimized plasmonic metal which is able to support an eigen SPP is designed. This proposed high-efficiency spoof SPP coupler based on a Huygens' metasurface may pave the way for the development of high-performance plasmonic devices.
Collapse
|
8
|
Venkatesh S, Sturm D, Lu X, Lang RJ, Sengupta K. Origami Microwave Imaging Array: Metasurface Tiles on a Shape-Morphing Surface for Reconfigurable Computational Imaging. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2022; 9:e2105016. [PMID: 35896946 PMCID: PMC9534976 DOI: 10.1002/advs.202105016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/04/2021] [Revised: 06/09/2022] [Indexed: 06/15/2023]
Abstract
Origami is the art of paper folding that allows a single flat piece of paper to assume different 3D shapes depending on the fold patterns and the sequence of folding. Using the principles of origami along with computation imaging technique the authors demonstrate a versatile shape-morphing microwave imaging array with reconfigurable field-of-view and scene-adaptive imaging capability. Microwave/millimeter-wave based array imaging systems are expected to be the workhorse for sensory perception of future autonomous intelligent systems. The imaging capability of a planar array-based systems operating in complex scattering conditions have limited field-of-view and lack the ability to adaptively reconfigure resolution. To overcome this, here, deviations from planarity and isometry are allowed, and a shape-morphing computational imaging system is demonstrated. Implemented on a reconfigurable Waterbomb origami surface with 22 active metasurface panels that radiate near-orthogonal modes across 17-27 GHz, capability to image complex 3D objects in full details minimizing the effects of specular reflections in diffraction-limited sparse imaging with scene adaptability, reconfigurable cross-range resolution, and field-of-view is demonstrated. Such electromagnetic origami surfaces, through simultaneous surface shape-morphing ability (potentially with shape-shifting electronic materials) and electromagnetic field programmability, opens up new avenues for intelligent and robust sensing and imaging systems for a wide range of applications.
Collapse
Affiliation(s)
- Suresh Venkatesh
- Department of Electrical and Computer EngineeringNorth Carolina State UniversityRaleighNC27606USA
| | - Daniel Sturm
- Department of Electrical and Computer EngineeringPrinceton UniversityPrincetonNJ08544USA
| | - Xuyang Lu
- University of Michigan‐Shanghai Jiao Tong University Joint InstituteShanghai200240China
| | | | - Kaushik Sengupta
- Department of Electrical and Computer EngineeringPrinceton UniversityPrincetonNJ08544USA
| |
Collapse
|
9
|
Liu Y, Ding H, Li J, Lou X, Yang M, Zheng Y. Light-driven single-cell rotational adhesion frequency assay. ELIGHT 2022; 2:13. [PMID: 35965781 DOI: 10.1186/s43593-022-00013-3] [Citation(s) in RCA: 56] [Impact Index Per Article: 28.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Revised: 06/28/2022] [Accepted: 07/07/2022] [Indexed: 05/23/2023]
Abstract
UNLABELLED The interaction between cell surface receptors and extracellular ligands is highly related to many physiological processes in living systems. Many techniques have been developed to measure the ligand-receptor binding kinetics at the single-cell level. However, few techniques can measure the physiologically relevant shear binding affinity over a single cell in the clinical environment. Here, we develop a new optical technique, termed single-cell rotational adhesion frequency assay (scRAFA), that mimics in vivo cell adhesion to achieve label-free determination of both homogeneous and heterogeneous binding kinetics of targeted cells at the subcellular level. Moreover, the scRAFA is also applicable to analyze the binding affinities on a single cell in native human biofluids. With its superior performance and general applicability, scRAFA is expected to find applications in study of the spatial organization of cell surface receptors and diagnosis of infectious diseases. SUPPLEMENTARY INFORMATION The online version contains supplementary material available at 10.1186/s43593-022-00020-4.
Collapse
Affiliation(s)
- Yaoran Liu
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX 78712 USA
| | - Hongru Ding
- Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX 78712 USA
| | - Jingang Li
- Materials Science & Engineering Program and Texas Materials Institute, The University of Texas at Austin, Austin, TX 78712 USA
| | - Xin Lou
- School of Physical Sciences, University of Chinese Academy of Sciences, Beijing, 100049 China
| | - Mingcheng Yang
- School of Physical Sciences, University of Chinese Academy of Sciences, Beijing, 100049 China
- Beijing National Laboratory for Condensed Matter Physics and Laboratory of Soft Matter Physics, Institute of Physics, Chinese Academy of Sciences, Beijing, 100190 China
- Songshan Lake Materials Laboratory, Dongguan, 523808 Guangdong China
| | - Yuebing Zheng
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX 78712 USA
- Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX 78712 USA
- Materials Science & Engineering Program and Texas Materials Institute, The University of Texas at Austin, Austin, TX 78712 USA
- Department of Biomedical Engineering, The University of Texas at Austin, Austin, TX 78712 USA
| |
Collapse
|
10
|
Usman M, Rains J, Cui TJ, Khan MZ, Kazim JUR, Imran MA, Abbasi QH. Intelligent wireless walls for contactless in-home monitoring. LIGHT, SCIENCE & APPLICATIONS 2022; 11:212. [PMID: 35798702 PMCID: PMC9262883 DOI: 10.1038/s41377-022-00906-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Revised: 05/25/2022] [Accepted: 06/20/2022] [Indexed: 06/15/2023]
Abstract
Human activity monitoring is an exciting research area to assist independent living among disabled and elderly population. Various techniques have been proposed to recognise human activities, such as exploiting sensors, cameras, wearables, and contactless microwave sensing. Among these, the microwave sensing has recently gained significant attention due to its merit to solve the privacy concerns of cameras and discomfort caused by wearables. However, the existing microwave sensing techniques have a basic disadvantage of requiring controlled and ideal settings for high-accuracy activity detections, which restricts its wide adoptions in non-line-of-sight (Non-LOS) environments. Here, we propose a concept of intelligent wireless walls (IWW) to ensure high-precision activity monitoring in complex environments wherein the conventional microwave sensing is invalid. The IWW is composed of a reconfigurable intelligent surface (RIS) that can perform beam steering and beamforming, and machine learning algorithms that can automatically detect the human activities with high accuracy. Two complex environments are considered: one is a corridor junction scenario with transmitter and receiver in separate corridor sections and the other is a multi-floor scenario wherein the transmitter and receiver are placed on two different floors of a building. In each of the aforementioned environments, three distinct body movements are considered namely, sitting, standing, and walking. Two subjects, one male and one female perform these activities in both environments. It is demonstrated that IWW provide a maximum detection gain of 28% in multi-floor scenario and 25% in corridor junction scenario as compared to traditional microwave sensing without RIS.
Collapse
Affiliation(s)
- Muhammad Usman
- University of Glasgow, James Watt School of Engineering, Glasgow, G12 8QQ, UK
| | - James Rains
- University of Glasgow, James Watt School of Engineering, Glasgow, G12 8QQ, UK
| | - Tie Jun Cui
- State Key Laboratory of Millimetre Waves, Southeast University, Nanjing, China
| | - Muhammad Zakir Khan
- University of Glasgow, James Watt School of Engineering, Glasgow, G12 8QQ, UK
| | | | - Muhammad Ali Imran
- University of Glasgow, James Watt School of Engineering, Glasgow, G12 8QQ, UK
| | - Qammer H Abbasi
- University of Glasgow, James Watt School of Engineering, Glasgow, G12 8QQ, UK.
| |
Collapse
|
11
|
Pan M, Fu Y, Zheng M, Chen H, Zang Y, Duan H, Li Q, Qiu M, Hu Y. Dielectric metalens for miniaturized imaging systems: progress and challenges. LIGHT, SCIENCE & APPLICATIONS 2022; 11:195. [PMID: 35764608 PMCID: PMC9240015 DOI: 10.1038/s41377-022-00885-7] [Citation(s) in RCA: 47] [Impact Index Per Article: 23.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/21/2022] [Revised: 06/03/2022] [Accepted: 06/10/2022] [Indexed: 05/25/2023]
Abstract
Lightweight, miniaturized optical imaging systems are vastly anticipated in these fields of aerospace exploration, industrial vision, consumer electronics, and medical imaging. However, conventional optical techniques are intricate to downscale as refractive lenses mostly rely on phase accumulation. Metalens, composed of subwavelength nanostructures that locally control light waves, offers a disruptive path for small-scale imaging systems. Recent advances in the design and nanofabrication of dielectric metalenses have led to some high-performance practical optical systems. This review outlines the exciting developments in the aforementioned area whilst highlighting the challenges of using dielectric metalenses to replace conventional optics in miniature optical systems. After a brief introduction to the fundamental physics of dielectric metalenses, the progress and challenges in terms of the typical performances are introduced. The supplementary discussion on the common challenges hindering further development is also presented, including the limitations of the conventional design methods, difficulties in scaling up, and device integration. Furthermore, the potential approaches to address the existing challenges are also deliberated.
Collapse
Affiliation(s)
- Meiyan Pan
- Jihua Laboratory, Foshan, 528200, China.
| | - Yifei Fu
- Jihua Laboratory, Foshan, 528200, China
| | | | - Hao Chen
- Jihua Laboratory, Foshan, 528200, China
| | | | - Huigao Duan
- College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China
- Greater Bay Area Institute for Innovation, Hunan University, Guangzhou, 511300, Guangdong Province, China
| | - Qiang Li
- State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou, 310027, China
| | - Min Qiu
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, 18 Shilongshan Road, Hangzhou, 310024, China
- Institute of Advanced Technology, Westlake Institute for Advanced Study, 18 Shilongshan Road, Hangzhou, 310024, China
| | - Yueqiang Hu
- College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China.
| |
Collapse
|
12
|
Abstract
Recent years have witnessed promising artificial intelligence (AI) applications in many disciplines, including optics, engineering, medicine, economics, and education. In particular, the synergy of AI and meta-optics has greatly benefited both fields. Meta-optics are advanced flat optics with novel functions and light-manipulation abilities. The optical properties can be engineered with a unique design to meet various optical demands. This review offers comprehensive coverage of meta-optics and artificial intelligence in synergy. After providing an overview of AI and meta-optics, we categorize and discuss the recent developments integrated by these two topics, namely AI for meta-optics and meta-optics for AI. The former describes how to apply AI to the research of meta-optics for design, simulation, optical information analysis, and application. The latter reports the development of the optical Al system and computation via meta-optics. This review will also provide an in-depth discussion of the challenges of this interdisciplinary field and indicate future directions. We expect that this review will inspire researchers in these fields and benefit the next generation of intelligent optical device design.
Collapse
Affiliation(s)
- Mu Ku Chen
- Department of Electrical Engineering, City University of Hong Kong, Kowloon, Hong Kong 999077.,Centre for Biosystems, Neuroscience, and Nanotechnology, City University of Hong Kong, Kowloon, Hong Kong 999077.,The State Key Laboratory of Terahertz and Millimeter Waves, City University of Hong Kong, Kowloon, Hong Kong 999077
| | - Xiaoyuan Liu
- Department of Electrical Engineering, City University of Hong Kong, Kowloon, Hong Kong 999077
| | - Yanni Sun
- Department of Electrical Engineering, City University of Hong Kong, Kowloon, Hong Kong 999077
| | - Din Ping Tsai
- Department of Electrical Engineering, City University of Hong Kong, Kowloon, Hong Kong 999077.,Centre for Biosystems, Neuroscience, and Nanotechnology, City University of Hong Kong, Kowloon, Hong Kong 999077.,The State Key Laboratory of Terahertz and Millimeter Waves, City University of Hong Kong, Kowloon, Hong Kong 999077
| |
Collapse
|
13
|
Luo X, Hu Y, Ou X, Li X, Lai J, Liu N, Cheng X, Pan A, Duan H. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible. LIGHT, SCIENCE & APPLICATIONS 2022; 11:158. [PMID: 35624107 PMCID: PMC9142536 DOI: 10.1038/s41377-022-00844-2] [Citation(s) in RCA: 41] [Impact Index Per Article: 20.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Revised: 05/06/2022] [Accepted: 05/10/2022] [Indexed: 05/16/2023]
Abstract
Replacing electrons with photons is a compelling route toward high-speed, massively parallel, and low-power artificial intelligence computing. Recently, diffractive networks composed of phase surfaces were trained to perform machine learning tasks through linear optical transformations. However, the existing architectures often comprise bulky components and, most critically, they cannot mimic the human brain for multitasking. Here, we demonstrate a multi-skilled diffractive neural network based on a metasurface device, which can perform on-chip multi-channel sensing and multitasking in the visible. The polarization multiplexing scheme of the subwavelength nanostructures is applied to construct a multi-channel classifier framework for simultaneous recognition of digital and fashionable items. The areal density of the artificial neurons can reach up to 6.25 × 106 mm-2 multiplied by the number of channels. The metasurface is integrated with the mature complementary metal-oxide semiconductor imaging sensor, providing a chip-scale architecture to process information directly at physical layers for energy-efficient and ultra-fast image processing in machine vision, autonomous driving, and precision medicine.
Collapse
Affiliation(s)
- Xuhao Luo
- National Research Center for High-Efficiency Grinding, College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China
- Institute of Precision Optical Engineering, School of Physics Science and Engineering, Tongji University, Shanghai, 200092, China
| | - Yueqiang Hu
- National Research Center for High-Efficiency Grinding, College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China.
- Advanced Manufacturing Laboratory of Micro-Nano Optical Devices, Shenzhen Research Institute, Hunan University, Shenzhen, 518000, China.
| | - Xiangnian Ou
- National Research Center for High-Efficiency Grinding, College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China
| | - Xin Li
- National Research Center for High-Efficiency Grinding, College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China
| | - Jiajie Lai
- National Research Center for High-Efficiency Grinding, College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China
| | - Na Liu
- 2nd Physics Institute, University of Stuttgart, Pfaffenwaldring 57, 70569, Stuttgart, Germany
- Max Planck Institute for Solid State Research, Heisenbergstrasse 1, 70569, Stuttgart, Germany
| | - Xinbin Cheng
- Institute of Precision Optical Engineering, School of Physics Science and Engineering, Tongji University, Shanghai, 200092, China
| | - Anlian Pan
- National Research Center for High-Efficiency Grinding, College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China
| | - Huigao Duan
- National Research Center for High-Efficiency Grinding, College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China.
- Greater Bay Area Institute for Innovation, Hunan University, Guangzhou, 511300, China.
| |
Collapse
|
14
|
Wu W, Hu D, Cong W, Shan H, Wang S, Niu C, Yan P, Yu H, Vardhanabhuti V, Wang G. Stabilizing deep tomographic reconstruction: Part A. Hybrid framework and experimental results. PATTERNS (NEW YORK, N.Y.) 2022; 3:100474. [PMID: 35607623 PMCID: PMC9122961 DOI: 10.1016/j.patter.2022.100474] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/26/2021] [Revised: 12/24/2021] [Accepted: 03/01/2022] [Indexed: 12/16/2022]
Abstract
A recent PNAS paper reveals that several popular deep reconstruction networks are unstable. Specifically, three kinds of instabilities were reported: (1) strong image artefacts from tiny perturbations, (2) small features missed in a deeply reconstructed image, and (3) decreased imaging performance with increased input data. Here, we propose an analytic compressed iterative deep (ACID) framework to address this challenge. ACID synergizes a deep network trained on big data, kernel awareness from compressed sensing (CS)-inspired processing, and iterative refinement to minimize the data residual relative to real measurement. Our study demonstrates that the ACID reconstruction is accurate, is stable, and sheds light on the converging mechanism of the ACID iteration under a bounded relative error norm assumption. ACID not only stabilizes an unstable deep reconstruction network but also is resilient against adversarial attacks to the whole ACID workflow, being superior to classic sparsity-regularized reconstruction and eliminating the three kinds of instabilities.
Collapse
Affiliation(s)
- Weiwen Wu
- Biomedical Imaging Center, Center for Biotechnology and Interdisciplinary Studies, Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
- School of Biomedical Engineering, Sun Yat-sen University, Shenzhen, Guangdong, China
- Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong SAR, China
| | - Dianlin Hu
- The Laboratory of Image Science and Technology, Southeast University, Nanjing, China
| | - Wenxiang Cong
- Biomedical Imaging Center, Center for Biotechnology and Interdisciplinary Studies, Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Hongming Shan
- Biomedical Imaging Center, Center for Biotechnology and Interdisciplinary Studies, Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
- Institute of Science and Technology for Brain-inspired Intelligence, Fudan University, Shanghai, China
| | - Shaoyu Wang
- Department of Electrical & Computer Engineering, University of Massachusetts Lowell, Lowell, MA, USA
| | - Chuang Niu
- Biomedical Imaging Center, Center for Biotechnology and Interdisciplinary Studies, Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Pingkun Yan
- Biomedical Imaging Center, Center for Biotechnology and Interdisciplinary Studies, Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
| | - Hengyong Yu
- Department of Electrical & Computer Engineering, University of Massachusetts Lowell, Lowell, MA, USA
| | - Varut Vardhanabhuti
- Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, University of Hong Kong, Hong Kong SAR, China
| | - Ge Wang
- Biomedical Imaging Center, Center for Biotechnology and Interdisciplinary Studies, Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA
| |
Collapse
|
15
|
Abstract
We present wave-based signal differentiation with unprecedented fidelity and flexibility by purposefully perturbing overmoded random scattering systems such that zeros of their scattering matrices lie exactly at the desired locations on the real frequency axis. Our technique overcomes limitations of hitherto existing approaches based on few-mode systems, both regarding their extreme vulnerability to fabrication inaccuracies or environmental perturbations and their inability to maintain high fidelity under in-situ adaptability. We demonstrate our technique experimentally by placing a programmable metasurface with hundreds of degrees of freedom inside a 3D disordered metallic box. Regarding the integrability of wave processors, such repurposing of existing enclosures is an enticing alternative to fabricating miniaturized devices. Our over-the-air differentiator can process in parallel multiple signals on distinct carriers and maintains high fidelity when reprogrammed to different carriers. We also perform programmable higher-order differentiation. Conceivable applications include segmentation or compression of communication or radar signals and machine vision.
Collapse
|
16
|
Zhu L, Zhou W, Dong L, Guan C, Shang G, Ding X, Burokur SN, Wu Q. Meta-hologram enabled by a double-face copper-cladded metasurface based on reflection-transmission amplitude coding. OPTICS LETTERS 2022; 47:174-177. [PMID: 34951910 DOI: 10.1364/ol.442464] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/06/2021] [Accepted: 11/24/2021] [Indexed: 06/14/2023]
Abstract
Here, we propose a double-face copper-cladded meta-hologram that can efficiently manipulate the amplitude of electromagnetic waves in both transmission and reflection spaces, depending on the polarization state of the incident electromagnetic wave. The proposed meta-hologram is validated by encoding the transmission-reflection amplitude information of two independent images into a single metasurface. The holographic images obtained from measurements agree qualitatively with simulation results. The proposed metasurface presents a novel, to the best of our knowledge, scheme for electromagnetic wavefront control in the whole space and overcomes the limitations of narrow frequency band operation.
Collapse
|
17
|
Del Hougne M, Gigan S, Del Hougne P. Deeply Subwavelength Localization with Reverberation-Coded Aperture. PHYSICAL REVIEW LETTERS 2021; 127:043903. [PMID: 34355940 DOI: 10.1103/physrevlett.127.043903] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/10/2021] [Revised: 06/10/2021] [Accepted: 06/14/2021] [Indexed: 05/18/2023]
Abstract
Accessing subwavelength information about a scene from the far-field without invasive near-field manipulations is a fundamental challenge in wave engineering. Yet it is well understood that the dwell time of waves in complex media sets the scale for the waves' sensitivity to perturbations. Modern coded-aperture imagers leverage the degrees of freedom (d.o.f.) offered by complex media as natural multiplexor but do not recognize and reap the fundamental difference between placing the object of interest outside or within the complex medium. Here, we show that the precision of localizing a subwavelength object can be improved by several orders of magnitude simply by enclosing it in its far field with a reverberant passive chaotic cavity. We identify deep learning as a suitable noise-robust tool to extract subwavelength localization information encoded in multiplexed measurements, achieving resolutions well beyond those available in the training data. We demonstrate our finding in the microwave domain: harnessing the configurational d.o.f. of a simple programmable metasurface, we localize a subwavelength object along a curved trajectory inside a chaotic cavity with a resolution of λ/76 using intensity-only single-frequency single-pixel measurements. Our results may have important applications in photoacoustic imaging as well as human-machine interaction based on reverberating elastic waves, sound, or microwaves.
Collapse
Affiliation(s)
| | - Sylvain Gigan
- Laboratoire Kastler Brossel, Université Pierre et Marie Curie, Ecole Normale Supérieure, CNRS, Collège de France, F-75005 Paris, France
| | | |
Collapse
|
18
|
Rapczyński M, Werner P, Handrich S, Al-Hamadi A. A Baseline for Cross-Database 3D Human Pose Estimation. SENSORS 2021; 21:s21113769. [PMID: 34071704 PMCID: PMC8198914 DOI: 10.3390/s21113769] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/17/2021] [Revised: 05/19/2021] [Accepted: 05/24/2021] [Indexed: 11/16/2022]
Abstract
Vision-based 3D human pose estimation approaches are typically evaluated on datasets that are limited in diversity regarding many factors, e.g., subjects, poses, cameras, and lighting. However, for real-life applications, it would be desirable to create systems that work under arbitrary conditions (“in-the-wild”). To advance towards this goal, we investigated the commonly used datasets HumanEva-I, Human3.6M, and Panoptic Studio, discussed their biases (that is, their limitations in diversity), and illustrated them in cross-database experiments (for which we used a surrogate for roughly estimating in-the-wild performance). For this purpose, we first harmonized the differing skeleton joint definitions of the datasets, reducing the biases and systematic test errors in cross-database experiments. We further proposed a scale normalization method that significantly improved generalization across camera viewpoints, subjects, and datasets. In additional experiments, we investigated the effect of using more or less cameras, training with multiple datasets, applying a proposed anatomy-based pose validation step, and using OpenPose as the basis for the 3D pose estimation. The experimental results showed the usefulness of the joint harmonization, of the scale normalization, and of augmenting virtual cameras to significantly improve cross-database and in-database generalization. At the same time, the experiments showed that there were dataset biases that could not be compensated and call for new datasets covering more diversity. We discussed our results and promising directions for future work.
Collapse
|
19
|
Cicirelli G, Marani R, Petitti A, Milella A, D’Orazio T. Ambient Assisted Living: A Review of Technologies, Methodologies and Future Perspectives for Healthy Aging of Population. SENSORS (BASEL, SWITZERLAND) 2021; 21:3549. [PMID: 34069727 PMCID: PMC8160803 DOI: 10.3390/s21103549] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Revised: 05/03/2021] [Accepted: 05/16/2021] [Indexed: 01/29/2023]
Abstract
Over the last decade, there has been considerable and increasing interest in the development of Active and Assisted Living (AAL) systems to support independent living. The demographic change towards an aging population has introduced new challenges to today's society from both an economic and societal standpoint. AAL can provide an arrary of solutions for improving the quality of life of individuals, for allowing people to live healthier and independently for longer, for helping people with disabilities, and for supporting caregivers and medical staff. A vast amount of literature exists on this topic, so this paper aims to provide a survey of the research and skills related to AAL systems. A comprehensive analysis is presented that addresses the main trends towards the development of AAL systems both from technological and methodological points of view and highlights the main issues that are worthy of further investigation.
Collapse
Affiliation(s)
- Grazia Cicirelli
- Institute of Intelligent Industrial Systems and Technologies for Advanced Manufacturing (STIIMA), National Research Council of Italy (CNR), Via G. Amendola 122, 70126 Bari, Italy; (R.M.); (A.P.); (A.M.); (T.D.)
| | | | | | | | | |
Collapse
|
20
|
Taghvaee H, Jain A, Timoneda X, Liaskos C, Abadal S, Alarcón E, Cabellos-Aparicio A. Radiation Pattern Prediction for Metasurfaces: A Neural Network-Based Approach. SENSORS 2021; 21:s21082765. [PMID: 33919861 PMCID: PMC8070797 DOI: 10.3390/s21082765] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/16/2021] [Revised: 04/02/2021] [Accepted: 04/02/2021] [Indexed: 11/16/2022]
Abstract
As the current standardization for the 5G networks nears completion, work towards understanding the potential technologies for the 6G wireless networks is already underway. One of these potential technologies for the 6G networks is reconfigurable intelligent surfaces. They offer unprecedented degrees of freedom towards engineering the wireless channel, i.e., the ability to modify the characteristics of the channel whenever and however required. Nevertheless, such properties demand that the response of the associated metasurface is well understood under all possible operational conditions. While an understanding of the radiation pattern characteristics can be obtained through either analytical models or full-wave simulations, they suffer from inaccuracy and extremely high computational complexity, respectively. Hence, in this paper, we propose a neural network-based approach that enables a fast and accurate characterization of the metasurface response. We analyze multiple scenarios and demonstrate the capabilities and utility of the proposed methodology. Concretely, we show that this method can learn and predict the parameters governing the reflected wave radiation pattern with an accuracy of a full-wave simulation (98.8–99.8%) and the time and computational complexity of an analytical model. The aforementioned result and methodology will be of specific importance for the design, fault tolerance, and maintenance of the thousands of reconfigurable intelligent surfaces that will be deployed in the 6G network environment.
Collapse
Affiliation(s)
- Hamidreza Taghvaee
- NaNoNetworking Center in Catalonia (N3Cat), Universitat Politècnica de Catalunya, 08034 Barcelona, Spain; (A.J.); (X.T.); (S.A.); (E.A.); (A.C.-A.)
- Correspondence:
| | - Akshay Jain
- NaNoNetworking Center in Catalonia (N3Cat), Universitat Politècnica de Catalunya, 08034 Barcelona, Spain; (A.J.); (X.T.); (S.A.); (E.A.); (A.C.-A.)
| | - Xavier Timoneda
- NaNoNetworking Center in Catalonia (N3Cat), Universitat Politècnica de Catalunya, 08034 Barcelona, Spain; (A.J.); (X.T.); (S.A.); (E.A.); (A.C.-A.)
| | - Christos Liaskos
- Foundation for Research and Technology Hellas, 71110 Heraklion, Greece;
| | - Sergi Abadal
- NaNoNetworking Center in Catalonia (N3Cat), Universitat Politècnica de Catalunya, 08034 Barcelona, Spain; (A.J.); (X.T.); (S.A.); (E.A.); (A.C.-A.)
| | - Eduard Alarcón
- NaNoNetworking Center in Catalonia (N3Cat), Universitat Politècnica de Catalunya, 08034 Barcelona, Spain; (A.J.); (X.T.); (S.A.); (E.A.); (A.C.-A.)
| | - Albert Cabellos-Aparicio
- NaNoNetworking Center in Catalonia (N3Cat), Universitat Politècnica de Catalunya, 08034 Barcelona, Spain; (A.J.); (X.T.); (S.A.); (E.A.); (A.C.-A.)
| |
Collapse
|
21
|
Luo J, Li X, Zhang X, Guo J, Liu W, Lai Y, Zhan Y, Huang M. Deep-learning-enabled inverse engineering of multi-wavelength invisibility-to-superscattering switching with phase-change materials. OPTICS EXPRESS 2021; 29:10527-10537. [PMID: 33820186 DOI: 10.1364/oe.422119] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Accepted: 03/11/2021] [Indexed: 06/12/2023]
Abstract
Inverse design of nanoparticles for desired scattering spectra and dynamic switching between the two opposite scattering anomalies, i.e. superscattering and invisibility, is important in realizing cloaking, sensing and functional devices. However, traditionally the design process is quite complicated, which involves complex structures with many choices of synthetic constituents and dispersions. Here, we demonstrate that a well-trained deep-learning neural network can handle these issues efficiently, which can not only forwardly predict scattering spectra of multilayer nanoparticles with high precision, but also inversely design the required structural and material parameters efficiently. Moreover, we show that the neural network is capable of finding out multi-wavelength invisibility-to-superscattering switching points at the desired wavelengths in multilayer nanoparticles composed of metals and phase-change materials. Our work provides a useful solution of deep learning for inverse design of nanoparticles with dynamic scattering spectra by using phase-change materials.
Collapse
|
22
|
Liu F, Zhang W, Sun Y, Liu J, Miao J, He F, Wu X. Secure Deep Learning for Intelligent Terahertz Metamaterial Identification. SENSORS (BASEL, SWITZERLAND) 2020; 20:E5673. [PMID: 33027897 PMCID: PMC7583053 DOI: 10.3390/s20195673] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/03/2020] [Revised: 10/01/2020] [Accepted: 10/02/2020] [Indexed: 11/18/2022]
Abstract
Metamaterials, artificially engineered structures with extraordinary physical properties, offer multifaceted capabilities in interdisciplinary fields. To address the looming threat of stealthy monitoring, the detection and identification of metamaterials is the next research frontier but have not yet been explored. Here, we show that the crypto-oriented convolutional neural network (CNN) makes possible the secure intelligent detection of metamaterials in mixtures. Terahertz signals were encrypted by homomorphic encryption and the ciphertext was submitted to the CNN directly for results, which can only be decrypted by the data owner. The experimentally measured terahertz signals were augmented and further divided into training sets and test sets using 5-fold cross-validation. Experimental results illustrated that the model achieved an accuracy of 100% on the test sets, which highly outperformed humans and the traditional machine learning. The CNN took 9.6 s to inference on 92 encrypted test signals with homomorphic encryption backend. The proposed method with accuracy and security provides private preserving paradigm for artificial intelligence-based material identification.
Collapse
Affiliation(s)
- Feifei Liu
- School of Cyber Science and Technology, Beihang University, Beijing 100191, China; (F.L.); (W.Z.); (J.L.); (X.W.)
| | - Weihao Zhang
- School of Cyber Science and Technology, Beihang University, Beijing 100191, China; (F.L.); (W.Z.); (J.L.); (X.W.)
| | - Yu Sun
- School of Cyber Science and Technology, Beihang University, Beijing 100191, China; (F.L.); (W.Z.); (J.L.); (X.W.)
| | - Jianwei Liu
- School of Cyber Science and Technology, Beihang University, Beijing 100191, China; (F.L.); (W.Z.); (J.L.); (X.W.)
| | - Jungang Miao
- School of Electronic and Information Engineering, Beihang University, Beijing 100191, China; (J.M.); (F.H.)
| | - Feng He
- School of Electronic and Information Engineering, Beihang University, Beijing 100191, China; (J.M.); (F.H.)
| | - Xiaojun Wu
- School of Cyber Science and Technology, Beihang University, Beijing 100191, China; (F.L.); (W.Z.); (J.L.); (X.W.)
- School of Electronic and Information Engineering, Beihang University, Beijing 100191, China; (J.M.); (F.H.)
| |
Collapse
|
23
|
Cui TJ, Li L, Liu S, Ma Q, Zhang L, Wan X, Jiang WX, Cheng Q. Information Metamaterial Systems. iScience 2020; 23:101403. [PMID: 32777776 PMCID: PMC7415848 DOI: 10.1016/j.isci.2020.101403] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2020] [Revised: 07/16/2020] [Accepted: 07/20/2020] [Indexed: 11/19/2022] Open
Abstract
Metamaterials have great capabilities and flexibilities in controlling electromagnetic (EM) waves because their subwavelength meta-atoms can be designed and tailored in desired ways. However, once the structure-only metamaterials (i.e., passive metamaterials) are fabricated, their functions will be fixed. To control the EM waves dynamically, active devices are integrated into the meta-atoms, yielding active metamaterials. Traditionally, the active metamaterials include tunable metamaterials and reconfigurable metamaterials, which have either small-range tunability or a few numbers of reconfigurability. Recently, a special kind of active metamaterials, digital coding and programmable metamaterials, have been presented, which can realize a large number of distinct functionalities and switch them in real time with the aid of field programmable gate array (FPGA). More importantly, the digital coding representations of metamaterials make it possible to bridge the digital world and physical world using the metamaterial platform and make the metamaterials process digital information directly, resulting in information metamaterials. In this review article, we firstly introduce the evolution of metamaterials and then present the concepts and basic principles of digital coding metamaterials and information metamaterials. With more details, we discuss a series of information metamaterial systems, including the programmable metamaterial systems, software metamaterial systems, intelligent metamaterial systems, and space-time-coding metamaterial systems. Finally, we introduce the current progress and predict the future trends of information metamaterials.
Collapse
Affiliation(s)
- Tie Jun Cui
- State Key Laboratory of Millimeter Waves, Southeast University, Nanjing 210096, China.
| | - Lianlin Li
- State Key Laboratory of Advanced Optical Communication Systems and Networks, Department of Electronics, Peking University, Beijing 100871, China
| | - Shuo Liu
- State Key Laboratory of Millimeter Waves, Southeast University, Nanjing 210096, China
| | - Qian Ma
- State Key Laboratory of Millimeter Waves, Southeast University, Nanjing 210096, China
| | - Lei Zhang
- State Key Laboratory of Millimeter Waves, Southeast University, Nanjing 210096, China
| | - Xiang Wan
- State Key Laboratory of Millimeter Waves, Southeast University, Nanjing 210096, China
| | - Wei Xiang Jiang
- State Key Laboratory of Millimeter Waves, Southeast University, Nanjing 210096, China
| | - Qiang Cheng
- State Key Laboratory of Millimeter Waves, Southeast University, Nanjing 210096, China
| |
Collapse
|
24
|
Zhao H, Shuang Y, Wei M, Cui TJ, Hougne PD, Li L. Metasurface-assisted massive backscatter wireless communication with commodity Wi-Fi signals. Nat Commun 2020; 11:3926. [PMID: 32764638 PMCID: PMC7413398 DOI: 10.1038/s41467-020-17808-y] [Citation(s) in RCA: 37] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2020] [Accepted: 07/16/2020] [Indexed: 11/20/2022] Open
Abstract
Conventional wireless communication architecture, a backbone of our modern society, relies on actively generated carrier signals to transfer information, leading to important challenges including limited spectral resources and energy consumption. Backscatter communication systems, on the other hand, modulate an antenna's impedance to encode information into already existing waves but suffer from low data rates and a lack of information security. Here, we introduce the concept of massive backscatter communication which modulates the propagation environment of stray ambient waves with a programmable metasurface. The metasurface's large aperture and huge number of degrees of freedom enable unprecedented wave control and thereby secure and high-speed information transfer. Our prototype leveraging existing commodity 2.4 GHz Wi-Fi signals achieves data rates on the order of hundreds of Kbps. Our technique is applicable to all types of wave phenomena and provides a fundamentally new perspective on the role of metasurfaces in future wireless communication.
Collapse
Affiliation(s)
- Hanting Zhao
- State Key Laboratory of Advanced Optical Communication Systems and Networks, Department of Electronics, Peking University, 100871, Beijing, China
| | - Ya Shuang
- State Key Laboratory of Advanced Optical Communication Systems and Networks, Department of Electronics, Peking University, 100871, Beijing, China
| | - Menglin Wei
- State Key Laboratory of Advanced Optical Communication Systems and Networks, Department of Electronics, Peking University, 100871, Beijing, China
| | - Tie Jun Cui
- State Key Laboratory of Millimeter Waves, Southeast University, 210096, Nanjing, China.
| | - Philipp Del Hougne
- Univ Rennes, CNRS, Institut d'Electronique et de Télécommunications de Rennes (IETR)-UMR 6164, 35000, Rennes, France.
| | - Lianlin Li
- State Key Laboratory of Advanced Optical Communication Systems and Networks, Department of Electronics, Peking University, 100871, Beijing, China.
| |
Collapse
|