Ultrathin Eardrum-Inspired Self-Powered Acoustic Sensor for Vocal Synchronization Recognition with the Assistance of Machine Learning

A Nanoparticle-Based Artificial Ear for Personalized Classification of Emotions in the Human Voice Using Deep Learning

Artificial intelligence and human-computer interaction advances demand bioinspired sensing modalities capable of comprehending human affective states and speech. However, endowing skin-like interfaces with such intricate perception abilities remains challenging. Here, we have developed a flexible piezoresistive artificial ear (AE) sensor based on gold nanoparticles, which can convert sound signals into electrical signals through changes in resistance. By testing the sensor's performance at both frequency and sound pressure level (SPL), the AE has a frequency response range of 20 Hz to 12 kHz and can sense sound signals from up to 5 m away at a frequency of 1 kHz and an SPL of 126 dB. Furthermore, through deep learning, the device achieves up to 96.9% and 95.0% accuracy in classification and recognition applications for seven emotional and eight urban environmental noises, respectively. Hence, on one hand, our device can monitor the patient's emotional state by their speech, such as sudden yelling and screaming, which can help healthcare workers understand patients' condition in time. On the other hand, the device could also be used for real-time monitoring of noise levels in aircraft, ships, factories, and other high-decibel equipment and environments.

Intelligent Song Recognition via a Hollow-Microstructure-Based, Ultrasensitive Artificial Eardrum

Artificial ears with intelligence, which can sensitively detect sound-a variant of pressure-and generate consciousness and logical decision-making abilities, hold great promise to transform life. However, despite the emerging flexible sensors for sound detection, most success is limited to very simple phonemes, such as a couple of letters or words, probably due to the lack of device sensitivity and capability. Herein, the construction of ultrasensitive artificial eardrums enabling intelligent song recognition is reported. This strategy employs novel geometric engineering of sensing units in the soft microstructure array (to significantly reduce effective modulus) along with complex song recognition exploration leveraging machine learning algorithms. Unprecedented pressure sensitivity (6.9 × 103 kPa-1) is demonstrated in a sensor with a hollow pyramid architecture with porous slants. The integrated device exhibits unparalleled (exceeding by 1-2 orders of magnitude compared with reported benchmark samples) sound detection sensitivity, and can accurately identify 100% (for training set) and 97.7% (for test set) of a database of the segments from 77 songs varying in language, style, and singer. Overall, the results highlight the outstanding performance of the hollow-microstructure-based sensor, indicating its potential applications in human-machine interaction and wearable acoustical technologies.

Emerging Trends of Nanofibrous Piezoelectric and Triboelectric Applications: Mechanisms, Electroactive Materials, and Designed Architectures

Over the past few decades, significant progress in piezo-/triboelectric nanogenerators (PTEGs) has led to the development of cutting-edge wearable technologies. Nanofibers with good designability, controllable morphologies, large specific areas, and unique physicochemical properties provide a promising platform for PTEGs for various advanced applications. However, the further development of nanofiber-based PTEGs is limited by technical difficulties, ranging from materials design to device integration. Herein, the current developments in PTEGs based on electrospun nanofibers are systematically reviewed. This review begins with the mechanisms of PTEGs and the advantages of nanofibers and nanodevices, including high breathability, waterproofness, scalability, and thermal-moisture comfort. In terms of materials and structural design, novel electroactive nanofibers and structure assemblies based on 1D micro/nanostructures, 2D bionic structures, and 3D multilayered structures are discussed. Subsequently, nanofibrous PTEGs in applications such as energy harvesters, personalized medicine, personal protective equipment, and human-machine interactions are summarized. Nanofiber-based PTEGs still face many challenges such as energy efficiency, material durability, device stability, and device integration. Finally, the research gap between research and practical applications of PTEGs is discussed, and emerging trends are proposed, providing some ideas for the development of intelligent wearables.

Expanding the Horizons of Machine Learning in Nanomaterials to Chiral Nanostructures

Machine learning holds significant research potential in the field of nanotechnology, enabling nanomaterial structure and property predictions, facilitating materials design and discovery, and reducing the need for time-consuming and labor-intensive experiments and simulations. In contrast to their achiral counterparts, the application of machine learning for chiral nanomaterials is still in its infancy, with a limited number of publications to date. This is despite the great potential of machine learning to advance the development of new sustainable chiral materials with high values of optical activity, circularly polarized luminescence, and enantioselectivity, as well as for the analysis of structural chirality by electron microscopy. In this review, an analysis of machine learning methods used for studying achiral nanomaterials is provided, subsequently offering guidance on adapting and extending this work to chiral nanomaterials. An overview of chiral nanomaterials within the framework of synthesis-structure-property-application relationships is presented and insights on how to leverage machine learning for the study of these highly complex relationships are provided. Some key recent publications are reviewed and discussed on the application of machine learning for chiral nanomaterials. Finally, the review captures the key achievements, ongoing challenges, and the prospective outlook for this very important research field.

Enhanced sensitivity and linear-response in iontronic pressure sensors for non-contact, high-frequency vibration recognition

Monitoring non-contact high-frequency vibrations requires improving the sensitivity and linear response of iontronic pressure sensors (IPSs). In this study, we incorporate composite electrodes comprising silver nanowires (Ag NWs) and MXene into IPSs to enhance electronic conduction and pseudocapacitance. Moreover, we utilize a novel surface-pillar microstructure, along with an internally randomized multi-bubble structure within the dielectric layer, to significantly expand the linear response range of the sensor. The resulting IPS device demonstrates exceptional linear sensitivity, measuring approximately 153.83 kPa-1, across a broad pressure range of up to 260 kPa. Additionally, it exhibits long-term stability, rapid response and recovery characteristics, and remains functional underwater. Notably, these devices exhibit remarkable capabilities in monitoring ultrasonic vibrations and accurately identifying sound wave vibrations. The integration of composite electrodes, microstructure designs, and their compatibility with underwater applications positions these IPSs as highly promising tools for precise measurements and advancements in flexible electronics technology.

Ultra-Broadband Flexible Thin-Film Sensor for Sound Monitoring and Ultrasonic Diagnosis

Small-scale and flexible acoustic probes are more desirable for exquisite objects like human bodies and complex-shaped components than conventional rigid ones. Herein, a thin-film flexible acoustic sensor (FA-TES) that can detect ultra-broadband acoustic signals in multiple applications is proposed. The device consists of two thin copper-coated polyvinyl chloride films, which are stimulated by acoustic waves and contact each other to generate the triboelectric signal. Interlocking nanocolumn arrays fabricated on the friction surfaces are regarded as a highly adaptive spacer enabling this device to respond to ultra-broadband acoustic signals (100 Hz-4 MHz) and enhance sensor sensitivity for film weak vibration. Benefiting from the characteristics of high shape adaptability and ultrawide response range, the FA-TES can precisely sense human physiological sounds and voice (≤10 kHz) for laryngeal health monitoring and interaction in real-time. Moreover, the FA-TES flexibly arranged on a 3D-printed vertebra model can effectively and accurately diagnose the inner defect by ultrasonic testing (≥1 MHz). It envisions that this work can provide new ideas for flexible acoustic sensor designs and optimize real-time acoustic detections of human bodies and complex components.

Insights into Materials, Physics, and Applications in Flexible and Wearable Acoustic Sensing Technology

Sound plays a crucial role in the perception of the world. It allows to communicate, learn, and detect potential dangers, diagnose diseases, and much more. However, traditional acoustic sensors are limited in their form factors, being rigid and cumbersome, which restricts their potential applications. Recently, acoustic sensors have made significant advancements, transitioning from rudimentary forms to wearable devices and smart everyday clothing that can conform to soft, curved, and deformable surfaces or surroundings. In this review, the latest scientific and technological breakthroughs with insightful analysis in materials, physics, design principles, fabrication strategies, functions, and applications of flexible and wearable acoustic sensing technology are comprehensively explored. The new generation of acoustic sensors that can recognize voice, interact with machines, control robots, enable marine positioning and localization, monitor structural health, diagnose human vital signs in deep tissues, and perform organ imaging is highlighted. These innovations offer unique solutions to significant challenges in fields such as healthcare, biomedicine, wearables, robotics, and metaverse. Finally, the existing challenges and future opportunities in the field are addressed, providing strategies to advance acoustic sensing technologies for intriguing real-world applications and inspire new research directions.

Quantitative Topic Analysis of Materials Science Literature Using Natural Language Processing

Materials science research has garnered extensive attention from industry, society, policy, and academia. However, understanding the research landscape and extracting strategic insights are challenging due to the increasing diversity and volume of publications. This study proposes a natural language processing-based protocol for extracting text-encoded topics from a large volume of scientific literature, uncovering research interests of scientific communities, as well as convergence trends. We report a topic map, representing the materials science research landscape with text-mined 257 topics regarding biocompatible materials, structural materials, electrochemistry, or photonics. We analyze the topic map in terms of national research interests in materials science, revealing competitive positions and strategies of active nations. For example, it is found that the increasing trend of research interest in machine learning topic was captured in the United States earlier than other nations. Similarly, our journal-level analyses serve as reference information for journal recommendations and trend guidance, showing that the main topics and research interests of materials science journals slightly changed over time. Moreover, we build the topic association network which can highlight the status and future potential of interdisciplinary research, revealing research fields with high centrality in the network such as machine learning-enabled composite modeling, energy policy, or wearable electronics. This study offers insightful results on current and near-future materials science research landscapes, facilitating the understanding of stakeholders, amidst the fast-evolving and diverse knowledge of materials science.

Ultrasensitive Acoustic Detection Using an Enlarged Fabry-Perot Cavity with a Graphene Diaphragm

For exerting high sensitivity of ultrathin graphene to detection deformation, an enlarged backing air cavity (EBC) structure is developed to further enhance the mechanical sensitivity (SM) of a graphene-based Fabry-Perot (F-P) acoustic sensor. COMSOL acoustic field simulation on the air cavity size-dependent SM confirms the optimal length and radius of the EBC of 0.2 and 1.5 mm, respectively, with the maximum simulation SM of 26.16 nm/Pa@1 kHz. Acoustic experiments further demonstrate that the frequency response of the fabricated graphene-based F-P acoustic sensor after the use of the EBC is enhanced by 5.73-79.33 times in the range of 0.5-18 kHz, compared with the conventional one without the EBC. Especially the maximum SM is up to 187.32 nm/Pa@16 kHz, which is at least 17% higher than the SM values ranging from 1.1 to 160 nm/Pa in previously reported F-P acoustic sensors using various diaphragm materials. More acoustic characteristics are examined to highlight various merits of the EBC structure, including a signal-to-noise ratio (SNR) of 60-75 dB@0.5-18 kHz, a time stability of less than ±1.3% for 90 min, a detection resolution of 0.01 Hz, and a high-fidelity speech detection with a cross-correlation coefficient of greater than 0.9, thereby revealing its high-performance weak acoustic sensing and speech recognition applications.

Emerging Trends in Soft Electronics: Integrating Machine Intelligence with Soft Acoustic/Vibration Sensors

In the last decade, soft acoustic/vibration sensors have gained tremendous research interest due to their unique ability to detect broadband acoustic/vibration stimuli, potentializing futuristic applications including voice biometrics, voice-controlled human-machine-interfaces, electronic skin, and skin-mountable healthcare devices. Importantly, to benefit most from these sensors, it is inevitable to use machine learning (ML) to process their output signals; with ML, a more accurate and efficient interpretation of original data is possible. This paper is dedicated to offering an overview of recent advances empowering the development of soft acoustic/vibration sensors and their signal processing using ML. First, the key performance parameters of the sensors are discussed. Second, popular transduction mechanisms for the sensors are addressed, followed by an in-depth overview of each type, covering materials used, structural designs, and sensing performances. Third, potential applications of the sensors are elaborated and fourth, a thorough discussion on ML is conducted, exploring different types of ML, specific ML algorithms suitable for processing acoustic/vibration signals, and current trends in ML-assisted applications. Finally, the challenges and potential opportunities in soft acoustic/vibration sensor and ML research are revealed to offer new insights into future prospects in these fields.

Human-Machine Interaction via Dual Modes of Voice and Gesture Enabled by Triboelectric Nanogenerator and Machine Learning

With the development of science and technology, human-machine interaction has brought great benefits to the society. Here, we design a voice and gesture signal translator (VGST), which can translate natural actions into electrical signals and realize efficient communication in human-machine interface. By spraying silk protein on the copper of the device, the VGST can achieve improved output and a wide frequency response of 20-2000 Hz with a high sensitivity of 167 mV/dB, and the resolution of frequency detection can reach 0.1 Hz. By designing its internal structure, its resonant frequency and output voltage can be adjusted. The VGST can be used as a high-fidelity platform to effectively recover recorded music and can also be combined with machine learning algorithms to realize the function of speech recognition with a high accuracy rate of 97%. It also has good antinoise performance to recognize speech correctly even in noisy environments. Meanwhile, in gesture recognition, the triboelectric translator is able to recognize simple hand gestures and to judge the distance between hand and the VGST based on the principle of electrostatic induction. This work demonstrates that triboelectric nanogenerator (TENG) technology can have great application prospects and significant advantages in human-machine interaction and high-fidelity platforms.

Multichannel Gradient Piezoelectric Transducer Assisted with Deep Learning for Broadband Acoustic Sensing

As an important part of human-machine interfaces, piezoelectric voice recognition has received extensive attention due to its unique self-powered nature. However, conventional voice recognition devices exhibit a limited response frequency band due to the intrinsic hardness and brittleness of piezoelectric ceramics or the flexibility of piezoelectric fibers. Here, we propose a cochlear-inspired multichannel piezoelectric acoustic sensor (MAS) based on gradient PVDF piezoelectric nanofibers for broadband voice recognition by a programmable electrospinning technique. Compared with the common electrospun PVDF membrane-based acoustic sensor, the developed MAS demonstrates the greatly 300%-broadened frequency band and the substantially 334.6%-enhanced piezoelectric output. More importantly, this MAS can serve as a high-fidelity auditory platform for music recording and human voice recognition, in which the classification accuracy rate can reach up to 100% in coordination with deep learning. The programmable bionic gradient piezoelectric nanofiber may provide a universal strategy for the development of intelligent bioelectronics.

Ultrastretchable, Self-Healing Conductive Hydrogel-Based Triboelectric Nanogenerators for Human-Computer Interaction

The rapid development of wearable electronic devices and virtual reality technology has revived interest in flexible sensing and control devices. Here, we report an ionic hydrogel (PTSM) prepared from polypropylene amine (PAM), tannic acid (TA), sodium alginate (SA), and MXene. Based on the multiple weak H-bonds, this hydrogel exhibits excellent stretchability (strain >4600%), adhesion, and self-healing. The introduction of MXene nanosheets endows the hydrogel sensor with a high gauge factor (GF) of 6.6. Meanwhile, it also enables triboelectric nanogenerators (PTSM-TENGs) fabricated from silicone rubber-encapsulated hydrogels to have excellent energy harvesting efficiency, with an instantaneous output power density of 54.24 mW/m². We build a glove-based human-computer interaction (HMI) system using PTSM-TENGs. The multidimensional signal features of PTSM-TENG are extracted and analyzed by the HMI system, and the functions of gesture visualization and robot hand control are realized. In addition, triboelectric signals can be used for object recognition with the help of machine learning techniques. The glove based on PTSM-TENG achieves the classification and recognition of five objects through contact, with an accuracy rate of 98.7%. Therefore, strain sensors and triboelectric nanogenerators based on hydrogels have broad application prospects in man-machine interface, intelligent recognition systems, auxiliary control systems, and other fields due to their excellent stretchable and high self-healing performance.