1
|
Wang X, Meng J, Liu Y, Zhan G, Tian Z. Self-supervised acoustic representation learning via acoustic-embedding memory unit modified space autoencoder for underwater target recognition. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2022; 152:2905. [PMID: 36456286 DOI: 10.1121/10.0015138] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/10/2022] [Accepted: 10/23/2022] [Indexed: 06/17/2023]
Abstract
Since the expensive annotation of high-quality signals obtained from passive sonars and the weak generalization ability of the single feature in the ocean, this paper proposes the self-supervised acoustic representation learning under acoustic-embedding memory unit modified space autoencoder (ASAE) and performs the underwater target recognition task. In the manner of the animal-like acoustic auditory system, the first step is to design a self-supervised representation learning method called space autoencoder (SAE) to merge Mel filter-bank (FBank) with the acoustic discrimination and gammatone filter-bank (GBank) with the anti-noise robustness into SAE spectrogram (SAE Spec). Meanwhile, due to poor high-level semantic information in SAE Spec, an acoustic-embedding memory unit (AEMU) is introduced as the strategy of adversarial enhancement. During the auxiliary task, more negative samples are joined in the improved contrastive loss function to obtain adversarial enhanced features called ASAE spectrogram (ASAE Spec). Ultimately, the comprehensive contrast experiments and ablation experiments on two underwater datasets show that ASAE Spec increases by more than 0.96% in accuracy, convergence rate, and anti-noise robustness of other mainstream acoustic features. The results prove the potential value of ASAE in practical applications.
Collapse
Affiliation(s)
- Xingmei Wang
- College of Computer Science and Technology, Harbin Engineering University, Harbin 150001, China
| | - Jiaxiang Meng
- College of Computer Science and Technology, Harbin Engineering University, Harbin 150001, China
| | - Yangtao Liu
- College of Computer Science and Technology, Harbin Engineering University, Harbin 150001, China
| | - Ge Zhan
- College of Computer Science and Technology, Harbin Engineering University, Harbin 150001, China
| | - Zhaonan Tian
- College of Computer Science and Technology, Harbin Engineering University, Harbin 150001, China
| |
Collapse
|
2
|
Chen J, Liu C, Xie J, An J, Huang N. Time-Frequency Mask-Aware Bidirectional LSTM: A Deep Learning Approach for Underwater Acoustic Signal Separation. SENSORS 2022; 22:s22155598. [PMID: 35898099 PMCID: PMC9332702 DOI: 10.3390/s22155598] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/07/2022] [Revised: 07/14/2022] [Accepted: 07/19/2022] [Indexed: 12/07/2022]
Abstract
Underwater acoustic signal separation is a key technique for underwater communications. The existing methods are mostly model-based, and cannot accurately characterize the practical underwater acoustic communication environment. They are only suitable for binary signal separation and cannot handle multivariate signal separation. However, recurrent neural networks (RNNs) show a powerful ability to extract the features of temporal sequences. Inspired by this, in this paper, we present a data-driven approach for underwater acoustic signal separation using deep learning technology. We use a bidirectional long short-term memory (Bi-LSTM) approach to explore the features of a time–frequency (T-F) mask, and propose a T-F-mask-aware Bi-LSTM for signal separation. Taking advantage of the sparseness of the T-F image, the designed Bi-LSTM network is able to extract the discriminative features for separation, which further improves the separation performance. In particular, this method breaks through the limitations of the existing methods and not only achieves good results in multivariate separation but also effectively separates signals when they are mixed with 40 dB Gaussian noise signals. The experimental results show that this method can achieve a 97% guarantee ratio (PSR), and the average similarity coefficient of the multivariate signal separation is stable above 0.8 under high noise conditions. It should be noted that our model can only handle known signals such as test signals for calibration.
Collapse
|
3
|
Han XC, Ren C, Wang L, Bai Y. Underwater acoustic target recognition method based on a joint neural network. PLoS One 2022; 17:e0266425. [PMID: 35486577 PMCID: PMC9053803 DOI: 10.1371/journal.pone.0266425] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2021] [Accepted: 03/20/2022] [Indexed: 12/04/2022] Open
Abstract
To improve the recognition accuracy of underwater acoustic targets by artificial neural network, this study presents a new recognition method that integrates a one-dimensional convolutional neural network and a long short-term memory network. This new network framework is constructed and applied to underwater acoustic target recognition for the first time. Ship acoustic data are used as input to evaluate the network performance. A visual analysis of the recognition results is performed. The results show that this method can realize the recognition and classification of underwater acoustic targets. Compared with a single neural network, the relevant indices, such as the recognition accuracy of the joint network are considerably higher. This provides a new direction for the application of deep learning in the field of underwater acoustic target recognition.
Collapse
Affiliation(s)
- Xing Cheng Han
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, China
- School of Information and Communication Engineering, North University of China, Taiyuan, China
| | - Chenxi Ren
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, China
- School of Information and Communication Engineering, North University of China, Taiyuan, China
| | - Liming Wang
- State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan, China
- School of Information and Communication Engineering, North University of China, Taiyuan, China
| | - Yunjiao Bai
- Department of Mechanics, Jinzhong University, Jinzhong, China
| |
Collapse
|
4
|
Underwater Target Recognition Based on Multi-Decision LOFAR Spectrum Enhancement: A Deep-Learning Approach. FUTURE INTERNET 2021. [DOI: 10.3390/fi13100265] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Underwater target recognition is an important supporting technology for the development of marine resources, which is mainly limited by the purity of feature extraction and the universality of recognition schemes. The low-frequency analysis and recording (LOFAR) spectrum is one of the key features of the underwater target, which can be used for feature extraction. However, the complex underwater environment noise and the extremely low signal-to-noise ratio of the target signal lead to breakpoints in the LOFAR spectrum, which seriously hinders the underwater target recognition. To overcome this issue and to further improve the recognition performance, we adopted a deep-learning approach for underwater target recognition, and a novel LOFAR spectrum enhancement (LSE)-based underwater target-recognition scheme was proposed, which consists of preprocessing, offline training, and online testing. In preprocessing, we specifically design a LOFAR spectrum enhancement based on multi-step decision algorithm to recover the breakpoints in LOFAR spectrum. In offline training, the enhanced LOFAR spectrum is adopted as the input of convolutional neural network (CNN) and a LOFAR-based CNN (LOFAR-CNN) for online recognition is developed. Taking advantage of the powerful capability of CNN in feature extraction, the recognition accuracy can be further improved by the proposed LOFAR-CNN. Finally, extensive simulation results demonstrate that the LOFAR-CNN network can achieve a recognition accuracy of 95.22%, which outperforms the state-of-the-art methods.
Collapse
|
5
|
Liang SHL, Saeedi S, Ojagh S, Honarparvar S, Kiaei S, Mohammadi Jahromi M, Squires J. An Interoperable Architecture for the Internet of COVID-19 Things (IoCT) Using Open Geospatial Standards-Case Study: Workplace Reopening. SENSORS 2020; 21:s21010050. [PMID: 33374208 PMCID: PMC7796058 DOI: 10.3390/s21010050] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/27/2020] [Revised: 12/21/2020] [Accepted: 12/22/2020] [Indexed: 11/29/2022]
Abstract
To safely protect workplaces and the workforce during and after the COVID-19 pandemic, a scalable integrated sensing solution is required in order to offer real-time situational awareness and early warnings for decision-makers. However, an information-based solution for industry reopening is ineffective when the necessary operational information is locked up in disparate real-time data silos. There is a lot of ongoing effort to combat the COVID-19 pandemic using different combinations of low-cost, location-based contact tracing, and sensing technologies. These ad hoc Internet of Things (IoT) solutions for COVID-19 were developed using different data models and protocols without an interoperable way to interconnect these heterogeneous systems and exchange data on people and place interactions. This research aims to design and develop an interoperable Internet of COVID-19 Things (IoCT) architecture that is able to exchange, aggregate, and reuse disparate IoT sensor data sources in order for informed decisions to be made after understanding the real-time risks in workplaces based on person-to-place interactions. The IoCT architecture is based on the Sensor Web paradigm that connects various Things, Sensors, and Datastreams with an indoor geospatial data model. This paper presents a study of what, to the best of our knowledge, is the first real-world integrated implementation of the Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE) and IndoorGML standards to calculate the risk of COVID-19 online using a workplace reopening case study. The proposed IoCT offers a new open standard-based information model, architecture, methodologies, and software tools that enable the interoperability of disparate COVID-19 monitoring systems with finer spatial-temporal granularity. A workplace cleaning use case was developed in order to demonstrate the capabilities of this proposed IoCT architecture. The implemented IoCT architecture included proximity-based contact tracing, people density sensors, a COVID-19 risky behavior monitoring system, and the contextual building geospatial data.
Collapse
Affiliation(s)
- Steve H. L. Liang
- Department of Geomatics Engineering, University of Calgary, Calgary, AB T2N1N4, Canada; (S.O.); (S.H.); (S.K.); (M.M.J.)
- SensorUp Inc., Calgary, AB T2L2K7, Canada;
- Correspondence: (S.H.L.L.); (S.S.); Tel.: +1-403-926-4030 (S.S.)
| | - Sara Saeedi
- Department of Geomatics Engineering, University of Calgary, Calgary, AB T2N1N4, Canada; (S.O.); (S.H.); (S.K.); (M.M.J.)
- Correspondence: (S.H.L.L.); (S.S.); Tel.: +1-403-926-4030 (S.S.)
| | - Soroush Ojagh
- Department of Geomatics Engineering, University of Calgary, Calgary, AB T2N1N4, Canada; (S.O.); (S.H.); (S.K.); (M.M.J.)
| | - Sepehr Honarparvar
- Department of Geomatics Engineering, University of Calgary, Calgary, AB T2N1N4, Canada; (S.O.); (S.H.); (S.K.); (M.M.J.)
| | - Sina Kiaei
- Department of Geomatics Engineering, University of Calgary, Calgary, AB T2N1N4, Canada; (S.O.); (S.H.); (S.K.); (M.M.J.)
| | - Mahnoush Mohammadi Jahromi
- Department of Geomatics Engineering, University of Calgary, Calgary, AB T2N1N4, Canada; (S.O.); (S.H.); (S.K.); (M.M.J.)
| | | |
Collapse
|
6
|
Shen S, Yang H, Yao X, Li J, Xu G, Sheng M. Ship Type Classification by Convolutional Neural Networks with Auditory-like Mechanisms. SENSORS (BASEL, SWITZERLAND) 2020; 20:E253. [PMID: 31906314 PMCID: PMC6983013 DOI: 10.3390/s20010253] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/01/2019] [Revised: 12/17/2019] [Accepted: 12/26/2019] [Indexed: 11/16/2022]
Abstract
Ship type classification with radiated noise helps monitor the noise of shipping around the hydrophone deployment site. This paper introduces a convolutional neural network with several auditory-like mechanisms for ship type classification. The proposed model mainly includes a cochlea model and an auditory center model. In cochlea model, acoustic signal decomposition at basement membrane is implemented by time convolutional layer with auditory filters and dilated convolutions. The transformation of neural patterns at hair cells is modeled by a time frequency conversion layer to extract auditory features. In the auditory center model, auditory features are first selectively emphasized in a supervised manner. Then, spectro-temporal patterns are extracted by deep architecture with multistage auditory mechanisms. The whole model is optimized with an objective function of ship type classification to form the plasticity of the auditory system. The contributions compared with an auditory inspired convolutional neural network include the improvements in dilated convolutions, deep architecture and target layer. The proposed model can extract auditory features from a raw hydrophone signal and identify types of ships under different working conditions. The model achieved a classification accuracy of 87.2% on four ship types and ocean background noise.
Collapse
Affiliation(s)
| | - Honghui Yang
- School of Marine Science and Technology, Northwestern Polytechnical University, Xi’an 710072, China; (S.S.); (X.Y.); (J.L.); (G.X.); (M.S.)
| | | | | | | | | |
Collapse
|
7
|
Auditory Inspired Convolutional Neural Networks for Ship Type Classification with Raw Hydrophone Data. ENTROPY 2018; 20:e20120990. [PMID: 33266713 PMCID: PMC7512589 DOI: 10.3390/e20120990] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/26/2018] [Revised: 12/13/2018] [Accepted: 12/14/2018] [Indexed: 11/24/2022]
Abstract
Detecting and classifying ships based on radiated noise provide practical guidelines for the reduction of underwater noise footprint of shipping. In this paper, the detection and classification are implemented by auditory inspired convolutional neural networks trained from raw underwater acoustic signal. The proposed model includes three parts. The first part is performed by a multi-scale 1D time convolutional layer initialized by auditory filter banks. Signals are decomposed into frequency components by convolution operation. In the second part, the decomposed signals are converted into frequency domain by permute layer and energy pooling layer to form frequency distribution in auditory cortex. Then, 2D frequency convolutional layers are applied to discover spectro-temporal patterns, as well as preserve locality and reduce spectral variations in ship noise. In the third part, the whole model is optimized with an objective function of classification to obtain appropriate auditory filters and feature representations that are correlative with ship categories. The optimization reflects the plasticity of auditory system. Experiments on five ship types and background noise show that the proposed approach achieved an overall classification accuracy of 79.2%, which improved by 6% compared to conventional approaches. Auditory filter banks were adaptive in shape to improve accuracy of classification.
Collapse
|