Transfer learning for efficient classification of grouper sound

Contrastive Learning vs. Self-Learning vs. Deformable Data Augmentation in Semantic Segmentation of Medical Images

To develop a robust segmentation model, encoding the underlying features/structures of the input data is essential to discriminate the target structure from the background. To enrich the extracted feature maps, contrastive learning and self-learning techniques are employed, particularly when the size of the training dataset is limited. In this work, we set out to investigate the impact of contrastive learning and self-learning on the performance of the deep learning-based semantic segmentation. To this end, three different datasets were employed used for brain tumor and hippocampus delineation from MR images (BraTS and Decathlon datasets, respectively) and kidney segmentation from CT images (Decathlon dataset). Since data augmentation techniques are also aimed at enhancing the performance of deep learning methods, a deformable data augmentation technique was proposed and compared with contrastive learning and self-learning frameworks. The segmentation accuracy for the three datasets was assessed with and without applying data augmentation, contrastive learning, and self-learning to individually investigate the impact of these techniques. The self-learning and deformable data augmentation techniques exhibited comparable performance with Dice indices of 0.913 ± 0.030 and 0.920 ± 0.022 for kidney segmentation, 0.890 ± 0.035 and 0.898 ± 0.027 for hippocampus segmentation, and 0.891 ± 0.045 and 0.897 ± 0.040 for lesion segmentation, respectively. These two approaches significantly outperformed the contrastive learning and the original model with Dice indices of 0.871 ± 0.039 and 0.868 ± 0.042 for kidney segmentation, 0.872 ± 0.045 and 0.865 ± 0.048 for hippocampus segmentation, and 0.870 ± 0.049 and 0.860 ± 0.058 for lesion segmentation, respectively. The combination of self-learning with deformable data augmentation led to a robust segmentation model with no outliers in the outcomes. This work demonstrated the beneficial impact of self-learning and deformable data augmentation on organ and lesion segmentation, where no additional training datasets are needed.

Efficient ship noise classification with positive incentive noise and fused features using a simple convolutional network

Ship noise analysis is a critical area of research in hydroacoustic remote sensing due to its practical implications in identifying ship direction, type, and even specific ship identities. However, the limited availability of data poses challenges in developing accurate ship noise classification models. Previous studies have mainly focused on small-sample learning approaches, resulting in complex network structures. Nonetheless, underwater robots often have limited computing power, making it essential to develop simpler recognition networks. In this paper, we address the issue of data scarcity by introducing positive incentive noise. We propose a CNN-based hydroacoustic signal recognition method that achieves comparable or superior performance to previous studies, using a simple network structure as a back-end decision system. We describe the feature extraction process using a dataset with added noise and compare the performance of various features. Additionally, we compare our proposed method with previous studies. Experimental results demonstrate that simple neural networks can achieve high performance and excellent generalizability without the need for complex network structures like adversarial learning models.

Denoising odontocete echolocation clicks using a hybrid model with convolutional neural network and long short-term memory network

Ocean noise negatively influences the recording of odontocete echolocation clicks. In this study, a hybrid model based on the convolutional neural network (CNN) and long short-term memory (LSTM) network-called a hybrid CNN-LSTM model-was proposed to denoise echolocation clicks. To learn the model parameters, the echolocation clicks were partially corrupted by adding ocean noise, and the model was trained to recover the original echolocation clicks. It can be difficult to collect large numbers of echolocation clicks free of ambient sea noise for training networks. Data augmentation and transfer learning were employed to address this problem. Based on Gabor functions, simulated echolocation clicks were generated to pre-train the network models, and the parameters of the networks were then fine-tuned using odontocete echolocation clicks. Finally, the performance of the proposed model was evaluated using synthetic data. The experimental results demonstrated the effectiveness of the proposed model for denoising two typical echolocation clicks-namely, narrowband high-frequency and broadband echolocation clicks. The denoising performance of hybrid models with the different number of convolution and LSTM layers was evaluated. Consequently, hybrid models with one convolutional layer and multiple LSTM layers are recommended, which can be adopted for denoising both types of echolocation clicks.

Environmental Sound Classification Based on Transfer-Learning Techniques with Multiple Optimizers

The last decade has seen increased interest in environmental sound classification (ESC) due to the increased complexity and rich information of ambient sounds. The state-of-the-art methods for ESC are based on transfer learning paradigms that often utilize learned representations from common image-classification problems. This paper aims to determine the effectiveness of employing pre-trained convolutional neural networks (CNNs) for audio categorization and the feasibility of retraining. This study investigated various hyper-parameters and optimizers, such as optimal learning rate, epochs, and Adam, Adamax, and RMSprop optimizers for several pre-trained models, such as Inception, and VGG, ResNet, etc. Firstly, the raw sound signals were transferred into an image format (log-Mel spectrogram). Then, the selected pre-trained models were applied to the obtained spectrogram data. In addition, the effect of essential retraining factors on classification accuracy and processing time was investigated during CNN training. Various optimizers (such as Adam, Adamax, and RMSprop) and hyperparameters were utilized for evaluating the proposed method on the publicly accessible sound dataset UrbanSound8K. The proposed method achieves 97.25% and 95.5% accuracy on the provided dataset using the pre-trained DenseNet201 and the ResNet50V2 CNN models, respectively.

Deep Learning for Drug Discovery: A Study of Identifying High Efficacy Drug Compounds Using a Cascade Transfer Learning Approach

In this research, we applied deep learning to rank the effectiveness of candidate drug compounds in combating viral cells, in particular, SARS-Cov-2 viral cells. For this purpose, two different datasets from Recursion Pharmaceuticals, a siRNA image dataset (RxRx1), which were used to build and calibrate our model for feature extraction, and a SARS-CoV-2 dataset (RxRx19a) was used to train our model for ranking efficacy of candidate drug compounds. The SARS-CoV-2 dataset contained healthy, uninfected control or “mock” cells, as well as “active viral” cells (cells infected with COVID-19), which were the two cell types used to train our deep learning model. In addition, it contains viral cells treated with different drug compounds, which were the cells not used to train but test our model. We devised a new cascade transfer learning strategy to construct our model. We first trained a deep learning model, the DenseNet, with the siRNA set, a dataset with characteristics similar to the SARS-CoV-2 dataset, for feature extraction. We then added additional layers, including a SoftMax layer as an output layer, and retrained the model with active viral cells and mock cells from the SARS-CoV-2 dataset. In the test phase, the SoftMax layer outputs probability (equivalently, efficacy) scores which allows us to rank candidate compounds, and to study the performance of each candidate compound statistically. With this approach, we identified several compounds with high efficacy scores which are promising for the therapeutic treatment of COVID-19. The compounds showing the most promise were GS-441524 and then Remdesivir, which overlapped with these reported in the literature and with these drugs that are approved by FDA, or going through clinical trials and preclinical trials. This study shows the potential of deep learning in its ability to identify promising compounds to aid rapid responses to future pandemic outbreaks.

Transfer learning for denoising the echolocation clicks of finless porpoise (Neophocaena phocaenoides sunameri) using deep convolutional autoencoders

Ocean noise has a negative impact on the acoustic recordings of odontocetes' echolocation clicks. In this study, deep convolutional autoencoders (DCAEs) are presented to denoise the echolocation clicks of the finless porpoise (Neophocaena phocaenoides sunameri). A DCAE consists of an encoder network and a decoder network. The encoder network is composed of convolutional layers and fully connected layers, whereas the decoder network consists of fully connected layers and transposed convolutional layers. The training scheme of the denoising autoencoder was applied to learn the DCAE parameters. In addition, transfer learning was employed to address the difficulty in collecting a large number of echolocation clicks that are free of ambient sea noise. Gabor functions were used to generate simulated clicks to pretrain the DCAEs; subsequently, the parameters of the DCAEs were fine-tuned using the echolocation clicks of the finless porpoise. The experimental results showed that a DCAE pretrained with simulated clicks achieved better denoising results than a DCAE trained only with echolocation clicks. Moreover, deep fully convolutional autoencoders, which are special DCAEs that do not contain fully connected layers, generally achieved better performance than the DCAEs that contain fully connected layers.