Learning Domain-Independent Deep Representations by Mutual Information Minimization

Fault Detection and Diagnosis for Plasticizing Process of Single-Base Gun Propellant Using Mutual Information Weighted MPCA under Limited Batch Samples Modelling

Aiming at the lack of reliable gradual fault detection and abnormal condition alarm and evaluation ability in the plasticizing process of single-base gun propellant, a fault detection and diagnosis method based on normalized mutual information weighted multiway principal component analysis (NMI-WMPCA) under limited batch samples modelling was proposed. In this method, the differences of coupling correlation among multi-dimensional process variables and the coupling characteristics of linear and nonlinear relationships in the process are considered. NMI-WMPCA utilizes the generalization ability of a multi-model to establish an accurate fault detection model in limited batch samples, and adopts fault diagnosis methods based on a multi-model SPE statistic contribution plot to identify the fault source. The experimental results demonstrate that the proposed method is effective, which can realize the rapid detection and diagnosis of multiple faults in the plasticizing process.

Anti-transfer learning for task invariance in convolutional neural networks for speech processing

We introduce the novel concept of anti-transfer learning for speech processing with convolutional neural networks. While transfer learning assumes that the learning process for a target task will benefit from re-using representations learned for another task, anti-transfer avoids the learning of representations that have been learned for an orthogonal task, i.e., one that is not relevant and potentially confounding for the target task, such as speaker identity for speech recognition or speech content for emotion recognition. This extends the potential use of pre-trained models that have become increasingly available. In anti-transfer learning, we penalize similarity between activations of a network being trained on a target task and another one previously trained on an orthogonal task, which yields more suitable representations. This leads to better generalization and provides a degree of control over correlations that are spurious or undesirable, e.g. to avoid social bias. We have implemented anti-transfer for convolutional neural networks in different configurations with several similarity metrics and aggregation functions, which we evaluate and analyze with several speech and audio tasks and settings, using six datasets. We show that anti-transfer actually leads to the intended invariance to the orthogonal task and to more appropriate features for the target task at hand. Anti-transfer learning consistently improves classification accuracy in all test cases. While anti-transfer creates computation and memory cost at training time, there is relatively little computation cost when using pre-trained models for orthogonal tasks. Anti-transfer is widely applicable and particularly useful where a specific invariance is desirable or where labeled data for orthogonal tasks are difficult to obtain on a given dataset but pre-trained models are available.