A Novel Approach to Large-Scale Dynamically Weighted Directed Network Representation

Context-Aware Poly(A) Signal Prediction Model via Deep Spatial-Temporal Neural Networks

Polyadenylation [Poly(A)] is an essential process during messenger RNA (mRNA) maturation in biological eukaryote systems. Identifying Poly(A) signals (PASs) from the genome level is the key to understanding the mechanism of translation regulation and mRNA metabolism. In this work, we propose a deep dual-dynamic context-aware Poly(A) signal prediction model, called multiscale convolution with self-attention networks (MCANet), to adaptively uncover the spatial-temporal contextual dependence information. Specifically, the model automatically learns and strengthens informative features from the temporalwise and the spatialwise dimension. The identity connectivity performs contextual feature maps of Poly(A) data by direct connections from previous layers to subsequent layers. Then, a fully parametric rectified linear unit (FP-RELU) with dual-dynamic coefficients is devised to make the training of the model easier and enhance the generalization ability. A cross-entropy loss (CL) function is designed to make the model focus on samples that are easy to misclassify. Experiments on different Poly(A) signals demonstrate the superior performance of the proposed MCANet, and an ablation study shows the effectiveness of the network design for the feature learning and prediction of Poly(A) signals.

Joint Anchor Graph Embedding and Discrete Feature Scoring for Unsupervised Feature Selection

The success of existing unsupervised feature selection (UFS) methods heavily relies on the assumption that the intrinsic relationships among original high-dimensional (HD) data samples exist in the discriminative low-dimension (LD) subspace. However, previous UFS methods commonly construct pairwise graphs and employ l2,1 -norm regularization to severally preserve the local structure and calculate the score of features, which is computationally complex and easy to get stuck into local optimum, so that those approaches cannot be applied in dealing with large-scale datasets in practice. To overcome this challenge, we propose a novel UFS method, in which a novel anchor graph embedding paradigm is designed to extract the local neighborhood relationships among data samples by reducing the computational complexity of graph construction to be linear in the number of data. Moreover, to improve the optimality of selected features as well as the performance of downstream tasks, we propose a discrete feature scoring mechanism, which imposes orthogonal l2,0 -norm constraints on learned projections, in order to enhance the distinction of feature scores as well as reduce the probability of falling into local optimum. In addition, solving the proposed nonconvex and nonsmooth NP-hard problem is challenging, and we present an efficient optimization algorithm to address it and acquire a closed-form solution of the transformation matrix. Extensive experiments demonstrate the effectiveness and efficiency of the proposed UFS by comparison with several state-of-the-art approaches to clustering and image segmentation tasks.

Differentiable channel pruning guided via attention mechanism: a novel neural network pruning approach

Neural network pruning offers great prospects for facilitating the deployment of deep neural networks on computational resource limited devices. Neural architecture search (NAS) provides an efficient way to automatically seek appropriate neural architecture design for compressed model. It is observed that, for existing NAS-based pruning methods, there is usually a lack of layer information when searching the optimal neural architecture. In this paper, we propose a new NAS approach, namely, differentiable channel pruning method guided via attention mechanism (DCP-A), where the adopted attention mechanism is able to provide layer information to guide the optimization of the pruning policy. The training process is differentiable with Gumbel-softmax sampling, while parameters are optimized under a two-stage training procedure. The neural network block with the shortcut is dedicatedly designed, which is of help to prune the network not only on its width but also on its depth. Extensive experiments are performed to verify the applicability and superiority of the proposed method. Detailed analysis with visualization of the pruned model architecture shows that our proposed DCP-A learns explainable pruning policies.

Metric Learning-Based Fault Diagnosis and Anomaly Detection for Industrial Data With Intraclass Variance

Industrial system monitoring includes fault diagnosis and anomaly detection, which have received extensive attention, since they can recognize the fault types and detect unknown anomalies. However, a separate fault diagnosis method or anomaly detection method cannot identify unknown faults and distinguish between different fault types simultaneously; thus, it is difficult to meet the increasing demand for safety and reliability of industrial systems. Besides, the actual system often operates in varying working conditions and is disturbed by the noise, which results in the intraclass variance of the raw data and degrades the performance of industrial system monitoring. To solve these problems, a metric learning-based fault diagnosis and anomaly detection method is proposed. Fault diagnosis and anomaly detection are adaptively fused in the proposed end-to-end model, where anomaly detection can prevent the model from misjudging the unknown anomaly as the known type, while fault diagnosis can identify the specific type of system fault. In addition, a novel multicenter loss is introduced to restrain the intraclass variance. Compared with manual feature extraction that can only extract suboptimal features, it can learn discriminant features automatically for both fault diagnosis and anomaly detection tasks. Experiments on three-phase flow (TPF) facility and Case Western Reserve University (CWRU) bearing have demonstrated that the proposed method can avoid the interference of intraclass variances and learn features that are effective for identifying tasks. Moreover, it achieves the best performance in both fault diagnosis and anomaly detection.

Iterative Named Entity Recognition with Conditional Random Fields

Named entity recognition (NER) constitutes an important step in the processing of unstructured text content for the extraction of information as well as for the computer-supported analysis of large amounts of digital data via machine learning methods. However, NER often relies on domain-specific knowledge, being conducted manually in a time- and human-resource-intensive process. These can be reduced with statistical models performing NER automatically. The current work investigates whether Conditional Random Fields (CRF) can be efficiently trained for NER in German texts, by means of an iterative procedure combining self-learning with a manual annotation–active learning–component. The training dataset increases continuously with the iterative procedure. Whilst self-learning did not markedly improve the performance of the CRF for NER, the manual annotation of sentences with the lowest probability of correct prediction clearly improved the model F1-score and simultaneously reduced the amount of manual annotation required to train the model. A model with an F1-score of 0.885 was able to be trained in 11.4 h.