Decision trees for hierarchical multi-label classification

Comparative Analysis of Deterministic and Nondeterministic Decision Trees for Decision Tables from Closed Classes

In this paper, we consider classes of decision tables with many-valued decisions closed under operations of the removal of columns, the changing of decisions, the permutation of columns, and the duplication of columns. We study relationships among three parameters of these tables: the complexity of a decision table (if we consider the depth of the decision trees, then the complexity of a decision table is the number of columns in it), the minimum complexity of a deterministic decision tree, and the minimum complexity of a nondeterministic decision tree. We consider the rough classification of functions characterizing relationships and enumerate all possible seven types of relationships.

Enhancing safety of construction workers in Korea: an integrated text mining and machine learning framework for predicting accident types

Construction workers face a high risk of various occupational accidents, many of which can result in fatalities. This study aims to develop a prediction model for nine prevalent types of construction accidents, utilizing construction tasks, activities, and tools/materials as input features, through the application of machine learning-based multi-class classification algorithms. 152,867 construction accident summary reports, composed of both structured (construction task, construction activity, accident type) and unstructured data (tools/materials) were used for the study. The study employed several data processing techniques, including keyword extraction through text mining, Boruta feature selection, and SMOTE data resampling enhance model accuracy. Three performance metrics (Multi-class area under the receiver operating characteristic curve (MAUC), Multi-class Matthews Correlation Coefficient (MMCC), Geometric-mean (G-mean)) were used to compare the predictive performance of four machine learning algorithms, including Decision tree, Random forest, Naïve bayes, and XGBoost. Of the four algorithms, XGBoost showed the highest performance in predicting accident type (MAUC: 0.8603, MMCC: 0.3523, G-mean: 0.5009). Furthermore, a Shapley additive explanation (SHAP) analysis was conducted to visualize feature importance. The findings of this study make a valuable contribution to improving construction safety by presenting a prediction model for accident types derived from real-world big data.

Multiple marine algae identification based on three-dimensional fluorescence spectroscopy and multi-label convolutional neural network

Accurate identification of algal populations plays a pivotal role in monitoring seawater quality. Fluorescence-based techniques are effective tools for quickly identifying different algae. However, multiple coexisting algae and their similar photosynthetic pigments can constrain the efficacy of fluorescence methods. This study introduces a multi-label classification model that combines a specific Excitation-Emission matric convolutional neural network (EEM-CNN) with three-dimensional (3D) fluorescence spectroscopy to detect single and mixed algal samples. Spectral data can be input directly into the model without transforming into images. Rectangular convolutional kernels and double convolutional layers are applied to enhance the extraction of balanced and comprehensive spectral features for accurate classification. A dataset comprising 3D fluorescence spectra from eight distinct algae species representing six different algal classes was obtained, preprocessed, and augmented to create input data for the classification model. The classification model was trained and validated using 4448 sets of test samples and 60 sets of test samples, resulting in an accuracy of 0.883 and an F1 score of 0.925. This model exhibited the highest recognition accuracy in both single and mixed algae samples, outperforming comparative methods such as ML-kNN and N-PLS-DA. Furthermore, the classification results were extended to three different algae species and mixed samples of skeletonema costatum to assess the impact of spectral similarity on multi-label classification performance. The developed classification models demonstrated robust performance across samples with varying concentrations and growth stages, highlighting CNN's potential as a promising tool for the precise identification of marine algae.

Partial label learning for automated classification of single-cell transcriptomic profiles

Single-cell RNA sequencing (scRNASeq) data plays a major role in advancing our understanding of developmental biology. An important current question is how to classify transcriptomic profiles obtained from scRNASeq experiments into the various cell types and identify the lineage relationship for individual cells. Because of the fast accumulation of datasets and the high dimensionality of the data, it has become challenging to explore and annotate single-cell transcriptomic profiles by hand. To overcome this challenge, automated classification methods are needed. Classical approaches rely on supervised training datasets. However, due to the difficulty of obtaining data annotated at single-cell resolution, we propose instead to take advantage of partial annotations. The partial label learning framework assumes that we can obtain a set of candidate labels containing the correct one for each data point, a simpler setting than requiring a fully supervised training dataset. We study and extend when needed state-of-the-art multi-class classification methods, such as SVM, kNN, prototype-based, logistic regression and ensemble methods, to the partial label learning framework. Moreover, we study the effect of incorporating the structure of the label set into the methods. We focus particularly on the hierarchical structure of the labels, as commonly observed in developmental processes. We show, on simulated and real datasets, that these extensions enable to learn from partially labeled data, and perform predictions with high accuracy, particularly with a nonlinear prototype-based method. We demonstrate that the performances of our methods trained with partially annotated data reach the same performance as fully supervised data. Finally, we study the level of uncertainty present in the partially annotated data, and derive some prescriptive results on the effect of this uncertainty on the accuracy of the partial label learning methods. Overall our findings show how hierarchical and non-hierarchical partial label learning strategies can help solve the problem of automated classification of single-cell transcriptomic profiles, interestingly these methods rely on a much less stringent type of annotated datasets compared to fully supervised learning methods.

Online Passive-Aggressive Multilabel Classification Algorithms

Most existing multilabel classification methods are batch learning methods, which may suffer from expensive retraining costs when dealing with new incoming data. In order to overcome the drawbacks of batch learning, we develop a family of online multilabel classification algorithms, which can update the model instantly and efficiently, and make a timely online prediction when new data arrive. Our algorithms all take a closed-form update, which is obtained by solving a constrained optimization problem in each round of online learning. Label correlation is explicitly modeled in our optimization problem. The label thresholding function, an important component of our online classifier, can also be learned online. Our algorithms can be easily generalized to the nonlinear prediction cases using Mercer kernels. The worst case loss bounds for our algorithms are provided. The bounds are relative to the cumulative loss suffered by the best fixed predictive model that can be attained in hindsight. Finally, we corroborate the merits of our algorithms in both linear and nonlinear predictions on nine open multilabel benchmark datasets.

Xerophilic fungi contaminating historically valuable easel paintings from Slovenia

Historically valuable canvas paintings are often exposed to conditions enabling microbial deterioration. Painting materials, mainly of organic origin, in combination with high humidity and other environmental conditions, favor microbial metabolism and growth. These preconditions are often present during exhibitions or storage in old buildings, such as churches and castles, and also in museum storage depositories. The accumulated dust serves as an inoculum for both indoor and outdoor fungi. In our study, we present the results on cultivable fungi isolated from 24 canvas paintings, mainly exhibited in Slovenian sacral buildings, dating from the 16th to 21st centuries. Fungi were isolated from the front and back of damaged and undamaged surfaces of the paintings using culture media with high- and low-water activity. A total of 465 isolates were identified using current taxonomic DNA markers and assigned to 37 genera and 98 species. The most abundant genus was Aspergillus, represented by 32 species, of which 9 xerophilic species are for the first time mentioned in contaminated paintings. In addition to the most abundant xerophilic A. vitricola, A. destruens, A. tardicrescens, and A. magnivesiculatus, xerophilic Wallemia muriae and W. canadensis, xerotolerant Penicillium chrysogenum, P. brevicompactum, P. corylophilum, and xerotolerant Cladosporium species were most frequent. When machine learning methods were used to predict the relationship between fungal contamination, damage to the painting, and the type of material present, proteins were identified as one of the most important factors and cracked paint was identified as a hotspot for fungal growth. Aspergillus species colonize paintings regardless of materials, while Wallemia spp. can be associated with animal fat. Culture media with low-water activity are suggested in such inventories to isolate and obtain an overview of fungi that are actively contaminating paintings stored indoors at low relative humidity.

Decision trees: from efficient prediction to responsible AI

This article provides a birds-eye view on the role of decision trees in machine learning and data science over roughly four decades. It sketches the evolution of decision tree research over the years, describes the broader context in which the research is situated, and summarizes strengths and weaknesses of decision trees in this context. The main goal of the article is to clarify the broad relevance to machine learning and artificial intelligence, both practical and theoretical, that decision trees still have today.

Applications of Depth Minimization of Decision Trees Containing Hypotheses for Multiple-Value Decision Tables

In this research, we consider decision trees that incorporate standard queries with one feature per query as well as hypotheses consisting of all features' values. These decision trees are used to represent knowledge and are comparable to those investigated in exact learning, in which membership queries and equivalence queries are used. As an application, we look into the issue of creating decision trees for two cases: the sorting of a sequence that contains equal elements and multiple-value decision tables which are modified from UCI Machine Learning Repository. We contrast the efficiency of several forms of optimal (considering the parameter depth) decision trees with hypotheses for the aforementioned applications. We also investigate the efficiency of decision trees built by dynamic programming and by an entropy-based greedy method. We discovered that the greedy algorithm produces very similar results compared to the results of dynamic programming algorithms. Therefore, since the dynamic programming algorithms take a long time, we may readily apply the greedy algorithms.

IPC prediction of patent documents using neural network with attention for hierarchical structure

International patent classifications (IPCs) are assigned to patent documents; however, since the procedure for assigning classifications is manually done by the patent examiner, it takes a lot of time and effort to select some IPCs from about 70,000 IPCs. Hence, some research has been conducted on patent classification with machine learning. However, patent documents are very voluminous, and learning with all the claims (the part describing the content of the patent) as input would run out of the necessary memory, even if the batch size is set to a very small size. Therefore, most of the existing methods learn by excluding some information, such as using only the first claim as input. In this study, we propose a model that considers the contents of all claims by extracting important information for input. In addition, we focus on the hierarchical structure of the IPC, and propose a new decoder architecture to consider it. Finally, we conducted an experiment using actual patent data to verify the accuracy of the prediction. The results showed a significant improvement in accuracy compared to existing methods, and the actual applicability of the method was also discussed.

Semantic Tagging for the Urdu Language: Annotated Corpus and Multi-Target Classification Methods

Extracting and analysing meaning-related information from natural language data has attracted the attention of researchers in various fields, such as natural language processing, corpus linguistics, information retrieval, and data science. An important aspect of such automatic information extraction and analysis is the annotation of language data using semantic tagging tools. Different semantic tagging tools have been designed to carry out various levels of semantic analysis, for instance, named entity recognition and disambiguation, sentiment analysis, word sense disambiguation, content analysis, and semantic role labelling. Common to all of these tasks, in the supervised setting, is the requirement for a manually semantically annotated corpus, which acts as a knowledge base from which to train and test potential word and phrase-level sense annotations. Many benchmark corpora have been developed for various semantic tagging tasks, but most are for English and other European languages. There is a dearth of semantically annotated corpora for the Urdu language, which is widely spoken and used around the world. To fill this gap, this study presents a large benchmark corpus and methods for the semantic tagging task for the Urdu language. The proposed corpus contains 8,000 tokens in the following domains or genres: news, social media, Wikipedia, and historical text (each domain having 2K tokens). The corpus has been manually annotated with 21 major semantic fields and 232 sub-fields with the USAS (UCREL Semantic Analysis System) semantic taxonomy which provides a comprehensive set of semantic fields for coarse-grained annotation. Each word in our proposed corpus has been annotated with at least one and up to nine semantic field tags to provide a detailed semantic analysis of the language data, which allowed us to treat the problem of semantic tagging as a supervised multi-target classification task. To demonstrate how our proposed corpus can be used for the development and evaluation of Urdu semantic tagging methods, we extracted local, topical and semantic features from the proposed corpus and applied seven different supervised multi-target classifiers to them. Results show an accuracy of 94% on our proposed corpus which is free and publicly available to download.

Deep learning based classification of multi-label chest X-ray images via dual-weighted metric loss

-Thoracic disease, like many other diseases, can lead to complications. Existing multi-label medical image learning problems typically include rich pathological information, such as images, attributes, and labels, which are crucial for supplementary clinical diagnosis. However, the majority of contemporary efforts exclusively focus on regression from input to binary labels, ignoring the relationship between visual features and semantic vectors of labels. In addition, there is an imbalance in data amount between diseases, which frequently causes intelligent diagnostic systems to make erroneous disease predictions. Therefore, we aim to improve the accuracy of the multi-label classification of chest X-ray images. Chest X-ray14 pictures were utilized as the multi-label dataset for the experiments in this study. By fine-tuning the ConvNeXt network, we got visual vectors, which we combined with semantic vectors encoded by BioBert to map the two different forms of features into a common metric space and made semantic vectors the prototype of each class in metric space. The metric relationship between images and labels is then considered from the image level and disease category level, respectively, and a new dual-weighted metric loss function is proposed. Finally, the average AUC score achieved in the experiment reached 0.826, and our model outperformed the comparison models.

Leveraging class hierarchy for detecting missing annotations on hierarchical multi-label classification

With the development of new sequencing technologies, availability of genomic data has grown exponentially. Over the past decade, numerous studies have used genomic data to identify associations between genes and biological functions. While these studies have shown success in annotating genes with functions, they often assume that genes are completely annotated and fail to take into account that datasets are sparse and noisy. This work proposes a method to detect missing annotations in the context of hierarchical multi-label classification. More precisely, our method exploits the relations of functions, represented as a hierarchy, by computing probabilities based on the paths of functions in the hierarchy. By performing several experiments on a variety of rice (Oriza sativa Japonica), we showcase that the proposed method accurately detects missing annotations and yields superior results when compared to state-of-art methods from the literature.

Determining Exception Context in Assembly Operations from Multimodal Data

Robot assembly tasks can fail due to unpredictable errors and can only continue with the manual intervention of a human operator. Recently, we proposed an exception strategy learning framework based on statistical learning and context determination, which can successfully resolve such situations. This paper deals with context determination from multimodal data, which is the key component of our framework. We propose a novel approach to generate unified low-dimensional context descriptions based on image and force-torque data. For this purpose, we combine a state-of-the-art neural network model for image segmentation and contact point estimation using force-torque measurements. An ensemble of decision trees is used to combine features from the two modalities. To validate the proposed approach, we have collected datasets of deliberately induced insertion failures both for the classic peg-in-hole insertion task and for an industrially relevant task of car starter assembly. We demonstrate that the proposed approach generates reliable low-dimensional descriptors, suitable as queries necessary in statistical learning.

Improving Node Classification through Convolutional Networks Built on Enhanced Message-Passing Graph

Enhancing message propagation is critical for solving the problem of node classification in sparse graph with few labels. The recently popularized Graph Convolutional Network (GCN) lacks the ability to propagate messages effectively to distant nodes because of over-smoothing. Besides, the GCN with numerous trainable parameters suffers from overfitting when the labeled nodes are scarce. This article addresses the problem via building GCN on Enhanced Message-Passing Graph (EMPG). The key idea is that node classification can benefit from various variants of the input graph that can propagate messages more efficiently, based on the assumption that the structure of each variant is reasonable when more unlabeled nodes are labeled properly. Specifically, the proposed method first maps the nodes to a latent space through graph embedding that captures the structural information of the input graph. Considering the node attributes together, the proposed method constructs the EMPG by adding connections between the nodes in close proximity in the latent space. With the help of the added connections, the EMPG allows a node to propagate its message to the right nodes at long distances, so that the GCN built on the EMPG need not stack multiple layers. As a result, over-smoothing is avoided. However, dense connections may cause message propagation saturation and lead to overfitting. Seeing the EMPG as an accumulation of some potential variants of the original graph, the proposed method utilizes dropout to extract a group of variants from the EMPG and then builds multichannel GCNs on them. The multichannel features learned from different dropout EMPGs are aggregated to compute the final prediction jointly. The proposed method is flexible, as a brod range of GCNs can be incorporated easily. Additionally, it is efficient and robust. Experimental results demonstrate that the proposed method yields improvements in node classification.

MLI: A multi-level inference mechanism for user attributes in social networks

In the social network, each user has attributes for self-description called user attributes which are semantically hierarchical. Attribute inference has become an essential way for social platforms to realize user classifications and targeted recommendations. Most existing approaches mainly focus on the flat inference problem neglecting the semantic hierarchy of user attributes which will cause serious inconsistency in multi-level tasks. In this article, we propose a multi-level model MLI, where information propagation part collects attribute information by mining the global graph structure, and the attribute correction part realizes the mutual correction between different levels of attributes. Further, we put forward the concept of generalized semantic tree, a way of representing the hierarchical structure of user attributes, whose nodes are allowed to have multiple parent nodes unlike the regular tree. Both regular and generalized semantic tree are commonly used in practice, and can be handled by our model. Besides, by making the inference start from sub-networks with sufficient attribute information, we design a “Ripple” algorithm to improve the efficiency and effectiveness of our model. For evaluation purposes, we conduct extensive verification experiments on DBLP datasets. The experimental results show the superior effect of MLI, compared with the state-of-the-art methods.

Feature ranking for semi-supervised learning

The data used for analysis are becoming increasingly complex along several directions: high dimensionality, number of examples and availability of labels for the examples. This poses a variety of challenges for the existing machine learning methods, related to analyzing datasets with a large number of examples that are described in a high-dimensional space, where not all examples have labels provided. For example, when investigating the toxicity of chemical compounds, there are many compounds available that can be described with information-rich high-dimensional representations, but not all of the compounds have information on their toxicity. To address these challenges, we propose methods for semi-supervised learning (SSL) of feature rankings. The feature rankings are learned in the context of classification and regression, as well as in the context of structured output prediction (multi-label classification, MLC, hierarchical multi-label classification, HMLC and multi-target regression, MTR) tasks. This is the first work that treats the task of feature ranking uniformly across various tasks of semi-supervised structured output prediction. To the best of our knowledge, it is also the first work on SSL of feature rankings for the tasks of HMLC and MTR. More specifically, we propose two approaches—based on predictive clustering tree ensembles and the Relief family of algorithms—and evaluate their performance across 38 benchmark datasets. The extensive evaluation reveals that rankings based on Random Forest ensembles perform the best for classification tasks (incl. MLC and HMLC tasks) and are the fastest for all tasks, while ensembles based on extremely randomized trees work best for the regression tasks. Semi-supervised feature rankings outperform their supervised counterparts across the majority of datasets for all of the different tasks, showing the benefit of using unlabeled in addition to labeled data.

Probabilistic risk analysis for coal mine gas overrun based on FAHP and BN: a case study

To analyze the risk of gas overrun in coal mines and improve the risk analysis, a novel risk analysis method was proposed based on FAHP and Bayesian network. The risk analysis framework consisted of causal reasoning, logical reasoning, and sensitivity analysis. The gas overrun risk analysis was conducted by taking the Laohutai Coal Mine in China as the research object. Specifically, based on prior knowledge and sample data, the probability of the gas overrun was 3.2%, belonging to a small probability event. However, the probability of gas concentration exceeding 1% was 12%, and there was still potential danger. Logical reasoning diagnosed and identified that wind speed and air leakage were the direct causes of gas overrun. Sensitivity analysis indicated that wind speed, human error, and ground stress were key factors of the gas overrun. The case study showed this fuzzy analytic hierarchy process (FAHP)-Bayesian network (BN)-based risk analysis method can provide real-time and dynamic decision support for gas overrun control and treatment in coal mines to ensure the safe and efficient mining.

Tree-based dynamic classifier chains

Classifier chains are an effective technique for modeling label dependencies in multi-label classification. However, the method requires a fixed, static order of the labels. While in theory, any order is sufficient, in practice, this order has a substantial impact on the quality of the final prediction. Dynamic classifier chains denote the idea that for each instance to classify, the order in which the labels are predicted is dynamically chosen. The complexity of a naïve implementation of such an approach is prohibitive, because it would require to train a sequence of classifiers for every possible permutation of the labels. To tackle this problem efficiently, we propose a new approach based on random decision trees which can dynamically select the label ordering for each prediction. We show empirically that a dynamic selection of the next label improves over the use of a static ordering under an otherwise unchanged random decision tree model. In addition, we also demonstrate an alternative approach based on extreme gradient boosted trees, which allows for a more target-oriented training of dynamic classifier chains. Our results show that this variant outperforms random decision trees and other tree-based multi-label classification methods. More importantly, the dynamic selection strategy allows to considerably speed up training and prediction.

TCMPR: TCM Prescription Recommendation Based on Subnetwork Term Mapping and Deep Learning

Traditional Chinese medicine (TCM) has played an indispensable role in clinical diagnosis and treatment. Based on a patient’s symptom phenotypes, computation-based prescription recommendation methods can recommend personalized TCM prescription using machine learning and artificial intelligence technologies. However, owing to the complexity and individuation of a patient’s clinical phenotypes, current prescription recommendation methods cannot obtain good performance. Meanwhile, it is very difficult to conduct effective representation for unrecorded symptom terms in an existing knowledge base. In this study, we proposed a subnetwork-based symptom term mapping method (SSTM) and constructed a SSTM-based TCM prescription recommendation method (termed TCMPR). Our SSTM can extract the subnetwork structure between symptoms from a knowledge network to effectively represent the embedding features of clinical symptom terms (especially the unrecorded terms). The experimental results showed that our method performs better than state-of-the-art methods. In addition, the comprehensive experiments of TCMPR with different hyperparameters (i.e., feature embedding, feature dimension, subnetwork filter threshold, and feature fusion) demonstrate that our method has high performance on TCM prescription recommendation and potentially promote clinical diagnosis and treatment of TCM precision medicine.

Exploration of Potential miRNA Biomarkers and Prediction for Ovarian Cancer Using Artificial Intelligence

Ovarian cancer is the second most dangerous gynecologic cancer with a high mortality rate. The classification of gene expression data from high-dimensional and small-sample gene expression data is a challenging task. The discovery of miRNAs, a small non-coding RNA with 18–25 nucleotides in length that regulates gene expression, has revealed the existence of a new array for regulation of genes and has been reported as playing a serious role in cancer. By using LASSO and Elastic Net as embedded algorithms of feature selection techniques, the present study identified 10 miRNAs that were regulated in ovarian serum cancer samples compared to non-cancer samples in public available dataset GSE106817: hsa-miR-5100, hsa-miR-6800-5p, hsa-miR-1233-5p, hsa-miR-4532, hsa-miR-4783-3p, hsa-miR-4787-3p, hsa-miR-1228-5p, hsa-miR-1290, hsa-miR-3184-5p, and hsa-miR-320b. Further, we implemented state-of-the-art machine learning classifiers, such as logistic regression, random forest, artificial neural network, XGBoost, and decision trees to build clinical prediction models. Next, the diagnostic performance of these models with identified miRNAs was evaluated in the internal (GSE106817) and external validation dataset (GSE113486) by ROC analysis. The results showed that first four prediction models consistently yielded an AUC of 100%. Our findings provide significant evidence that the serum miRNA profile represents a promising diagnostic biomarker for ovarian cancer.

Automated Diagnostics: Advances in the Diagnosis of Intestinal Parasitic Infections in Humans and Animals

The increasingly close proximity between people and animals is of great concern for public health, given the risk of exposure to infectious diseases transmitted through animals, which are carriers of more than 60 zoonotic agents. These diseases, which are included in the list of Neglected Tropical Diseases, cause losses in countries with tropical and subtropical climates, and in regions with temperate climates. Indeed, they affect more than a billion people around the world, a large proportion of which are infected by one or more parasitic helminths, causing annual losses of billions of dollars. Several studies are being conducted in search for differentiated, more sensitive diagnostics with fewer errors. These studies, which involve the automated examination of intestinal parasites, still face challenges that must be overcome in order to ensure the proper identification of parasites. This includes a protocol that allows for elimination of most of the debris in samples, satisfactory staining of parasite structures, and a robust image database. Our objective here is therefore to offer a critical description of the techniques currently in use for the automated diagnosis of intestinal parasites in fecal samples, as well as advances in these techniques.

VinegarScan: A Computer Tool Based on Ultraviolet Spectroscopy for a Rapid Authentication of Wine Vinegars

Ultraviolet-visible (UV-vis) spectroscopy has shown successful results in the last few years to characterize and classify wine vinegar according to its quality, particularly those with a protected designation of origin (PDO). Due to these promising results, together with the simplicity, price, speed, portability of this technique and its ability to create robust hierarchical classification models, the objective of this work was the development of a computer tool or software, named VinegarScan, which uses the UV-vis spectra to be able to perform quality control and authentication of wine vinegar in a quick and user-friendly way. This software was based on the open-source GUI created in C++ using several data mining algorithms (e.g., decision trees, classification algorithms) on UV-vis spectra. This software achieved satisfactory prediction results with the available analytical UV-vis data. The future idea of utility is to combine the VinegarScan tool with a portable UV-vis device that could be used by control bodies of the wine vinegar industry to achieve a clear differentiation from their competitors to avoid fraud.

Real-Time Hierarchical Classification of Time Series Data for Locomotion Mode Detection

OBJECTIVE

Accurate real-time estimation of motion intent is critical for rendering useful assistance using wearable robotic prosthetic and exoskeleton devices during user-initiated motions. We aim to evaluate hierarchical classification as a strategy for real-time locomotion mode recognition for the control of wearable robotic prosthetics and exoskeletons during user-intiated motions.

METHODS

We collect motion data from 8 subjects using a set of 7 inertial sensors for 16 lower limb locomotion modes of different specificities. A CNN based hierarchical classifier is trained to classify the modes into a specified label hierarchy. We measure the accuracy, stability, behaviour during mode transitions and suitability for real-time inference of the classifier.

RESULTS

The method achieves stable classification of locomotion modes using 1280 ms of time history data. It achieves average classification accuracy of 94.34% and an average AU(PRC) of 0.773 - comparable to similar classifiers. The method produces more informative classifications at transitions between modes. Less specific classes are classified earlier than more specific classes in the hierarchy. The inference step of the classifier can be executed in less than 2 ms on embedded hardware, indicating suitability for real-time operation.

CONCLUSION

Hierarchical classification can achieve accurate detection of locomotion modes and can break up mode transitions into multiple transitions between modes of different specificity.

SIGNIFICANCE

Multi-specific hierarchical classification of locomotion modes could lead to smoother, more fine grained control adaptation of wearable robots during locomotion mode transitions.

Classification of Ataxic Gait

Gait disorders accompany a number of neurological and musculoskeletal disorders that significantly reduce the quality of life. Motion sensors enable high-quality modelling of gait stereotypes. However, they produce large volumes of data, the evaluation of which is a challenge. In this publication, we compare different data reduction methods and classification of reduced data for use in clinical practice. The best accuracy achieved between a group of healthy individuals and patients with ataxic gait extracted from the records of 43 participants (23 ataxic, 20 healthy), forming 418 segments of straight gait pattern, is 98% by random forest classifier preprocessed by t-distributed stochastic neighbour embedding.

Long-term Cognitive Network-based architecture for multi-label classification

This paper presents a neural system to deal with multi-label classification problems that might involve sparse features. The architecture of this model involves three sequential blocks with well-defined functions. The first block consists of a multilayered feed-forward structure that extracts hidden features, thus reducing the problem dimensionality. This block is useful when dealing with sparse problems. The second block consists of a Long-term Cognitive Network-based model that operates on features extracted by the first block. The activation rule of this recurrent neural network is modified to prevent the vanishing of the input signal during the recurrent inference process. The modified activation rule combines the neurons' state in the previous abstract layer (iteration) with the initial state. Moreover, we add a bias component to shift the transfer functions as needed to obtain good approximations. Finally, the third block consists of an output layer that adapts the second block's outputs to the label space. We propose a backpropagation learning algorithm that uses a squared hinge loss function to maximize the margins between labels to train this network. The results show that our model outperforms the state-of-the-art algorithms in most datasets.

Evolutionary Multilabel Classification Algorithm Based on Cultural Algorithm

As one of the common methods to construct classifiers, naïve Bayes has become one of the most popular classification methods because of its solid theoretical basis, strong prior knowledge learning characteristics, unique knowledge expression forms, and high classification accuracy. This classification method has a symmetry phenomenon in the process of data classification. Although the naïve Bayes classifier has high classification performance in single-label classification problems, it is worth studying whether the multilabel classification problem is still valid. In this paper, with the naïve Bayes classifier as the basic research object, in view of the naïve Bayes classification algorithm’s shortage of conditional independence assumptions and label class selection strategies, the characteristics of weighted naïve Bayes is given a better label classifier algorithm framework; the introduction of cultural algorithms to search for and determine the optimal weights is proposed as the weighted naïve Bayes multilabel classification algorithm. Experimental results show that the algorithm proposed in this paper is superior to other algorithms in classification performance.

Incremental predictive clustering trees for online semi-supervised multi-target regression

In many application settings, labeling data examples is a costly endeavor, while unlabeled examples are abundant and cheap to produce. Labeling examples can be particularly problematic in an online setting, where there can be arbitrarily many examples that arrive at high frequencies. It is also problematic when we need to predict complex values (e.g., multiple real values), a task that has started receiving considerable attention, but mostly in the batch setting. In this paper, we propose a method for online semi-supervised multi-target regression. It is based on incremental trees for multi-target regression and the predictive clustering framework. Furthermore, it utilizes unlabeled examples to improve its predictive performance as compared to using just the labeled examples. We compare the proposed iSOUP-PCT method with supervised tree methods, which do not use unlabeled examples, and to an oracle method, which uses unlabeled examples as though they were labeled. Additionally, we compare the proposed method to the available state-of-the-art methods. The method achieves good predictive performance on account of increased consumption of computational resources as compared to its supervised variant. The proposed method also beats the state-of-the-art in the case of very few labeled examples in terms of performance, while achieving comparable performance when the labeled examples are more common.

Deep hiearchical multi-label classification applied to chest X-ray abnormality taxonomies

Chest X-rays (CXRs) are a crucial and extraordinarily common diagnostic tool, leading to heavy research for computer-aided diagnosis (CAD) solutions. However, both high classification accuracy and meaningful model predictions that respect and incorporate clinical taxonomies are crucial for CAD usability. To this end, we present a deep hierarchical multi-label classification (HMLC) approach for CXR CAD. Different than other hierarchical systems, we show that first training the network to model conditional probability directly and then refining it with unconditional probabilities is key in boosting performance. In addition, we also formulate a numerically stable cross-entropy loss function for unconditional probabilities that provides concrete performance improvements. Finally, we demonstrate that HMLC can be an effective means to manage missing or incomplete labels. To the best of our knowledge, we are the first to apply HMLC to medical imaging CAD. We extensively evaluate our approach on detecting abnormality labels from the CXR arm of the Prostate, Lung, Colorectal and Ovarian (PLCO) dataset, which comprises over 198,000 manually annotated CXRs. When using complete labels, we report a mean area under the curve (AUC) of 0.887, the highest yet reported for this dataset. These results are supported by ancillary experiments on the PadChest dataset, where we also report significant improvements, 1.2% and 4.1% in AUC and average precision, respectively over strong "flat" classifiers. Finally, we demonstrate that our HMLC approach can much better handle incompletely labelled data. These performance improvements, combined with the inherent usefulness of taxonomic predictions, indicate that our approach represents a useful step forward for CXR CAD.

Bonsai: diverse and shallow trees for extreme multi-label classification

Extreme multi-label classification (XMC) refers to supervised multi-label learning involving hundreds of thousands or even millions of labels. In this paper, we develop a suite of algorithms, called , which generalizes the notion of label representation in XMC, and partitions the labels in the representation space to learn shallow trees. We show three concrete realizations of this label representation space including: (i) the input space which is spanned by the input features, (ii) the output space spanned by label vectors based on their co-occurrence with other labels, and (iii) the joint space by combining the input and output representations. Furthermore, the constraint-free multi-way partitions learnt iteratively in these spaces lead to shallow trees. By combining the effect of shallow trees and generalized label representation, achieves the best of both worlds—fast training which is comparable to state-of-the-art tree-based methods in XMC, and much better prediction accuracy, particularly on tail-labels. On a benchmark Amazon-3M dataset with 3 million labels, outperforms a state-of-the-art one-vs-rest method in terms of prediction accuracy, while being approximately 200 times faster to train. The code for is available at https://github.com/xmc-aalto/bonsai.

UDSMProt: universal deep sequence models for protein classification

MOTIVATION

Inferring the properties of a protein from its amino acid sequence is one of the key problems in bioinformatics. Most state-of-the-art approaches for protein classification are tailored to single classification tasks and rely on handcrafted features, such as position-specific-scoring matrices from expensive database searches. We argue that this level of performance can be reached or even be surpassed by learning a task-agnostic representation once, using self-supervised language modeling, and transferring it to specific tasks by a simple fine-tuning step.

RESULTS

We put forward a universal deep sequence model that is pre-trained on unlabeled protein sequences from Swiss-Prot and fine-tuned on protein classification tasks. We apply it to three prototypical tasks, namely enzyme class prediction, gene ontology prediction and remote homology and fold detection. The proposed method performs on par with state-of-the-art algorithms that were tailored to these specific tasks or, for two out of three tasks, even outperforms them. These results stress the possibility of inferring protein properties from the sequence alone and, on more general grounds, the prospects of modern natural language processing methods in omics. Moreover, we illustrate the prospects for explainable machine learning methods in this field by selected case studies.

AVAILABILITY AND IMPLEMENTATION

Source code is available under https://github.com/nstrodt/UDSMProt.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.