Multiclass support vector machines with example-dependent costs applied to plankton biomass estimation

One-step Bayesian example-dependent cost classification: The OsC-MLP method

Example-dependent cost classification problems are those where the decision costs depend not only on the true and the attributed classes but also on the sample features. Discriminative algorithms that carry out such classification tasks must take this dependence into account. In some applications, the decision costs are known for the training set but not in production, which complicates the problem. In this paper, we introduce a new one-step Bayesian formulation to train Neural Networks and solve the above limitation for binary cases with one-step Learning Machines, avoiding the drawbacks that unknown analytical forms of the example-dependent costs create. The formulation is based on defining an artificial likelihood ratio by using the available training classification costs in its definition, and proposes a test that does not require the values of the costs for unseen samples. Furthermore, it also includes Bayesian rebalancing mechanisms to combat the negative effects of class imbalance. Experimental results support the consistency and effectiveness of the corresponding algorithms.

A Bayes Risk Minimization Machine for Example-Dependent Cost Classification

A new method for example-dependent cost (EDC) classification is proposed. The method constitutes an extension of a recently introduced training algorithm for neural networks. The surrogate cost function is an estimate of the Bayesian risk, where the estimates of the conditional probabilities for each class are defined in terms of a 1-D Parzen window estimator of the output of (discriminative) neural networks. This probability density is modeled with the objective of allowing an easy minimization of a sampled version of the Bayes risk. The conditional probabilities included in the definition of the risk are not explicitly estimated, but the risk is minimized by a gradient-descent algorithm. The proposed method has been evaluated using linear classifiers and neural networks, with both shallow (a single hidden layer) and deep (multiple hidden layers) architectures. The experimental results show the potential and flexibility of the proposed method, which can handle EDC classification under imbalanced data situations that commonly appear in this kind of problems.

Privacy-Preserving Cost-Sensitive Learning

Cost-sensitive learning methods guaranteeing privacy are becoming crucial nowadays in many applications where increasing use of sensitive personal information is observed. However, there has no optimal learning scheme developed in the literature to learn cost-sensitive classifiers under constraint of enforcing differential privacy. Our approach is to first develop a unified framework for existing cost-sensitive learning methods by incorporating the weight constant and weight functions into the classical regularized empirical risk minimization framework. Then, we propose two privacy-preserving algorithms with output perturbation and objective perturbation methods, respectively, to be integrated with the cost-sensitive learning framework. We showcase how this general framework can be used analytically by deriving the privacy-preserving cost-sensitive extensions of logistic regression and support vector machine. Experimental evidence on both synthetic and real data sets verifies that the proposed algorithms can reduce the misclassification cost effectively while satisfying the privacy requirement. A theoretical investigation is also conducted, revealing a very interesting analytic relation, i.e., that the choice of the weight constant and weight functions does not only influence the Fisher-consistent property (population minimizer of expected risk with a specific loss function leads to the Bayes optimal decision rule) but also interacts with privacy-preserving levels to affect the performance of classifiers significantly.

Modeling species richness and abundance of phytoplankton and zooplankton in radioactively contaminated water bodies

Water bodies polluted by the Mayak nuclear plant in Russia provide valuable information on multi-generation effects of radioactive contamination on freshwater organisms. For example, lake Karachay was probably the most radioactive lake in the world: its water contained ∼2 × 10⁷ Bq/L of radionuclides and estimated dose rates to plankton exceeded 5 Gy/h. We performed quantitative modeling of radiation effects on phytoplankton and zooplankton species richness and abundance in Mayak-contaminated water bodies. Due to collinearity between radioactive contamination, water body size and salinity, we combined these variables into one (called HabitatFactors). We employed a customized machine learning approach, where synthetic noise variables acted as benchmarks of predictor performance. HabitatFactors was the only predictor that outperformed noise variables and, therefore, we used it for parametric modeling of plankton responses. Best-fit model predictions suggested 50% species richness reduction at HabitatFactors values corresponding to dose rates of 10⁴-10⁵ μGy/h for phytoplankton, and 10³-10⁴ μGy/h for zooplankton. Under conditions similar to those in lake Karachay, best-fit models predicted 81-98% species richness reductions for various taxa (Cyanobacteria, Bacillariophyta, Chlorophyta, Rotifera, Cladocera and Copepoda), ∼20-300-fold abundance reduction for total zooplankton, but no abundance reduction for phytoplankton. Rotifera was the only taxon whose fractional abundance increased with contamination level, reaching 100% in lake Karachay, but Rotifera species richness declined with contamination level, as in other taxa. Under severe radioactive and chemical contamination, one species of Cyanobacteria (Geitlerinema amphibium) dominated phytoplankton, and rotifers from the genus Brachionus dominated zooplankton. The modeling approaches proposed here are applicable to other radioecological data sets. The results provide quantitative information and easily interpretable model parameter estimates for the shapes and magnitudes of freshwater plankton responses to a wide range of radioactive contamination levels.

Structural Minimax Probability Machine

Minimax probability machine (MPM) is an interesting discriminative classifier based on generative prior knowledge. It can directly estimate the probabilistic accuracy bound by minimizing the maximum probability of misclassification. The structural information of data is an effective way to represent prior knowledge, and has been found to be vital for designing classifiers in real-world problems. However, MPM only considers the prior probability distribution of each class with a given mean and covariance matrix, which does not efficiently exploit the structural information of data. In this paper, we use two finite mixture models to capture the structural information of the data from binary classification. For each subdistribution in a finite mixture model, only its mean and covariance matrix are assumed to be known. Based on the finite mixture models, we propose a structural MPM (SMPM). SMPM can be solved effectively by a sequence of the second-order cone programming problems. Moreover, we extend a linear model of SMPM to a nonlinear model by exploiting kernelization techniques. We also show that the SMPM can be interpreted as a large margin classifier and can be transformed to support vector machine and maxi-min margin machine under certain special conditions. Experimental results on both synthetic and real-world data sets demonstrate the effectiveness of SMPM.