Backpropagation uses prior information efficiently

ADAPTIVE FEATURE SPACES FOR LAND COVER CLASSIFICATION WITH LIMITED GROUND TRUTH DATA

Classification of land cover based on hyperspectral data is very challenging because typically tens of classes with uneven priors are involved, the inputs are high dimensional, and there is often scarcity of labeled data. Several researchers have observed that it is often preferable to decompose a multiclass problem into multiple two-class problems, solve each such subproblem using a suitable binary classifier, and then combine the outputs of this collection of classifiers in a suitable manner to obtain the answer to the original multiclass problem. This approach is taken by the popular error correcting output codes (ECOC) technique, as well by the binary hierarchical classifier (BHC). Classical techniques for dealing with small sample sizes include regularization of covariance matrices and feature reduction. In this paper we address the twin problems of small sample sizes and multiclass settings by proposing a feature reduction scheme that adaptively adjusts to the amount of labeled data available. This scheme can be used in conjunction with ECOC and the BHC, as well as other approaches such as round-robin classification that decompose a multiclass problem into a number of two (meta)-class problems. In particular, we develop the best-basis binary hierarchical classifier (BB-BHC) and best basis ECOC (BB-ECOC) families of models that are adapted to "small sample size" situations. Currently, there are few studies that compare the efficacy of different approaches to multiclass problems in general settings as well as in the specific context of small sample sizes. Our experiments on two sets of remote sensing data show that both BB-BHC and BB-ECOC methods are superior to their nonadaptive versions when faced with limited data, with the BB-BHC showing a slight edge in terms of classification accuracy as well as interpretability.

Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance

This study investigates the effect of class imbalance in training data when developing neural network classifiers for computer-aided medical diagnosis. The investigation is performed in the presence of other characteristics that are typical among medical data, namely small training sample size, large number of features, and correlations between features. Two methods of neural network training are explored: classical backpropagation (BP) and particle swarm optimization (PSO) with clinically relevant training criteria. An experimental study is performed using simulated data and the conclusions are further validated on real clinical data for breast cancer diagnosis. The results show that classifier performance deteriorates with even modest class imbalance in the training data. Further, it is shown that BP is generally preferable over PSO for imbalanced training data especially with small data sample and large number of features. Finally, it is shown that there is no clear preference between oversampling and no compensation approach and some guidance is provided regarding a proper selection.

Cascaded linear shift-invariant processors in optical pattern recognition

We study a cascade of linear shift-invariant processing modules (correlators), each augmented with a nonlinear threshold as a means to increase the performance of high-speed optical pattern recognition. This configuration is a special class of multilayer, feed-forward neural networks and has been proposed in the literature as a relatively fast best-guess classifier. However, it seems that, although cascaded correlation has been proposed in a number of specific pattern recognition problems, the importance of the configuration has been largely overlooked. We prove that the cascaded architecture is the exact structure that must be adopted if a multilayer feed-forward neural network is trained to produce a shift-invariant output. In contrast with more generalized multilayer networks, the approach is easily implemented in practice with optical techniques and is therefore ideally suited to the high-speed analysis of large images. We have trained a digital model of the system using a modified backpropagation algorithm with optimization using simulated annealing techniques. The resulting cascade has been applied to a defect recognition problem in the canning industry as a benchmark for comparison against a standard linear correlation filter, the minimum average correlation energy (MACE) filter. We show that the nonlinear performance of the cascade is a significant improvement over that of the linear MACE filter in this case.

Pattern recognition in flow cytometry

BACKGROUND

Analytical flow cytometry (AFC), by quantifying sometimes more than 10 optical parameters on cells at rates of approximately 10(3) cells/s, rapidly generates vast quantities of multidimensional data, which provides a considerable challenge for data analysis. We review the application of multivariate data analysis and pattern recognition techniques to flow cytometry.

METHODS

Approaches were divided into two broad types depending on whether the aim was identification or clustering. Multivariate statistical approaches, supervised artificial neural networks (ANNs), problems of overlapping character distributions, unbounded data sets, missing parameters, scaling up, and estimating proportions of different types of cells comprised the first category. Classic clustering methods, fuzzy clustering, and unsupervised ANNs comprised the second category. We demonstrate the state of the art by using AFC data on marine phytoplankton populations.

RESULTS AND CONCLUSIONS

Information held within the large quantities of data generated by AFC was tractable using ANNs, but for field studies the problem of obtaining suitable training data needs to be resolved, and coping with an almost infinite number of cell categories needs further research.

Training radial basis function neural networks: effects of training set size and imbalanced training sets

Obtaining training data for constructing artificial neural networks (ANNs) to identify microbiological taxa is not always easy. Often, only small data sets with different numbers of observations per taxon are available. Here, the effect of both size of the training data set and of an imbalanced number of training patterns for different taxa is investigated using radial basis function ANNs to identify up to 60 species of marine microalgae. The best networks trained to discriminate 20, 40 and 60 species respectively gave overall percentage correct identification of 92, 84 and 77%. From 100 to 200 patterns per species was sufficient in networks trained to discriminate 20, 40 or 60 species. For 40 and 60 species data sets an imbalance in the number of training patterns per species always affected training success, the greater the imbalance the greater the effect. However, this could be largely compensated for by adjusting the networks using a posteriori probabilities, estimated as network output values.

Statistical performance of cascaded linear shift-invariant processing

The cascaded correlator architecture comprises a series of traditional linear correlators separated by nonlinear threshold functions, trained with neural-network techniques. We investigate the shift-invariant classification performance of cascaded correlators in comparison with optimum Bayes classifiers. Inputs are formulated as randomly generated sample members of known statistical class distributions. It is shown that when the separability of true and false classes is varied in both the first and the second orders, the two-stage cascaded correlator shows performance similar to that of the optimum quadratic Bayes classifier throughout the studied range. It is shown that this is due to the similar decision boundaries implemented by the two nonlinear classifiers.