Performance analysis of LVQ algorithms: A statistical physics approach

Enhancing the Speed of the Learning Vector Quantization (LVQ) Algorithm by Adding Partial Distance Computation

Learning Vector Quantization (LVQ) is one of the most widely used classification approaches. LVQ faces a problem as when the size of data grows large it becomes slower. In this paper, a modified version of LVQ, which is called PDLVQ is proposed to accelerate the traditional version. The proposed scheme aims to avoid unnecessary computations by applying an efficient Partial Distance (PD) computation strategy. Three different benchmark datasets are used in the experiments. The comparisons have been done between LVQ and PDLVQ in terms of runtime and in result, it turns out that PDLVQ shows better efficiency than LVQ. PDLVQ has achieved up to 37% efficiency in runtime compared to LVQ when the dimensions have increased. Also, the enhanced algorithm (PDLVQ) shows clear enhancement to decrease runtime when the size of dimensions, the number of clusters, or the size of data becomes increased compared with the traditional one which is LVQ.

Energy Management Strategy for Plug-In Hybrid Electric Vehicles Based on Driving Condition Recognition: A Review

Appropriate energy management strategies (EMSs) have been selected for plug-in hybrid electric vehicles (PHEVs) based on driving-condition recognition (DCR) according to the acquired driving information, so as to minimize the target parameters. With online control and offline optimization, the strategy is suitable for real-time applications and is of great significance to repetitive routes, owing to its simplicity and ease of implementation. This paper aims to identity the DCR-based EMSs, develop efficient EMSs, and invite researchers involved in this field to discover new solutions. This paper presents a comprehensive analysis of EMSs based on DCR in terms of principles, the scope of application as well as their advantages and disadvantages, and provides a thorough survey of the latest progress in EMSs. We concluded that DCR-based EMSs show an improvement in energy saving and the pollutant-discharging effect.

Supervised learning in the presence of concept drift: a modelling framework

We present a modelling framework for the investigation of supervised learning in non-stationary environments. Specifically, we model two example types of learning systems: prototype-based learning vector quantization (LVQ) for classification and shallow, layered neural networks for regression tasks. We investigate so-called student–teacher scenarios in which the systems are trained from a stream of high-dimensional, labeled data. Properties of the target task are considered to be non-stationary due to drift processes while the training is performed. Different types of concept drift are studied, which affect the density of example inputs only, the target rule itself, or both. By applying methods from statistical physics, we develop a modelling framework for the mathematical analysis of the training dynamics in non-stationary environments. Our results show that standard LVQ algorithms are already suitable for the training in non-stationary environments to a certain extent. However, the application of weight decay as an explicit mechanism of forgetting does not improve the performance under the considered drift processes. Furthermore, we investigate gradient-based training of layered neural networks with sigmoidal activation functions and compare with the use of rectified linear units. Our findings show that the sensitivity to concept drift and the effectiveness of weight decay differs significantly between the two types of activation function.

Statistical Mechanics of On-Line Learning Under Concept Drift

We introduce a modeling framework for the investigation of on-line machine learning processes in non-stationary environments. We exemplify the approach in terms of two specific model situations: In the first, we consider the learning of a classification scheme from clustered data by means of prototype-based Learning Vector Quantization (LVQ). In the second, we study the training of layered neural networks with sigmoidal activations for the purpose of regression. In both cases, the target, i.e., the classification or regression scheme, is considered to change continuously while the system is trained from a stream of labeled data. We extend and apply methods borrowed from statistical physics which have been used frequently for the exact description of training dynamics in stationary environments. Extensions of the approach allow for the computation of typical learning curves in the presence of concept drift in a variety of model situations. First results are presented and discussed for stochastic drift processes in classification and regression problems. They indicate that LVQ is capable of tracking a classification scheme under drift to a non-trivial extent. Furthermore, we show that concept drift can cause the persistence of sub-optimal plateau states in gradient based training of layered neural networks for regression.

Window-Based Example Selection in Learning Vector Quantization

A variety of modifications have been employed to learning vector quantization (LVQ) algorithms using either crisp or soft windows for selection of data. Although these schemes have been shown in practice to improve performance, a theoretical study on the influence of windows has so far been limited. Here we rigorously analyze the influence of windows in a controlled environment of gaussian mixtures in high dimensions. Concepts from statistical physics and the theory of online learning allow an exact description of the training dynamics, yielding typical learning curves, convergence properties, and achievable generalization abilities. We compare the performance and demonstrate the advantages of various algorithms, including LVQ 2.1, generalized LVQ (GLVQ), Learning from Mistakes (LFM) and Robust Soft LVQ (RSLVQ). We find that the selection of the window parameter highly influences the learning curves but not, surprisingly, the asymptotic performances of LVQ 2.1 and RSLVQ. Although the prototypes of LVQ 2.1 exhibit divergent behavior, the resulting decision boundary coincides with the optimal decision boundary, thus yielding optimal generalization ability.

Adaptive relevance matrices in learning vector quantization

We propose a new matrix learning scheme to extend relevance learning vector quantization (RLVQ), an efficient prototype-based classification algorithm, toward a general adaptive metric. By introducing a full matrix of relevance factors in the distance measure, correlations between different features and their importance for the classification scheme can be taken into account and automated, and general metric adaptation takes place during training. In comparison to the weighted Euclidean metric used in RLVQ and its variations, a full matrix is more powerful to represent the internal structure of the data appropriately. Large margin generalization bounds can be transferred to this case, leading to bounds that are independent of the input dimensionality. This also holds for local metrics attached to each prototype, which corresponds to piecewise quadratic decision boundaries. The algorithm is tested in comparison to alternative learning vector quantization schemes using an artificial data set, a benchmark multiclass problem from the UCI repository, and a problem from bioinformatics, the recognition of splice sites for C. elegans.