Least squares large margin distribution machine for regression

Identification effect of least square fitting method in archives management

Archives management plays an important role in the current information age. Solving the problem of identifying and classifying archives is essential for promoting the development of archives management. The Least Squares Support Vector Machine (LS-SVM) is obtained by introducing the least squares fitting method into SVM, which is good at solving nonlinear classification. A new wavelet function is used to improve the classifier. At the same time, the cross-validation method is used to optimize the kernel parameters. Finally, the fuzzy theory and LS-SVM are combined to obtain Fuzzy Least Squares Support Vector Machines (FLS-SVM). This FLS-SVM classifier can use the distance between the data points and the classification hyperplane to classify the data in the non-separable region. The performance of FLS-SVM is verified by simulation experiments. The experimental results show that the classification accuracy of FLS-SVM classifier in archive data sets is 98.7%, and the loss rate is only 0.26%. When the wavelet function is used as the kernel function, the average accuracy of the classifier reaches 98.38%. Experiments show that the proposed method has good classification performance. It verifies the feasibility and effectiveness of the least squares fitting method in file management identification and classification.

Least squares support vector regression for complex censored data

Least squares support vector regression (LS-SVR) is a robust machine learning algorithm for small sample data. Its solution is derived from solving a set of linear equations, making the calculation process straightforward. In order to overcome the difficulties of the regression estimations when the responses are subject to interval censoring or left truncation and right censoring, two LS-SVR methods are proposed. For interval-censored data, one can easily estimate the regression functions by combining the imputation techniques and LS-SVR for right-censored data. For left-truncated and right-censored data, a weight is used to reduce the effects of truncation and censoring on the LS-SVR procedure. Simulation results show that the proposed methods can reduce regression error and yield high accuracy and stability.

A novel multi-innovation gradient support vector machine regression method

For the regression problem of support vector machine, the solution processes of the most existing methods use offline datasets, which cannot be realized online. For this problem, this paper presents a new online approach to identify these unknown parameters contained in the support vector machine. A new cost function is constructed by substituting the error term into the standard cost function, which is different from the standard support vector machine, and the gradient descent approach is then used to minimize the newly created loss function, thus proposing a stochastic gradient support vector machine algorithm to estimate the unknown parameters based on the recursive identification methods. Furthermore, to advance the property of the stochastic gradient support vector machine algorithm, a moving data window is used to widen the scalar information into a fixed-length innovation vector, thereby increasing the amount of information used in the parameter estimation based on the multi-innovation identification theory. In addition, the forgetting factor is brought into the proposed algorithms, and the corresponding forgetting factor recursive algorithms are derived. These methods are recursive identification methods, which may be implemented online and are more efficient in terms of computing. Finally, utilizing the MatLab platform, the validity and usefulness of the explored methodologies are proven using several numerical simulation examples.

Improving the Stability of the Variable Selection with Small Datasets in Classification and Regression Tasks

Within the design of a machine learning-based solution for classification or regression problems, variable selection techniques are often applied to identify the input variables, which mainly affect the considered target. The selection of such variables provides very interesting advantages, such as lower complexity of the model and of the learning algorithm, reduction of computational time and improvement of performances. Moreover, variable selection is useful to gain a profound knowledge of the considered problem. High correlation in variables often produces multiple subsets of equally optimal variables, which makes the traditional method of variable selection unstable, leading to instability and reducing the confidence of selected variables. Stability identifies the reproducibility power of the variable selection method. Therefore, having a high stability is as important as the high precision of the developed model. The paper presents an automatic procedure for variable selection in classification (binary and multi-class) and regression tasks, which provides an optimal stability index without requiring any a priori information on data. The proposed approach has been tested on different small datasets, which are unstable by nature, and has achieved satisfactory results.