On optimization based extreme learning machine in primal for regression and classification by functional iterative method

A novel multi-innovation gradient support vector machine regression method

For the regression problem of support vector machine, the solution processes of the most existing methods use offline datasets, which cannot be realized online. For this problem, this paper presents a new online approach to identify these unknown parameters contained in the support vector machine. A new cost function is constructed by substituting the error term into the standard cost function, which is different from the standard support vector machine, and the gradient descent approach is then used to minimize the newly created loss function, thus proposing a stochastic gradient support vector machine algorithm to estimate the unknown parameters based on the recursive identification methods. Furthermore, to advance the property of the stochastic gradient support vector machine algorithm, a moving data window is used to widen the scalar information into a fixed-length innovation vector, thereby increasing the amount of information used in the parameter estimation based on the multi-innovation identification theory. In addition, the forgetting factor is brought into the proposed algorithms, and the corresponding forgetting factor recursive algorithms are derived. These methods are recursive identification methods, which may be implemented online and are more efficient in terms of computing. Finally, utilizing the MatLab platform, the validity and usefulness of the explored methodologies are proven using several numerical simulation examples.

Selection of optimal models for predicting growth stress in Artemisia desertorum by comparison of linear regression and multiple neural networks: Take the construction of a green mine in the Bayan Obo mine as an example

In recent years, more and more countries are focusing on the control of mining sites and the surrounding ecological environment, and the new environmental concept of green mines has been proposed. By investigating the ecological background of a mine site, pollution and ecological imbalances in the mine can be predicted, managed or transformed. This study investigated the effects of rare earth elements on plant growth in the Baotou Bayan Obo Rare Earth Mine and evaluated soil contamination and subsequent remediation through the measured plant height. Using linear regression, BP(Back Propagation) neural networks, GA-BP(Genetic Algorithm- Back Propagation) neural networks, ELM(Extreme Learning Machine) and GA-ELM(Genetic Algorithm- Extreme Learning Machine) model prediction instruments, the different rare earth solution concentrations were set as input values and the heights of Artemisia desertorum, which as the model plant, were set as output values in the prediction. The results showed that the linear regression predicted the standard error of single La(III), Ce(III) solution and compound La(III) + Ce(III) solution for Artemisia desertorum growth stress was on the high side, 7.02%- 8.92%; the efficiency range of each group of models under BP neural network, GA-BP neural network and ELM neural network were 1.15%- 2.53%, 0.85%- 1.28%, 1.76%- 3.53%; while the efficiency range under GA-ELM neural network was 0.59%- 0.68%, with average error values and predicted values close to the true values. Among them, the MAPE of GA-ELM neural network are significantly lower than other models, and the error decreases with increasing concentration of the compound solution. So GA-ELM neural network can be used as an efficient, fast and reasonable optimal model for predicting the growth stress of Artemisia desertorum in Bayan Obo mining area. The experimental results can provide a theoretical basis for assessing the risk of soil rare earth contamination in the area, evaluating the expectation of later remediation, and provide a degree of new ideas for the construction of green mines.

Random vector functional link with ε-insensitive Huber loss function for biomedical data classification

BACKGROUND AND OBJECTIVE

Biomedical data classification has been a trending topic among researchers during the last decade. Biomedical datasets may contain several features noises. Hence, the conventional machine learning model cannot efficiently handle the presence of noise in datasets. Among the several machine learning model, the random vector functional link (RVFL) is one of the most popular and efficient models for task related to both classification and regression. Despite its excellent classification performance, its performance degrades while dealing with the datasets with noise. Researchers are searching for powerful models to minimize the influence of noise in datasets. Therefore, to enhance the classification ability of RVFL on noisy datasets, this paper suggests a novel random vector functional link with ε-insensitive Huber loss function (ε-HRVFL) for biomedical data classification problems.

METHODS

The optimization problem of ε-HRVFL is reformulated as strongly convex minimization problems with a simple function iterative approach to find solutions. To have a better understanding of the scope of the biomedical data classification problem and potential solutions, we conducted experiments with three different types of label noise in biomedical datasets as well as a few non-biomedical datasets. The classification accuracy of the proposed ε-HRVFL model is compared statistically using Friedman test with the support vector machine, extreme learning machine with radial basis function (RBF) and sigmoid activation functions and RVFL with RBF and sigmoid activation functions.

RESULTS

For non-biomedical datasets, the proposed model showed the highest accuracy of 98.1332%. Moreover, for the biomedical datasets, the proposed model showed the best accuracy of 96.5229%. The proposed ε-HRVFL model with sigmoid activation function reveals the best mean ranks among the reported classifiers for both, biomedical and non-biomedical datasets.

CONCLUSION

Numerical results show the applicability of the proposed ε-HRVFL model. In future, the proposed ε-HRVFL can be developed to solve multiclass biomedical data classification problems. Moreover, ε-insensitive asymmetric Huber loss function based RVFL model can be developed for dealing more efficiently with these noisy biomedical datasets.

1-Norm random vector functional link networks for classification problems

This paper presents a novel random vector functional link (RVFL) formulation called the 1-norm RVFL (1N RVFL) networks, for solving the binary classification problems. The solution to the optimization problem of 1N RVFL is obtained by solving its exterior dual penalty problem using a Newton technique. The 1-norm makes the model robust and delivers sparse outputs, which is the fundamental advantage of this model. The sparse output indicates that most of the elements in the output matrix are zero; hence, the decision function can be achieved by incorporating lesser hidden nodes compared to the conventional RVFL model. 1N RVFL produces a classifier that is based on a smaller number of input features. To put it another way, this method will suppress the neurons in the hidden layer. Statistical analyses have been carried out on several real-world benchmark datasets. The proposed 1N RVFL with two activation functions viz., ReLU and sine are used in this work. The classification accuracies of 1N RVFL are compared with the extreme learning machine (ELM), kernel ridge regression (KRR), RVFL, kernel RVFL (K-RVFL) and generalized Lagrangian twin RVFL (GLTRVFL) networks. The experimental results with comparable or better accuracy indicate the effectiveness and usability of 1N RVFL for solving binary classification problems.

Support vector regression-based QSAR models for prediction of antioxidant activity of phenolic compounds

The Support vector regression (SVR) was used to investigate quantitative structure-activity relationships (QSAR) of 75 phenolic compounds with Trolox-equivalent antioxidant capacity (TEAC). Geometric structures were optimized at the EF level of the MOPAC software program. Using Pearson correlation coefficient analysis, four molecular descriptors [n(OH), Cosmo Area (CA), Core-Core Repulsion (CCR) and Final Heat of Formation (FHF)] were selected as independent variables. The QSAR model was developed from the training set consisting of 57 compounds and then used the leave-one-out cross-validation (LOOCV) correlation coefficient to evaluate the prediction ability of the QSAR model. Used Artificial neural network (ANN) and multiple linear regression (MLR) for comparing. The RMSE (root mean square error) values of LOOCV in SVR, ANN and MLR models were 0.44, 0.46 and 0.54. The RMSE values of prediction of external 18 compounds were 0.41, 0.39 and 0.54 for SVR, ANN and MLR models, respectively. The obtained result indicated that the SVR models exhibited excellent predicting performance and competent for predicting the TEAC of phenolic compounds.

Multi-Agent System Based on the Extreme Learning Machine and Fuzzy Control for Intelligent Energy Management in Microgrid

Renewable energies constitute an alternative to fossil energies for several reasons. The microgrid can be assumed as the ideal way to integrate a renewable energy source in the production of electricity and give the consumer the opportunity to participate in the electricity market not just like a consumer but also like a producer. In this paper, we present a multi-agent system based on wind and photovoltaic power prediction using the extreme learning machine algorithm. This algorithm was tested on real weather data taken from the region of Tetouan City in Morocco. The process aimed to implement a microgrid located in Tetouan City and composed of different generation units (solar and wind energies were combined together to increase the efficiency of the system) and storage units (batteries were used to ensure the availability of power on demand as much as possible). In the proposed architecture, the microgrid can exchange electricity with the main grid; therefore, it can buy or sell electricity. Thus, the goal of our multi-agent system is to control the amount of power delivered or taken from the main grid in order to reduce the cost and maximize the benefit. To address uncertainties in the system, we use fuzzy logic control to manage the flow of energy, to ensure the availability of power on demand, and to make a reasonable decision about storing or selling electricity.

Probabilistic Regularized Extreme Learning Machine for Robust Modeling of Noise Data

The extreme learning machine (ELM) has been extensively studied in the machine learning field and has been widely implemented due to its simplified algorithm and reduced computational costs. However, it is less effective for modeling data with non-Gaussian noise or data containing outliers. Here, a probabilistic regularized ELM is proposed to improve modeling performance with data containing non-Gaussian noise and/or outliers. While traditional ELM minimizes modeling error by using a worst-case scenario principle, the proposed method constructs a new objective function to minimize both mean and variance of this modeling error. Thus, the proposed method considers the modeling error distribution. A solution method is then developed for this new objective function and the proposed method is further proved to be more robust when compared with traditional ELM, even when subject to noise or outliers. Several experimental cases demonstrate that the proposed method has better modeling performance for problems with non-Gaussian noise or outliers.

Discovering the Relationship Between Generalization and Uncertainty by Incorporating Complexity of Classification

The generalization ability of a classifier learned from a training set is usually dependent on the classifier's uncertainty, which is often described by the fuzziness of the classifier's outputs on the training set. Since the exact dependency relation between generalization and uncertainty of a classifier is quite complicated, it is difficult to clearly or explicitly express this relation in general. This paper shows a specific study on this relation from the viewpoint of complexity of classification by choosing extreme learning machines as the classification algorithms. It concludes that the generalization ability of a classifier is statistically becoming better with the increase of uncertainty when the complexity of the classification problem is relatively high, and the generalization ability is statistically becoming worse with the increase of uncertainty when the complexity is relatively low. This paper tries to provide some useful guidelines for improving the generalization ability of classifiers by adjusting uncertainty based on the problem complexity.

An Efficient Leave-One-Out Cross-Validation-Based Extreme Learning Machine (ELOO-ELM) With Minimal User Intervention

It is well known that the architecture of the extreme learning machine (ELM) significantly affects its performance and how to determine a suitable set of hidden neurons is recognized as a key issue to some extent. The leave-one-out cross-validation (LOO-CV) is usually used to select a model with good generalization performance among potential candidates. The primary reason for using the LOO-CV is that it is unbiased and reliable as long as similar distribution exists in the training and testing data. However, the LOO-CV has rarely been implemented in practice because of its notorious slow execution speed. In this paper, an efficient LOO-CV formula and an efficient LOO-CV-based ELM (ELOO-ELM) algorithm are proposed. The proposed ELOO-ELM algorithm can achieve fast learning speed similar to the original ELM without compromising the reliability feature of the LOO-CV. Furthermore, minimal user intervention is required for the ELOO-ELM, thus it can be easily adopted by nonexperts and implemented in automation processes. Experimentation studies on benchmark datasets demonstrate that the proposed ELOO-ELM algorithm can achieve good generalization with limited user intervention while retaining the efficiency feature.