Curriculum Reinforcement Learning Based on K-Fold Cross Validation

With the continuous development of deep reinforcement learning in intelligent control, combining automatic curriculum learning and deep reinforcement learning can improve the training performance and efficiency of algorithms from easy to difficult. Most existing automatic curriculum learning algorithms perform curriculum ranking through expert experience and a single network, which has the problems of difficult curriculum task ranking and slow convergence speed. In this paper, we propose a curriculum reinforcement learning method based on K-Fold Cross Validation that can estimate the relativity score of task curriculum difficulty. Drawing lessons from the human concept of curriculum learning from easy to difficult, this method divides automatic curriculum learning into a curriculum difficulty assessment stage and a curriculum sorting stage. Through parallel training of the teacher model and cross-evaluation of task sample difficulty, the method can better sequence curriculum learning tasks. Finally, simulation comparison experiments were carried out in two types of multi-agent experimental environments. The experimental results show that the automatic curriculum learning method based on K-Fold cross-validation can improve the training speed of the MADDPG algorithm, and at the same time has a certain generality for multi-agent deep reinforcement learning algorithm based on the replay buffer mechanism.

Analysis of input set characteristics and variances on k-fold cross validation for a Recurrent Neural Network model on waste disposal rate estimation

The use of machine learning techniques in waste management studies is increasingly popular. Recent literature suggests k-fold cross validation may reduce input dataset partition uncertainties and minimize overfitting issues. The objectives are to quantify the benefits of k-fold cross validation for municipal waste disposal prediction and to identify the relationship of testing dataset variance on predictive neural network model performance. It is hypothesized that the dataset characteristics and variances may dictate the necessity of k-fold cross validation on neural network waste model construction. Seven RNN-LSTM predictive models were developed using historical landfill waste records and climatic and socio-economic data. The performance of all trials was acceptable in the training and validation stages, with MAPE all less than 10%. In this study, the 7-fold cross validation reduced the bias in selection of testing sets as it helps to reduce MAPE by up to 44.57%, MSE by up to 54.15%, and increased R value by up to 8.33%. Correlation analysis suggests that fewer outliers and less variance of the testing dataset correlated well with lower modeling error. The length of the continuous high waste season and length of total high waste period appear not important to the model performance. The result suggests that k-fold cross validation should be applied to testing datasets with higher variances. The use of MSE as an evaluation index is recommended.

A robust elastic net-ℓ1ℓ2reconstruction method for x-ray luminescence computed tomography

Objective. X-ray luminescence computed tomography (XLCT) has played a crucial role in pre-clinical research and effective diagnosis of disease. However, due to the ill-posed of the XLCT inverse problem, the generalization of reconstruction methods and the selection of appropriate regularization parameters are still challenging in practical applications. In this research, an robust Elastic net-ℓ₁ℓ₂reconstruction method is proposed aiming to the challenge.Approach. Firstly, our approach consists of ℓ₁and ℓ₂regularization to enhance the sparsity and suppress the smoothness. Secondly, through optimal approximation of the optimization problem, double modification of Landweber algorithm is adopted to solve the Elastic net-ℓ₁ℓ₂regulazation. Thirdly, drawing on the ideal of supervised learning, multi-parameter K-fold cross validation strategy is proposed to determin the optimal parameters adaptively.Main results. To evaluate the performance of the Elastic net-ℓ₁ℓ₂method, numerical simulations, phantom and in vivo experiments were conducted. In these experiments, the Elastic net-ℓ₁ℓ₂method achieved the minimum reconstruction error (with smallest location error, fluorescent yield relative error, normalized root-mean-square error) and the best image reconstruction quality (with largest contrast-to-noise ratio and Dice similarity) among all methods. The results demonstrated that Elastic net-ℓ₁ℓ₂can obtain superior reconstruction performance in terms of location accuracy, dual source resolution, robustness and in vivo practicability.Significance. It is believed that this study will further benefit preclinical applications with a view to provide a more reliable reference for the later researches on XLCT.

Construction of AQHI based on the exposure relationship between air pollution and YLL in northern China

The current air quality index (AQI) has been argued for failing to respond to the combined health effects of multiple air pollutants. Thus, it is a challenge to construct a new indicator, air quality health index (AQHI) to comprehensively assess and predict air quality and the health effects caused by air pollution. Here, we have comprehensively considered the relationship between six air pollutants and the total mortality. And we constructed AQHI using the principal component analysis (FCA) by taking into account of the associations between six main air pollutants and YLL in Tianjin, China from 2014 to 2017. Then, we used the K-fold cross-validation method and the method of comparing AQHI with AQI to assess the validity of AQHI, respectively. Two principal components (F₁ and F₂) were used to construct AQHI; the cumulative contribution rate of variance for them was >70% (53.6% and 16.4%, respectively). With each unit increase of F₁, the total daily YLL increased by 18.420 person-years. With each unit increase of F₂, the total daily YLL increased by 22.409 person-years. The correlation between the predicted and actual values of total mortality and total YLL of AQHI was 0.742 (P < 0.001) and 0.700 (P < 0.001), respectively. The difference between AQI and AQHI was statistically significant (χ² = 103.15, P < 0.001). There was a correlation between AQHI and AQI (r = 0.807, P < 0.01), and the grading was also correlated (r_s = 0.580, P < 0.01). With an increase of interquartile range (IQR) for AQHI, the daily YLL increased by 32.797 (95% CI: 14.559, 51.034), while for the AQI, the daily YLL increased by 22.367 (95% CI: 6.619, 38.116), which was less than AQHI. These results imply that AQHI can comprehensively consider the impact of various pollutants on disease mortality and YLL, and can comprehensively reflect air quality, which has an important practical significance.

Beware of External Validation! - A Comparative Study of Several Validation Techniques used in QSAR Modelling

BACKGROUND

Proper validation is an important aspect of QSAR modelling. External validation is one of the widely used validation methods in QSAR where the model is built on a subset of the data and validated on the rest of the samples. However, its effectiveness for datasets with a small number of samples but a large number of predictors remains suspect.

OBJECTIVE

Calculating hundreds or thousands of molecular descriptors using currently available software has become the norm in QSAR research, owing to computational advances in the past few decades. Thus, for n chemical compounds and p descriptors calculated for each molecule, the typical chemometric dataset today has a high value of p but small n (i.e. n << p). Motivated by the evidence of inadequacies of external validation in estimating the true predictive capability of a statistical model in recent literature, this paper performs an extensive and comparative study of this method with several other validation techniques.

METHODOLOGY

We compared four validation methods: Leave-one-out, K-fold, external and multi-split validation, using statistical models built using the LASSO regression, which simultaneously performs variable selection and modelling. We used 300 simulated datasets and one real dataset of 95 congeneric amine mutagens for this evaluation.

RESULTS

External validation metrics have high variation among different random splits of the data, hence are not recommended for predictive QSAR models. LOO has the overall best performance among all validation methods applied in our scenario.

CONCLUSION

Results from external validation are too unstable for the datasets we analyzed. Based on our findings, we recommend using the LOO procedure for validating QSAR predictive models built on high-dimensional small-sample data.

Alumina Concentration Detection Based on the Kernel Extreme Learning Machine

The concentration of alumina in the electrolyte is of great significance during the production of aluminum. The amount of the alumina concentration may lead to unbalanced material distribution and low production efficiency and affect the stability of the aluminum reduction cell and current efficiency. The existing methods cannot meet the needs for online measurement because industrial aluminum electrolysis has the characteristics of high temperature, strong magnetic field, coupled parameters, and high nonlinearity. Currently, there are no sensors or equipment that can detect the alumina concentration on line. Most companies acquire the alumina concentration from the electrolyte samples which are analyzed through an X-ray fluorescence spectrometer. To solve the problem, the paper proposes a soft sensing model based on a kernel extreme learning machine algorithm that takes the kernel function into the extreme learning machine. K-fold cross validation is used to estimate the generalization error. The proposed soft sensing algorithm can detect alumina concentration by the electrical signals such as voltages and currents of the anode rods. The predicted results show that the proposed approach can give more accurate estimations of alumina concentration with faster learning speed compared with the other methods such as the basic ELM, BP, and SVM.

A neural network based model to analyze rice parboiling process with small dataset

In this study, milling recovery, head rice yield, degree of milling and whiteness were utilized to characterize the milling quality of Tarom parboiled rice variety. The parboiled rice was prepared with three soaking temperatures and steaming times. Then the samples were dried to three levels of final moisture contents [8, 10 and 12% (w.b)]. Modeling of process and validating of the results with small dataset are always challenging. So, the aim of this study was to develop models based on the milling quality data in parboiling process by means of multivariate regression and artificial neural network. In order to validate the neural network model with a little dataset, K-fold cross validation method was applied. The ANN structure with one hidden layer and Tansig transfer function by 18 neurons in the hidden layer was selected as the best model in this study. The results indicated that the neural network could model the parboiling process with higher degree of accuracy. This method was a promising procedure to create accuracy and can be used as a reliable model to select the best parameters for the parboiling process with little experiment dataset.