Integration of extreme gradient boosting feature selection approach with machine learning models: application of weather relative humidity prediction

Hybrid noise reduction-based data-driven modeling of relative humidity in Khulna, Bangladesh

In this study, a hybrid Machine Learning (ML) approach is proposed for Relative Humidity (RH) prediction with a combination of Empirical Mode Decomposition (EMD) to improve the prediction accuracy over the traditional prediction technique using a Machine Learning (ML) algorithm called Support Vector Machine (SVM). The main objective of proposing this hybrid technique is to deal with the extremely nonlinear and noisy humidity pattern in Khulna, Bangladesh, which is experiencing rapid urbanization and environmental change. To develop the model, data on temperature, relative humidity, rainfall, and wind speed were collected from the Bangladesh Meteorological Department (BMD), and the data was divided into three phases: 70 % of the historical dataset as training data for the model, 15 % of the data set as the validation phase, and the remaining 15 % of the data set as the test phase of the model. Employing the Particle Swarm Optimization (PSO) algorithm, the SVM model determines its best hypermeters within this research. In the present research, performance analysis is carried out utilizing the Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Coefficient of Determination (R2). Results show that the increase in R2 values resulting from the EMD-based approach is significant: 21.05 % in H1(Traditional model), 19.48 % in H2 (Traditional model), 76.92 % in H3 (Traditional model), 55.93 % in H4 (Traditional model), and 64.29 % in H5 (Traditional model) and H6 (Traditional model). The analytical results show that the proposed EMD-based technique efficiently filters and processes noisy, highly nonlinear humidity data during prediction in the Khulna region. It is recommended that this technique could be applied to other geological areas.

Nested ensemble selection: An effective hybrid feature selection method

It has been shown that while feature selection algorithms are able to distinguish between relevant and irrelevant features, they fail to differentiate between relevant and redundant and correlated features. To address this issue, we propose a highly effective approach, called Nested Ensemble Selection (NES), that is based on a combination of filter and wrapper methods. The proposed feature selection algorithm differs from the existing filter-wrapper hybrid methods in its simplicity and efficiency as well as precision. The new algorithm is able to separate the relevant variables from the irrelevant as well as the redundant and correlated features. Furthermore, we provide a robust heuristic for identifying the optimal number of selected features which remains one of the greatest challenges in feature selection. Numerical experiments on synthetic and real-life data demonstrate the effectiveness of the proposed method. The NES algorithm achieves perfect precision on the synthetic data and near optimal accuracy on the real-life data. The proposed method is compared against several popular algorithms including mRMR, Boruta, genetic, recursive feature elimination, Lasso, and Elastic Net. The results show that NES significantly outperforms the benchmarks algorithms especially on multi-class datasets.

Estimation of urban AQI based on interpretable machine learning

Air pollution is an increasingly serious problem. Accurate and efficient prediction of air quality can effectively prevent air pollution and improve the quality of human life. The air quality index (AQI) is a dimensionless tool to describe air quality quantitatively. In this study, the machine learning (ML) method was used to estimate AQI for Shijiazhuang, China, as the research object, and pollutants and meteorological factors as data models. Specifically, eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), and Random Forest (RF) models were used. The experimental results show that XGBoost model captures the AQI variation trend well, and the R2 of XGBoost model is 0.929, which is 0.3% and 2.3% higher than the R2 of RF model and LightGBM model, respectively. In addition, through the SHAP-based model interpretation method, the study reveals the key factors of AQI variation, that is PM2.5 and PM10, play positive roles in the variation of AQI and AQI is less sensitive to meteorological factors. Finally, Beijing, Shanghai, Xi'an, and Guangzhou were selected to test the model's validity, and the model performance remained good. Our study shows that applying ML approach to air quality prediction is beneficial for efficiently assessing cities' future air quality.

Improving the accuracy of air relative humidity prediction using hybrid machine learning based on empirical mode decomposition: a comparative study

This paper proposes a hybrid air relative humidity prediction based on preprocessing signal decomposition. New modelling strategy was introduced based on the use of the empirical mode decomposition, variational mode decomposition, and the empirical wavelet transform, combined with standalone machine learning to increase their numerical performances. First, standalone models, i.e., extreme learning machine, multilayer perceptron neural network, and random forest regression, were used for predicting daily air relative humidity using various daily meteorological variables, i.e., maximal and minimal air temperatures, precipitation, solar radiation, and wind speed, measured at two meteorological stations located in Algeria. Second, meteorological variables are decomposed into several intrinsic mode functions and presented as new input variables to the hybrid models. The comparison between the models was achieved based on numerical and graphical indices, and obtained results demonstrate the superiority of the proposed hybrid models compared to the standalone models. Further analysis revealed that using standalone models, the best performances are obtained using the multilayer perceptron neural network with Pearson correlation coefficient, Nash-Sutcliffe efficiency, root-mean-square error, and mean absolute error of approximately ≈0.939, ≈0.882, ≈7.44, and ≈5.62 at Constantine station, and ≈0.943, ≈0.887, ≈7.72, and ≈5.93 at Sétif station, respectively. The hybrid models based on the empirical wavelet transform decomposition exhibited high performances with Pearson correlation coefficient, Nash-Sutcliffe efficiency, root-mean-square error, and mean absolute error of approximately ≈0.950, ≈0.902, ≈6.79, and ≈5.24, at Constantine station, and ≈0.955, ≈0.912, ≈6.82, and ≈5.29, at Sétif station. Finally, we show that the new hybrid approaches delivered high predictive accuracies of air relative humidity, and it was concluded that the contribution of the signal decomposition was demonstrated and justified.

Development of new computational machine learning models for longitudinal dispersion coefficient determination: case study of natural streams, United States

Natural streams longitudinal dispersion coefficient (Kx) is an essential indicator for pollutants transport and its determination is very important. Kx is influenced by several parameters, including river hydraulic geometry, sediment properties, and other morphological characteristics, and thus its calculation is a highly complex engineering problem. In this research, three relatively explored machine learning (ML) models, including Random Forest (RF), Gradient Boosting Decision Tree (GTB), and XGboost-Grid, were proposed for the Kx determination. The modeling scheme on building the prediction matrix was adopted from the well-established literature. Several input combinations were tested for better predictability performance for the Kx. The modeling performance was tested based on the data division for the training and testing (70-30% and 80-20%). Based on the attained modeling results, XGboost-Grid reported the best prediction results over the training and testing phase compared to RF and GTB models. The development of the newly established machine learning model revealed an excellent computed-aided technology for the Kx simulation.

Integrative artificial intelligence models for Australian coastal sediment lead prediction: An investigation of in-situ measurements and meteorological parameters effects

Heavy metals (HMs) such as Lead (Pb) have played a vital role in increasing the sediments of the Australian bay's ecosystem. Several meteorological parameters (i.e., minimum, maximum and average temperature (T_min, T_max and T_avg^oC), rainfall (R_n mm) and their interactions with the other batch HMs, are hypothesized to have high impact for the decision-making strategies to minimize the impacts of Pb. Three feature selection (FS) algorithms namely the Boruta method, genetic algorithm (GA) and extreme gradient boosting (XGBoost) were investigated to select the highly important predictors for Pb concentration in the coastal bay sediments of Australia. These FS algorithms were statistically evaluated using principal component analysis (PCA) Biplot along with the correlation metrics describing the statistical characteristics that exist in the input and output parameter space of the models. To ensure a high accuracy attained by the applied predictive artificial intelligence (AI) models i.e., XGBoost, support vector machine (SVM) and random forest (RF), an auto-hyper-parameter tuning process using a Grid-search approach was also implemented. Cu, Ni, Ce, and Fe were selected by all the three applied FS algorithms whereas the T_avg and R_n inputs remained the essential parameters identified by GA and Boruta. The order of the FS outcome was XGBoost > GA > Boruta based on the applied statistical examination and the PCA Biplot results and the order of applied AI predictive models was XGBoost-SVM > GA-SVM > Boruta-SVM, where the SVM model remained at the top performance among the other statistical metrics. Based on the Taylor diagram for model evaluation, the RF model was reflected only marginally different so overall, the proposed integrative AI model provided an evidence a robust and reliable predictive technique used for coastal sediment Pb prediction.

Forecasting of monthly relative humidity in Delhi, India, using SARIMA and ANN models

Relative humidity plays an important role in climate change and global warming, making it a research area of greater concern in recent decades. The present study attempted to implement seasonal autoregressive moving average (SARIMA) and artificial neural network (ANN) with multilayer perceptron (MLP) models to forecast the monthly relative humidity in Delhi, India during 2017-2025. The average monthly relative humidity data for the period 2000-2016 have been used to carry out the objectives of the proposed study. The forecast trend in relative humidity declines from 2017 to 2025. The accuracy of the models has been measured using root mean squared error (RMSE) and mean absolute error (MAE). The results showed that the SARIMA model provides the forecasted relative humidity with RMSE of 6.04 and MAE of 4.56. On the other hand, MLP model reported the forecasted relative humidity with RMSE of 4.65 and MAE of 3.42. This study concluded that the ANN model was more reliable for predicting relative humidity than SARIMA model.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40808-022-01385-8.