Multilayer perceptron neural network-based approach for modeling phycocyanin pigment concentrations: case study from lower Charles River buoy, USA

Application of a hybrid algorithm of LSTM and Transformer based on random search optimization for improving rainfall-runoff simulation

Flood forecasting using traditional physical hydrology models requires consideration of multiple complex physical processes including the spatio-temporal distribution of rainfall, the spatial heterogeneity of watershed sub-surface characteristics, and runoff generation and routing behaviours. Data-driven models offer novel solutions to these challenges, though they are hindered by difficulties in hyperparameter selection and a decline in prediction stability as the lead time extends. This study introduces a hybrid model, the RS-LSTM-Transformer, which combines Random Search (RS), Long Short-Term Memory networks (LSTM), and the Transformer architecture. Applied to the typical Jingle watershed in the middle reaches of the Yellow River, this model utilises rainfall and runoff data from basin sites to simulate flood processes, and its outcomes are compared against those from RS-LSTM, RS-Transformer, RS-BP, and RS-MLP models. It was evaluated against RS-LSTM, RS-Transformer, RS-BP, and RS-MLP models using the Nash-Sutcliffe Efficiency Coefficient (NSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Bias percentage as metrics. At a 1-h lead time during calibration and validation, the RS-LSTM-Transformer model achieved NSE, RMSE, MAE, and Bias values of 0.970, 14.001m3/s, 5.304m3/s, 0.501% and 0.953, 14.124m3/s, 6.365m3/s, 0.523%, respectively. These results demonstrate the model's superior simulation capabilities and robustness, providing more accurate peak flow forecasts as the lead time increases. The study highlights the RS-LSTM-Transformer model's potential in flood forecasting and the advantages of integrating various data-driven approaches for innovative modelling.

W-WaveNet: A multi-site water quality prediction model incorporating adaptive graph convolution and CNN-LSTM

Water quality prediction is of great significance in pollution control, prevention, and management. Deep learning models have been applied to water quality prediction in many recent studies. However, most existing deep learning models for water quality prediction are used for single-site data, only considering the time dependency of water quality data and ignoring the spatial correlation among multi-sites. This research defines and analyzes the non-aligned spatial correlations that exist in multi-site water quality data. Then deploy spatial-temporal graph convolution to process water quality data, which takes into account both the temporal and spatial correlation of multi-site water quality data. A multi-site water pollution prediction method called W-WaveNet is proposed that integrates adaptive graph convolution and Convolutional Neural Network, Long Short-Term Memory (CNN-LSTM). It integrates temporal and spatial models by interleaved stacking. Theoretical analysis shows that the method can deal with non-aligned spatial correlations in different time spans, which is suitable for water quality data processing. The model validates water quality data generated on two real river sections that have multiple sites. The experimental results were compared with the results of Support Vector Regression, CNN-LSTM, and Spatial-Temporal Graph Convolutional Networks (STGCN). It shows that when W-WaveNet predicts water quality over two river sections, the average Mean Absolute Error is 0.264, which is 45.2% lower than the commonly used CNN-LSTM model and 23.8% lower than the STGCN. The comparison experiments also demonstrate that W-WaveNet has a more stable performance in predicting longer sequences.

Prediction and sensitivity analysis of chlorophyll a based on a support vector machine regression algorithm

Outbreaks of planktonic algae seriously affect the water quality of rivers and are difficult to control. Based on the analysis of the temporal and spatial variation characteristics of environmental factors, this study uses a support vector machine regression (SVR) algorithm to establish a chlorophyll a (Chl-a) prediction model and conduct Chl-a sensitivity analysis. In 2018, the average Chl-a content was 126.25 ug/L. The maximum total nitrogen (TN) content was 16.68 mg/L and high year-round. The average NH₄⁺-N and total phosphorous (TP) contents were only 0.78 and 0.18 mg/L. The content of NH₄⁺-N was higher in spring and increased significantly along the water flow, while TP decreased slightly along the water flow. We used a radial basis function kernel SVR model and tenfold cross-validation method to optimize parameters. The penalty parameter c was 1.4142, the kernel function parameter g was 1, and the training and verification errors were only 0.032 and 0.067, respectively, indicating a good model fit. Based on a sensitivity analysis of the SVR prediction model, the maximum sensitivity coefficients of Chl-a to TP and WT were 0.571 and 0.394, respectively, and the contributions were 33% and 22%, respectively. The next highest sensitivity coefficients were those of DO (0.28, 16%) and pH (0.243, 14%). The sensitivity coefficients of TN and NH₄⁺-N were the lowest. According to the current water environment pollution conditions, TP is the limiting factor of Chl-a in the Qingshui River, and it is also the main prevention and control factor of phytoplankton outbreak.

TLT: Recurrent fine-tuning transfer learning for water quality long-term prediction

The water quality long-term prediction is essential to water environment management decisions. In recent years, although water quality prediction methods based on deep learning have achieved excellent performance in short-term prediction, these methods are unsuitable for long-term prediction because the accumulation use of short-term prediction will easily introduce noise. Furthermore, The long-term prediction task requires a large amount of data to train the model to obtain accurate prediction results. For some monitoring stations with limited historical data, it is challenging to fully exploit the performance of deep learning models. To this end, we introduce a transfer learning framework into water quality prediction to improve the prediction performance in data-constrained scenarios. We propose a deep Transfer Learning based on Transformer (TLT) model to enable time dependency perception and facilitate long-term water quality prediction. In TLT, we innovatively introduce a recurrent fine-tuning transfer learning method, which can transfer the knowledge learned from source monitoring stations to the target station, while preventing the deep learning model from overfitting the source data during the pre-training phase. So, TLT can fully exert the performance of deep learning models with limited samples. We conduct experiments on data from 120 monitoring stations in major rivers and lakes in China to verify the effectiveness of TLT. The results show that TLT can effectively improve the long-term prediction accuracy of four water quality indicators (pH, DO, NH₃-N, and COD_Mn) from monitoring stations with limited samples.

Water Quality Prediction Based on Multi-Task Learning

Water pollution seriously endangers people's lives and restricts the sustainable development of the economy. Water quality prediction is essential for early warning and prevention of water pollution. However, the nonlinear characteristics of water quality data make it challenging to accurately predicted by traditional methods. Recently, the methods based on deep learning can better deal with nonlinear characteristics, which improves the prediction performance. Still, they rarely consider the relationship between multiple prediction indicators of water quality. The relationship between multiple indicators is crucial for the prediction because they can provide more associated auxiliary information. To this end, we propose a prediction method based on exploring the correlation of water quality multi-indicator prediction tasks in this paper. We explore four sharing structures for the multi-indicator prediction to train the deep neural network models for constructing the highly complex nonlinear characteristics of water quality data. Experiments on the datasets of more than 120 water quality monitoring sites in China show that the proposed models outperform the state-of-the-art baselines.

The use of statistical and machine learning tools to accurately quantify the energy performance of residential buildings

Prediction of building energy consumption is key to achieving energy efficiency and sustainability. Nowadays, the analysis or prediction of building energy consumption using building energy simulation tools facilitates the design and operation of energy-efficient buildings. The collection and generation of building data are essential components of machine learning models; however, there is still a lack of such data covering certain weather conditions. Such as those related to arid climate areas. This paper fills this identified gap with the creation of a new dataset for energy consumption of 3,840 records of typical residential buildings of the Saudi Arabia region of Qassim, and investigates the impact of residential buildings' eight input variables (Building Size, Floor Height, Glazing Area, Wall Area, window to wall ratio (WWR), Win Glazing U-value, Roof U-value, and External Wall U-value) on the heating load (HL) and cooling load (CL) output variables. A number of classical and non-parametric statistical tools are used to uncover the most strongly associated input variables with each one of the output variables. Then, the machine learning Multiple linear regression (MLR) and Multilayer perceptron (MLP) methods are used to estimate HL and CL, and their results compared using the Mean Absolute Error (MAE), the Root Mean Square Error (RMSE), and coefficient of determination (R²) performance measures. The use of the IES simulation software on the new dataset concludes that MLP accurately estimates both HL and CL with low MAE, RMSE, and R², which evidences the feasibility and accuracy of applying machine learning methods to estimate building energy consumption.

A new hybrid model based on relevance vector machine with flower pollination algorithm for phycocyanin pigment concentration estimation

Precise monitoring of cyanobacteria concentration in water resources is a daunting task. The development of reliable tools to monitor this contamination is an important research topic in water resources management. Indirect methods such as chlorophyll-a determination, cell counting, and toxin measurement of the cyanobacteria are tedious, cumbersome, and often lead to inaccurate results. The quantity of phycocyanin (PC) pigment is considered more appropriate for cyanobacteria monitoring. Traditional approaches for PC estimation are time-consuming, expensive, and require high expertise. Recently, some studies have proposed the application of artificial intelligence (AI) techniques to predict the amount of PC concentration. Nonetheless, most of these researches are limited to standalone modeling schemas such as artificial neural network (ANN), multilayer perceptron (MLP), and support vector machine (SVM). The independent schema provides imprecise results when faced with highly nonlinear systems and data uncertainties resulting from environmental disturbances. To alleviate the limitations of the existing models, this study proposes the first application of a hybrid AI model that integrates the potentials of relevance vector machine (RVM) and flower pollination algorithm (RVM-FPA) to predict the PC concentration in water resources. The performance of the hybrid model is compared with the standalone RVM model. The prediction performance of the proposed models was evaluated at two stations (stations 508 and 478) using different statistical and graphical performance evaluation methods. The results showed that the hybrid models exhibited higher performance at both stations compared to the standalone RVM model. The proposed hybrid RVM-FPA can therefore serve as a reliable predictive tool for PC concentration in water resources.

Using multi-layer perceptron with Laplacian edge detector for bladder cancer diagnosis

In this paper, the urinary bladder cancer diagnostic method which is based on Multi-Layer Perceptron and Laplacian edge detector is presented. The aim of this paper is to investigate the implementation possibility of a simpler method (Multi-Layer Perceptron) alongside commonly used methods, such as Deep Learning Convolutional Neural Networks, for the urinary bladder cancer detection. The dataset used for this research consisted of 1997 images of bladder cancer and 986 images of non-cancer tissue. The results of the conducted research showed that using Multi-Layer Perceptron trained and tested with images pre-processed with Laplacian edge detector are achieving AUC value up to 0.99. When different image sizes are compared it can be seen that the best results are achieved if 50×50 and 100×100 images were used.

Comparing artificial intelligence techniques for chlorophyll-a prediction in US lakes

Chlorophyll-a (CHLA) is a key indicator to represent eutrophication status in lakes. In this study, CHLA, total phosphorus (TP), total nitrogen (TN), turbidity (TB), and Secchi depth (SD) collected by the United States Environmental Protection Agency for the National Lakes Assessment in the continental USA were analyzed. Statistical analysis showed that water quality variables in natural lakes have strong patterns of autocorrelations than man-made lakes, indicating the perturbation of anthropogenic stresses on man-made lake ecosystems. Meanwhile, adaptive neuro-fuzzy inference systems (ANFIS) with fuzzy c-mean-clustering algorithm (ANFIS_FC), ANFIS with grid partition method (ANFIS_GP), and ANFIS with subtractive clustering method (ANFIS_SC) were implemented to model CHLA in lakes, and modeling results were compared with the multilayer perceptron neural network models (MLPNN). Results showed that ANFIS_FC models outperformed other models for natural lakes, while for man-made lakes, MLPNN models performed the best. ANFIS_GP models have the lowest accuracies in general. The results indicated that ANFIS models can be screening tools for an overall estimation of CHLA levels of lakes in large scales, especially for natural lakes.

Modeling daily water temperature for rivers: comparison between adaptive neuro-fuzzy inference systems and artificial neural networks models

River water temperature is a key control of many physical and bio-chemical processes in river systems, which theoretically depends on multiple factors. Here, four different machine learning models, including multilayer perceptron neural network models (MLPNN), adaptive neuro-fuzzy inference systems (ANFIS) with fuzzy c-mean clustering algorithm (ANFIS_FC), ANFIS with grid partition method (ANFIS_GP), and ANFIS with subtractive clustering method (ANFIS_SC), were implemented to simulate daily river water temperature, using air temperature (T_a), river flow discharge (Q), and the components of the Gregorian calendar (CGC) as predictors. The proposed models were tested in various river systems characterized by different hydrological conditions. Results showed that including the three inputs as predictors (T_a, Q, and the CGC) yielded the best accuracy among all the developed models. In particular, model performance improved considerably compared to the case where only T_a is used as predictor, which is the typical approach of most of previous machine learning applications. Additionally, it was found that Q played a relevant role mainly in snow-fed and regulated rivers with higher-altitude hydropower reservoirs, while it improved to a lower extent model performance in lowland rivers. In the validation phase, the MLPNN model was generally the one providing the highest performances, although in some river stations ANFIS_FC and ANFIS_GP were slightly more accurate. Overall, the results indicated that the machine learning models developed in this study can be effectively used for river water temperature simulation.