Estimation of water quality by UV/Vis spectrometry in the framework of treated wastewater reuse

Information extraction of UV-NIR spectral data in waste water based on Large Language Model

In recent years, with the rise of various machine learning methods, the Ultraviolet and Near Infrared (UV-NIR) spectral analysis has been impressive in the determination of intricate systems. However, the UV-NIR spectral analysis based on traditional machine learning requires independent training with tedious parameter tuning for different samples or tasks. As a result, training a high-quality model is often complicated and time-consuming. Large language model (LLM) is one of the cutting-edge achievements in deep learning, with the parameter size of the order of billion. LLM can extract abstract information from input and use it effectively. Even without any additional training, using only simple natural language prompts, LLM can accomplish tasks that have never been seen before in completely new domains. We look forward to utilizing this capability in spectral analysis to reduce the time-consuming and operational difficulties. In this study, we used UV-NIR spectral analysis to predict the concentration of Chemical Oxygen Demand (COD) in three different water samples, including a complex wastewater. By extracting the characteristic bands in the spectrum, we input them into LLM for concentration prediction. We compared the COD prediction results of different models on water samples and discussed the effects of different experiments setting on LLM. The results show that even with brief prompts, the prediction of LLM in wastewater achieved the best performance, with R2 and RMSE equal to 0.931 and 10.966, which exceed the best results of traditional models, where R2 and RMSE correspond to 0.920 and 11.854. This result indicates that LLM, with simpler operation and less time-consuming, has ability to approach or even surpass traditional machine learning models in UV-NIR spectral analysis. In conclusion, our study proposed a new method for the UV-NIR spectral analysis based on LLM and preliminary demonstrated the potential of LLM for application.

Spectrophotometric determination of COD based on synergistic photocatalysis redox reaction using titanium dioxide nanoparticles and phosphomolybdic heteropoly acid

Chemical oxygen demand (COD) is one of the important indicators to measure the degree of organic pollution in water. In this work, a rapid spectrophotometric method for detection of COD was achieved based on the oxidation of organics in water by photogenerated holes or free radicals and the reduction of phosphomolybdic heteropolyacid by photogenerated electrons by using TiO2 nanoparticles as a photocatalyst. Taking potassium hydrogen phthalate as the COD standard, under the optimal conditions, the absorbance of reduced phosphomolybdic heteropoly acid was linear with COD in the range of 0.50-100 mg L -1. The detection limit for was COD detection was 0.171 mg L -1. The proposed methods was used for the determination of COD in real water samples, and the results were in general agreement with the national standard method. Compared with the direct photo initiated reduction of phosphomolybdic heteropoly acid without TiO2 nanoparticles, the photocatalytic reaction has better stability and higher efficiency.

HUM: A review of hydrochemical analysis using ultraviolet-visible absorption spectroscopy and machine learning

There is a need to develop improved methods for water quality analysis. Traditionally, water quality analysis is performed in a laboratory on discrete samples or in the field with simple sensors, but these methods have inherent limitations. Ultraviolet-visible absorption spectroscopy (UVAS) is a commonly used laboratory technique for water quality analysis and is being applied more broadly in combination with machine learning (ML) to allow for the detection of multiple analytes without sample pretreatments. This methodology (referred to here as Hydrochemical analysis using Ultraviolet-visible absorption spectroscopy and Machine learning; 'HUM') can be applied in the laboratory or in situ while requiring less time, labor, and materials compared to traditional laboratory analysis. HUM has been used for the quantification of a variety of chemicals in a variety of settings, but information is lacking related to instrumental setup, sample requirements, and data analysis procedures. For instance, there is a need to investigate the influence of spectral parameters (e.g., sensitivity, signal-to-noise ratio, and spectral resolution) on measurement error. There is also a lack of research aimed at developing ML algorithms specifically for HUM. Finally, there are emerging concepts such as sensor fusion and model-sensor fusion which have been applied to similar fields but are not common in studies involving HUM. This review suggests the need for further studies to better understand the factors that influence HUM measurement accuracy along with the need for hardware and software developments so that the methodology can ultimately become more robust and standardized. This, in turn, could increase its adoption in both academic and non-academic settings. Once the HUM methodology has matured, it could help to reduce the environmental impacts of society by improving our understanding and management of environmental systems through high-frequency data collection and automated control of water quality in environmentally relevant systems.

A straightforward approach utilizing an exponential model to compensate for turbidity in chemical oxygen demand measurements using UV-vis spectrometry

Recently, ultraviolet-visible (UV-vis) absorption spectrometry has garnered considerable attention because it enables real-time and unpolluted detection of chemical oxygen demand (COD) and plays a crucial role in the early warning of emerging organic contaminants. However, the accuracy of detection is inevitably constrained by the co-absorption of organic pollutants and turbidity at UV wavelengths. To ensure accurate detection of COD, it is necessary to directly subtract the absorbance caused by turbidity from the overlaid spectrum using the principle of superposition. The absorbance of COD is confined to the UV range, whereas that of turbidity extends across the entire UV-vis spectrum. Therefore, based on its visible absorbance, the UV absorbance of turbidity can be predicted. In this way, the compensation for turbidity is achieved by subtracting the predicted absorbance from the overlaid spectrum. Herein, a straightforward yet robust exponential model was employed based on this principle to predict the corresponding absorbance of turbidity at UV wavelengths. The model was used to analyze the overlaid absorption spectra of synthetic water samples containing COD and turbidity. The partial least squares (PLS) method was employed to predict the COD concentrations in synthetic water samples based on the compensated spectra, and the corresponding root mean square error (RMSE) values were recorded. The results indicated that the processed spectra yielded a considerably lower RMSE value (9.51) than the unprocessed spectra (29.9). Furthermore, the exponential model outperformed existing turbidity compensation models, including the Lambert-Beer law-based model (RMSE = 12.53) and multiple-scattering cluster method (RMSE = 79.34). Several wastewater samples were also analyzed to evaluate the applicability of the exponential model to natural water. UV analysis yielded undesirable results owing to filtration procedures. However, the consistency between the compensated spectra and filtered wastewater samples in the visible range demonstrated that the model is applicable to natural water. Therefore, this proposed method is advantageous for improving the accuracy of COD measurement in turbid water.

Towards highly economical and accurate wastewater sensors by reduced parts of the LED-visible spectrum

Interest is growing in simple, fast and inexpensive systems to analyze urban wastewater quality in real time. In this research project, a methodology is presented for the characterization of COD, BOD₅, TSS, TN, and TP of wastewater samples, without the need to alter the samples or use chemical reagents, from a few wavelengths, belonging to the different color groups that compose the visible spectrum in isolation: (380-700 nm): violet (380-427 nm), blue (427-476 nm), cyan (476-497 nm), green (497-570 nm), yellow (570-581 nm), orange (581-618 nm), and red (618-700 nm). In this study, about 650 raw and treated urban wastewater samples from over 43 WWTPs and a total of 36 estimation models based on genetic algorithms have been calculated. Seven models were calculated for each pollutant parameter; one model for each color group of the visible spectrum, except for TN, which includes an additional model combining the wavelengths of the violet and red region of the spectrum. All the calculated models showed high accuracy, with an R² between 80 and 85 % for COD, BOD₅ and TSS, and 66-74 % for TN and TP. The tests carried out have shown the accuracy of the models of the different color groups to be very close to each other. However, it is noted that the models making use of the wavelengths between 497 and 570 nm (green) were the ones that showed the best performance in all the parameters under study. This research work lays the foundations for the development of cheaper, faster, and simpler wastewater monitoring and characterization equipment.

Sensor setpoints that ensure compliance with microbial water quality targets for membrane bioreactor and chlorination treatment in on-site water reuse systems

Widespread implementation of on-site water reuse systems is hindered by the limited ability to ensure the level of treatment and protection of human health during operation. In this study, we tested the ability of five commercially available online sensors (free chlorine (FC), oxidation-reduction potential (ORP), pH, turbidity, UV absorbance at 254 nm) to predict the microbial water quality in membrane bioreactors followed by chlorination using logistic regression-based and mechanism-based models. The microbial water quality was assessed in terms of removal of enteric bacteria from the wastewater, removal of enteric viruses, and regrowth of bacteria in the treated water. We found that FC and ORP alone could predict the microbial water quality well, with ORP-based models generally performing better. We further observed that prediction accuracy did not increase when data from multiple sensors were integrated. We propose a methodology to link online sensor measurements to risk-based water quality targets, providing operation setpoints protective of human health for specific combinations of wastewaters and reuse applications. For instance, we recommend a minimum ORP of 705 mV to ensure a virus log-removal of 5, and an ORP of 765 mV for a log-removal of 6. These setpoints were selected to ensure that the percentage of events where the water is predicted to meet the quality target but it does not remains below 5%. Such a systematic approach to set sensor setpoints could be used in the development of water reuse guidelines and regulations that aim to cover a range of reuse applications with differential risks to human health.

Evaluation of genetic models for COD and TSS estimation in wastewater through its spectrophotometric response

In an urban wastewater treatment plant (WWTP), early knowledge of the pollutant load levels throughout the plant is key to optimize its processes and achieve better purification levels. Molecular spectrophotometry has begun to gain prominence in this wastewater characterization process, as it is a simple, fast, inexpensive and non-invasive technique. In this research work, different mathematical models based on genetic algorithms have been developed for the estimation of chemical oxygen demand (COD) and total suspended solids (TSS) from the spectral response of the samples, measured in the 380-700 nm range by means of a light-emitting diode (LED) spectrophotometer developed by the researchers. A field campaign was carried out in Mapocho-Trebal WWTP (Chile), where 550 samples were obtained in three different parts of the plant: at the inlet (raw wastewater), at the outlet (secondary treated wastewater) and at the outlet of the primary clarifier. A total of 18 estimation models have been calculated by mean of HeuristicLab software, which have presented a high accuracy, with a Pearson's coefficient between 80 and 90% in most cases. In order to achieve the most accurate models possible to characterize each part of the plant, specific models have also been developed, as well as combined models that are valid for all types of wastewater.

Implementing Early Warning Systems in WWTP. An investigation with cost-effective LED-VIS spectroscopy-based genetic algorithms

Measuring how the pollution load evolves in real time along sewer networks is key for proper management of water resources and protecting the environment. The technique of molecular spectroscopy for water characterization has increasingly widespread use, as it is a non-invasive technique that leads to the correlation of the physical-chemical conditions of wastewater with spectroscopic surrogates by a series of mathematical estimation models. In the present research work, different symbolic regression models obtained with evolutive genetic algorithms are evaluated for the estimation of chemical oxygen demand (COD); five-day biochemical oxygen demand (BOD₅); total suspended solids (TSS); total phosphorus (TP); and total nitrogen (TN), from the spectral response of samples measured between 380 and 700 nm and without the use of chemicals or pre-treatment. Around 650 wastewater samples were used in the campaign, from 43 different wastewater treatment plants (WWTP) in which both, raw/influent and treated/effluent, were examined through 18 models composed of Classical Genetic Algorithm (CGA), the Age-Layered Population Structure (ALPS), and Offspring Selection (OS) by mean of HeuristicLab software, to make a comparison among them and to determine which models and wavelengths are most suitable for the correlation. Models are proposed considering both raw and treated samples together (15) and only with tertiary treated wastewater reclaimed for agriculture irrigation effluent (3). The Pearson correlation coefficients were in the range of 67-91% for the test data in the case of the combined models. The results conform the first steps for a real-time monitoring of WWTP.

Spectrophotometric Online Detection of Drinking Water Disinfectant: A Machine Learning Approach

The spectra fingerprint of drinking water from a water treatment plant (WTP) is characterised by a number of light-absorbing substances, including organic, nitrate, disinfectant, and particle or turbidity. Detection of disinfectant (monochloramine) can be better achieved by separating its spectra from the combined spectra. In this paper, two major focuses are (i) the separation of monochloramine spectra from the combined spectra and (ii) assessment of the application of the machine learning algorithm in real-time detection of monochloramine. The support vector regression (SVR) model was developed using multi-wavelength ultraviolet-visible (UV-Vis) absorbance spectra and online amperometric monochloramine residual measurement data. The performance of the SVR model was evaluated by using four different kernel functions. Results show that (i) particles or turbidity in water have a significant effect on UV-Vis spectral measurement and improved modelling accuracy is achieved by using particle compensated spectra; (ii) modelling performance is further improved by compensating the spectra for natural organic matter (NOM) and nitrate (NO₃) and (iii) the choice of kernel functions greatly affected the SVR performance, especially the radial basis function (RBF) appears to be the highest performing kernel function. The outcomes of this research suggest that disinfectant residual (monochloramine) can be measured in real time using the SVR algorithm with a precision level of ± 0.1 mg L⁻¹.

Turbidity compensation method based on Mie scattering theory for water chemical oxygen demand determination by UV-Vis spectrometry

In view of the problem that chemical oxygen demand (COD) measurement in water using UV-Vis spectrometry was easily affected by turbidity, this paper proposed an analytical method for determining the complex refractive index of particles in water based on Lambert-Beer's law and K-K (Kramers-Kronig) relationship. The obtained complex refractive index was used to establish the turbidity compensation model in the COD characteristic spectral region, and the COD concentration inversion were achieved by using the PLS algorithm. The results show that the turbidity compensation method based on Mie scattering theory can improve the accuracy of COD measurement by UV-Vis spectroscopy. Compared with before turbidity compensation, R² (determination coefficient) between true values and predicted values of COD increased from 0.2274 to 0.9629, and RMSE (root mean square error) of predicted values decreased from 21.73 to 3.12 mg L^-1. Compared with 350 nm PC, derivative method, and improved MSC method, the turbidity compensation method for COD measurement based on Mie scattering theory is simple, fast, and highly accurate. And the calculated spectrum can represent the scattering characteristics of the measured spectrum. The average relative error between the fitted spectrum and the original normalized spectrum in the 55 mixed solutions was 0.52%, and the maximum relative error was 6.65%. This method can be useful for online COD measurement. Graphical abstract.

Global calibration model of UV-Vis spectroscopy for COD estimation in the effluent of rural sewage treatment facilities

In recent years, rural sewage treatment facilities have grown rapidly in China, and yet the water quality of the effluent has not been well monitored. The detection of chemical oxygen demand (COD) via ultraviolet-visible (UV-Vis) spectroscopy is an emerging technology with advantages of low cost and easy maintenance, which make it appropriate for the on-line monitoring of effluents from rural sewage treatment facilities. Because there are numerous sewage treatment devices in rural regions and as their locations are usually very scattered, it is difficult to calibrate the COD estimation model for each monitoring site. Hence, a COD estimation model with global calibration is a specific problem for application in rural regions. However, little research was performed on real rural sewage, yet much is desired in terms of the model accuracy and robustness. Consequently, a practical COD detection method with UV-Vis spectroscopy was established in this study. The COD estimation model was globally calibrated with effluents from rural sewage treatment devices. In order to avoid misleading data for evaluating the model performance caused by the differences in the COD concentration range of training sets, two new criteria, namely the Root Mean Square Relative Error (RMSRE) and Relative Error Variance (REV), were proposed to evaluate the model accuracy and robustness. Differences in the organic composition as characterized by excitation–emission matrix (EEM) fluorescence spectroscopy were shown to significantly affect the accuracy of the global calibration model. Through comparison among the methods of the partial least squares (PLS), support vector machine (SVM), and back-propagation neural network, PLS was verified to be able to attain sufficient accuracy and to be suitable for applying to the modeling with global calibration. A simplified modeling method was proposed to replace the absorption spectra at the full wavelength band with the absorbance at some specific wavelengths that were selected by interval partial least-squares regression (iPLSR) and synergy interval partial least-squares regression (siPLSR). In this study, the simplified model was proven to be reliable with three specific wavelengths: 251, 356, and 363 nm. An on-line COD monitor utilizing UV-Vis spectroscopy was thus developed for combining with the global calibration model.

In recent years, rural sewage treatment facilities have grown rapidly in China, and yet the water quality of the effluent has not been well monitored.

Predicting the redshift on the ultraviolet spectrum using the peak area method

The ultraviolet (UV) absorption peaks of NaCl, NaOH, and $\beta $β-phenylethylamine (PEA) in an aqueous solution move toward redshift. We proposed the peak area method for the quantitative analysis of PEA, NaCl, and NaOH. First, we obtained the predictable regularities of the redshift of the single component sample. Then, we obtained the regularities of the redshift of the UV spectrum for the mixture by the peak height and peak area methods. Finally, the BP-ANN algorithm was applied to determine the concentration of the mixture using the peak height and peak area method. The results of the testing set showed that correlation coefficients (${{\rm R}^2}$R²) of 0.992, 0.993, and 0.992 were obtained by peak height method and 0.997, 0.998, and 0.998 were obtained by peak area method for NaCl, NaOH, and PEA, respectively. Meanwhile, the relative errors of the prediction of NaCl, NaOH, and PEA obtained by peak area method were less than 3%, whereas the REP of NaCl, NaOH, and PEA obtained by peak height method were more than 5%. Compared with the results of the peak height method, the results showed that the peak area was more accurate than the peak height in predicting the redshift of the UV spectrum.