Kernel functions embedded in support vector machine learning models for rapid water pollution assessment via near-infrared spectroscopy

On-line measurement of COD and nitrate in water against stochastic background interference based on ultraviolet-visible spectroscopy and physics-informed multi-task learning

Traditional ultraviolet-visible spectroscopic quantitative analytical methods face challenges in simultaneous and long-term accurate measurement of chemical oxygen demand (COD) and nitrate due to spectral overlap and the interference from stochastic background caused by turbidity and chromaticity in water. Addressing these limitations, a compact dual optical path spectrum detection sensor is introduced, and a novel ultraviolet-visible spectroscopic quantitative analysis model based on physics-informed multi-task learning (PI-MTL) is designed. Incorporating a physics-informed block, the PI-MTL model integrates pre-existing physical knowledge for enhanced feature extraction specific to each task. A multi-task loss wrapper strategy is also employed, facilitating comprehensive loss evaluation and adaptation to stochastic backgrounds. This novel approach significantly outperforms conventional models in COD and nitrate measurement under stochastic background interference, achieving impressive prediction R2 values of 0.941 for COD and 0.9575 for nitrate, while reducing root mean squared error (RMSE) by 60.89 % for COD and 77.3 % for nitrate in comparison to the conventional chemometric model partial least squares regression (PLSR), and by 30.59 % and 65.96 %, respectively, in comparison to a benchmark convolutional neural network (CNN) model. The promising results emphasize its potential as a spectroscopic instrument designed for online multi-parameter water quality monitoring against stochastic background interference, enabling long-term accurate measurement of COD and nitrate levels.

NIR sensors combined with chemometric algorithms in intelligent quality evaluation of sweetpotato roots from 'Farm' to 'Table': Progresses, challenges, trends, and prospects

NIR sensors, in conjunction with advanced chemometric algorithms, have proven to be a powerful and efficient tool for intelligent quality evaluation of sweetpotato roots throughout the entire supply chain. By leveraging NIR data in different wavelength ranges, the physicochemical, nutritional and antioxidant compositions, as well as variety classification of sweetpotato roots during the different stages were adequately evaluated, and all findings involving quantitative and qualitative investigations from the beginning to the present were summarized and analyzed comprehensively. All chemometric algorithms including both linear and nonlinear employed in NIR analysis of sweetpotato roots were introduced in detail and their calibration performances in terms of regression and classification were assessed and discussed. The challenges and limitations of current NIR application in quality evaluation of sweetpotato roots are emphasized. The prospects and trends covering the ongoing advancements in software and hardware are suggested to support the sustainable and efficient sweetpotato processing and utilization.

Graphormer-IR: Graph Transformers Predict Experimental IR Spectra Using Highly Specialized Attention

Infrared (IR) spectroscopy is an important analytical tool in various chemical and forensic domains and a great deal of effort has gone into developing in silico methods for predicting experimental spectra. A key challenge in this regard is generating highly accurate spectra quickly to enable real-time feedback between computation and experiment. Here, we employ Graphormer, a graph neural network (GNN) transformer, to predict IR spectra using only simplified molecular-input line-entry system (SMILES) strings. Our data set includes 53,528 high-quality spectra, measured in five different experimental media (i.e., phases), for molecules containing the elements H, C, N, O, F, Si, S, P, Cl, Br, and I. When using only atomic numbers for node encodings, Graphormer-IR achieved a mean test spectral information similarity (SISμ) value of 0.8449 ± 0.0012 (n = 5), which surpasses that the current state-of-the-art model Chemprop-IR (SISμ = 0.8409 ± 0.0014, n = 5) with only 36% of the encoded information. Augmenting node embeddings with additional node-level descriptors in learned embeddings generated through a multilayer perceptron improves scores to SISμ = 0.8523 ± 0.0006, a total improvement of 19.7σ (t = 19). These improved scores show how Graphormer-IR excels in capturing long-range interactions like hydrogen bonding, anharmonic peak positions in experimental spectra, and stretching frequencies of uncommon functional groups. Scaling our architecture to 210 attention heads demonstrates specialist-like behavior for distinct IR frequencies that improves model performance. Our model utilizes novel architectures, including a global node for phase encoding, learned node feature embeddings, and a one-dimensional (1D) smoothing convolutional neural network (CNN). Graphormer-IR's innovations underscore its value over traditional message-passing neural networks (MPNNs) due to its expressive embeddings and ability to capture long-range intramolecular relationships.

Near-Infrared Spectroscopy Analysis of the Phytic Acid Content in Fuzzy Cottonseed Based on Machine Learning Algorithms

Cottonseed is rich in oil and protein. However, its antinutritional factor content, of phytic acid (PA), has limited its utilization. Near-infrared (NIR) spectroscopy, combined with chemometrics, is an efficient and eco-friendly analytical technique for crop quality analysis. Despite its potential, there are currently no established NIR models for measuring the PA content in fuzzy cottonseeds. In this research, a total of 456 samples of fuzzy cottonseed were used as the experimental materials. Spectral pre-treatments, including first derivative (1D) and standard normal variable transformation (SNV), were applied, and the linear partial least squares (PLS), nonlinear support vector machine (SVM), and random forest (RF) methods were utilized to develop accurate calibration models for predicting the content of PA in fuzzy cottonseed. The results showed that the spectral pre-treatment significantly improved the prediction performance of the models, with the RF model exhibiting the best prediction performance. The RF model had a coefficient of determination in prediction (R2p) of 0.9114, and its residual predictive deviation (RPD) was 3.9828, which indicates its high accuracy in measuring the PA content in fuzzy cottonseed. Additionally, this method avoids the costly and time-consuming delinting and crushing of cottonseeds, making it an economical and environmentally friendly alternative.

Empirical study of college students' extracurricular reading preference by functional data analysis of the library book borrowing behavior

Library data contains many students' reading records that reflect their general knowledge acquisition. The purpose of this study is to deeply mine the library book-borrowing data, with concerns on different book catalogues and properties to predict the students' extracurricular interests. An intelligent computing framework is proposed by the fusion of a neural network architecture and a partial differential equations (PDE) function module. In model designs, the architecture is constructed as an adaptive learning backpropagation neural network (BPNN), with automatic tuning of its hyperparameters. The PDE module is embedded into the network structure to enhance the loss functions of each neural perceptron. For model evaluation, a novel comprehensive index is designed using the calculus of information entropy. Empirical experiments are conducted on a diverse and multimodal time-series dataset of library book borrowing records to demonstrate the effectiveness of the proposed methodology. Results validate that the proposed framework is capable of revealing the students' extracurricular reading interests by processing related book borrowing records, and expected to be applied to "big data" analysis for a wide range of various libraries.

Water treatment and artificial intelligence techniques: a systematic literature review research

As clean water can be considered among the essentials of human life, there is always a requirement to seek its foremost and high quality. Water primarily becomes polluted due to organic as well as inorganic pollutants, including nutrients, heavy metals, and constant contamination with organic materials. Predicting the quality of water accurately is essential for its better management along with controlling pollution. With stricter laws regarding water treatment to remove organic and biologic materials along with different pollutants, looking for novel technologic procedures will be necessary for improved control of the treatment processes by water utilities. Linear regression-based models with relative simplicity considering water prediction have been typically used as available statistical models. Nevertheless, in a majority of real problems, particularly those associated with modeling of water quality, non-linear patterns will be observed, requiring non-linear models to address them. Thus, artificial intelligence (AI) can be a good candidate in modeling and optimizing the elimination of pollutants from water in empirical settings with the ability to generate ideal operational variables, due to its recent considerable advancements. Management and operation of water treatment procedures are supported technically by these technologies, leading to higher efficiency compared to sole dependence on human operations. Thus, establishing predictive models for water quality and subsequently, more efficient management of water resources would be critically important, serving as a strong tool. A systematic review methodology has been employed in the present work to investigate the previous studies over the time interval of 2010-2020, while analyzing and synthesizing the literature, particularly regarding AI application in water treatment. A total number of 92 articles had addressed the topic under study using AI. Based on the conclusions, the application of AI can obviously facilitate operations, process automation, and management of water resources in significantly volatile contexts.

Forecasting and Optimizing Dual Media Filter Performance via Machine Learning

Four different machine learning algorithms, including Decision Tree (DT), Random Forest (RF), Multivariable Linear Regression (MLR), Support Vector Regressions (SVR), and Gaussian Process Regressions (GPR), were applied to predict the performance of a multi-media filter operating as a function of raw water quality and plant operating variables. The models were trained using data collected over a seven year period covering water quality and operating variables, including true colour, turbidity, plant flow, and chemical dose for chlorine, KMnO₄, FeCl₃, and Cationic Polymer (PolyDADMAC). The machine learning algorithms have shown that the best prediction is at a 1-day time lag between input variables and unit filter run volume (UFRV). Furthermore, the RF algorithm with grid search using the input metrics mentioned above with a 1-day time lag has provided the highest reliability in predicting UFRV with a RMSE and R² of 31.58 and 0.98, respectively. Similarly, RF with grid search has shown the shortest training time, prediction accuracy, and forecasting events using a ROC-AUC curve analysis (AUC over 0.8) in extreme wet weather events. Therefore, Random Forest with grid search and a 1-day time lag is an effective and robust machine learning algorithm that can predict the filter performance to aid water treatment operators in their decision makings by providing real-time warning of the potential turbidity breakthrough from the filters.

Adopting improved Adam optimizer to train dendritic neuron model for water quality prediction

As one of continuous concern all over the world, the problem of water quality may cause diseases and poisoning and even endanger people's lives. Therefore, the prediction of water quality is of great significance to the efficient management of water resources. However, existing prediction algorithms not only require more operation time but also have low accuracy. In recent years, neural networks are widely used to predict water quality, and the computational power of individual neurons has attracted more and more attention. The main content of this research is to use a novel dendritic neuron model (DNM) to predict water quality. In DNM, dendrites combine synapses of different states instead of simple linear weighting, which has a better fitting ability compared with traditional neural networks. In addition, a recent optimization algorithm called AMSGrad (Adaptive Gradient Method) has been introduced to improve the performance of the Adam dendritic neuron model (ADNM). The performance of ADNM is compared with that of traditional neural networks, and the simulation results show that ADNM is better than traditional neural networks in mean square error, root mean square error and other indicators. Furthermore, the stability and accuracy of ADNM are better than those of other conventional models. Based on trained neural networks, policymakers and managers can use the model to predict the water quality. Real-time water quality level at the monitoring site can be presented so that measures can be taken to avoid diseases caused by water quality problems.

Facile and highly precise pH-value estimation using common pH paper based on machine learning techniques and supported mobile devices

Numerous scientific, health care, and industrial applications are showing increasing interest in developing optical pH sensors with low-cost, high precision that cover a wide pH range. Although serious efforts, the development of high accuracy and cost-effectiveness, remains challenging. In this perspective, we present the implementation of the machine learning technique on the common pH paper for precise pH-value estimation. Further, we develop a simple, flexible, and free precise mobile application based on a machine learning algorithm to predict the accurate pH value of a solution using an available commercial pH paper. The common light conditions were studied under different light intensities of 350, 200, and 20 Lux. The models were trained using 2689 experimental values without a special instrument control. The pH range of 1: 14 is covered by an interval of ~ 0.1 pH value. The results show a significant relationship between pH values and both the red color and green color, in contrast to the poor correlation by the blue color. The K Neighbors Regressor model improves linearity and shows a significant coefficient of determination of 0.995 combined with the lowest errors. The free, publicly accessible online and mobile application was developed and enables the highly precise estimation of the pH value as a function of the RGB color code of typical pH paper. Our findings could replace higher expensive pH instruments using handheld pH detection, and an intelligent smartphone system for everyone, even the chef in the kitchen, without the need for additional costly and time-consuming experimental work.

A Review of Machine Learning for Near-Infrared Spectroscopy

The analysis of infrared spectroscopy of substances is a non-invasive measurement technique that can be used in analytics. Although the main objective of this study is to provide a review of machine learning (ML) algorithms that have been reported for analyzing near-infrared (NIR) spectroscopy from traditional machine learning methods to deep network architectures, we also provide different NIR measurement modes, instruments, signal preprocessing methods, etc. Firstly, four different measurement modes available in NIR are reviewed, different types of NIR instruments are compared, and a summary of NIR data analysis methods is provided. Secondly, the public NIR spectroscopy datasets are briefly discussed, with links provided. Thirdly, the widely used data preprocessing and feature selection algorithms that have been reported for NIR spectroscopy are presented. Then, the majority of the traditional machine learning methods and deep network architectures that are commonly employed are covered. Finally, we conclude that developing the integration of a variety of machine learning algorithms in an efficient and lightweight manner is a significant future research direction.

Zinc Oxide/Phosphorus-Doped Carbon Nitride Composite as Potential Scaffold for Electrochemical Detection of Nitrofurantoin

Herein, we present an electrocatalyst constructed by zinc oxide hexagonal prisms/phosphorus-doped carbon nitride wrinkles (ZnO HPs/P-CN) prepared via a facile sonochemical method towards the detection of nitrofurantoin (NF). The ZnO HPs/P-CN-sensing platform showed amplified response and low-peak potential compared with other electrodes. The exceptional electrochemical performance could be credited to ideal architecture, rapid electron/charge transfer, good conductivity, and abundant active sites in the ZnO HPs/P-CN composite. Resulting from these merits, the ZnO HPs/P-CN-modified electrode delivered rapid response (2 s), a low detection limit (2 nM), good linear range (0.01-111 µM), high sensitivity (4.62 µA µM^-1 cm²), better selectivity, decent stability (±97.6%), and reproducibility towards electrochemical detection of NF. We further demonstrated the feasibility of the proposed ZnO HPs/P-CN sensor for detecting NF in samples of water and human urine. All the above features make our proposed ZnO HPs/P-CN sensor a most promising probe for detecting NF in natural samples.

Detection of Water pH Using Visible Near-Infrared Spectroscopy and One-Dimensional Convolutional Neural Network

pH is an important parameter for water quality detection. This study proposed a novel calibration regression strategy based on a one-dimensional convolutional neural network (1D-CNN) for water pH detection using visible near-infrared (Vis-NIR) spectroscopy. Two groups of Vis-NIR spectral analysis experiments of water pH detection were employed to evaluate the performance of 1D-CNN. Two conventional multivariate regression calibration methods, including partial least squares (PLS) and least squares support vector machine (LS-SVM), were introduced for comparative analysis with 1D-CNN. The successive projections algorithm (SPA) was adopted to select the feature variables. In addition, the learning mechanism of 1D-CNN was interpreted through visual feature maps by convolutional layers. The results showed that the 1D-CNN models obtained the highest prediction accuracy based on full spectra for the two experiments. For the spectrophotometer experiment, the root mean square error of prediction (RMSEP) was 0.7925, and the determination coefficient of prediction (Rp2) was 0.8515. For the grating spectrograph experiment, the RMSEP was 0.5128 and the Rp2 was 0.9273. The convolutional layers could automatically preprocess the spectra and effectively extract the spectra features. Compared with the traditional regression methods, 1D-CNN does not need complex spectra pretreatment and variable selection. Therefore, 1D-CNN is a promising regression approach, with higher prediction accuracy and better modeling convenience for rapid water pH detection using Vis-NIR spectroscopy.

Data-driven soft computing modeling of groundwater quality parameters in southeast Nigeria: comparing the performances of different algorithms

In recent decades, the simulation and modeling of water quality parameters have been useful for monitoring and assessment of the quality of water resources. Moreover, the use of multiple modeling techniques, rather than a standalone model, tends to provide more robust and reliable insights. In this present paper, several soft computing techniques were integrated and compared for the modeling of groundwater quality parameters (pH, electrical conductivity (EC), total dissolved solids (TDS), total hardness (TH), modified heavy metal index (MHMI), pollution load index (PLI), and synthetic pollution index (SPI)) in Ojoto area, SE Nigeria. Standard methods were employed in the physicochemical analysis of the groundwater resources. It was found that anthropogenic and non-anthropogenic activities influenced the concentrations of the water quality parameters. The PLI, MHMI, and SPI revealed that about 20-25% of the groundwater samples are unsuitable for drinking. Simple linear regression indicated that strong agreements exist between the results of the water quality indices. Principal component and Varimax-rotated factor analyses showed that Pb, Ni, and Zn influenced the judgment of the water quality indices most. Q-mode hierarchical and K-means clustering  algorithms grouped the water samples based on their pH, EC, TDS, TH, MHMI, PLI, and SPI values. Multiple linear regression (MLR) and artificial neural network (ANN) algorithms were used for the simulation and prediction of  the pH, EC, TDS, TH, PLI, MHMI, and SPI. The MLR performed better than the ANN model in predicting EC, TH, and TDS. Nevertheless, the ANN model predicted the pH better than the MLR model. Meanwhile, both MLR and ANN performed equally in the prediction of PLI, MHMI, and SPI.

Smart Earth Technologies: a pressing need for abating pollution for a better tomorrow

Standing at the cusp of an augmented age facilitates a glance into the future of a cybernetic world aligned with planetary wellbeing. The era of exponential technological revolutions has brought with it a plethora of opportunities expanding in a multi-faceted dimension with an added emphasis towards nurturing a mutual synergy of nature with a daily dose of digitalization. The paper is written with an intent to lay out an accumulated comprehensive review of different literary works which lay the grounds for how different Smart Earth Technologies aid in monitoring and tackling the degradation of air and water resources. If an intertwined state-of-the-art centralized research source could be created, it would become a boon for seasoned researchers and neophytes succeeding portion of the article expands itself to a wide variety of research literature complimented with real-time models, case, and empirical studies which help heighten the previous limit to the research done on these Technologies tinkering the present monitoring systems. The primary aim of this work is to fuel the need of theoretical, practical, and empirical evolution in the ways the intelligent technologies help blossom a pollution-free environment. The secondary intention was to ensure that in-depth study of Smart Environmental Pollution the Monitoring Systems provisioned a multitude of prospects for upgrading one's knowledge on environmental management through current world technologies. By looking at these trends of the past, the enthusiast of technology could collaborate with the researchers of Environmental Pollution to assist in proliferation of diverse 'smart' solutions creating a Smarter, Greener, and Brighter future for research and developments in Sustainable Technologies devising a pollution-free environment.

Multi-Features-Based Automated Breast Tumor Diagnosis Using Ultrasound Image and Support Vector Machine

Breast ultrasound examination is a routine, fast, and safe method for clinical diagnosis of breast tumors. In this paper, a classification method based on multi-features and support vector machines was proposed for breast tumor diagnosis. Multi-features are composed of characteristic features and deep learning features of breast tumor images. Initially, an improved level set algorithm was used to segment the lesion in breast ultrasound images, which provided an accurate calculation of characteristic features, such as orientation, edge indistinctness, characteristics of posterior shadowing region, and shape complexity. Simultaneously, we used transfer learning to construct a pretrained model as a feature extractor to extract the deep learning features of breast ultrasound images. Finally, the multi-features were fused and fed to support vector machine for the further classification of breast ultrasound images. The proposed model, when tested on unknown samples, provided a classification accuracy of 92.5% for cancerous and noncancerous tumors.

The global research trend on cadmium in freshwater: a bibliometric review

Cadmium pollution turns out to be a global environmental problem. This study conducted a quantitative and qualitative bibliometric analysis based on 9188 research items from the Web of Science Core Collection published in the last 20 years (2000-2020), presenting an in-depth statistical investigation of global freshwater cadmium research progress and developing trend. Our results demonstrated that the researchers from China, the USA, and India contribute the most to this field. The primary sources of cadmium are mining, industry, wastewater, sedimentation, and agricultural activities. In developing countries, cadmium exposure occurs mainly through the air, freshwater, and food. Fish and vegetables are the main food sources of cadmium for humans because of their high accumulation capability. Source evaluation, detection, and remediation represent the main technologies used to clean up cadmium-contaminated sites. To mitigate the risk of cadmium contamination in freshwater, biomarker-based cadmium monitoring methods and integrated policies/strategies to reduce cadmium exposure merit further concern.

Early Diagnosis of Type 2 Diabetes Based on Near-Infrared Spectroscopy Combined With Machine Learning and Aquaphotomics

Early diagnosis is important to reduce the incidence and mortality rate of diabetes. The feasibility of early diagnosis of diabetes was studied via near-infrared spectra (NIRS) combined with a support vector machine (SVM) and aquaphotomics. Firstly, the NIRS of entire blood samples from the population of healthy, pre-diabetic, and diabetic patients were obtained. The spectral data of the entire spectra in the visible and near-infrared region (400–2,500 nm) were used as the research object of the qualitative analysis. Secondly, several preprocessing steps including multiple scattering correction, variable standardization, and first derivative and second derivative steps were performed and the best pretreatment method was selected. Finally, for the early diagnosis of diabetes, models were established using SVM. The first overtone of water (1,300–1,600 nm) was used as the research object for an aquaphotomics model, and the aquagram of the healthy group, pre-diabetes, and diabetes groups were drawn using 12 water absorption patterns for the early diagnosis of diabetes. The results of SVM showed that the highest accuracy was 97.22% and the specificity and sensitivity were 95.65 and 100%, respectively when the pretreatment method of the first derivative was used, and the best model parameters were c = 18.76 and g = 0.008583.The results of the aquaphotomics model showed clear differences in the 1,400–1,500 nm region, and the number of hydrogen bonds in water species (1,408, 1,416, 1,462, and 1,522 nm) was evidently correlated with the occurrence and development of diabetes. The number of hydrogen bonds was the smallest in the healthy group and the largest in the diabetes group. The suggested reason is that the water matrix of blood changes with the worsening of blood glucose metabolic dysfunction. The number of hydrogen bonds could be used as biomarkers for the early diagnosis of diabetes. The result show that it is effective and feasible to establish an accurate and rapid early diagnosis model of diabetes via NIRS combined with SVM and aquaphotomics.

Spectrophotometric Online Detection of Drinking Water Disinfectant: A Machine Learning Approach

The spectra fingerprint of drinking water from a water treatment plant (WTP) is characterised by a number of light-absorbing substances, including organic, nitrate, disinfectant, and particle or turbidity. Detection of disinfectant (monochloramine) can be better achieved by separating its spectra from the combined spectra. In this paper, two major focuses are (i) the separation of monochloramine spectra from the combined spectra and (ii) assessment of the application of the machine learning algorithm in real-time detection of monochloramine. The support vector regression (SVR) model was developed using multi-wavelength ultraviolet-visible (UV-Vis) absorbance spectra and online amperometric monochloramine residual measurement data. The performance of the SVR model was evaluated by using four different kernel functions. Results show that (i) particles or turbidity in water have a significant effect on UV-Vis spectral measurement and improved modelling accuracy is achieved by using particle compensated spectra; (ii) modelling performance is further improved by compensating the spectra for natural organic matter (NOM) and nitrate (NO₃) and (iii) the choice of kernel functions greatly affected the SVR performance, especially the radial basis function (RBF) appears to be the highest performing kernel function. The outcomes of this research suggest that disinfectant residual (monochloramine) can be measured in real time using the SVR algorithm with a precision level of ± 0.1 mg L⁻¹.

The statistical fusion identification of dairy products based on extracted Raman spectroscopy

At present, practical and rapid identification techniques for dairy products are still scarce. Taking different brands of pasteurized milk as an example, they are all milky white in appearance, and their Raman spectra are very similar, so it is not feasible to identify them directly using the naked eye. In the current work, a clear feature extraction and fusion strategy based on a combination of Raman spectroscopy and a support vector machine (SVM) algorithm was demonstrated. The results showed a 58% average recognition accuracy rate for dairy products as based on the original Raman full spectral data and up to nearly 70% based on a single spectral interval. Data normalization processing effectively improved the recognition accuracy rate. The average recognition accuracy rate of dairy products reached 91% based on the normalized Raman full spectral data or nearly 85% based on a normalized single spectral interval. The fusion of multispectral feature regions yielded high accuracy and operation efficiency. After screening and optimizing based on SVM algorithm, the best spectral feature intervals were determined to be 335–354 cm⁻¹, 435–454 cm⁻¹, 485–540 cm⁻¹, 820–915 cm⁻¹, 1155–1185 cm⁻¹, 1300–1414 cm⁻¹, and 1415–1520 cm⁻¹ under the experimental conditions, and the average identification accuracy rate here reached 93%. The developed scheme has the advantages of clear feature extraction and fusion, and short identification time, and it provides a technical reference for food quality control.

At present, practical and rapid identification techniques for dairy products are still scarce.