Outlier detection and gap filling methodologies for low-cost air quality measurements

An Image-Based Sensor System for Low-Cost Airborne Particle Detection in Citizen Science Air Quality Monitoring

Air pollution poses significant public health risks, necessitating accurate and efficient monitoring of particulate matter (PM). These organic compounds may be released from natural sources like trees and vegetation, as well as from anthropogenic, or human-made sources including industrial activities and motor vehicle emissions. Therefore, measuring PM concentrations is paramount to understanding people's exposure levels to pollutants. This paper introduces a novel image processing technique utilizing photographs/pictures of Do-it-Yourself (DiY) sensors for the detection and quantification of PM10 particles, enhancing community involvement and data collection accuracy in Citizen Science (CS) projects. A synthetic data generation algorithm was developed to overcome the challenge of data scarcity commonly associated with citizen-based data collection to validate the image processing technique. This algorithm generates images by precisely defining parameters such as image resolution, image dimension, and PM airborne particle density. To ensure these synthetic images mimic real-world conditions, variations like Gaussian noise, focus blur, and white balance adjustments and combinations were introduced, simulating the environmental and technical factors affecting image quality in typical smartphone digital cameras. The detection algorithm for PM10 particles demonstrates robust performance across varying levels of noise, maintaining effectiveness in realistic mobile imaging conditions. Therefore, the methodology retains sufficient accuracy, suggesting its practical applicability for environmental monitoring in diverse real-world conditions using mobile devices.

Data-driven evolution of water quality models: An in-depth investigation of innovative outlier detection approaches-A case study of Irish Water Quality Index (IEWQI) model

Recently, there has been a significant advancement in the water quality index (WQI) models utilizing data-driven approaches, especially those integrating machine learning and artificial intelligence (ML/AI) technology. Although, several recent studies have revealed that the data-driven model has produced inconsistent results due to the data outliers, which significantly impact model reliability and accuracy. The present study was carried out to assess the impact of data outliers on a recently developed Irish Water Quality Index (IEWQI) model, which relies on data-driven techniques. To the author's best knowledge, there has been no systematic framework for evaluating the influence of data outliers on such models. For the purposes of assessing the outlier impact of the data outliers on the water quality (WQ) model, this was the first initiative in research to introduce a comprehensive approach that combines machine learning with advanced statistical techniques. The proposed framework was implemented in Cork Harbour, Ireland, to evaluate the IEWQI model's sensitivity to outliers in input indicators to assess the water quality. In order to detect the data outlier, the study utilized two widely used ML techniques, including Isolation Forest (IF) and Kernel Density Estimation (KDE) within the dataset, for predicting WQ with and without these outliers. For validating the ML results, the study used five commonly used statistical measures. The performance metric (R2) indicates that the model performance improved slightly (R2 increased from 0.92 to 0.95) in predicting WQ after removing the data outlier from the input. But the IEWQI scores revealed that there were no statistically significant differences among the actual values, predictions with outliers, and predictions without outliers, with a 95 % confidence interval at p < 0.05. The results of model uncertainty also revealed that the model contributed <1 % uncertainty to the final assessment results for using both datasets (with and without outliers). In addition, all statistical measures indicated that the ML techniques provided reliable results that can be utilized for detecting outliers and their impacts on the IEWQI model. The findings of the research reveal that although the data outliers had no significant impact on the IEWQI model architecture, they had moderate impacts on the rating schemes' of the model. This finding indicated that detecting the data outliers could improve the accuracy of the IEWQI model in rating WQ as well as be helpful in mitigating the model eclipsing problem. In addition, the results of the research provide evidence of how the data outliers influenced the data-driven model in predicting WQ and reliability, particularly since the study confirmed that the IEWQI model's could be effective for accurately rating WQ despite the presence of the data outliers in the input. It could occur due to the spatio-temporal variability inherent in WQ indicators. However, the research assesses the influence of data input outliers on the IEWQI model and underscores important areas for future investigation. These areas include expanding temporal analysis using multi-year data, examining spatial outlier patterns, and evaluating detection methods. Moreover, it is essential to explore the real-world impacts of revised rating categories, involve stakeholders in outlier management, and fine-tune model parameters. Analysing model performance across varying temporal and spatial resolutions and incorporating additional environmental data can significantly enhance the accuracy of WQ assessment. Consequently, this study offers valuable insights to strengthen the IEWQI model's robustness and provides avenues for enhancing its utility in broader WQ assessment applications. Moreover, the study successfully adopted the framework for evaluating how data input outliers affect the data-driven model, such as the IEWQI model. The current study has been carried out in Cork Harbour for only a single year of WQ data. The framework should be tested across various domains for evaluating the response of the IEWQI model's in terms of the spatio-temporal resolution of the domain. Nevertheless, the study recommended that future research should be conducted to adjust or revise the IEWQI model's rating schemes and investigate the practical effects of data outliers on updated rating categories. However, the study provides potential recommendations for enhancing the IEWQI model's adaptability and reveals its effectiveness in expanding its applicability in more general WQ assessment scenarios.

Anomaly Detection and Repairing for Improving Air Quality Monitoring

Clean air in cities improves our health and overall quality of life and helps fight climate change and preserve our environment. High-resolution measures of pollutants' concentrations can support the identification of urban areas with poor air quality and raise citizens' awareness while encouraging more sustainable behaviors. Recent advances in Internet of Things (IoT) technology have led to extensive use of low-cost air quality sensors for hyper-local air quality monitoring. As a result, public administrations and citizens increasingly rely on information obtained from sensors to make decisions in their daily lives and mitigate pollution effects. Unfortunately, in most sensing applications, sensors are known to be error-prone. Thanks to Artificial Intelligence (AI) technologies, it is possible to devise computationally efficient methods that can automatically pinpoint anomalies in those data streams in real time. In order to enhance the reliability of air quality sensing applications, we believe that it is highly important to set up a data-cleaning process. In this work, we propose AIrSense, a novel AI-based framework for obtaining reliable pollutant concentrations from raw data collected by a network of low-cost sensors. It enacts an anomaly detection and repairing procedure on raw measurements before applying the calibration model, which converts raw measurements to concentration measurements of gasses. There are very few studies of anomaly detection in raw air quality sensor data (millivolts). Our approach is the first that proposes to detect and repair anomalies in raw data before they are calibrated by considering the temporal sequence of the measurements and the correlations between different sensor features. If at least some previous measurements are available and not anomalous, it trains a model and uses the prediction to repair the observations; otherwise, it exploits the previous observation. Firstly, a majority voting system based on three different algorithms detects anomalies in raw data. Then, anomalies are repaired to avoid missing values in the measurement time series. In the end, the calibration model provides the pollutant concentrations. Experiments conducted on a real dataset of 12,000 observations produced by 12 low-cost sensors demonstrated the importance of the data-cleaning process in improving calibration algorithms' performances.

Effects of air pollution on clinical pregnancy rates after in vitro fertilisation (IVF): a retrospective cohort study

OBJECTIVE

To evaluate the effect of air pollution, from oocyte retrieval to embryo transfer, on the results of in vitro fertilisation (IVF) in terms of clinical pregnancy rates, at two fertility centres, from 2013 to 2019.

DESIGN

Exploratory retrospective cohort study.

SETTING

This retrospective cohort study was performed in the Reproductive Biology Department of Bordeaux University Hospital localised in Bordeaux, France and the Jean Villar Fertility Center localised in Bruges, France.

PARTICIPANTS

This study included 10 763 IVF attempts occurring between January 2013 and December 2019, 2194 of which resulted in a clinical pregnancy.

PRIMARY AND SECONDARY OUTCOME MEASURES

The outcome of the IVF attempt was recorded as the presence or absence of a clinical pregnancy; exposure to air pollution was assessed by calculating the cumulative exposure of suspended particulate matter, fine particulate matter, black carbon, nitrogen dioxide and ozone (O₃₎, over the period from oocyte retrieval to embryo transfer, together with secondary exposure due to the presence of the biomass boiler room, which was installed in 2016, close to the Bordeaux University Hospital laboratory. The association between air pollution and IVF outcome was evaluated by a random-effects logistic regression analysis.

RESULTS

We found negative associations between cumulative O₃ exposure and clinical pregnancy rate (OR=0.92, 95% CI = (0.86 to 0.98)), and between biomass boiler room exposure and clinical pregnancy rate (OR=0.75, 95% CI = (0.61 to 0.91)), after adjustment for potential confounders.

CONCLUSION

Air pollution could have a negative effect on assisted reproductive technology results and therefore precautions should be taken to minimise the impact of outdoor air on embryo culture.

Immission levels and identification of sulfur dioxide sources in La Oroya city, Peruvian Andes

La Oroya is a city in the Peruvian Andes that has suffered a serious deterioration in its air quality, especially due to the high rate of sulfur dioxide (SO2) emissions, which underlines the importance of knowing its sources of contamination and variation over the years. In this sense, this study aimed to evaluate the immission levels and determine the sources of SO2 contamination in La Oroya. This analysis was performed using the hourly concentration data of SO2, and meteorological variables (wind speed and direction), which were analyzed for a period of three years (2018-2020). Graphs of time series, wind and pollutant roses, bivariate polar graphs, clustering k-means, nonparametric statistical tests, and the application of the conditional bivariate probability function were performed to analyze the data and identify the emission sources. The mean concentration of SO2 was 264.2 μg m-3 for the study period, where 55.66 and 2.37% of the evaluated days exceeded the guideline values recommended by the World Health Organization and the Peruvian Environmental Quality Standard for air for 24 h, respectively. The results showed a defined pattern for the daily and monthly variations, with peaks in the morning hours (0900-1000 h LT) and at the end of the year (December), respectively. The main sources of SO2 emissions identified were light and heavy vehicles that travel through the Central Highway, the La Oroya Metallurgical Complex, the transit of vehicles within the city, and the diesel-electric locomotives that provide cargo transportation services and tourism passenger transportation. The article attempts to contribute to the development of adequate air quality management policies.

Household PM2.5 pollution in rural Chinese homes: Levels, dynamic characteristics and seasonal variations

Humans generally spend most of their time indoors, and fine particulate matter (PM_2.5) in indoor air can have seriously adverse effects on human health due to the long exposure time. This study conducted field measurements to explore seasonal variations of PM_2.5 concentrations in household air by revisiting the same rural homes in southern China and factors influencing indoor PM_2.5 concentrations were explored mainly by one-way ANOVA. The PM_2.5 concentrations of outdoor, kitchen and living room air were 38.9 ± 12.2, 47.1 ± 20.3 and 50.8 ± 24.1 μg/m³ in summer, respectively, which were 2.3 to 2.9 times lower than those in winter (p < 0.05). The lower indoor PM_2.5 pollution in summer was attributed to the transition to clean household energy and better ventilation. Fuel type can significantly affect PM_2.5 concentrations in the kitchen, with greater PM_2.5 pollution associated with wood combustion than electricity. Our study firstly found mosquito coil emission was an important contributor to PM_2.5 in the living room of rural households, which should be investigated further. Dynamic variations of PM_2.5 suggested that cooking, heating and mosquito coil emission can rapidly increase indoor PM_2.5 concentrations (up to one order of magnitude higher than baseline values), as well as the indoor/outdoor PM_2.5 ratios. This study had the first insight of seasonal differences of household PM_2.5 in the same rural homes using real-time monitors, confirming the different patterns and characteristics of household PM_2.5 pollution in different seasons.

Malaysia PM10 Air Quality Time Series Clustering Based on Dynamic Time Warping

Air quality monitoring is important in the management of the environment and pollution. In this study, time series of PM10 from air quality monitoring stations in Malaysia were clustered based on similarity in terms of time series patterns. The identified clusters were analyzed to gain meaningful information regarding air quality patterns in Malaysia and to identify characterization for each cluster. PM10 time series data from 5 July 2017 to 31 January 2019, obtained from the Malaysian Department of Environment and Dynamic Time Warping as the dissimilarity measure were used in this study. At the same time, k-Means, Partitioning Around Medoid, agglomerative hierarchical clustering, and Fuzzy k-Means were the algorithms used for clustering. The results portray that the categories and activities of locations of the monitoring stations do not directly influence the pattern of the PM10 values, instead, the clusters formed are mainly influenced by the region and geographical area of the locations.

Building Low-Cost Sensing Infrastructure for Air Quality Monitoring in Urban Areas Based on Fog Computing

Because the number of air quality measurement stations governed by a public authority is limited, many methodologies have been developed in order to integrate low-cost sensors and to improve the spatial density of air quality measurements. However, at the large-scale level, the integration of a huge number of sensors brings many challenges. The volume, velocity and processing requirements regarding the management of the sensor life cycle and the operation of system services overcome the capabilities of the centralized cloud model. In this paper, we present the methodology and the architectural framework for building large-scale sensing infrastructure for air quality monitoring applicable in urban scenarios. The proposed tiered architectural solution based on the adopted fog computing model is capable of handling the processing requirements of a large-scale application, while at the same time sustaining real-time performance. Furthermore, the proposed methodology introduces the collection of methods for the management of edge-tier node operation through different phases of the node life cycle, including the methods for node commission, provision, fault detection and recovery. The related sensor-side processing is encapsulated in the form of microservices that reside on the different tiers of system architecture. The operation of system microservices and their collaboration was verified through the presented experimental case study.

Low-cost sensor outlier detection framework for on-line monitoring of particle pollutants in multiple scenarios

Monitoring the concentration of particle pollutants is very important for industrial production control and workers' health protection. Low-cost sensors are widely used to reduce deployment costs. The outliers in the observed data of pollutant concentration can be eliminated by outlier detection algorithms. However, it is difficult to meet the actual needs of changing working conditions or scene migration in factories by building a single algorithm for specific scenarios. It is a feasible scheme to identify the changing characteristics of data and adaptively adjust the outlier detection algorithm. From the point of view of data characteristics, we creatively match typical data types with high-performance algorithms. The framework proposed in this paper provides a general process including five basic tasks and uses a modular structure to complete the outlier detection target. The actual pollutant data of the workshops are used to evaluate the performance of our framework. At last, we compare eight different strategies under this framework and analyze the contribution of each step to outlier detection from the perspective of algorithm principle. The results show that low-cost sensors following the framework can meet the outlier detection requirements in the field of pollutant monitoring, thus greatly reducing the cost of algorithm selection and data adaptation.

Evaluation of field calibration methods and performance of AQMesh, a low-cost air quality monitor

Field calibrations of NO₂, NO, and PM₁₀ from AQMesh Air Quality Monitors (AQMs) were conducted during a summer and an autumn period in a busy street in a midsize Swedish city. All the three linear calibration procedures studied (postscaled, bisquare, and orthogonal data) significantly reduced the ranges and magnitudes of the performance indicators to yield more reliable results than the raw data. The improvements were sufficient to satisfy the European Union (EU) Data Quality Objective (DQO) for indicative measurements as compared to reference data only for NO₂ (above 50 µg m^-3) and NO (above 30 µg m^-3) during the autumn calibration period. The relatively simple bisquare procedure had the best performance overall. The bisquare procedure improved the root mean square error by the same amount as other studies using complex multivariate calibration methods. Low concentrations of pollutants were measured, far below the EU Environmental Quality Standard thresholds and even satisfying the future goals for the Environmental Quality Objectives. Cleaning the raw data by removing data points in the reference data that were below the reference station limit of detections (and the synchronous data points in the AQM prescaled data) was found to improve the performances of the calibration procedures appreciably. Many NO₂ and almost all PM₁₀ data points in this study fell below the AQM limit of detection. These low concentrations will probably be a common problem in many field studies, at least in areas with relatively low air pollution. However, the relative errors were sufficiently low for these data points that they could be interpreted as accurately representing low concentrations and did not need to be removed from the datasets. For the NO₂ measurements, a slight periodic error correlated with sunlight and increased ambient temperature was noted. NO measurements correlated strongly with increased traffic.

Low-Cost Air Quality Sensing towards Smart Homes

The evolution of low-cost sensors (LCSs) has made the spatio-temporal mapping of indoor air quality (IAQ) possible in real-time but the availability of a diverse set of LCSs make their selection challenging. Converting individual sensors into a sensing network requires the knowledge of diverse research disciplines, which we aim to bring together by making IAQ an advanced feature of smart homes. The aim of this review is to discuss the advanced home automation technologies for the monitoring and control of IAQ through networked air pollution LCSs. The key steps that can allow transforming conventional homes into smart homes are sensor selection, deployment strategies, data processing, and development of predictive models. A detailed synthesis of air pollution LCSs allowed us to summarise their advantages and drawbacks for spatio-temporal mapping of IAQ. We concluded that the performance evaluation of LCSs under controlled laboratory conditions prior to deployment is recommended for quality assurance/control (QA/QC), however, routine calibration or implementing statistical techniques during operational times, especially during long-term monitoring, is required for a network of sensors. The deployment height of sensors could vary purposefully as per location and exposure height of the occupants inside home environments for a spatio-temporal mapping. Appropriate data processing tools are needed to handle a huge amount of multivariate data to automate pre-/post-processing tasks, leading to more scalable, reliable and adaptable solutions. The review also showed the potential of using machine learning technique for predicting spatio-temporal IAQ in LCS networked-systems.

Low-cost sensor system for monitoring the oil mist concentration in a workshop

Metalworking fluids used in industrial workshops may present a major threat to the health of workers who have been exposed to a high oil mist concentration over a long period of time. Therefore, monitoring the temporal and spatial distribution of particulate matter concentration has great practical significance for the control of oil mist. Traditional particle monitors are generally cumbersome, expensive, and difficult to maintain, which to some extent restricts their extensive use in workshops. Recent years have witnessed tremendous developments in the area of low-cost sensors, which are of great help in obtaining high-density pollution data. In this paper, we evaluate the performance of an inexpensive laser sensor (A4-CG) during long-term oil mist monitoring in a machine shop for the first time. With the use of Lora technology, we developed an online oil mist monitoring network to access real-time concentration, temperature, and humidity information from distributed monitors. According to the results, the sensor data correlated well with measurements by the reference instrument (R² = 0.96), which means that the distributed sensor network can accurately detect the concentration level of oil mist in the workshop. In fact, most of the sensors demonstrated stable operation for up to half a year according to cluster analysis, while several sensors exhibited serious data drift. Furthermore, the results indicate that the peak oil mist concentration in most areas during production exceeded the value of 0.5 mg m^-3 recommended by NIOSH, and it was found that appropriately lowering the relative humidity can make sampling more accurate, while lowering the temperature can reduce the oil mist concentration in the workshop. Thus, measures to control oil mist such as generation and distribution of pollution sources should be on the agenda.

Community-Based Measurements Reveal Unseen Differences during Air Pollution Episodes

Short-term exposure to fine particulate matter (PM_2.5) pollution is linked to numerous adverse health effects. Pollution episodes, such as wildfires, can lead to substantial increases in PM_2.5 levels. However, sparse regulatory measurements provide an incomplete understanding of pollution gradients. Here, we demonstrate an infrastructure that integrates community-based measurements from a network of low-cost PM_2.5 sensors with rigorous calibration and a Gaussian process model to understand neighborhood-scale PM_2.5 concentrations during three pollution episodes (July 4, 2018, fireworks; July 5 and 6, 2018, wildfire; Jan 3-7, 2019, persistent cold air pool, PCAP). The firework/wildfire events included 118 sensors in 84 locations, while the PCAP event included 218 sensors in 138 locations. The model results accurately predict reference measurements during the fireworks (n: 16, hourly root-mean-square error, RMSE, 12.3-21.5 μg/m³, n(normalized)RMSE: 14.9-24%), the wildfire (n: 46, RMSE: 2.6-4.0 μg/m³; nRMSE: 13.1-22.9%), and the PCAP (n: 96, RMSE: 4.9-5.7 μg/m³; nRMSE: 20.2-21.3%). They also revealed dramatic geospatial differences in PM_2.5 concentrations that are not apparent when only considering government measurements or viewing the US Environmental Protection Agency's AirNow visualizations. Complementing the PM_2.5 estimates and visualizations are highly resolved uncertainty maps. Together, these results illustrate the potential for low-cost sensor networks that combined with a data-fusion algorithm and appropriate calibration and training can dynamically and with improved accuracy estimate PM_2.5 concentrations during pollution episodes. These highly resolved uncertainty estimates can provide a much-needed strategy to communicate uncertainty to end users.

Long-term calibration models to estimate ozone concentrations with a metal oxide sensor

Ozone (O₃) is a potent oxidant associated with adverse health effects. Low-cost O₃ sensors, such as metal oxide (MO) sensors, can complement regulatory O₃ measurements and enhance the spatiotemporal resolution of measurements. However, the quality of MO sensor data remains a challenge. The University of Utah has a network of low-cost air quality sensors (called AirU) that primarily measures PM_2.5 concentrations around the Salt Lake City valley (Utah, U.S.). The AirU package also contains a low-cost MO sensor ($8) that measures oxidizing/reducing species. These MO sensors exhibited excellent laboratory response to O₃ although they exhibited some intra-sensor variability. Field performance was evaluated by placing eight AirUs at two Division of Air Quality (DAQ) monitoring stations with O₃ federal equivalence methods for one year to develop long-term multiple linear regression (MLR) and artificial neural network (ANN) calibration models to predict O₃ concentrations. Six sensors served as train/test sets. The remaining two sensors served as a holdout set to evaluate the applicability of the new calibration models in predicting O₃ concentrations for other sensors of the same type. A rigorous variable selection method was also performed by least absolute shrinkage and selection operator (LASSO), MLR and ANN models. The variable selection indicated that the AirU's MO oxidizing species and temperature measurements and DAQ's solar radiation measurements were the most important variables. The MLR calibration model exhibited moderate performance (R² = 0.491), and the ANN exhibited good performance (R² = 0.767) for the holdout set. We also evaluated the performance of the MLR and ANN models in predicting O₃ for five months after the calibration period and the results showed moderate correlations (R²s of 0.427 and 0.567, respectively). These low-cost MO sensors combined with a long-term ANN calibration model can complement reference measurements to understand geospatial and temporal differences in O₃ levels.

Temporary reduction in fine particulate matter due to 'anthropogenic emissions switch-off' during COVID-19 lockdown in Indian cities

The COVID-19 pandemic elicited a global response to limit associated mortality, with social distancing and lockdowns being imposed. In India, human activities were restricted from late March 2020. This 'anthropogenic emissions switch-off' presented an opportunity to investigate impacts of COVID-19 mitigation measures on ambient air quality in five Indian cities (Chennai, Delhi, Hyderabad, Kolkata, and Mumbai), using in-situ measurements from 2015 to 2020. For each year, we isolated, analysed and compared fine particulate matter (PM2.5) concentration data from 25 March to 11 May, to elucidate the effects of the lockdown. Like other global cities, we observed substantial reductions in PM2.5 concentrations, from 19 to 43% (Chennai), 41-53% (Delhi), 26-54% (Hyderabad), 24-36% (Kolkata), and 10-39% (Mumbai). Generally, cities with larger traffic volumes showed greater reductions. Aerosol loading decreased by 29% (Chennai), 11% (Delhi), 4% (Kolkata), and 1% (Mumbai) against 2019 data. Health and related economic impact assessments indicated 630 prevented premature deaths during lockdown across all five cities, valued at 0.69 billion USD. Improvements in air quality may be considered a temporary lockdown benefit as revitalising the economy could reverse this trend. Regulatory bodies must closely monitor air quality levels, which currently offer a baseline for future mitigation plans.

Assessment of Daily Personal PM2.5 Exposure Level According to Four Major Activities among Children

Particulate matters less than 2.5 micrometers in diameter (PM2.5), whose concentration has increased in Korea, has a considerable impact on health. From a risk management point of view, there has been interest in understanding the variations in real-time PM2.5 concentrations per activity in different microenvironments. We analyzed personal monitoring data collected from 15 children aged 6 to 11 years engaged in different activities such as commuting in a car, visiting a commercial building, attending an education institute, and resting inside home from October 2018 to March 2019. The fraction of daily mean exposure duration per activity was 72.7 ± 18.7% for resting inside home, 27.2 ± 14.4% for attending an education institute, and 11.5 ± 9.6% and 5.3 ± 5.9% for visiting a commercial building, commuting in a car, respectively. Daily median (interquartile range) PM2.5 exposure amount was 88.9 (55.9–159.7) μg in houses and that in education buildings was 43.3 (22.9–55.6) μg. Real-time PM2.5 exposure levels varied by person and time of day (p-value < 0.05). This study demonstrated that our real-time personal monitoring and data analysis methodologies were effective in detecting polluted microenvironments and provided a potential person-specific management strategy to reduce a person’s exposure level to PM2.5.