Webcrawling and machine learning as a new approach for the spatial distribution of atmospheric emissions

Improving residential wood burning emission inventories with the integration of readily available data sources

Residential wood burning (RWB) is the largest anthropogenic source of PM2.5 in many North American and European cities in the winter. The current lack of information on the locations, types, and intensity of use of wood burning appliances limits the ability to accurately assess the exposure of the population to RWB emissions. In this study, we generated a high spatial resolution emission inventory for RWB in the province of Quebec, Canada using a novel data driven approach. The method first combines real estate and socioeconomic census data through machine learning models to estimate ownership rates of fireplaces and wood stoves. These ownership rates are then combined with household survey data (on wood consumption and types of appliances), emission factors and building registry data to generate the emission inventory at a 1Km2 resolution. Our proposed approach was able to capture spatial patterns in RWB appliance ownership and intensity of use, which may be overlooked by using simple urban vs rural or population based spatial proxies. The machine learning models explained 80.3 % and 63 % of the variability in wood stove and fireplace ownership rates with each appliance type exhibiting different spatial trends. Wood stoves were common in rural areas and among lower income households, whereas fireplaces were more common in urban areas. Additionally, we observed large spatial and regional variability in emissions per residence due to differences in wood consumption, appliance ownership rates, and appliance mixes (e.g. conventional vs certified). Our results suggest that using simple spatial proxies based on population, urbanization levels or residence type are not enough to explain the spatial distribution of RWB emissions as they might overlook other factors such as socioeconomic factors or regional heating preferences. Finally, our spatially distributed emissions were strongly correlated (r = 0.86) with the increase in PM2.5 concentrations during peak-RWB hours on winter weekends at 42 reference stations across the province of Quebec.

IHWC: intelligent hidden web crawler for harvesting data in urban domains

Due to the massive size of the hidden web, searching, retrieving and mining rich and high-quality data can be a daunting task. Moreover, with the presence of forms, data cannot be accessed easily. Forms are dynamic, heterogeneous and spread over trillions of web pages. Significant efforts have addressed the problem of tapping into the hidden web to integrate and mine rich data. Effective techniques, as well as application in special cases, are required to be explored to achieve an effective harvest rate. One such special area is atmospheric science, where hidden web crawling is least implemented, and crawler is required to crawl through the huge web to narrow down the search to specific data. In this study, an intelligent hidden web crawler for harvesting data in urban domains (IHWC) is implemented to address the relative problems such as classification of domains, prevention of exhaustive searching, and prioritizing the URLs. The crawler also performs well in curating pollution-related data. The crawler targets the relevant web pages and discards the irrelevant by implementing rejection rules. To achieve more accurate results for a focused crawl, ICHW crawls the websites on priority for a given topic. The crawler has fulfilled the dual objective of developing an effective hidden web crawler that can focus on diverse domains and to check its integration in searching pollution data in smart cities. One of the objectives of smart cities is to reduce pollution. Resultant crawled data can be used for finding the reason for pollution. The crawler can help the user to search the level of pollution in a specific area. The harvest rate of the crawler is compared with pioneer existing work. With an increase in the size of a dataset, the presented crawler can add significant value to emission accuracy. Our results are demonstrating the accuracy and harvest rate of the proposed framework, and it efficiently collect hidden web interfaces from large-scale sites and achieve higher rates than other crawlers.

Supporting the improvement of air quality management practices: The "FAIRMODE pilot" activity

This paper presents the first outcomes of the "FAIRMODE pilot" activity, aiming at improving the way in which air quality models are used in the frame of the European "Air Quality Directive". Member States may use modelling, combined with measurements, to "assess" current levels of air quality and estimate future air quality under different scenarios. In case of current and potential exceedances of the Directive limit values, it is also requested that they "plan" and implement emission reductions measures to avoid future exceedances. In both "assessment" and "planning", air quality models can and should be used; but to do so, the used modelling chain has to be fit-for-purpose and properly checked and verified. FAIRMODE has developed in the recent years a suite of methodologies and tools to check if emission inventories, model performance, source apportionment techniques and planning activities are fit-for-purpose. Within the "FAIRMODE pilot", these tools are used and tested by regional/local authorities, with the two-fold objective of improving management practices at regional/local scale, and providing valuable feedback to the FAIRMODE community. Results and lessons learnt from this activity are presented in this paper, as a showcase that can potentially benefit other authorities in charge of air quality assessment and planning.