The Grids Python Tool for Querying Spatiotemporal Multidimensional Water Data

Scientific datasets from global-scale earth science models and remote sensing instruments are becoming available at greater spatial and temporal resolutions with shorter lag times. Water data are frequently stored as multidimensional arrays, also called gridded or raster data, and span two or three spatial dimensions, the time dimension, and other dimensions which vary by the specific dataset. Water engineers and scientists need these data as inputs for models and generate data in these formats as results. A myriad of file formats and organizational conventions exist for storing these array datasets. The variety does not make the data unusable but does add considerable difficulty in using them because the structure can vary. These storage formats are largely incompatible with common geographic information system (GIS) software. This introduces additional complexity in extracting values, analyzing results, and otherwise working with multidimensional data since they are often spatial data. We present a Python package which provides a central interface for efficient access to multidimensional water data regardless of the file format. This research builds on and unifies existing file formats and software rather than suggesting entirely new alternatives. We present a summary of the code design and validate the results using common water-related datasets and software.

A 20-Year MODIS-Based Snow Cover Dataset for Svalbard and Its Link to Phenological Timing and Sea Ice Variability

The climate in Svalbard has been warming dramatically compared with the global average for the last few decades. Seasonal snow cover, which is sensitive to temperature and precipitation changes, is therefore expected to undergo both spatial and temporal changes in response to the changing climate in Svalbard. This will in turn have implications for timing of terrestrial productivity, which is closely linked to the disappearance of seasonal snow. We have produced a 20-year snow cover fraction time series for the Svalbard archipelago, derived from MODIS (Moderate Resolution Imaging Spectroradiometer) Terra data to map and identify changes in the timing of the first snow-free day (FSFD) for the period 2000–2019. Moreover, we investigate the influence of sea ice concentration (SIC) variations on FSFD and how FSFD is related to the start of the phenological growing season in Svalbard. Our results revealed clear patterns of earlier FSFD in the southern and central parts of the archipelago, while the northernmost parts exhibit little change or trend toward later FSFD, resulting in weaker trends in summer and winter duration. We found that FSFD preceded the onset of the phenological growing season with an average difference of 12.4 days for the entire archipelago, but with large regional variations that are indicative of temperature dependence. Lastly, we found a significant correlation between variations of time-integrated SIC and variations in FSFD, which maximizes when correlating SIC northeast of Svalbard with FSFD averaged over Nordaustlandet. Prolonged sea ice cover in the spring was correlated with late snow disappearance, while lower-than-average sea ice cover correlated with early snow disappearance, indicating that proximity to sea ice plays an important role in regulating the timing of snow disappearance on land through influencing the regional air temperature and therefore rate of spring snowmelt.

A Cloud-Based Framework for Large-Scale Log Mining through Apache Spark and Elasticsearch

The volume, variety, and velocity of different data, e.g., simulation data, observation data, and social media data, are growing ever faster, posing grand challenges for data discovery. An increasing trend in data discovery is to mine hidden relationships among users and metadata from the web usage logs to support the data discovery process. Web usage log mining is the process of reconstructing sessions from raw logs and finding interesting patterns or implicit linkages. The mining results play an important role in improving quality of search-related components, e.g., ranking, query suggestion, and recommendation. While researches were done in the data discovery domain, collecting and analyzing logs efficiently remains a challenge because (1) the volume of web usage logs continues to grow as long as users access the data; (2) the dynamic volume of logs requires on-demand computing resources for mining tasks; (3) the mining process is compute-intensive and time-intensive. To speed up the mining process, we propose a cloud-based log-mining framework using Apache Spark and Elasticsearch. In addition, a data partition paradigm, logPartitioner, is designed to solve the data imbalance problem in data parallelism. As a proof of concept, oceanographic data search and access logs are chosen to validate performance of the proposed parallel log-mining framework.

Data and tools to integrate climate and environmental information into public health

BACKGROUND

During the last 30 years, the development of geographical information systems and satellites for Earth observation has made important progress in the monitoring of the weather, climate, environmental and anthropogenic factors that influence the reduction or the reemergence of vector-borne diseases. Analyses resulting from the combination of geographical information systems (GIS) and remote sensing have improved knowledge of climatic, environmental, and biodiversity factors influencing vector-borne diseases (VBDs) such as malaria, visceral leishmaniasis, dengue, Rift Valley fever, schistosomiasis, Chagas disease and leptospirosis. These knowledge and products developed using remotely sensed data helped and continue to help decision makers to better allocate limited resources in the fight against VBDs.

MAIN BODY

Because VBDs are linked to climate and environment, we present here our experience during the last four years working with the projects under the, World Health Organization (WHO)/ The Special Programme for Research and Training in Tropical Diseases (TDR)-International Development Research Centre (IDRC) Research Initiative on VBDs and Climate Change to integrate climate and environmental information into research and decision-making processes. The following sections present the methodology we have developed, which uses remote sensing to monitor climate variability, environmental conditions, and their impacts on the dynamics of infectious diseases. We then show how remotely sensed data can be accessed and evaluated and how they can be integrated into research and decision-making processes for mapping risks, and creating Early Warning Systems, using two examples from the WHO TDR projects based on schistosomiasis analysis in South Africa and Trypanosomiasis in Tanzania.

CONCLUSIONS

The tools presented in this article have been successfully used by the projects under the WHO/TDR-IDRC Research Initiative on VBDs and Climate Change. Combined with capacity building, they are an important piece of work which can significantly contribute to the goals of WHO Global Vector Control Response and to the Sustainable Development Goals especially those on health and climate action.

Progress and challenges in coupled hydrodynamic-ecological estuarine modeling

Numerical modeling has emerged over the last several decades as a widely accepted tool for investigations in environmental sciences. In estuarine research, hydrodynamic and ecological models have moved along parallel tracks with regard to complexity, refinement, computational power, and incorporation of uncertainty. Coupled hydrodynamic-ecological models have been used to assess ecosystem processes and interactions, simulate future scenarios, and evaluate remedial actions in response to eutrophication, habitat loss, and freshwater diversion. The need to couple hydrodynamic and ecological models to address research and management questions is clear, because dynamic feedbacks between biotic and physical processes are critical interactions within ecosystems. In this review we present historical and modern perspectives on estuarine hydrodynamic and ecological modeling, consider model limitations, and address aspects of model linkage, skill assessment, and complexity. We discuss the balance between spatial and temporal resolution and present examples using different spatiotemporal scales. Finally, we recommend future lines of inquiry, approaches to balance complexity and uncertainty, and model transparency and utility. It is idealistic to think we can pursue a "theory of everything" for estuarine models, but recent advances suggest that models for both scientific investigations and management applications will continue to improve in terms of realism, precision, and accuracy.