Earth Observation Open Science: Enhancing Reproducible Science Using Data Cubes

How well do concentric radii approximate population exposure to volcanic hazards?

Effective risk management requires accurate assessment of population exposure to volcanic hazards. Assessment of this exposure at the large-scale has often relied on circular footprints of various sizes around a volcano to simplify challenges associated with estimating the directionality and distribution of the intensity of volcanic hazards. However, to date, exposure values obtained from circular footprints have never been compared with modelled hazard footprints. Here, we compare hazard and population exposure estimates calculated from concentric radii of 10, 30 and 100 km with those calculated from the simulation of dome- and column-collapse pyroclastic density currents (PDCs), large clasts, and tephra fall across Volcanic Explosivity Index (VEI) 3, 4 and 5 scenarios for 40 volcanoes in Indonesia and the Philippines. We found that a 10 km radius-considered by previous studies to capture hazard footprints and populations exposed for VEI ≤ 3 eruptions-generally overestimates the extent for most simulated hazards, except for column collapse PDCs. A 30 km radius - considered representative of life-threatening VEI ≤ 4 hazards-overestimates the extent of PDCs and large clasts but underestimates the extent of tephra fall. A 100 km radius encapsulates most simulated life-threatening hazards, although there are exceptions for certain combinations of scenario, source parameters, and volcano. In general, we observed a positive correlation between radii- and model-derived population exposure estimates in southeast Asia for all hazards except dome collapse PDC, which is very dependent upon topography. This study shows, for the first time, how and why concentric radii under- or over-estimate hazard extent and population exposure, providing a benchmark for interpreting radii-derived hazard and exposure estimates.

Supplementary information

The online version contains supplementary material available at 10.1007/s00445-023-01686-5.

CloudSEN12, a global dataset for semantic understanding of cloud and cloud shadow in Sentinel-2

Accurately characterizing clouds and their shadows is a long-standing problem in the Earth Observation community. Recent works showcase the necessity to improve cloud detection methods for imagery acquired by the Sentinel-2 satellites. However, the lack of consensus and transparency in existing reference datasets hampers the benchmarking of current cloud detection methods. Exploiting the analysis-ready data offered by the Copernicus program, we created CloudSEN12, a new multi-temporal global dataset to foster research in cloud and cloud shadow detection. CloudSEN12 has 49,400 image patches, including (1) Sentinel-2 level-1C and level-2A multi-spectral data, (2) Sentinel-1 synthetic aperture radar data, (3) auxiliary remote sensing products, (4) different hand-crafted annotations to label the presence of thick and thin clouds and cloud shadows, and (5) the results from eight state-of-the-art cloud detection algorithms. At present, CloudSEN12 exceeds all previous efforts in terms of annotation richness, scene variability, geographic distribution, metadata complexity, quality control, and number of samples.

China Data Cube (CDC) for Big Earth Observation Data: Practices and Lessons Learned

In the face of tight natural resources and complex as well as volatile environments, and in order to meet the pressure brought by population growth, we need to overcome a series of challenges. As a new data management paradigm, the Earth Observation Data Cube simplifies the way that users manage and use earth observation data, and provides an analysis-ready form to access big spatiotemporal data, so as to realize the greater potential of earth observation data. Based on the Open Data Cube (ODC) framework, combined with analysis-ready data (ARD) generation technology, the design and implementation of CDC_DLTool, extending the support for data loading and the processing of international and Chinese imagery data covering China, this study eventually constructs the China Data Cube (CDC) framework. In the framework of this CDC grid, this study carried out case studies of water change monitoring based on international satellite imagery data of Landsat 8 in addition to vegetation change monitoring based on Chinese satellite imagery data of GF-1. The experimental results show that, compared with traditional scene-based data organization, the minimum management unit of this framework is a pixel, which makes the unified organization and management of multisource heterogeneous satellite imagery data more convenient and faster.

GF-1 Satellite Imagery Data Service and Application Based on Open Data Cube

With the application of big data in Earth observation, satellite imagery data are gradually becoming important means of observation for monitoring changes in vegetation, water bodies, and urbanization. Therefore, new satellite imagery data organization and management paradigms are urgently needed to fully mine the useful information from these data and provide new ways to better quantify and serve the sustainable development of resources and the environment. In this paper, a framework for processing and analyzing Chinese GF-1 satellite imagery data was developed using the latest technologies such as Open Data Cube (ODC) grids, Analysis Ready Data (ARD) generation, and space subdivision, which extended the data loading and processing capacities of the ODC grids for Chinese satellite imagery data. Using the proposed framework, we conducted a case study to investigate the spatial and temporal changes in vegetation and water mapping with GF-1 data collected from 2014 to 2021 covering the Miyun Reservoir, Beijing, China. The experimental results showed that the proposed framework had significantly improved temporal and spatial efficiency compared with the traditional scene-based data management approach, thus demonstrating the advantages and potential of the ODC grids as a new data management paradigm.

Tabular strategies for metadata in ecology, evolution, and the environmental sciences

Data support knowledge development and theory advances in ecology and evolution. We are increasingly reusing data within our teams and projects and through the global, openly archived datasets of others. Metadata can be challenging to write and interpret, but it is always crucial for reuse. The value metadata cannot be overstated-even as a relatively independent research object because it describes the work that has been done in a structured format. We advance a new perspective and classify methods for metadata curation and development with tables. Tables with templates can be effectively used to capture all components of an experiment or project in a single, easy-to-read file familiar to most scientists. If coupled with the R programming language, metadata from tables can then be rapidly and reproducibly converted to publication formats including extensible markup language files suitable for data repositories. Tables can also be used to summarize existing metadata and store metadata across many datasets. A case study is provided and the added benefits of tables for metadata, a priori, are developed to ensure a more streamlined publishing process for many data repositories used in ecology, evolution, and the environmental sciences. In ecology and evolution, researchers are often highly tabular thinkers from experimental data collection in the lab and/or field, and representations of metadata as a table will provide novel research and reuse insights.

The Swiss data cube, analysis ready data archive using earth observations of Switzerland

Since the opening of Earth Observation (EO) archives (USGS/NASA Landsat and EC/ESA Sentinels), large collections of EO data are freely available, offering scientists new possibilities to better understand and quantify environmental changes. Fully exploiting these satellite EO data will require new approaches for their acquisition, management, distribution, and analysis. Given rapid environmental changes and the emergence of big data, innovative solutions are needed to support policy frameworks and related actions toward sustainable development. Here we present the Swiss Data Cube (SDC), unleashing the information power of Big Earth Data for monitoring the environment, providing Analysis Ready Data over the geographic extent of Switzerland since 1984, which is updated on a daily basis. Based on a cloud-computing platform allowing to access, visualize and analyse optical (Sentinel-2; Landsat 5, 7, 8) and radar (Sentinel-1) imagery, the SDC minimizes the time and knowledge required for environmental analyses, by offering consistent calibrated and spatially co-registered satellite observations. SDC derived analysis ready data supports generation of environmental information, allowing to inform a variety of environmental policies with unprecedented timeliness and quality.

Satellite Image Time Series Analysis for Big Earth Observation Data

The development of analytical software for big Earth observation data faces several challenges. Designers need to balance between conflicting factors. Solutions that are efficient for specific hardware architectures can not be used in other environments. Packages that work on generic hardware and open standards will not have the same performance as dedicated solutions. Software that assumes that its users are computer programmers are flexible but may be difficult to learn for a wide audience. This paper describes sits, an open-source R package for satellite image time series analysis using machine learning. To allow experts to use satellite imagery to the fullest extent, sits adopts a time-first, space-later approach. It supports the complete cycle of data analysis for land classification. Its API provides a simple but powerful set of functions. The software works in different cloud computing environments. Satellite image time series are input to machine learning classifiers, and the results are post-processed using spatial smoothing. Since machine learning methods need accurate training data, sits includes methods for quality assessment of training samples. The software also provides methods for validation and accuracy measurement. The package thus comprises a production environment for big EO data analysis. We show that this approach produces high accuracy for land use and land cover maps through a case study in the Cerrado biome, one of the world’s fast moving agricultural frontiers for the year 2018.

Earth Observation Data Cubes for Brazil: Requirements, Methodology and Products

Recently, remote sensing image time series analysis has being widely used to investigate the dynamics of environments over time. Many studies have combined image time series analysis with machine learning methods to improve land use and cover change mapping. In order to support image time series analysis, analysis-ready data (ARD) image collections have been modeled and organized as multidimensional data cubes. Data cubes can be defined as sets of time series associated with spatially aligned pixels. Based on lessons learned in the research project e-Sensing, related to national demands for land use and cover monitoring and related to state-of-the-art studies on relevant topics, we define the requirements to build Earth observation data cubes for Brazil. This paper presents the methodology to generate ARD and multidimensional data cubes from remote sensing images for Brazil. We describe the computational infrastructure that we are developing in the Brazil Data Cube project, composed of software applications and Web services to create, integrate, discover, access, and process the data sets. We also present how we are producing land use and cover maps from data cubes using image time series analysis and machine learning techniques.

Essential Variables for Environmental Monitoring: What Are the Possible Contributions of Earth Observation Data Cubes?

Environmental sustainability is nowadays a major global issue that requires efficient and effective responses from governments. Essential variables (EV) have emerged in different scientific communities as a means to characterize and follow environmental changes through a set of measurements required to support policy evidence. To help track these changes, our planet has been under continuous observation from satellites since 1972. Currently, petabytes of satellite Earth observation (EO) data are freely available. However, the full information potential of EO data has not been yet realized because many big data challenges and complexity barriers hinder their effective use. Consequently, facilitating the production of EVs using the wealth of satellite EO data can be beneficial for environmental monitoring systems. In response to this issue, a comprehensive list of EVs that can take advantage of consistent time-series satellite data has been derived. In addition, a set of use-cases, using an Earth Observation Data Cube (EODC) to process large volumes of satellite data, have been implemented to demonstrate the practical applicability of EODC to produce EVs. The proposed approach has been successfully tested showing that EODC can facilitate the production of EVs at different scales and benefiting from the spatial and temporal dimension of satellite EO data for enhanced environmental monitoring.

Improved Estimations of Nitrate and Sediment Concentrations Based on SWAT Simulations and Annual Updated Land Cover Products from a Deep Learning Classification Algorithm

The agricultural sector and natural resources are heavily interdependent, comprising a coherent but complex system. The soil and water assessment tool (SWAT) is widely used in assessing these interdependencies for regional watershed management. However, long-term simulations of agricultural watersheds are considered as not realistic since they have often been performed assuming constant land use over time and are based on the coarse resolution of the existing global or national data. This work presents the first insights of the synergy among SWAT model and deep learning classification algorithms to provide annually updated and realistic model’s parameterization and simulations. The proposed hybrid modelling approach couples the physical process SWAT model with the versatility of Earth observation data-driven non-linear deep learning algorithms for land use classification (Overall Accuracy (OA) = 79.58% and Kappa = 0.79), giving a strong advantage to decision makers for efficient management planning. A validation case at an agricultural watershed located in Northern Greece is provided to demonstrate their synergistic use to estimate nitrate and sediment concentrations that load in Zazari Lake. The SWAT model has been implemented under two different simulations; one with the use of a static coarse land use map and the other with the use of the annual updated land use maps for three consecutive years (2017–2019). The results indicate that the land use changes affect the final estimations resulting to an enhanced prediction performance of 1% and 2% for sediment and nitrate, respectively, when the annual land use maps are incorporated into SWAT simulations. In this context, a hybrid approach could further contribute to addressing challenges and support a data-centric scheme for informed decision making with regard to environmental and agricultural issues on the river basin scale.

Recent Applications of Landsat 8/OLI and Sentinel-2/MSI for Land Use and Land Cover Mapping: A Systematic Review

Recent applications of Landsat 8 Operational Land Imager (L8/OLI) and Sentinel-2 MultiSpectral Instrument (S2/MSI) data for acquiring information about land use and land cover (LULC) provide a new perspective in remote sensing data analysis. Jointly, these sources permit researchers to improve operational classification and change detection, guiding better reasoning about landscape and intrinsic processes, as deforestation and agricultural expansion. However, the results of their applications have not yet been synthesized in order to provide coherent guidance on the effect of their applications in different classification processes, as well as to identify promising approaches and issues which affect classification performance. In this systematic review, we present trends, potentialities, challenges, actual gaps, and future possibilities for the use of L8/OLI and S2/MSI for LULC mapping and change detection. In particular, we highlight the possibility of using medium-resolution (Landsat-like, 10–30 m) time series and multispectral optical data provided by the harmonization between these sensors and data cube architectures for analysis-ready data that are permeated by publicizations, open data policies, and open science principles. We also reinforce the potential for exploring more spectral bands combinations, especially by using the three Red-edge and the two Near Infrared and Shortwave Infrared bands of S2/MSI, to calculate vegetation indices more sensitive to phenological variations that were less frequently applied for a long time, but have turned on since the S2/MSI mission. Summarizing peer-reviewed papers can guide the scientific community to the use of L8/OLI and S2/MSI data, which enable detailed knowledge on LULC mapping and change detection in different landscapes, especially in agricultural and natural vegetation scenarios.

Leveraging Container Technologies in a GIScience Project: A Perspective from Open Reproducible Research

Scientific reproducibility is essential for the advancement of science. It allows the results of previous studies to be reproduced, validates their conclusions and develops new contributions based on previous research. Nowadays, more and more authors consider that the ultimate product of academic research is the scientific manuscript, together with all the necessary elements (i.e., code and data) so that others can reproduce the results. However, there are numerous difficulties for some studies to be reproduced easily (i.e., biased results, the pressure to publish, and proprietary data). In this context, we explain our experience in an attempt to improve the reproducibility of a GIScience project. According to our project needs, we evaluated a list of practices, standards and tools that may facilitate open and reproducible research in the geospatial domain, contextualising them on Peng’s reproducibility spectrum. Among these resources, we focused on containerisation technologies and performed a shallow review to reflect on the level of adoption of these technologies in combination with OSGeo software. Finally, containerisation technologies proved to enhance the reproducibility and we used UML diagrams to describe representative work-flows deployed in our GIScience project.