Towards global flood mapping onboard low cost satellites with machine learning

Global flood extent segmentation in optical satellite images

Floods are among the most destructive extreme events that exist, being the main cause of people affected by natural disasters. In the near future, estimated flood intensity and frequency are projected to increase. In this context, automatic and accurate satellite-derived flood maps are key for fast emergency response and damage assessment. However, current approaches for operational flood mapping present limitations due to cloud coverage on acquired satellite images, the accuracy of flood detection, and the generalization of methods across different geographies. In this work, a machine learning framework for operational flood mapping from optical satellite images addressing these problems is presented. It is based on a clouds-aware segmentation model trained in an extended version of the WorldFloods dataset. The model produces accurate and fast water segmentation masks even in areas covered by semitransparent clouds, increasing the coverage for emergency response scenarios. The proposed approach can be applied to both Sentinel-2 and Landsat 8/9 data, which enables a much higher revisit of the damaged region, also key for operational purposes. Detection accuracy and generalization of proposed model is carefully evaluated in a novel global dataset composed of manually labeled flood maps. We provide evidence of better performance than current operational methods based on thresholding spectral indices. Moreover, we demonstrate the applicability of our pipeline to map recent large flood events that occurred in Pakistan, between June and September 2022, and in Australia, between February and April 2022. Finally, the high-resolution (10-30m) flood extent maps are intersected with other high-resolution layers of cropland, building delineations, and population density. Using this workflow, we estimated that approximately 10 million people were affected and 700k buildings and 25,000 km[Formula: see text] of cropland were flooded in 2022 Pakistan floods.

Semantic segmentation of methane plumes with hyperspectral machine learning models

Methane is the second most important greenhouse gas contributor to climate change; at the same time its reduction has been denoted as one of the fastest pathways to preventing temperature growth due to its short atmospheric lifetime. In particular, the mitigation of active point-sources associated with the fossil fuel industry has a strong and cost-effective mitigation potential. Detection of methane plumes in remote sensing data is possible, but the existing approaches exhibit high false positive rates and need manual intervention. Machine learning research in this area is limited due to the lack of large real-world annotated datasets. In this work, we are publicly releasing a machine learning ready dataset with manually refined annotation of methane plumes. We present labelled hyperspectral data from the AVIRIS-NG sensor and provide simulated multispectral WorldView-3 views of the same data to allow for model benchmarking across hyperspectral and multispectral sensors. We propose sensor agnostic machine learning architectures, using classical methane enhancement products as input features. Our HyperSTARCOP model outperforms strong matched filter baseline by over 25% in F1 score, while reducing its false positive rate per classified tile by over 41.83%. Additionally, we demonstrate zero-shot generalisation of our trained model on data from the EMIT hyperspectral instrument, despite the differences in the spectral and spatial resolution between the two sensors: in an annotated subset of EMIT images HyperSTARCOP achieves a 40% gain in F1 score over the baseline.

Development of High-Precision Urban Flood-Monitoring Technology for Sustainable Smart Cities

Owing to rapid climate change, large-scale floods have occurred yearly in cities worldwide, causing serious damage. We propose a real-time urban flood-monitoring technology as an urban disaster prevention technology for sustainable and secure smart cities. Our method takes advantage of the characteristic that water flow is regularly detected at a certain distance with a constant Doppler velocity within the radar observation area. Therefore, a pure flow energy detection algorithm in this technology can accurately and immediately detect water flow due to flooding by effectively removing dynamic obstacles such as cars, people, and animals that cause changes in observation distance, and static obstacles that do not cause Doppler velocities. Specifically, in this method, the pure flow energy is detected by generating a two-dimensional range-Doppler relation map using 1 s periodic radar observation data and performing statistical analysis on the energy detected on the successive maps. Experiments to verify the proposed technology are conducted indoors and in real river basins. As a result of conducting experiments in a narrow indoor space that could be considered an urban underpass or underground facility, it was found that this method can detect flooding situations with centimeter-level accuracy by measuring water level and flow velocity in real time from the time of flood occurrence. And the experimental results in various river environments showed that our technology could accurately detect changes in distance and flow speed from the river surface. We also confirmed that this method could effectively eliminate moving obstacles within the observation range and detect only pure flow energy. Finally, we expect that our method will be able to build a high-density urban flood-monitoring network and a high-precision digital flood twin.

In-orbit demonstration of a re-trainable machine learning payload for processing optical imagery

Cognitive cloud computing in space (3CS) describes a new frontier of space innovation powered by Artificial Intelligence, enabling an explosion of new applications in observing our planet and enabling deep space exploration. In this framework, machine learning (ML) payloads-isolated software capable of extracting high level information from onboard sensors-are key to accomplish this vision. In this work we demonstrate, in a satellite deployed in orbit, a ML payload called 'WorldFloods' that is able to send compressed flood maps from sensed images. In particular, we perform a set of experiments to: (1) compare different segmentation models on different processing variables critical for onboard deployment, (2) show that we can produce, onboard, vectorised polygons delineating the detected flood water from a full Sentinel-2 tile, (3) retrain the model with few images of the onboard sensor downlinked to Earth and (4) demonstrate that this new model can be uplinked to the satellite and run on new images acquired by its camera. Overall our work demonstrates that ML-based models deployed in orbit can be updated if new information is available, paving the way for agile integration of onboard and onground processing and "on the fly" continuous learning.

CloudSEN12, a global dataset for semantic understanding of cloud and cloud shadow in Sentinel-2

Accurately characterizing clouds and their shadows is a long-standing problem in the Earth Observation community. Recent works showcase the necessity to improve cloud detection methods for imagery acquired by the Sentinel-2 satellites. However, the lack of consensus and transparency in existing reference datasets hampers the benchmarking of current cloud detection methods. Exploiting the analysis-ready data offered by the Copernicus program, we created CloudSEN12, a new multi-temporal global dataset to foster research in cloud and cloud shadow detection. CloudSEN12 has 49,400 image patches, including (1) Sentinel-2 level-1C and level-2A multi-spectral data, (2) Sentinel-1 synthetic aperture radar data, (3) auxiliary remote sensing products, (4) different hand-crafted annotations to label the presence of thick and thin clouds and cloud shadows, and (5) the results from eight state-of-the-art cloud detection algorithms. At present, CloudSEN12 exceeds all previous efforts in terms of annotation richness, scene variability, geographic distribution, metadata complexity, quality control, and number of samples.

RaVÆn: unsupervised change detection of extreme events using ML on-board satellites

Applications such as disaster management enormously benefit from rapid availability of satellite observations. Traditionally, data analysis is performed on the ground after being transferred-downlinked-to a ground station. Constraints on the downlink capabilities, both in terms of data volume and timing, therefore heavily affect the response delay of any downstream application. In this paper, we introduce RaVÆn, a lightweight, unsupervised approach for change detection in satellite data based on Variational Auto-Encoders (VAEs), with the specific purpose of on-board deployment. RaVÆn pre-processes the sampled data directly on the satellite and flags changed areas to prioritise for downlink, shortening the response time. We verified the efficacy of our system on a dataset-which we release alongside this publication-composed of time series containing a catastrophic event, demonstrating that RaVÆn outperforms pixel-wise baselines. Finally, we tested our approach on resource-limited hardware for assessing computational and memory limitations, simulating deployment on real hardware.

Application of Remote-Sensing-Based Hydraulic Model and Hydrological Model in Flood Simulation

Floods are one of the main natural disaster threats to the safety of people’s lives and property. Flood hazards intensify as the global risk of flooding increases. The control of flood disasters on the basin scale has always been an urgent problem to be solved that is firmly associated with the sustainable development of water resources. As important nonengineering measures for flood simulation and flood control, the hydrological and hydraulic models have been widely applied in recent decades. In our study, on the basis of sufficient remote-sensing and hydrological data, a hydrological (Xin’anjiang (XAJ)) and a two-dimensional hydraulic (2D) model were constructed to simulate flood events and provide support for basin flood management. In the Chengcun basin, the two models were applied, and the model parameters were calibrated by the parameter estimation (PEST) automatic calibration algorithm in combination with the measured data of 10 typical flood events from 1990 to 1996. Results show that the two models performed well in the Chengcun basin. The average Nash–Sutcliffe efficiency (NSE), percentage error of peak discharge (PE), and percentage error of flood volume (RE) were 0.79, 16.55%, and 18.27%, respectively, for the XAJ model, and those values were 0.76, 12.83%, and 11.03% for 2D model. These results indicate that the models had high accuracy, and hydrological and hydraulic models both had good application performance in the Chengcun basin. The study can a provide decision-making basis and theoretical support for flood simulation, and the formulation of flood control and disaster mitigation measures in the basin.

Urban Flood-Risk Assessment: Integration of Decision-Making and Machine Learning

Urban flood-risk mapping is an important tool for the mitigation of flooding in view of continuing urbanization and climate change. However, many developing countries lack sufficiently detailed data to produce reliable risk maps with existing methods. Thus, improved methods are needed that can help managers and decision makers to combine existing data with more soft semi-subjective data, such as citizen observations of flood-prone and vulnerable areas in view of existing settlements. Thus, we present an innovative approach using the semi-subjective Analytic Hierarchy Process (AHP), which integrates both subjective and objective assessments, to help organize the problem framework. This approach involves measuring the consistency of decision makers’ judgments, generating pairwise comparisons for choosing a solution, and considering criteria and sub-criteria to evaluate possible options. An urban flood-risk map was created according to the vulnerabilities and hazards of different urban areas using classification and regression-tree models, and the map can serve both as a first stage in advancing flood-risk mitigation approaches and in allocating warning and forecasting systems. The findings show that machine-learning methods are efficient in urban flood zoning. Using the city Rasht in Iran, it is shown that distance to rivers, urban drainage density, and distance to vulnerable areas are the most significant parameters that influence flood hazards. Similarly, for urban flood vulnerability, population density, land use, dwelling quality, household income, distance to cultural heritage, and distance to medical centers and hospitals are the most important factors. The integrated technique for both objective and semi-subjective data as outlined in the present study shows credible results that can be obtained without complicated modeling and costly field surveys. The proposed method is especially helpful in areas with little data to describe and display flood hazards to managers and decision makers.

An object-based sparse representation model for spatiotemporal image fusion

Many algorithms have been proposed for spatiotemporal image fusion on simulated data, yet only a few deal with spectral changes in real satellite images. An innovative spatiotemporal sparse representation (STSR) image fusion approach is introduced in this study to generate global dense high spatial and temporal resolution images from real satellite images. It aimed to minimize the data gap, especially when fine spatial resolution images are unavailable for a specific period. The proposed approach uses a set of real coarse- and fine-spatial resolution satellite images acquired simultaneously and another coarse image acquired at a different time to predict the corresponding unknown fine image. During the fusion process, pixels located between object classes with different spectral responses are more vulnerable to spectral distortion. Therefore, firstly, a rule-based fuzzy classification algorithm is used in STSR to classify input data and extract accurate edge candidates. Then, an object-based estimation of physical constraints and brightness shift between input data is utilized to construct the proposed sparse representation (SR) model that can deal with real input satellite images. Initial rules to adjust spatial covariance and equalize spectral response of object classes between input images are introduced as prior information to the model, followed by an optimization step to improve the STSR approach. The proposed method is applied to real fine Sentinel-2 and coarse Landsat-8 satellite data. The results showed that introducing objects in the fusion process improved spatial detail, especially over the edge candidates, and eliminated spectral distortion by preserving the spectral continuity of extracted objects. Experiments revealed the promising performance of the proposed object-based STSR image fusion approach based on its quantitative results, where it preserved almost 96.9% and 93.8% of the spectral detail over the smooth and urban areas, respectively.

Using Data from Earth Observation to Support Sustainable Development Indicators: An Analysis of the Literature and Challenges for the Future

The Sustainable Development Goals (SDG) framework aims to end poverty, improve health and education, reduce inequality, design sustainable cities, support economic growth, tackle climate change and leave no one behind. To monitor and report the progress on the 231 unique SDGs indicators in all signatory countries, data play a key role. Here, we reviewed the data challenges and costs associated with obtaining traditional data and satellite data (particularly for developing countries), emphasizing the benefits of using satellite data, alongside their portal and platforms in data access. We then assessed, under the maturity matrix framework (MMF 2.0), the current potential of satellite data applications on the SDG indicators that were classified into the sustainability pillars. Despite the SDG framework having more focus on socio-economic aspects of sustainability, there has been a rapidly growing literature in the last few years giving practical examples in using earth observation (EO) to monitor both environmental and socio-economic SDG indicators; there is a potential to populate 108 indicators by using EO data. EO also has a wider potential to support the SDGs beyond the existing indicators.

Integrating EfficientNet into an HAFNet Structure for Building Mapping in High-Resolution Optical Earth Observation Data

Automated extraction of buildings from Earth observation (EO) data is important for various applications, including updating of maps, risk assessment, urban planning, and policy-making. Combining data from different sensors, such as high-resolution multispectral images (HRI) and light detection and ranging (LiDAR) data, has shown great potential in building extraction. Deep learning (DL) is increasingly used in multi-modal data fusion and urban object extraction. However, DL-based multi-modal fusion networks may under-perform due to insufficient learning of “joint features” from multiple sources and oversimplified approaches to fusing multi-modal features. Recently, a hybrid attention-aware fusion network (HAFNet) has been proposed for building extraction from a dataset, including co-located Very-High-Resolution (VHR) optical images and light detection and ranging (LiDAR) joint data. The system reported good performances thanks to the adaptivity of the attention mechanism to the features of the information content of the three streams but suffered from model over-parametrization, which inevitably leads to long training times and heavy computational load. In this paper, the authors propose a restructuring of the scheme, which involved replacing VGG-16-like encoders with the recently proposed EfficientNet, whose advantages counteract exactly the issues found with the HAFNet scheme. The novel configuration was tested on multiple benchmark datasets, reporting great improvements in terms of processing times, and also in terms of accuracy. The new scheme, called HAFNetE (HAFNet with EfficientNet integration), appears indeed capable of achieving good results with less parameters, translating into better computational efficiency. Based on these findings, we can conclude that, given the current advancements in single-thread schemes, the classical multi-thread HAFNet scheme could be effectively transformed by the HAFNetE scheme by replacing VGG-16 with EfficientNet blocks on each single thread. The remarkable reduction achieved in computational requirements moves the system one step closer to on-board implementation in a possible, future “urban mapping” satellite constellation.