Research on Post-Earthquake Landslide Extraction Algorithm Based on Improved U-Net Model

Endoscopic Image Classification Based on Explainable Deep Learning

Deep learning has achieved remarkably positive results and impacts on medical diagnostics in recent years. Due to its use in several proposals, deep learning has reached sufficient accuracy to implement; however, the algorithms are black boxes that are hard to understand, and model decisions are often made without reason or explanation. To reduce this gap, explainable artificial intelligence (XAI) offers a huge opportunity to receive informed decision support from deep learning models and opens the black box of the method. We conducted an explainable deep learning method based on ResNet152 combined with Grad-CAM for endoscopy image classification. We used an open-source KVASIR dataset that consisted of a total of 8000 wireless capsule images. The heat map of the classification results and an efficient augmentation method achieved a high positive result with 98.28% training and 93.46% validation accuracy in terms of medical image classification.

Multisource Data Fusion and Adversarial Nets for Landslide Extraction from UAV-Photogrammetry-Derived Data

Most traditional methods have difficulty detecting landslide boundary accurately, and the existing methods based on deep learning often lead to insufficient training or overfitting due to insufficient samples. An end-to-end, semi-supervised adversarial network, which fully considers spectral and topographic features derived using unmanned aerial vehicle (UAV) photogrammetry, is proposed to extract landslides by semantic segmentation to address the abovementioned problem. In the generative network, a generator similar to pix2pix is introduced into the proposed adversarial nets to learn semantic features from UAV-photogrammetry-derived data by semi-supervised operation and a confrontational strategy to reduce the requirement of the number of labeled samples. In the discriminative network, DeepLabv3+ is improved by inserting multilevel skip connection architecture with upsampling operation to obtain the contextual information and retain the boundary information of landslides at all levels, and a topographic convolutional neural network is proposed to be inserted into the encoder to concatenate topographic features together with spectral features. Then, transfer learning with the pre-trained parameters and weights, shared with pix2pix and DeepLabv3+, is used to perform landslide extraction training and validation. In our experiments, the UAV-photogrammetry-derived data of a typical landslide located at Meilong gully in China are collected to test the proposed method. The experimental results show that our method can accurately detect the area of a landslide and achieve satisfactiory results based on several indicators including the Precision, Recall, F1 score, and mIoU, which are 13.07%, 15.65%, 16.96%, and 18.23% higher than those of the DeepLabV3+. Compared with state-of-the-art methods such as U-Net, PSPNet, and pix2pix, the proposed adversarial nets considering multidimensional information such as topographic factors can perform better and significantly improve the accuracy of landslide extraction.

L-Unet: A Landslide Extraction Model Using Multi-Scale Feature Fusion and Attention Mechanism

At present, it is challenging to extract landslides from high-resolution remote-sensing images using deep learning. Because landslides are very complex, the accuracy of traditional extraction methods is low. To improve the efficiency and accuracy of landslide extraction, a new model is proposed based on the U-Net model to automatically extract landslides from remote-sensing images: L-Unet. The main innovations are as follows: (1) A multi-scale feature-fusion (MFF) module is added at the end of the U-Net encoding network to improve the model’s ability to extract multi-scale landslide information. (2) A residual attention network is added to the U-Net model to deepen the network and improve the model’s ability to represent landslide features. (3) The bilinear interpolation algorithm in the decoding network of the U-Net model is replaced by data-dependent upsampling (DUpsampling) to improve the quality of the feature maps. Experimental results showed that the precision, recall, MIoU and F1 values of the L-Unet model are 4.15%, 2.65%, 4.82% and 3.37% higher than that of the baseline U-Net model, respectively. It was proven that the new model can extract landslides accurately and effectively.

Landslide Extraction Using Mask R-CNN with Background-Enhancement Method

The application of deep learning methods has brought improvements to the accuracy and automation of landslide extractions based on remote sensing images because deep learning techniques have independent feature learning and powerful computing ability. However, in application, the quality of training samples often fails the requirement for training deep networks, causing insufficient feature learning. Furthermore, some background objects (e.g., river, bare land, building) share similar shapes, colors, and textures with landslides. They can be confusing to automatic tasks, contributing false and missed extractions. To solve the above problems, a background-enhancement method was proposed to enrich the complexity of samples. Models can learn the differences between landslides and background objects more efficiently through background-enhanced samples, then reduce false extractions on background objects. Considering that the environments of disaster areas play dominant roles in the formation of landslides, landslide-inducing attributes (DEM, slope, distance from river) were used as supplements, providing additional information for landslide extraction models to further improve the accuracy of extraction results. The proposed methods were applied to extract landslides that occurred in Ludian county, Yunnan Province, in August 2014. Comparative experiments were conducted using a mask R-CNN model. The experiment using both background-enhanced samples and landslide-inducing information showed a satisfying result with an F1 score of 89.08%. Compared with the F1 score from the experiment using only satellite images as input data, it was significantly improved by 22.38%, underscoring the applicability and effectiveness of our background-enhancement method.

DRs-UNet: A Deep Semantic Segmentation Network for the Recognition of Active Landslides from InSAR Imagery in the Three Rivers Region of the Qinghai–Tibet Plateau

At present, Synthetic Aperture Radar Interferometry (InSAR) has been an important technique for active landslides recognition in the geological survey field. However, the traditional interpretation method through human–computer interaction highly relies on expert experience, which is time-consuming and subjective. To solve the problem, this study designed an end-to-end semantic segmentation network, called deep residual shrinkage U-Net (DRs-UNet), to automatically extract potential active landslides in InSAR imagery. The proposed model was inspired by the structure of U-Net and adopted a residual shrinkage building unit (RSBU) as the feature extraction block in its encoder part. The method of this study has three main advantages: (1) The RSBU in the encoder part incorporated with soft thresholding can reduce the influence of noise from InSAR images. (2) The residual connection of the RSBU makes the training of the network easier and accelerates the convergency process. (3) The feature fusion of the corresponding layers between the encoder and decoder effectively improves the classification accuracy. Two widely used networks, U-Net and SegNet, were trained under the same experiment environment to compare with the proposed method. The experiment results in the test set show that our method achieved the best performance; specifically, the F1 score is 1.48% and 4.1% higher than U-Net and SegNet, which indicates a better balance between precision and recall. Additionally, our method has the best IoU score of over 90%. Furthermore, we applied our network to a test area located in Zhongxinrong County along Jinsha River where landslides are highly evolved. The quantitative evaluation results prove that our method is effective for the automatic recognition of potential active landslide hazards from InSAR imagery.

Rapid Mapping of Landslides on SAR Data by Attention U-Net

Multiple landslide events are common around the globe. They can cause severe damage to both human lives and infrastructures. Although a huge quantity of research has been shaped to address rapid mapping of landslides by optical Earth Observation (EO) data, various gaps and uncertainties are still present when dealing with cloud obscuration and 24/7 operativity. To address the issue, we explore the usage of SAR data over the eastern Iburi sub-prefecture of Hokkaido, Japan. In the area, about 8000 co-seismic landslides were triggered by an Mw 6.6 earthquake on 6 September 2018, at 03.08 local time (JST). In the following study, we modify a Deep Learning (DL) convolutional neural network (CNN) architecture suited for pixel-based classification purposes, the so-called Attention U-Net (Attn-U-Net) and we employ it to evaluate the potential of bi- and tri-temporal SAR amplitude data from the Sentinel-1 satellite and slope angle to map landslides even under thick cloud cover. Four different datasets, composed of two different band combinations per two satellite orbits (ascending and descending) are analyzed. Moreover, the impact of augmentations is evaluated independently for each dataset. The models’ predictions are compared against an accurate landslide inventory obtained by manual mapping on pre-and post-event PlanetScope imagery through F1-score and other common metrics. The best result was yielded by the augmented ascending tri-temporal SAR composite image (61% F1-score). Augmentations have a positive impact on the ascending Sentinel-1 orbit, while metrics decrease when augmentations are applied on descending path. Our findings demonstrate that combining SAR data with other data sources may help to map landslides quickly, even during storms and under deep cloud cover. However, further investigations and improvements are still needed, this being one of the first attempts in which the combination of SAR data and DL algorithms are employed for landslide mapping purposes.

Landslide Extraction from High-Resolution Remote Sensing Imagery Using Fully Convolutional Spectral–Topographic Fusion Network

Considering the complexity of landslide hazards, their manual investigation lacks efficiency and is time-consuming, especially in high-altitude plateau areas. Therefore, extracting landslide information using remote sensing technology has great advantages. In this study, comprehensive research was carried out on the landslide features of high-resolution remote sensing images on the Mangkam dataset. Based on the idea of feature-driven classification, the landslide extraction model of a fully convolutional spectral–topographic fusion network (FSTF-Net) based on a deep convolutional neural network of multi-source data fusion is proposed, which takes into account the topographic factor (slope and aspect) and the normalized difference vegetation index (NDVI) as multi-source data input by which to train the model. In this paper, a high-resolution remote sensing image classification method based on a fully convolutional network was used to extract the landslide information, thereby realizing the accurate extraction of the landslide and surrounding ground-object information. With Mangkam County in the southeast of the Qinghai–Tibet Plateau China as the study area, the proposed method was evaluated based on the high-precision digital elevation model (DEM) generated from stereoscopic images of Resources Satellite-3 and multi-source high-resolution remote sensing image data (Beijing-2, Worldview-3, and SuperView-1). Results show that our method had a landslide detection precision of 0.85 and an overall classification accuracy of 0.89. Compared with the latest DeepLab_v3+, our model increases the landslide detection precision by 5%. Thus, the proposed FSTF-Net model has high reliability and robustness.

A comprehensive transferability evaluation of U-Net and ResU-Net for landslide detection from Sentinel-2 data (case study areas from Taiwan, China, and Japan)

Earthquakes and heavy rainfalls are the two leading causes of landslides around the world. Since they often occur across large areas, landslide detection requires rapid and reliable automatic detection approaches. Currently, deep learning (DL) approaches, especially different convolutional neural network and fully convolutional network (FCN) algorithms, are reliably achieving cutting-edge accuracies in automatic landslide detection. However, these successful applications of various DL approaches have thus far been based on very high resolution satellite images (e.g., GeoEye and WorldView), making it easier to achieve such high detection performances. In this study, we use freely available Sentinel-2 data and ALOS digital elevation model to investigate the application of two well-known FCN algorithms, namely the U-Net and residual U-Net (or so-called ResU-Net), for landslide detection. To our knowledge, this is the first application of FCN for landslide detection only from freely available data. We adapt the algorithms to the specific aim of landslide detection, then train and test with data from three different case study areas located in Western Taitung County (Taiwan), Shuzheng Valley (China), and Eastern Iburi (Japan). We characterize three different window size sample patches to train the algorithms. Our results also contain a comprehensive transferability assessment achieved through different training and testing scenarios in the three case studies. The highest f1-score value of 73.32% was obtained by ResU-Net, trained with a dataset from Japan, and tested on China's holdout testing area using the sample patch size of 64 × 64 pixels.

A new strategy to map landslides with a generalized convolutional neural network

Rapid mapping of event landslides is crucial to identify the areas affected by damages as well as for effective disaster response. Traditionally, such maps are generated with visual interpretation of remote sensing imagery (manned/unmanned airborne systems or spaceborne sensors) and/or using pixel-based and object-based methods exploiting data-intensive machine learning algorithms. Recent works have explored the use of convolutional neural networks (CNN), a deep learning algorithm, for mapping landslides from remote sensing data. These methods follow a standard supervised learning workflow that involves training a model using a landslide inventory covering a relatively small area. The trained model is then used to predict landslides in the surrounding regions. Here, we propose a new strategy, i.e., a progressive CNN training relying on combined inventories to build a generalized model that can be applied directly to a new, unexplored area. We first prove the effectiveness of CNNs by training and validating on event landslides inventories in four regions after earthquakes and/or extreme meteorological events. Next, we use the trained CNNs to map landslides triggered by new events spread across different geographic regions. We found that CNNs trained on a combination of inventories have a better generalization performance, with a bias towards high precision and low recall scores. In our tests, the combined training model achieved the highest (Matthews correlation coefficient) MCC score of 0.69 when mapping landslides in new unseen regions. The mapping was done on images from different optical sensors, resampled to a spatial resolution of 6 m, 10 m, and 30 m. Despite a slightly reduced performance, the main advantage of combined training is to overcome the requirement of a local inventory for training a new deep learning model. This implementation can facilitate automated pipelines providing fast response for the generation of landslide maps in the post-disaster phase. In this study, the study areas were selected from seismically active zones with a high hydrological hazard distribution and vegetation coverage. Hence, future works should also include regions from less vegetated geographic locations.

Automated Mapping of Ms 7.0 Jiuzhaigou Earthquake (China) Post-Disaster Landslides Based on High-Resolution UAV Imagery

The Ms 7.0 Jiuzhaigou earthquake that occurred on 8 August 2017 triggered hundreds of landslides in the Jiuzhaigou valley scenic and historic-interest area in Sichuan, China, causing heavy casualties and serious property losses. Quick and accurate mapping of post-disaster landslide distribution is of paramount importance for earthquake emergency rescue and the analysis of post-seismic landslides distribution characteristics. The automatic identification of landslides is mostly based on medium- and low-resolution satellite-borne optical remote-sensing imageries, and the high-accuracy interpretation of earthquake-triggered landslides still relies on time-consuming manual interpretation. This paper describes a methodology based on the use of 1 m high-resolution unmanned aerial vehicle (UAV) imagery acquired after the earthquake, and proposes a support vector machine (SVM) classification method combining the roads and villages mask from pre-seismic remote sensing imagery to accurately and automatically map the landslide inventory. Compared with the results of manual visual interpretation, the automatic recognition accuracy could reach 99.89%, and the Kappa coefficient was higher than 0.9, suggesting that the proposed method and 1 m high-resolution UAV imagery greatly improved the mapping accuracy of the landslide area. We also analyzed the spatial-distribution characteristics of earthquake-triggered landslides with the influenced factors of altitude, slope gradient, slope aspect, and the nearest faults, which provided important support for the further study of post-disaster landslide distribution characteristics, susceptibility prediction, and risk assessment.

A Research on Landslides Automatic Extraction Model Based on the Improved Mask R-CNN

Landslides are the most common and destructive secondary geological hazards caused by earthquakes. It is difficult to extract landslides automatically based on remote sensing data, which is import for the scenario of disaster emergency rescue. The literature review showed that the current landslides extraction methods mostly depend on expert interpretation which was low automation and thus was unable to provide sufficient information for earthquake rescue in time. To solve the above problem, an end-to-end improved Mask R-CNN model was proposed. The main innovations of this paper were (1) replacing the feature extraction layer with an effective ResNeXt module to extract the landslides. (2) Increasing the bottom-up channel in the feature pyramid network to make full use of low-level positioning and high-level semantic information. (3) Adding edge losses to the loss function to improve the accuracy of the landslide boundary detection accuracy. At the end of this paper, Jiuzhaigou County, Sichuan Province, was used as the study area to evaluate the new model. Results showed that the new method had a precision of 95.8%, a recall of 93.1%, and an overall accuracy (OA) of 94.7%. Compared with the traditional Mask R-CNN model, they have been significantly improved by 13.9%, 13.4%, and 9.9%, respectively. It was proved that the new method was effective in the landslides automatic extraction.

A Novel Squeeze-and-Excitation W-Net for 2D and 3D Building Change Detection with Multi-Source and Multi-Feature Remote Sensing Data

Building Change Detection (BCD) is one of the core issues in earth observation and has received extensive attention in recent years. With the rapid development of earth observation technology, the data source of remote sensing change detection is continuously enriched, which provides the possibility to describe the spatial details of the ground objects more finely and to characterize the ground objects with multiple perspectives and levels. However, due to the different physical mechanisms of multi-source remote sensing data, BCD based on heterogeneous data is a challenge. Previous studies mostly focused on the BCD of homogeneous remote sensing data, while the use of multi-source remote sensing data and considering multiple features to conduct 2D and 3D BCD research is sporadic. In this article, we propose a novel and general squeeze-and-excitation W-Net, which is developed from U-Net and SE-Net. Its unique advantage is that it can not only be used for BCD of homogeneous and heterogeneous remote sensing data respectively but also can input both homogeneous and heterogeneous remote sensing data for 2D or 3D BCD by relying on its bidirectional symmetric end-to-end network architecture. Moreover, from a unique perspective, we use image features that are stable in performance and less affected by radiation differences and temporal changes. We innovatively introduced the squeeze-and-excitation module to explicitly model the interdependence between feature channels so that the response between the feature channels is adaptively recalibrated to improve the information mining ability and detection accuracy of the model. As far as we know, this is the first proposed network architecture that can simultaneously use multi-source and multi-feature remote sensing data for 2D and 3D BCD. The experimental results in two 2D data sets and two challenging 3D data sets demonstrate that the promising performances of the squeeze-and-excitation W-Net outperform several traditional and state-of-the-art approaches. Moreover, both visual and quantitative analyses of the experimental results demonstrate competitive performance in the proposed network. This demonstrates that the proposed network and method are practical, physically justified, and have great potential application value in large-scale 2D and 3D BCD and qualitative and quantitative research.

Automatic Extraction of Seismic Landslides in Large Areas with Complex Environments Based on Deep Learning: An Example of the 2018 Iburi Earthquake, Japan

After a major earthquake, the rapid identification and mapping of co-seismic landslides in the whole affected area is of great significance for emergency rescue and loss assessment of seismic hazards. In recent years, researchers have achieved good results in research on a small scale and single environment characteristics of this issue. However, for the whole earthquake-affected area with large scale and complex environments, the correct rate of extracting co-seismic landslides remains low, and there is no ideal method to solve this problem. In this paper, Planet Satellite images with a spatial resolution of 3 m are used to train a seismic landslide recognition model based on the deep learning method to carry out rapid and automatic extraction of landslides triggered by the 2018 Iburi earthquake, Japan. The study area is about 671.87 km2, of which 60% is used to train the model, and the remaining 40% is used to verify the accuracy of the model. The results show that most of the co-seismic landslides can be identified by this method. In this experiment, the verification precision of the model is 0.7965 and the F1 score is 0.8288. This method can intelligently identify and map landslides triggered by earthquakes from Planet images. It has strong practicability and high accuracy. It can provide assistance for earthquake emergency rescue and rapid disaster assessment.

Mapping Post-Earthquake Landslide Susceptibility: A U-Net Like Approach

A serious earthquake could trigger thousands of landslides and produce some slopes more sensitive to slide in future. Landslides could threaten human’s lives and properties, and thus mapping the post-earthquake landslide susceptibility is very valuable for a rapid response to landslide disasters in terms of relief resource allocation and posterior earthquake reconstruction. Previous researchers have proposed many methods to map landslide susceptibility but seldom considered the spatial structure information of the factors that influence a slide. In this study, we first developed a U-net like model suitable for mapping post-earthquake landslide susceptibility. The post-earthquake high spatial airborne images were used for producing a landslide inventory. Pre-earthquake Landsat TM (Thematic Mapper) images and the influencing factors such as digital elevation model (DEM), slope, aspect, multi-scale topographic position index (mTPI), lithology, fault, road network, streams network, and macroseismic intensity (MI) were prepared as the input layers of the model. Application of the model to the heavy-hit area of the destructive 2008 Wenchuan earthquake resulted in a high validation accuracy (precision 0.77, recall 0.90, F1 score 0.83, and AUC 0.90). The performance of this U-net like model was also compared with those of traditional logistic regression (LR) and support vector machine (SVM) models on both the model area and independent testing area with the former being stronger than the two traditional models. The U-net like model introduced in this paper provides us the inspiration that balancing the environmental influence of a pixel itself and its surrounding pixels to perform a better landslide susceptibility mapping (LSM) task is useful and feasible when using remote sensing and GIS technology.

Automatic Mapping of Landslides by the ResU-Net

Massive landslides over large regions can be triggered by heavy rainfalls or major seismic events. Mapping regional landslides quickly is important for disaster mitigation. In recent years, deep learning methods have been successfully applied in many fields, including landslide automatic identification. In this work, we proposed a deep learning approach, the ResU-Net, to map regional landslides automatically. This method and a baseline model (U-Net) were collectively tested in Tianshui city, Gansu province, where a heavy rainfall triggered more than 10,000 landslides in July 2013. All models were performed on a 3-band (near infrared, red, and green) GeoEye-1 image with a spatial resolution of 0.5 m. At such a fine spatial resolution, the study area is spatially heterogeneous. The tested study area is 128 km2, 80% of which was used to train models and the remaining 20% was used to validate accuracy of the models. This proposed ResU-Net achieved higher accuracy than the baseline U-Net model in this mountain region, where F1 improved by 0.09. Compared with the U-Net model, this proposed model (ResU-Net) performs better in discriminating landslides from bare floodplains along river valleys and unplanted terraces. By incorporating environmental information, this ResU-Net may also be applied to other landslide mapping, such as landslide susceptibility and hazard assessment.