Multi-Temporal SAR Data Large-Scale Crop Mapping Based on U-Net Model

On the design of deep learning-based control algorithms for visually guided UAVs engaged in power tower inspection tasks

This paper focuses on the design of Convolution Neural Networks to visually guide an autonomous Unmanned Aerial Vehicle required to inspect power towers. The network is required to precisely segment images taken by a camera mounted on a UAV in order to allow a motion module to generate collision-free and inspection-relevant manoeuvres of the UAV along different types of towers. The images segmentation process is particularly challenging not only because of the different structures of the towers but also because of the enormous variability of the background, which can vary from the uniform blue of the sky to the multi-colour complexity of a rural, forest, or urban area. To be able to train networks that are robust enough to deal with the task variability, without incurring into a labour-intensive and costly annotation process of physical-world images, we have carried out a comparative study in which we evaluate the performances of networks trained either with synthetic images (i.e., the synthetic dataset), physical-world images (i.e., the physical-world dataset), or a combination of these two types of images (i.e., the hybrid dataset). The network used is an attention-based U-NET. The synthetic images are created using photogrammetry, to accurately model power towers, and simulated environments modelling a UAV during inspection of different power towers in different settings. Our findings reveal that the network trained on the hybrid dataset outperforms the networks trained with the synthetic and the physical-world image datasets. Most notably, the networks trained with the hybrid dataset demonstrates a superior performance on multiples evaluation metrics related to the image-segmentation task. This suggests that, the combination of synthetic and physical-world images represents the best trade-off to minimise the costs related to capturing and annotating physical-world images, and to maximise the task performances. Moreover, the results of our study demonstrate the potential of photogrammetry in creating effective training datasets to design networks to automate the precise movement of visually-guided UAVs.

Using Sentinel-1, Sentinel-2, and Planet satellite data to map field-level tillage practices in smallholder systems

Remote sensing can be used to map tillage practices at large spatial and temporal scales. However, detecting such management practices in smallholder systems is challenging given that the size of fields is smaller than historical readily-available satellite imagery. In this study we used newer, higher-resolution satellite data from Sentinel-1, Sentinel-2, and Planet to map tillage practices in the Eastern Indo-Gangetic Plains in India. We specifically tested the classification performance of single sensor and multiple sensor random forest models, and the impact of spatial, temporal, or spectral resolution on classification accuracy. We found that when considering a single sensor, the model that used Planet imagery (3 m) had the highest classification accuracy (86.55%) while the model that used Sentinel-1 data (10 m) had the lowest classification accuracy (62.28%). When considering sensor combinations, the model that used data from all three sensors achieved the highest classification accuracy (87.71%), though this model was not statistically different from the Planet only model when considering 95% confidence intervals from bootstrap analyses. We also found that high levels of accuracy could be achieved by only using imagery from the sowing period. Considering the impact of spatial, temporal, and spectral resolution on classification accuracy, we found that improved spatial resolution from Planet contributed the most to improved classification accuracy. Overall, it is possible to use readily-available, high spatial resolution satellite data to map tillage practices of smallholder farms, even in heterogeneous systems with small field sizes.

Rice Mapping in a Subtropical Hilly Region Based on Sentinel-1 Time Series Feature Analysis and the Dual Branch BiLSTM Model

Timely and accurate information on rice cultivation makes important contributions to the profound reform of the global food and agricultural system, and promotes the development of global sustainable agriculture. With all-day and all-weather observing ability, synthetic aperture radar (SAR) can monitor the distribution of rice in tropical and subtropical areas. To solve the problem of misclassification of rice with no marked signal during the flooding period in subtropical hilly areas, this paper proposes a new feature combination and dual branch bi-directional long short-term memory (DB-BiLSTM) model to achieve high-precision rice mapping using Sentinel-1 time series data. Based on field investigation data, the backscatter time series curves of the rice area were analyzed, and a characteristic index (VV − VH)/(VV + VH) (VV: vertical emission and vertical receipt of polarization, VH: vertical emission and horizontal receipt of polarization) for small areas of hilly land was proposed to effectively distinguish rice and non-rice crops with no marked flooding period. The DB-BiLSTM model was designed, ensuring the independent learning of multiple features and effectively combining the time series information of both (VV − VH)/(VV + VH) and VH features. The city of Shanwei, Guangdong Province, China, was selected as the study area. Experimental results showed that the overall accuracy of the rice mapping results was 97.29%, and the kappa coefficient reached 0.9424. Compared to other methods, the rice mapping results obtained by the proposed method maintained good integrity and had less misclassification, which demonstrated the proposed method’s practical value in accurate and effective rice mapping tasks.

Water Body Mapping Using Long Time Series Sentinel-1 SAR Data in Poyang Lake

Mapping water bodies with a high accuracy is necessary for water resource assessment, and mapping them rapidly is necessary for flood monitoring. Poyang Lake is the largest freshwater lake in China, and its wetland is one of the most important in the world. Poyang Lake is affected by floods from the Yangtze River basin every year, and the fluctuation of the water area and water level directly or indirectly affects the ecological environment of Poyang Lake. Synthetic Aperture Radar (SAR) is particularly suitable for large-scale water body mapping, as SAR allows data acquisition regardless of illumination and weather conditions. The two-satellite Sentinel-1 constellation, providing C-Band SAR data, passes over the Poyang Lake about five times a month. With its high temporal-spatial resolution, the Sentinel-1 SAR data can be used to accurately monitor the water body. After acquiring all the Sentinel-1 (1A and 1B) SAR data, to ensure the consistency of data processing, we propose the use of a Python and SeNtinel Application Platform (SNAP)-based engine (SARProcMod) to process the data and construct a Poyang Lake Sentinel-1 SAR dataset with a 10 m resolution. To extract water body information from Sentinel-1 SAR data, we propose an automatic classification engine based on a modified U-Net convolutional neural network (WaterUNet), which classifies all data using artificial sample datasets with a high validation accuracy. The results show that the maximum and minimum water areas in our study area were 2714.08 km2 on 20 July 2020, and 634.44 km2 on 4 January 2020. Compared to the water level data from the Poyang gauging station, the water area was highly correlated with the water level, with the correlation coefficient being up to 0.92 and the R2 from quadratic polynomial fitting up to 0.88; thus, the resulting relationship results can be used to estimate the water area or water level of Poyang Lake. According to the results, we can conclude that Sentinel-1 SAR and WaterUNet are very suitable for water body monitoring as well as emergency flood mapping.

Collaborative Extraction of Paddy Planting Areas with Multi-Source Information Based on Google Earth Engine: A Case Study of Cambodia

High-precision spatial mapping of paddy planting areas is very important for food security risk assessment and agricultural monitoring. Previous studies have mainly been based on multi-source satellite imagery, the fusion of Synthetic Aperture Radar (SAR) with optical data, and the combined use of multi-scale and multi-source sensors. However, there have been few studies on paddy spatial mapping using collaborative multi-source remote sensing product information, especially in tropical regions such as Southeast Asia. Therefore, based on the Google Earth Engine (GEE) platform, in this study, Cambodia, which is dominated by agriculture, was taken as the study area, and an extraction scheme for paddy planting areas was developed from collaborative multi-source information, including multi-source remote sensing images (Sentinel-1 and Sentinel-2), multi-source remote sensing land cover products (GFSAD30SEACE, GLC_FCS30-2015, FROM_GLC2015, SERVIR MEKONG, and GUF), paddy phenology information, and topographic features. Evaluation and analysis of the extraction results and the SERVIR MEKONG and ESACCI-LC paddy products revealed that the accuracy of the paddy planting areas extracted using the proposed method is the highest, with an overall accuracy of 89.90%. The results of the proposed method are better than those of the other products in terms of the outline of the paddy planting areas and the description of the road information. The results of this study provide a reference for future high-precision paddy mapping.

Identification of Crop Type Based on C-AENN Using Time Series Sentinel-1A SAR Data

Crop type identification is the initial stage and an important part of the agricultural monitoring system. It is well known that synthetic aperture radar (SAR) Sentinel-1A imagery provides a reliable data source for crop type identification. However, a single-temporal SAR image does not contain enough features, and the unique physical characteristics of radar images are relatively lacking, which limits its potential in crop mapping. In addition, current methods may not be applicable for time -series SAR data. To address the above issues, a new crop type identification method was proposed. Specifically, a farmland mask was firstly generated by the object Markov random field (OMRF) model to remove the interference of non-farmland factors. Then, the features of the standard backscatter coefficient, Sigma-naught (σ0), and the normalized backscatter coefficient by the incident angle, Gamma-naught (γ0), were extracted for each type of crop, and the optimal feature combination was found from time -series SAR images by means of Jeffries-Matusita (J-M) distance analysis. Finally, to make efficient utilization of optimal multi-temporal feature combination, a new network, the convolutional-autoencoder neural network (C-AENN), was developed for the crop type identification task. In order to prove the effectiveness of the method, several classical machine learning methods such as support vector machine (SVM), random forest (RF), etc., and deep learning methods such as one dimensional convolutional neural network (1D-CNN) and stacked auto-encoder (SAE), etc., were used for comparison. In terms of quantitative assessment, the proposed method achieved the highest accuracy, with a macro-F1 score of 0.9825, an overall accuracy (OA) score of 0.9794, and a Kappa coefficient (Kappa) score of 0.9705. In terms of qualitative assessment, four typical regions were chosen for intuitive comparison with the sample maps, and the identification result covering the study area was compared with a contemporaneous optical image, which indicated the high accuracy of the proposed method. In short, this study enables the effective identification of crop types, which demonstrates the importance of multi-temporal radar images in feature combination and the necessity of deep learning networks to extract complex features.

Large-Scale Rice Mapping Using Multi-Task Spatiotemporal Deep Learning and Sentinel-1 SAR Time Series

Timely and accurate cropland information at large spatial scales can improve crop management and support the government in decision making. Mapping the spatial extent and distribution of crops on a large spatial scale is challenging work due to the spatial variability. A multi-task spatiotemporal deep learning model, named LSTM-MTL, was developed in this study for large-scale rice mapping by utilizing time-series Sentinel-1 SAR data. The model showed a reasonable rice classification accuracy in the major rice production areas of the U.S. (OA = 98.3%, F1 score = 0.804), even when it only utilized SAR data. The model learned region-specific and common features simultaneously, and yielded a significant improved performance compared with RF and AtBiLSTM in both global and local training scenarios. We found that the LSTM-MTL model achieved a regional F1 score up to 10% higher than both global and local baseline models. The results demonstrated that the consideration of spatial variability via LSTM-MTL approach yielded an improved crop classification performance at a large spatial scale. We analyzed the input-output relationship through gradient backpropagation and found that low VH value in the early period and high VH value in the latter period were critical for rice classification. The results of in-season analysis showed that the model was able to yield a high accuracy (F1 score = 0.746) two months before rice maturity. The integration between multi-task learning and multi-temporal deep learning approach provides a promising approach for crop mapping at large spatial scales.

A Review of Deep Learning in Multiscale Agricultural Sensing

Population growth, climate change, and the worldwide COVID-19 pandemic are imposing increasing pressure on global agricultural production. The challenge of increasing crop yield while ensuring sustainable development of environmentally friendly agriculture is a common issue throughout the world. Autonomous systems, sensing technologies, and artificial intelligence offer great opportunities to tackle this issue. In precision agriculture (PA), non-destructive and non-invasive remote and proximal sensing methods have been widely used to observe crops in visible and invisible spectra. Nowadays, the integration of high-performance imagery sensors (e.g., RGB, multispectral, hyperspectral, thermal, and SAR) and unmanned mobile platforms (e.g., satellites, UAVs, and terrestrial agricultural robots) are yielding a huge number of high-resolution farmland images, in which rich crop information is compressed. However, this has been accompanied by challenges, i.e., ways to swiftly and efficiently making full use of these images, and then, to perform fine crop management based on information-supported decision making. In the past few years, deep learning (DL) has shown great potential to reshape many industries because of its powerful capabilities of feature learning from massive datasets, and the agriculture industry is no exception. More and more agricultural scientists are paying attention to applications of deep learning in image-based farmland observations, such as land mapping, crop classification, biotic/abiotic stress monitoring, and yield prediction. To provide an update on these studies, we conducted a comprehensive investigation with a special emphasis on deep learning in multiscale agricultural remote and proximal sensing. Specifically, the applications of convolutional neural network-based supervised learning (CNN-SL), transfer learning (TL), and few-shot learning (FSL) in crop sensing at land, field, canopy, and leaf scales are the focus of this review. We hope that this work can act as a reference for the global agricultural community regarding DL in PA and can inspire deeper and broader research to promote the evolution of modern agriculture.

A Dual Attention Convolutional Neural Network for Crop Classification Using Time-Series Sentinel-2 Imagery

Accurate and timely mapping of crop types and having reliable information about the cultivation pattern/area play a key role in various applications, including food security and sustainable agriculture management. Remote sensing (RS) has extensively been employed for crop type classification. However, accurate mapping of crop types and extents is still a challenge, especially using traditional machine learning methods. Therefore, in this study, a novel framework based on a deep convolutional neural network (CNN) and a dual attention module (DAM) and using Sentinel-2 time-series datasets was proposed to classify crops. A new DAM was implemented to extract informative deep features by taking advantage of both spectral and spatial characteristics of Sentinel-2 datasets. The spectral and spatial attention modules (AMs) were respectively applied to investigate the behavior of crops during the growing season and their neighborhood properties (e.g., textural characteristics and spatial relation to surrounding crops). The proposed network contained two streams: (1) convolution blocks for deep feature extraction and (2) several DAMs, which were employed after each convolution block. The first stream included three multi-scale residual convolution blocks, where the spectral attention blocks were mainly applied to extract deep spectral features. The second stream was built using four multi-scale convolution blocks with a spatial AM. In this study, over 200,000 samples from six different crop types (i.e., alfalfa, broad bean, wheat, barley, canola, and garden) and three non-crop classes (i.e., built-up, barren, and water) were collected to train and validate the proposed framework. The results demonstrated that the proposed method achieved high overall accuracy and a Kappa coefficient of 98.54% and 0.981, respectively. It also outperformed other state-of-the-art classification methods, including RF, XGBOOST, R-CNN, 2D-CNN, 3D-CNN, and CBAM, indicating its high potential to discriminate different crop types.

Monitoring System for the Management of the Common Agricultural Policy Using Machine Learning and Remote Sensing

The European Commission promotes new technologies and data generated by the Copernicus Programme. These technologies are intended to improve the management of the Common Agricultural Policy aid, implement new monitoring controls to replace on-the-spot checks, and apply up to 100% of the applications continuously for an agricultural year. This paper presents a generic methodology developed for implementing monitoring controls. To achieve this, the dataset provided by the Sentinel-2 time series is transformed into information through the combination of classifications with machine learning using random forest and remote sensing-based biophysical indices. This work focuses on monitoring the helpline associated with rice cultivation, using 13 Sentinel-2 images whose grouping and characteristics change depending on the event or landmark being sought. Moreover, the functionality to check, before harvesting the crop, that the area declared is equal to the area cultivated is added. The 2020 results are around 96% for most of the metrics analysed, demonstrating the potential of Sentinel-2 for controlling subsidies, particularly for rice. After the quality assessment, the hit rate is 98%. The methodology is transformed into a tool for regular use to improve decision making by determining which declarants comply with the crop-specific aid obligations, contributing to optimising the administrations’ resources and a fairer distribution of funds.

Rice Mapping in Training Sample Shortage Regions Using a Deep Semantic Segmentation Model Trained on Pseudo-Labels

A deep semantic segmentation model-based method can achieve state-of-the-art accuracy and high computational efficiency in large-scale crop mapping. However, the model cannot be widely used in actual large-scale crop mapping applications, mainly because the annotation of ground truth data for deep semantic segmentation model training is time-consuming. At the operational level, it is extremely difficult to obtain a large amount of ground reference data by photointerpretation for the model training. Consequently, in order to solve this problem, this study introduces a workflow that aims to extract rice distribution information in training sample shortage regions, using a deep semantic segmentation model (i.e., U-Net) trained on pseudo-labels. Based on the time series Sentinel-1 images, Cropland Data Layer (CDL) and U-Net model, the optimal multi-temporal datasets for rice mapping were summarized, using the global search method. Then, based on the optimal multi-temporal datasets, the proposed workflow (a combination of K-Means and random forest) was directly used to extract the rice-distribution information of Jiangsu (i.e., the K–RF pseudo-labels). For comparison, the optimal well-trained U-Net model acquired from Arkansas (i.e., the transfer model) was also transferred to Jiangsu to extract local rice-distribution information (i.e., the TF pseudo-labels). Finally, the pseudo-labels with high confidences generated from the two methods were further used to retrain the U-Net models, which were suitable for rice mapping in Jiangsu. For different rice planting pattern regions of Jiangsu, the final results showed that, compared with the U-Net model trained on the TF pseudo-labels, the rice area extraction errors of pseudo-labels could be further reduced by using the U-Net model trained on the K–RF pseudo-labels. In addition, compared with the existing rule-based rice mapping methods, he U-Net model trained on the K–RF pseudo-labels could robustly extract the spatial distribution information of rice. Generally, this study could provide new options for applying a deep semantic segmentation model to training sample shortage regions.

Mapping Crop Rotation by Using Deeply Synergistic Optical and SAR Time Series

Crop rotations, the farming practice of growing crops in sequential seasons, occupy a core position in agriculture management, showing a key influence on food security and agro-ecosystem sustainability. Despite the improvement in accuracy of identifying mono-agricultural crop distribution, crop rotation patterns remain poorly mapped. In this study, a hybrid convolutional neural network (CNN) and long short-term memory (LSTM) architecture, namely crop rotation mapping (CRM), were proposed to synergize the synthetic aperture radar (SAR) and optical time series in a rotational mapping task. The proposed end-to-end architecture had reasonable accuracies (i.e., accuracy > 0.85) in mapping crop rotation, which outperformed other state-of-the-art non-deep or deep-learning solutions. For some confusing rotation types, such as fallow-single rice and crayfish-single rice, CRM showed substantial improvements from traditional methods. Furthermore, the deeply synergistic SAR-optical, time-series data, with a corresponding attention mechanism, were effective in extracting crop rotation features, with an overall gain of accuracy of four points compared with ablation models. Therefore, our proposed method added wisdom to dynamic crop rotation mapping and yields important information for the agro-ecosystem management of the study area.

Paddy Rice Mapping in Thailand Using Time-Series Sentinel-1 Data and Deep Learning Model

The elimination of hunger is the top concern for developing countries and is the key to maintain national stability and security. Paddy rice occupies an essential status in food supply, whose accurate monitoring is of great importance for human sustainable development. As one of the most important paddy rice production countries in the world, Thailand has a favorable hot and humid climate for paddy rice growing, but the growth patterns of paddy rice are too complicated to construct promising growth models for paddy rice discrimination. To solve this problem, this study proposes a large-scale paddy rice mapping scheme, which uses time-series Sentinel-1 data to generate a convincing annual paddy rice map of Thailand. The proposed method extracts temporal statistical features of the time-series SAR images to overcome the intra-class variability due to different management practices and modifies the U-Net model with the fully connected Conditional Random Field (CRF) to maintain the edge of the fields. In this study, 758 Sentinel-1 images that covered the whole country from the end of 2018 to 2019 were acquired to generate the annual paddy rice map. The accuracy, precision, and recall of the resultant paddy rice map reached 91%, 87%, and 95%, respectively. Compared to SVM classifier and the U-Net model based on feature selection strategy (FS-U-Net), the proposed scheme achieved the best overall performance, which demonstrated the capability of overcoming the complex cultivation conditions and accurately identifying the fragmented paddy rice fields in Thailand. This study provides a promising tool for large-scale paddy rice monitoring in tropical production regions and has great potential in the global sustainable development of food and environment management.

Sentinel-1 Time Series for Crop Identification in the Framework of the Future CAP Monitoring

In this upcoming Common Agricultural Policy (CAP) reform, the use of satellite imagery is taking an increasing role for improving the Integrated Administration and Control System (IACS). Considering the operational aspect of the CAP monitoring process, the use of Sentinel-1 SAR (Synthetic Aperture Radar) images is highly relevant, especially in regions with a frequent cloud cover, such as Belgium. Indeed, SAR imagery does not depend on sunlight and is barely affected by the presence of clouds. Moreover, the SAR signal is particularly sensitive to the geometry and the water content of the target. Crop identification is often a pre-requisite to monitor agriculture at parcel level (ploughing, harvest, grassland mowing, intercropping, etc.) The main goal of this study is to assess the performances and constraints of a SAR-based crop classification in an operational large-scale application. The Random Forest object-oriented classification model is built on Sentinel-1 time series from January to August 2020 only. It can identify crops in the Walloon Region (south part of Belgium) with high performance: 93.4% of well-classified area, representing 88.4% of the parcels. Among the 48 crop groups, the six most represented ones get a F1-score higher or equal to 84%. Additionally, this research documents how the classification performance is affected by different parameters: the SAR orbit, the size of the training dataset, the use of different internal buffers on parcel polygons before signal extraction, the set of explanatory variables, and the period of the time series. In an operational context, this allows to choose the right balance between classification accuracy and model complexity. A key result is that using a training dataset containing only 3.2% of the total number of parcels allows to correctly classify 91.7% of the agricultural area. The impact of rain and snow is also discussed. Finally, this research analyses how the classification accuracy depends on some characteristics of the parcels like their shape or size. This allows to assess the relevance of the classification depending on those characteristics, as well as to identify a subset of parcels for which the global accuracy is higher.

Machine Learning in Agriculture: A Comprehensive Updated Review

The digital transformation of agriculture has evolved various aspects of management into artificial intelligent systems for the sake of making value from the ever-increasing data originated from numerous sources. A subset of artificial intelligence, namely machine learning, has a considerable potential to handle numerous challenges in the establishment of knowledge-based farming systems. The present study aims at shedding light on machine learning in agriculture by thoroughly reviewing the recent scholarly literature based on keywords' combinations of "machine learning" along with "crop management", "water management", "soil management", and "livestock management", and in accordance with PRISMA guidelines. Only journal papers were considered eligible that were published within 2018-2020. The results indicated that this topic pertains to different disciplines that favour convergence research at the international level. Furthermore, crop management was observed to be at the centre of attention. A plethora of machine learning algorithms were used, with those belonging to Artificial Neural Networks being more efficient. In addition, maize and wheat as well as cattle and sheep were the most investigated crops and animals, respectively. Finally, a variety of sensors, attached on satellites and unmanned ground and aerial vehicles, have been utilized as a means of getting reliable input data for the data analyses. It is anticipated that this study will constitute a beneficial guide to all stakeholders towards enhancing awareness of the potential advantages of using machine learning in agriculture and contributing to a more systematic research on this topic.

Plot-Based Classification of Macronutrient Levels in Oil Palm Trees with Landsat-8 Images and Machine Learning

Oil palm crops are essential for ensuring sustainable edible oil production, in which production is highly dependent on fertilizer applications. Using Landsat-8 imageries, the feasibility of macronutrient level classification with Machine Learning (ML) was studied. Variable rates of compost and inorganic fertilizer were applied to experimental plots and the following nutrients were studied: nitrogen (N), phosphorus (P), potassium (K), magnesium (Mg) and calcium (Ca). By applying image filters, separability metrics, vegetation indices (VI) and feature selection, spectral features for each plot were acquired and used with ML models to classify macronutrient levels of palm stands from chemical foliar analysis of their 17th frond. The models were calibrated and validated with 30 repetitions, with the best mean overall accuracy reported for N and K at 79.7 ± 4.3% and 76.6 ± 4.1% respectively, while accuracies for P, Mg and Ca could not be accurately classified due to the limitations of the dataset used. The study highlighted the effectiveness of separability metrics in quantifying class separability, the importance of indices for N and K level classification, and the effects of filter and feature selection on model performance, as well as concluding RF or SVM models for excessive N and K level detection. Future improvements should focus on further model validation and the use of higher-resolution imaging.

Potential of Hybrid CNN-RF Model for Early Crop Mapping with Limited Input Data

When sufficient time-series images and training data are unavailable for crop classification, features extracted from convolutional neural network (CNN)-based representative learning may not provide useful information to discriminate crops with similar spectral characteristics, leading to poor classification accuracy. In particular, limited input data are the main obstacles to obtain reliable classification results for early crop mapping. This study investigates the potential of a hybrid classification approach, i.e., CNN-random forest (CNN-RF), in the context of early crop mapping, that combines the automatic feature extraction capability of CNN with the superior discrimination capability of an RF classifier. Two experiments on incremental crop classification with unmanned aerial vehicle images were conducted to compare the performance of CNN-RF with that of CNN and RF with respect to the length of the time-series and training data sizes. When sufficient time-series images and training data were used for the classification, the accuracy of CNN-RF was slightly higher or comparable with that of CNN. In contrast, when fewer images and the smallest training data were used at the early crop growth stage, CNN-RF was substantially beneficial and the overall accuracy increased by maximum 6.7%p and 4.6%p in the two study areas, respectively, compared to CNN. This is attributed to its ability to discriminate crops from features with insufficient information using a more sophisticated classifier. The experimental results demonstrate that CNN-RF is an effective classifier for early crop mapping when only limited input images and training samples are available.

Microplastics pollution in the soil mulched by dust-proof nets: A case study in Beijing, China

As a driving factor of global changes, microplastics have gradually attracted widespread attention. Although MPs are extensively studied in aquatic systems, their presence and fate in terrestrial systems and soil are not fully understood. In China, construction-land must be mulched by dust-proof nets to prevent and control fine particulate pollution, which may cause MPs pollution and increase ecological risks. In order to understand the pollution characteristics and sources of MP in the soil covered by dust nets, we conducted a case study in Beijing. Our results revealed that the abundance of MPs in soil mulched by dust-proof nets ranged from 272 to 13,752 items/kg. Large-sized particles (>1000 μm) made up a significant proportion (49.83%) of MPs in the study area. The dominant MP polymer types were polyethylene (50.12%) and polypropylene (41.25%). The accumulation of MPs in construction-site soil mulched by dust-proof nets (average, 4910.2 items/kg) was significantly higher (P < 0.05) than that in unmulched soil (average, 840.8 items/kg), which indicates a dust-proof nets as an essential source of microplastics in the soil of construction land. We applied a remote-sensing data analysis technique based on remote imagery acquired from a high-resolution remote-sensing satellite combined with deep-learning convolutional neural networks to automatically detect and segment dust-proof nets. Based on high-resolution remote sensing images and using a U-net convolutional neural network, we extract the coverage area of Beijing's dust-proof nets (18.6 km²). Combined the abundance of MPs and the dust-proof nets' coverage area, we roughly estimate that 7.616 × 10⁹ to 3.581 × 10¹¹ MPs accumulated in the soil mulched by the dust-proof nets in Beijing. Such a large amount of MPs may cause a series of environmental problems. This study will highlight the understanding of soil MPs pollution and its potential environmental impacts for scientists and policymakers. It provides suggestions for decision-makers to formulate effective legislation and policies, so as to protect human health and protect the soil and the wider environment.

Spaceborne SAR Data for Regional Urban Mapping Using a Robust Building Extractor

With the rapid development of urbanization, timely and accurate information on the spatial distribution of urban areas is essential for urban planning, environmental protection and sustainable urban development. To date, the main problem of urban mapping using synthetic aperture radar (SAR) data are that nonbuilding objects with high backscattering cause high false alarms, while small-scale buildings with low backscattering result in omission errors. In this paper, a robust building-area extraction extractor is proposed to solve the above problems. The specific work includes (1) building a multiscale and multicategory building area dataset to learn enough building features in various areas; (2) designing a multiscale extraction network based on the residual convolutional block (ResNet50) and a pyramid-based pooling module to extract more discriminative features of building areas and introducing the focal loss item as the object function of the network to further extract the small-scale building areas and (3) eliminating the false alarms using the Normalized Difference Vegetation Index (NDVI) and Modified Normalized Difference Water Index (MNDWI) index. GF-3 SAR data with a 10-m resolution of four regions in China are used to validate our method, and the regional building-area mapping results with overall accuracy above 85% and kappa coefficient not less than 0.73 are obtained. Compared with the current popular networks and the global human settlement layer (GHSL) product, our method shows better extraction results and higher accuracy in multiscale building areas. The experiments using Sentinel-1 and ALOS-2/PALSAR-2 data show that the proposed method has good robustness with different SAR data sources.

Performance Analysis of Deep Convolutional Autoencoders with Different Patch Sizes for Change Detection from Burnt Areas

Fire is one of the primary sources of damages to natural environments globally. Estimates show that approximately 4 million km2 of land burns yearly. Studies have shown that such estimates often underestimate the real extent of burnt land, which highlights the need to find better, state-of-the-art methods to detect and classify these areas. This study aimed to analyze the use of deep convolutional Autoencoders in the classification of burnt areas, considering different sample patch sizes. A simple Autoencoder and the U-Net and ResUnet architectures were evaluated. We collected Landsat 8 OLI+ data from three scenes in four consecutive dates to detect the changes specifically in the form of burnt land. The data were sampled according to four different sampling strategies to evaluate possible performance changes related to sampling window sizes. The training stage used two scenes, while the validation stage used the remaining scene. The ground truth change mask was created using the Normalized Burn Ratio (NBR) spectral index through a thresholding approach. The classifications were evaluated according to the F1 index, Kappa index, and mean Intersection over Union (mIoU) value. Results have shown that the U-Net and ResUnet architectures offered the best classifications with average F1, Kappa, and mIoU values of approximately 0.96, representing excellent classification results. We have also verified that a sampling window size of 256 by 256 pixels offered the best results.

Cloud Detection for Satellite Imagery Using Attention-Based U-Net Convolutional Neural Network

Cloud detection is an important and difficult task in the pre-processing of satellite remote sensing data. The results of traditional cloud detection methods are often unsatisfactory in complex environments or the presence of various noise disturbances. With the rapid development of artificial intelligence technology, deep learning methods have achieved great success in many fields such as image processing, speech recognition, autonomous driving, etc. This study proposes a deep learning model suitable for cloud detection, Cloud-AttU, which is based on a U-Net network and incorporates an attention mechanism. The Cloud-AttU model adopts the symmetric Encoder-Decoder structure, which achieves the fusion of high-level features and low-level features through the skip-connection operation, making the output results contain richer multi-scale information. This symmetrical network structure is concise and stable, significantly enhancing the effect of image segmentation. Based on the characteristics of cloud detection, the model is improved by introducing an attention mechanism that allows model to learn more effective features and distinguish between cloud and non-cloud pixels more accurately. The experimental results show that the method proposed in this paper has a significant accuracy advantage over the traditional cloud detection method. The proposed method is also able to achieve great results in the presence of snow/ice disturbance and other bright non-cloud objects, with strong resistance to disturbance. The Cloud-AttU model proposed in this study has achieved excellent results in the cloud detection tasks, indicating that this symmetric network architecture has great potential for application in satellite image processing and deserves further research.

The Delineation and Grading of Actual Crop Production Units in Modern Smallholder Areas Using RS Data and Mask R-CNN

The extraction and evaluation of crop production units are important foundations for agricultural production and management in modern smallholder regions, which are very significant to the regulation and sustainable development of agriculture. Crop areas have been recognized efficiently and accurately via remote sensing (RS) and machine learning (ML), especially deep learning (DL), which are too rough for modern smallholder production. In this paper, a delimitation-grading method for actual crop production units (ACPUs) based on RS images was explored using a combination of a mask region-based convolutional neural network (Mask R-CNN), spatial analysis, comprehensive index evaluation, and cluster analysis. Da’an City, Jilin province, China, was chosen as the study region to satisfy the agro-production demands in modern smallholder areas. Firstly, the ACPUs were interpreted from perspectives such as production mode, spatial form, and actual productivity. Secondly, cultivated land plots (C-plots) were extracted by Mask R-CNN with high-resolution RS images, which were used to delineate contiguous cultivated land plots (CC-plots) on the basis of auxiliary data correction. Then, the refined delimitation-grading results of the ACPUs were obtained through comprehensive evaluation of spatial characteristics and real productivity clustering. For the conclusion, the effectiveness of the Mask R-CNN model in C-plot recognition (loss = 0.16, mean average precision (mAP) = 82.29%) and a reasonable distance threshold (20 m) for CC-plot delimiting were verified. The spatial features were evaluated with the scale-shape dimensions of nine specific indicators. Real productivities were clustered by the incorporation of two-step cluster and K-Means cluster. Furthermore, most of the ACPUs in the study area were of a reasonable scale and an appropriate shape, holding real productivities at a medium level or above. The proposed method in this paper can be adjusted according to the changes of the study area with flexibility to assist agro-supervision in many modern smallholder regions.

Change Detection of Deforestation in the Brazilian Amazon Using Landsat Data and Convolutional Neural Networks

Mapping deforestation is an essential step in the process of managing tropical rainforests. It lets us understand and monitor both legal and illegal deforestation and its implications, which include the effect deforestation may have on climate change through greenhouse gas emissions. Given that there is ample room for improvements when it comes to mapping deforestation using satellite imagery, in this study, we aimed to test and evaluate the use of algorithms belonging to the growing field of deep learning (DL), particularly convolutional neural networks (CNNs), to this end. Although studies have been using DL algorithms for a variety of remote sensing tasks for the past few years, they are still relatively unexplored for deforestation mapping. We attempted to map the deforestation between images approximately one year apart, specifically between 2017 and 2018 and between 2018 and 2019. Three CNN architectures that are available in the literature—SharpMask, U-Net, and ResUnet—were used to classify the change between years and were then compared to two classic machine learning (ML) algorithms—random forest (RF) and multilayer perceptron (MLP)—as points of reference. After validation, we found that the DL models were better in most performance metrics including the Kappa index, F1 score, and mean intersection over union (mIoU) measure, while the ResUnet model achieved the best overall results with a value of 0.94 in all three measures in both time sequences. Visually, the DL models also provided classifications with better defined deforestation patches and did not need any sort of post-processing to remove noise, unlike the ML models, which needed some noise removal to improve results.

A Study on Establishing a Microstructure-Related Hardness Model with Precipitate Segmentation Using Deep Learning Method

This paper established a microstructure-related hardness model of a polycrystalline Ni-based superalloy GH4720Li, and the sizes and area fractions of γ' precipitates were extracted from scanning electron microscope (SEM) images using a deep learning method. The common method used to obtain morphological parameters of γ' precipitates is the thresholding method. However, this method is not suitable for distinguishing different generations of γ' precipitates with similar gray values in SEM images, which needs many manual interventions. In this paper, we employ SEM with ATLAS (AuTomated Large Area Scanning) module to automatically and quickly detect a much wider range of microstructures. A deep learning method of U-Net is firstly applied to automatically and accurately segment different generations of γ' precipitates and extract their parameters from the large-area SEM images. Then the obtained sizes and area fractions of γ' precipitates are used to study the precipitate stability and microstructure-related hardness of GH4720Li alloy at long-term service temperatures. The experimental results show that primary and secondary γ' precipitates show good stability under long-term service temperatures. Tertiary γ' precipitates coarsen selectively, and their coarsening behavior can be predicted by the Lifshitz-Slyozov encounter modified (LSEM) model. The hardness decreases as a result of γ' coarsening. A microstructure-related hardness model for correlating the hardness of the γ'/γ coherent structures and the microstructure is established, which can effectively predict the hardness of the alloy with different microstructures.

Feature Dimension Reduction Using Stacked Sparse Auto-Encoders for Crop Classification with Multi-Temporal, Quad-Pol SAR Data

Crop classification in agriculture is one of important applications for polarimetric synthetic aperture radar (PolSAR) data. For agricultural crop discrimination, compared with single-temporal data, multi-temporal data can dramatically increase crop classification accuracies since the same crop shows different external phenomena as it grows up. In practice, the utilization of multi-temporal data encounters a serious problem known as a “dimension disaster”. Aiming to solve this problem and raise the classification accuracy, this study developed a feature dimension reduction method using stacked sparse auto-encoders (S-SAEs) for crop classification. First, various incoherent scattering decomposition algorithms were employed to extract a variety of detailed and quantitative parameters from multi-temporal PolSAR data. Second, based on analyzing the configuration and main parameters for constructing an S-SAE, a three-hidden-layer S-SAE network was built to reduce the dimensionality and extract effective features to manage the “dimension disaster” caused by excessive scattering parameters, especially for multi-temporal, quad-pol SAR images. Third, a convolutional neural network (CNN) was constructed and employed to further enhance the crop classification performance. Finally, the performances of the proposed strategy were assessed with the simulated multi-temporal Sentinel-1 data for two experimental sites established by the European Space Agency (ESA). The experimental results showed that the overall accuracy with the proposed method was raised by at least 17% compared with the long short-term memory (LSTM) method in the case of a 1% training ratio. Meanwhile, for a CNN classifier, the overall accuracy was almost 4% higher than those of the principle component analysis (PCA) and locally linear embedded (LLE) methods. The comparison studies clearly demonstrated the advantage of the proposed multi-temporal crop classification methodology in terms of classification accuracy, even with small training ratios.

One View Per City for Buildings Segmentation in Remote-Sensing Images via Fully Convolutional Networks: A Proof-of-Concept Study

The segmentation of buildings in remote-sensing (RS) images plays an important role in monitoring landscape changes. Quantification of these changes can be used to balance economic and environmental benefits and most importantly, to support the sustainable urban development. Deep learning has been upgrading the techniques for RS image analysis. However, it requires a large-scale data set for hyper-parameter optimization. To address this issue, the concept of “one view per city” is proposed and it explores the use of one RS image for parameter settings with the purpose of handling the rest images of the same city by the trained model. The proposal of this concept comes from the observation that buildings of a same city in single-source RS images demonstrate similar intensity distributions. To verify the feasibility, a proof-of-concept study is conducted and five fully convolutional networks are evaluated on five cities in the Inria Aerial Image Labeling database. Experimental results suggest that the concept can be explored to decrease the number of images for model training and it enables us to achieve competitive performance in buildings segmentation with decreased time consumption. Based on model optimization and universal image representation, it is full of potential to improve the segmentation performance, to enhance the generalization capacity, and to extend the application of the concept in RS image analysis.

Synchronous Response Analysis of Features for Remote Sensing Crop Classification Based on Optical and SAR Time-Series Data

Accurate crop classification is the basis of agricultural research, and remote sensing is the only effective measuring technique to classify crops over large areas. Optical remote sensing is effective in regions with good illumination; however, it usually fails to meet requirements for highly accurate crop classification in cloud-covered areas and rainy regions. Synthetic aperture radar (SAR) can achieve active data acquisition by transmitting signals; thus, it has strong resistance to cloud and rain interference. In this study, we designed an improved crop planting structure mapping framework for cloudy and rainy regions by combining optical data and SAR data, and we revealed the synchronous-response relationship of these two data types. First, we extracted geo-parcels from optical images with high spatial resolution. Second, we built a recurrent neural network (RNN)-based classifier suitable for remote sensing images on the geo-parcel scale. Third, we classified crops based on the two datasets and established the network. Fourth, we analyzed the synchronous response relationships of crops based on the results of the two classification schemes. This work is the basis for the application of remote sensing data for the fine mapping and growth monitoring of crop planting structures in cloudy and rainy areas in the future.

Ship Detection Using a Fully Convolutional Network with Compact Polarimetric SAR Images

Compact polarimetric synthetic aperture radar (CP SAR), as a new technique or observation system, has attracted much attention in recent years. Compared with quad-polarization SAR (QP SAR), CP SAR provides an observation with a wider swath, while, compared with linear dual-polarization SAR, retains more polarization information in observations. These characteristics make CP SAR a useful tool in marine environmental applications. Previous studies showed the potential of CP SAR images for ship detection. However, false alarms, caused by ocean clutter and the lack of detailed information about ships, largely hinder traditional methods from feature selection for ship discrimination. In this paper, a segmentation method designed specifically for ship detection from CP SAR images is proposed. The pixel-wise detection is based on a fully convolutional network (i.e., U-Net). In particular, three classes (ship, land, and sea) were considered in the classification scheme. To extract features, a series of down-samplings with several convolutions were employed. Then, to generate classifications, deep semantic and shallow high-resolution features were used in up-sampling. Experiments on several CP SAR images simulated from Gaofen-3 QP SAR images demonstrate the effectiveness of the proposed method. Compared with Faster RCNN (region-based convolutional neural network), which is considered a popular and effective deep learning network for object detection, the newly proposed method, with precision and recall greater than 90% and a F1 score of 0.912, performs better at ship detection. Additionally, findings verify the advantages of the CP configuration compared with single polarization and linear dual-polarization.

Urban Building Change Detection in SAR Images Using Combined Differential Image and Residual U-Net Network

With the rapid development of urbanization in China, monitoring urban changes is of great significance to city management, urban planning, and cadastral map updating. Spaceborne synthetic aperture radar (SAR) sensors can capture a large area of radar images quickly with fine spatiotemporal resolution and are not affected by weather conditions, making multi-temporal SAR images suitable for change detection. In this paper, a new urban building change detection method based on an improved difference image and residual U-Net network is proposed. In order to overcome the intensity compression problem of the traditional log-ratio method, the spatial distance and intensity similarity are combined to generate a weighting function to obtain a weighted difference image. By fusing the weighted difference image and the bitemporal original images, the three-channel color difference image is generated for building change detection. Due to the complexity of urban environments and the small scale of building changes, the residual U-Net network is used instead of fixed statistical models and the construction and classifier of the network are modified to distinguish between different building changes. Three scenes of Sentinel-1 interferometric wide swath data are used to validate the proposed method. The experimental results and comparative analysis show that our proposed method is effective for urban building change detection and is superior to the original U-Net and SVM method.

A Novel Spatio-Temporal FCN-LSTM Network for Recognizing Various Crop Types Using Multi-Temporal Radar Images

In recent years, analyzing Synthetic Aperture Radar (SAR) data has turned into one of the challenging and interesting topics in remote sensing. Radar sensors are capable of imaging Earth’s surface independently of the weather conditions, local time of day, penetrating of waves through clouds, and containing spatial information on agricultural crop types. Based on these characteristics, the main goal sought in this research is to reveal the SAR imaging data capability in recognizing various agricultural crops in the main growth season in a more clarified and detailed way by using a deep-learning-based method. In the present research, the multi-temporal C-band Sentinel 1 images were used to classify 14 major classes of agricultural crops plus background in Denmark. By considering the capability of a deep learning method in analyzing satellite images, a novel, optimal, and lightweight network structure was developed and implemented based on a combination of a fully convolutional network (FCN) and a convolutional long short-term memory (ConvLSTM) network. The average pixel-based accuracy and Intersection over Union obtained from the proposed network were 86% and 0.64, respectively. Winter rapeseed, winter barley, winter wheat, spring barley, and sugar beet had the highest pixel-based accuracies of 95%, 94%, 93%, 90%, and 90%; respectively. The pixel-based accuracies for eight crop types and the background class were more than 84%. The network prediction showed that in field borders the classification confidence was lower than the center regions of the fields. However, the proposed structure has been able to identify different crops in multi-temporal Sentinel 1 data of a large area of around 254 thousand hectares with high performance.

Robinia pseudoacacia L. in Short Rotation Coppice: Seed and Stump Shoot Reproduction as well as UAS-based Spreading Analysis

Varying reproduction strategies are an important trait that tree species need in order both to survive and to spread. Black locust is able to reproduce via seeds, stump shoots, and root suckers. However, little research has been conducted on the reproduction and spreading of black locust in short rotation coppices. This research study focused on seed germination, stump shoot resprout, and spreading by root suckering of black locust in ten short rotation coppices in Germany. Seed experiments and sample plots were analyzed for the study. Spreading was detected and measured with unmanned aerial system (UAS)-based images and classification technology—object-based image analysis (OBIA). Additionally, the classification of single UAS images was tested by applying a convolutional neural network (CNN), a deep learning model. The analyses showed that seed germination increases with increasing warm-cold variety and scarification. Moreover, it was found that the number of shoots per stump decreases as shoot age increases. Furthermore, spreading increases with greater light availability and decreasing tillage. The OBIA and CNN image analysis technologies achieved 97% and 99.5% accuracy for black locust classification in UAS images. All in all, the three reproduction strategies of black locust in short rotation coppices differ with regards to initialization, intensity, and growth performance, but all play a role in the survival and spreading of black locust.