Evaluation of Three Deep Learning Models for Early Crop Classification Using Sentinel-1A Imagery Time Series—A Case Study in Zhanjiang, China

Mapping Paddy Rice in Complex Landscapes with Landsat Time Series Data and Superpixel-Based Deep Learning Method

The spatial pattern and temporal variation in paddy rice areas captured by remote sensing imagery provide an effective way of performing crop management and developing suitable agricultural policies. However, fragmented and scattered rice paddies due to undulating and varied topography, and the availability and quality of remote sensing images (e.g., frequent cloud coverage) pose significant challenges to accurate long-term rice mapping, especially for traditional pixel and phenological methods in subtropical monsoon regions. This study proposed a superpixel and deep-learning-based time series method to analyze Landsat time series data for paddy rice classification in complex landscape regions. First, a superpixel segmentation map was generated using a dynamic-time-warping-based simple non-iterative clustering algorithm with preprocessed spectral indices (SIs) time series data. Second, the SI images were overlaid onto the superpixel map to construct mean SIs time series for each superpixel. Third, a multivariate long short-term memory full convolution neural network (MLSTM-FCN) classifier was employed to learn time series features of rice paddies to produce accurate paddy rice maps. The method was evaluated using Landsat imagery from 2000 to 2020 in Cengong County, Guizhou Province, China. Results indicate that the superpixel MLSTM-FCN achieved a high performance with an overall accuracy varying from 0.9547 to 0.9721, which presents an 0.17–1.23% improvement compared to the random forest method. This study showed that combining spectral, spatial, and temporal features with deep learning methods can generate accurate paddy rice maps in complex landscape regions.

Multiscale Superpixel-Based Fine Classification of Crops in the UAV-Manned Hyperspectral Imagery

As an effective approach to obtaining agricultural information, the remote sensing technique has been applied in the classification of crop types. The unmanned aerial vehicle (UAV)-manned hyperspectral sensors provide imagery with high spatial and high spectral resolutions. Moreover, the detailed spatial information, as well as abundant spectral properties of UAV-manned hyperspectral imagery, opens a new avenue to the fine classification of crops. In this manuscript, multiscale superpixel-based approaches are proposed for the fine identification of crops in the UAV-manned hyperspectral imagery. Specifically, the multiscale superpixel segmentation is performed and a series of superpixel maps can be obtained. Then, the multiscale information is integrated into image classification by two strategies, namely pre-processing and post-processing. For the pre-processing strategy, the superpixel is regarded as the minimum unit for image classification, whose feature is obtained by using the average of spectral values of pixels within it. At each scale, the classification is performed on the basis of the superpixel. Then, the multiscale classification results are combined to generate the final map. For the post-processing strategy, the pixel-wise classification is implemented to obtain the label and posterior probabilities of each pixel. Subsequently, the superpixel-based voting is conducted at each scale, and these obtained voting results are fused to generate the multiscale voting result. To evaluate the effectiveness of the proposed approaches, three open-sourced hyperspectral UAV-manned datasets are employed in the experiments. Meanwhile, seven training sets with different numbers of labeled samples and two classifiers are taken into account for further analysis. The results demonstrate that the multiscale superpixel-based approaches outperform the single-scale approaches. Meanwhile, the post-processing strategy is superior to the pre-processing strategy in terms of higher classification accuracies in all the datasets.

Performance and the Optimal Integration of Sentinel-1/2 Time-Series Features for Crop Classification in Northern Mongolia

Accurate and early crop-type maps are essential for agricultural policy development and food production assessment at regional and national levels. This study aims to produce a crop-type map with acceptable accuracy and spatial resolution in northern Mongolia by optimizing the combination of Sentinel-1 (S1) and Sentinel-2 (S2) images with the Google Earth Engine (GEE) environment. A total of three satellite data combination scenarios are set, including S1 alone, S2 alone, and the combination of S1 and S2. In order to avoid the impact of data gaps caused by clouds on crop classification, this study reconstructed the time series of S1 and S2 with a 10-day interval using the median composite method, linear moving interpolation, and Savitzky–Golay (SG) filter. Our results indicated that crop-type classification accuracy increased with the increase in data length to all three data combination scenarios. S2 alone has higher accuracy than S1 alone and the combination of S1 and S2. The crop-type map with the highest accuracy was generated using S2 data from 150 days of the year (DOY) (11 May) to 260 DOY (18 September). The OA and kappa were 0.93 and 0.78, respectively, and the F1-score for spring wheat and rapeseed were 0.96 and 0.80, respectively. The classification accuracy of the crop increased rapidly from 210 DOY (end of July) to 260 DOY (August to mid-September), and then it remained stable after 260 DOY. Based on our analysis, we filled the gap of the crop-type map with 10 m spatial resolution in northern Mongolia, revealing the best satellite combination and the best period for crop-type classification, which can benefit the achievement of sustainable development goals 2 (SDGs2).

Identification of Crop Type Based on C-AENN Using Time Series Sentinel-1A SAR Data

Crop type identification is the initial stage and an important part of the agricultural monitoring system. It is well known that synthetic aperture radar (SAR) Sentinel-1A imagery provides a reliable data source for crop type identification. However, a single-temporal SAR image does not contain enough features, and the unique physical characteristics of radar images are relatively lacking, which limits its potential in crop mapping. In addition, current methods may not be applicable for time -series SAR data. To address the above issues, a new crop type identification method was proposed. Specifically, a farmland mask was firstly generated by the object Markov random field (OMRF) model to remove the interference of non-farmland factors. Then, the features of the standard backscatter coefficient, Sigma-naught (σ0), and the normalized backscatter coefficient by the incident angle, Gamma-naught (γ0), were extracted for each type of crop, and the optimal feature combination was found from time -series SAR images by means of Jeffries-Matusita (J-M) distance analysis. Finally, to make efficient utilization of optimal multi-temporal feature combination, a new network, the convolutional-autoencoder neural network (C-AENN), was developed for the crop type identification task. In order to prove the effectiveness of the method, several classical machine learning methods such as support vector machine (SVM), random forest (RF), etc., and deep learning methods such as one dimensional convolutional neural network (1D-CNN) and stacked auto-encoder (SAE), etc., were used for comparison. In terms of quantitative assessment, the proposed method achieved the highest accuracy, with a macro-F1 score of 0.9825, an overall accuracy (OA) score of 0.9794, and a Kappa coefficient (Kappa) score of 0.9705. In terms of qualitative assessment, four typical regions were chosen for intuitive comparison with the sample maps, and the identification result covering the study area was compared with a contemporaneous optical image, which indicated the high accuracy of the proposed method. In short, this study enables the effective identification of crop types, which demonstrates the importance of multi-temporal radar images in feature combination and the necessity of deep learning networks to extract complex features.

Deep learning techniques to classify agricultural crops through UAV imagery: a review

During the last few years, Unmanned Aerial Vehicles (UAVs) technologies are widely used to improve agriculture productivity while reducing drudgery, inspection time, and crop management cost. Moreover, they are able to cover large areas in a matter of a few minutes. Due to the impressive technological advancement, UAV-based remote sensing technologies are increasingly used to collect valuable data that could be used to achieve many precision agriculture applications, including crop/plant classification. In order to process these data accurately, we need powerful tools and algorithms such as Deep Learning approaches. Recently, Convolutional Neural Network (CNN) has emerged as a powerful tool for image processing tasks achieving remarkable results making it the state-of-the-art technique for vision applications. In the present study, we reviewed the recent CNN-based methods applied to the UAV-based remote sensing image analysis for crop/plant classification to help researchers and farmers to decide what algorithms they should use accordingly to their studied crops and the used hardware. Fusing different UAV-based data and deep learning approaches have emerged as a powerful tool to classify different crop types accurately. The readers of the present review could acquire the most challenging issues facing researchers to classify different crop types from UAV imagery and their potential solutions to improve the performance of deep learning-based algorithms.

Temporal Characteristics of Stress Signals Using GRU Algorithm for Heavy Metal Detection in Rice Based on Sentinel-2 Images

Heavy metal stress, which is a serious environmental problem, affects both animal and human health through the food chain. However, such subtle stress information is difficult to detect in remote sensing images. Therefore, enhancing the stress signal is key to accurately identifying heavy metal contamination in crops. The aim of this study was to identify heavy metal stress in rice at a regional scale by mining the time-series characteristics of rice growth under heavy metal stress using the gated recurrent unit (GRU) algorithm. The experimental area was located in Zhuzhou City, Hunan Province, China. We collected situ-measured data and Sentinel-2A images corresponding to the 2019-2021 period. First, the spatial distribution of the rice in the study area was extracted using the random forest algorithm based on the Sentinel 2 images. Second, the time-series characteristics were analyzed, sensitive parameters were selected, and a GRU classification model was constructed. Third, the model was used to identify the heavy metals in rice and then assess the accuracy of the classification results using performance metrics such as the accuracy rate, precision, recall rate (recall), and F1-score (F1-score). The results showed that the GRU model based on the time series of the red-edge location feature index has a good classification performance with an overall accuracy of 93.5% and a Kappa coefficient of 85.6%. This study shows that regional heavy metal stress in crops can be accurately detected using the GRU algorithm. A combination of spectrum and temporal information appears to be a promising method for monitoring crops under various types of stress.

Multi-Dimensional Dataset of Open Data and Satellite Images for Characterization of Food Security and Nutrition

Background

Nutrition is one of the main factors affecting the development and quality of life of a person. From a public health perspective, food security is an essential social determinant for promoting healthy nutrition. Food security embraces four dimensions: physical availability of food, economic and physical access to food, food utilization, and the sustainability of the dimensions above. Integrally addressing the four dimensions is vital. Surprisingly most of the works focused on a single dimension of food security: the physical availability of food.

Objective

The paper proposes a multi-dimensional dataset of open data and satellite images to characterize food security in the department of Cauca, Colombia.

Methods

The food security dataset integrates multiple open data sources; therefore, the Cross-Industry Standard Process for Data Mining methodology was used to guide the construction of the dataset. It includes sources such as population and agricultural census, nutrition surveys, and satellite images.

Results

An open multidimensional dataset for the Department of Cauca with 926 attributes and 9 rows (each row representing a Municipality) from multiple sources in Colombia, is configured. Then, machine learning models were used to characterize food security and nutrition in the Cauca Department. As a result, The Food security index calculated for Cauca using a linear regression model (Mean Absolute Error of 0.391) is 57.444 in a range between 0 and 100, with 100 the best score. Also, an approach for extracting four features (Agriculture, Habitation, Road, Water) of satellite images were tested with the ResNet50 model trained from scratch, having the best performance with a macro-accuracy, macro-precision, macro-recall, and macro-F1-score of 91.7, 86.2, 66.91, and 74.92%, respectively.

Conclusion

It shows how the CRISP-DM methodology can be used to create an open public health data repository. Furthermore, this methodology could be generalized to other types of problems requiring the creation of a dataset. In addition, the use of satellite images presents an alternative for places where data collection is challenging. The model and methodology proposed based on open data become a low-cost and effective solution that could be used by decision-makers, especially in developing countries, to support food security planning.

Comparison of Long Short-Term Memory Networks and Random Forest for Sentinel-1 Time Series Based Large Scale Crop Classification

To ensure future food security, improved agricultural management approaches are required. For many of those applications, precise knowledge of the distribution of crop types is essential. Various machine and deep learning models have been used for automated crop classification using microwave remote sensing time series. However, the application of these approaches on a large spatial and temporal scale is barely investigated. In this study, the performance of two frequently used algorithms, Long Short-Term Memory (LSTM) networks and Random Forest (RF), for crop classification based on Sentinel-1 time series and meteorological data on a large spatial and temporal scale is assessed. For data from Austria, the Netherlands, and France and the years 2015–2019, scenarios with different spatial and temporal scales were defined. To quantify the complexity of these scenarios, the Fisher Discriminant measurement F1 (FDR1) was used. The results demonstrate that both classifiers achieve similar results for simple classification tasks with low FDR1 values. With increasing FDR1 values, however, LSTM networks outperform RF. This suggests that the ability of LSTM networks to learn long-term dependencies and identify the relation between radar time series and meteorological data becomes increasingly important for more complex applications. Thus, the study underlines the importance of deep learning models, including LSTM networks, for large-scale applications.

High Resolution Distribution Dataset of Double-Season Paddy Rice in China

Although China is the largest producer of rice, accounting for about 25% of global production, there are no high-resolution maps of paddy rice covering the entire country. Using time-weighted dynamic time warping (TWDTW), this study developed a pixel- and phenology-based method to identify planting areas of double-season paddy rice in China, by comparing temporal variations of synthetic aperture radar (SAR) signals of unknown pixels to those of known double-season paddy rice fields. We conducted a comprehensive evaluation of the method’s performance at pixel and regional scales. Based on 145,210 field surveyed samples from 2018 to 2020, the producer’s and user’s accuracy are 88.49% and 87.02%, respectively. Compared to county-level statistical data from 2016 to 2019, the relative mean absolute errors are 34.11%. This study produced distribution maps of double-season rice at 10 m spatial resolution from 2016 to 2020 over nine provinces in South China, which account for more than 99% of the planting areas of double-season paddy rice of China. The maps are expected to contribute to timely monitoring and evaluating rice growth and yield.

Temperature Prediction Using Multivariate Time Series Deep Learning in the Lining of an Electric Arc Furnace for Ferronickel Production

The analysis of data from sensors in structures subjected to extreme conditions such as the ones used in smelting processes is a great decision tool that allows knowing the behavior of the structure under different operational conditions. In this industry, the furnaces and the different elements are fully instrumented, including sensors to measure variables such as temperature, pressure, level, flow, power, electrode positions, among others. From the point of view of engineering and data analytics, this quantity of data presents an opportunity to understand the operation of the system under normal conditions or to explore new ways of operation by using information from models provided by using deep learning approaches. Although some approaches have been developed with application to this industry, it is still an open research area. As a contribution, this paper presents an applied deep learning temperature prediction model for a 75 MW electric arc furnace, which is used for ferronickel production. In general, the methodology proposed considers two steps: first, a data cleaning process to increase the quality of the data, eliminating both redundant information as well as atypical and unusual data, and second, a multivariate time series deep learning model to predict the temperatures in the furnace lining. The developed deep learning model is a sequential one based on GRU (gated recurrent unit) layer plus a dense layer. The GRU + Dense model achieved an average root mean square error (RMSE) of 1.19 °C in the test set of 16 different thermocouples radially distributed on the furnace.

Mapping Maize Area in Heterogeneous Agricultural Landscape with Multi-Temporal Sentinel-1 and Sentinel-2 Images Based on Random Forest

Accurate estimation of crop area is essential to adjusting the regional crop planting structure and the rational planning of water resources. However, it is quite challenging to map crops accurately by high-resolution remote sensing images because of the ecological gradient and ecological convergence between crops and non-crops. The purpose of this study is to explore the combining application of high-resolution multi-temporal Sentinel-1 (S1) radar backscatter and Sentinel-2 (S2) optical reflectance images for maize mapping in highly complex and heterogeneous landscapes in the middle reaches of Heihe River, northwest China. We proposed a new two-step method of vegetation extraction and followed by maize extraction, that is, extract the vegetation-covered areas first to reduce the inter-class variance by using a Random Forest (RF) classifier based on S2 data, and then extract the maize distribution in the vegetation area by using another RF classifier based on S1 and/or S2 data. The results demonstrate that the vegetation extraction classifier successfully identified vegetation-covered regions with an overall accuracy above 96% in the study area, and the accuracy of the maize extraction classifier constructed by the combined multi-temporal S1 and S2 images is significantly improved compared with that S1 (alone) or S2 (alone), with an overall accuracy of 87.63%, F1_Score of 0.86, and Kappa coefficient of 0.75. In addition, with the introduction of multi-temporal S1 and/or S2 images in crop growing season, the constructed RF model is more beneficial to maize mapping.

Evaluation of Five Deep Learning Models for Crop Type Mapping Using Sentinel-2 Time Series Images with Missing Information

Accurate crop type maps play an important role in food security due to their widespread applicability. Optical time series data (TSD) have proven to be significant for crop type mapping. However, filling in missing information due to clouds in optical imagery is always needed, which will increase the workload and the risk of error transmission, especially for imagery with high spatial resolution. The development of optical imagery with high temporal and spatial resolution and the emergence of deep learning algorithms provide solutions to this problem. Although the one-dimensional convolutional neural network (1D CNN), long short-term memory (LSTM), and gate recurrent unit (GRU) models have been used to classify crop types in previous studies, their ability to identify crop types using optical TSD with missing information needs to be further explored due to their different mechanisms for handling invalid values in TSD. In this research, we designed two groups of experiments to explore the performances and characteristics of the 1D CNN, LSTM, GRU, LSTM-CNN, and GRU-CNN models for crop type mapping using unfilled Sentinel-2 (Sentinel-2) TSD and to discover the differences between unfilled and filled Sentinel-2 TSD based on the same algorithm. A case study was conducted in Hengshui City, China, of which 70.3% is farmland. The results showed that the 1D CNN, LSTM-CNN, and GRU-CNN models achieved acceptable classification accuracies (above 85%) using unfilled TSD, even though the total missing rate of the sample values was 43.5%; these accuracies were higher and more stable than those obtained using filled TSD. Furthermore, the models recalled more samples on crop types with small parcels when using unfilled TSD. Although LSTM and GRU models did not attain accuracies as high as the other three models using unfilled TSD, their results were almost close to those with filled TSD. This research showed that crop types could be identified by deep learning features in Sentinel-2 dense time series images with missing information due to clouds or cloud shadows randomly, which avoided spending a lot of time on missing information reconstruction.

Machine Learning in Agriculture: A Comprehensive Updated Review

The digital transformation of agriculture has evolved various aspects of management into artificial intelligent systems for the sake of making value from the ever-increasing data originated from numerous sources. A subset of artificial intelligence, namely machine learning, has a considerable potential to handle numerous challenges in the establishment of knowledge-based farming systems. The present study aims at shedding light on machine learning in agriculture by thoroughly reviewing the recent scholarly literature based on keywords' combinations of "machine learning" along with "crop management", "water management", "soil management", and "livestock management", and in accordance with PRISMA guidelines. Only journal papers were considered eligible that were published within 2018-2020. The results indicated that this topic pertains to different disciplines that favour convergence research at the international level. Furthermore, crop management was observed to be at the centre of attention. A plethora of machine learning algorithms were used, with those belonging to Artificial Neural Networks being more efficient. In addition, maize and wheat as well as cattle and sheep were the most investigated crops and animals, respectively. Finally, a variety of sensors, attached on satellites and unmanned ground and aerial vehicles, have been utilized as a means of getting reliable input data for the data analyses. It is anticipated that this study will constitute a beneficial guide to all stakeholders towards enhancing awareness of the potential advantages of using machine learning in agriculture and contributing to a more systematic research on this topic.

Mapping Smallholder Maize Farms Using Multi-Temporal Sentinel-1 Data in Support of the Sustainable Development Goals

Reducing food insecurity in developing countries is one of the crucial targets of the Sustainable Development Goals (SDGs). Smallholder farmers play a crucial role in combating food insecurity. However, local planning agencies and governments do not have adequate spatial information on smallholder farmers, and this affects the monitoring of the SDGs. This study utilized Sentinel-1 multi-temporal data to develop a framework for mapping smallholder maize farms and to estimate maize production area as a parameter for supporting the SDGs. We used Principal Component Analysis (PCA) to pixel fuse the multi-temporal data to only three components for each polarization (vertical transmit and vertical receive (VV), vertical transmit and horizontal receive (VH), and VV/VH), which explained more than 70% of the information. The Support Vector Machine (SVM) and Extreme Gradient Boosting (Xgboost) algorithms were used at model-level feature fusion to classify the data. The results show that the adopted strategy of two-stage image fusion was sufficient to map the distribution and estimate production areas for smallholder farms. An overall accuracy of more than 90% for both SVM and Xgboost algorithms was achieved. There was a 3% difference in production area estimation observed between the two algorithms. This framework can be used to generate spatial agricultural information in areas where agricultural survey data are limited and for areas that are affected by cloud coverage. We recommend the use of Sentinel-1 multi-temporal data in conjunction with machine learning algorithms to map smallholder maize farms to support the SDGs.

Delineating Smallholder Maize Farms from Sentinel-1 Coupled with Sentinel-2 Data Using Machine Learning

Rural communities rely on smallholder maize farms for subsistence agriculture, the main driver of local economic activity and food security. However, their planted area estimates are unknown in most developing countries. This study explores the use of Sentinel-1 and Sentinel-2 data to map smallholder maize farms. The random forest (RF), support vector (SVM) machine learning algorithms and model stacking (ST) were applied. Results show that the classification of combined Sentinel-1 and Sentinel-2 data improved the RF, SVM and ST algorithms by 24.2%, 8.7%, and 9.1%, respectively, compared to the classification of Sentinel-1 data individually. Similarities in the estimated areas (7001.35 ± 1.2 ha for RF, 7926.03 ± 0.7 ha for SVM and 7099.59 ± 0.8 ha for ST) show that machine learning can estimate smallholder maize areas with high accuracies. The study concludes that the single-date Sentinel-1 data were insufficient to map smallholder maize farms. However, single-date Sentinel-1 combined with Sentinel-2 data were sufficient in mapping smallholder farms. These results can be used to support the generation and validation of national crop statistics, thus contributing to food security.

Potential of Hybrid CNN-RF Model for Early Crop Mapping with Limited Input Data

When sufficient time-series images and training data are unavailable for crop classification, features extracted from convolutional neural network (CNN)-based representative learning may not provide useful information to discriminate crops with similar spectral characteristics, leading to poor classification accuracy. In particular, limited input data are the main obstacles to obtain reliable classification results for early crop mapping. This study investigates the potential of a hybrid classification approach, i.e., CNN-random forest (CNN-RF), in the context of early crop mapping, that combines the automatic feature extraction capability of CNN with the superior discrimination capability of an RF classifier. Two experiments on incremental crop classification with unmanned aerial vehicle images were conducted to compare the performance of CNN-RF with that of CNN and RF with respect to the length of the time-series and training data sizes. When sufficient time-series images and training data were used for the classification, the accuracy of CNN-RF was slightly higher or comparable with that of CNN. In contrast, when fewer images and the smallest training data were used at the early crop growth stage, CNN-RF was substantially beneficial and the overall accuracy increased by maximum 6.7%p and 4.6%p in the two study areas, respectively, compared to CNN. This is attributed to its ability to discriminate crops from features with insufficient information using a more sophisticated classifier. The experimental results demonstrate that CNN-RF is an effective classifier for early crop mapping when only limited input images and training samples are available.

Multitemporal Water Extraction of Dongting Lake and Poyang Lake Based on an Automatic Water Extraction and Dynamic Monitoring Framework

Timely and accurate large-scale water body mapping and dynamic monitoring are of great significance for water resource planning, flood control, and disaster reduction applications. Synthetic aperture radar (SAR) systems have the characteristics of strong operability, wide coverage, and all-weather data availability, and play a key role in large-scale water monitoring applications. However, there are still some challenges in the application of highly efficient, high-precision water extraction and dynamic monitoring methods. In this paper, a framework for the automatic extraction and long-term change monitoring of water bodies is proposed. First, a multitemporal water sample dataset is produced based on the bimodal threshold segmentation method. Second, attention block and pyramid module are introduced into the UNet (encoder-decoder) model to construct a robust water extraction network (PA-UNet). Then, GIS modeling is used for the automatic postprocessing of the water extraction results. Finally, the results are mapped and statistically analyzed. The whole process realizes end-to-end input and output. Sentinel-1 data covering Dongting Lake and Poyang Lake are selected for water extraction and dynamic monitoring analysis from 2017 to 2020, and Sentinel-2 images from a similar time frame are selected for verification. The results show that the proposed framework can realize high-precision (the extraction accuracy is higher than 95%), highly efficient automatic water extraction. Multitemporal monitoring results show that Dongting Lake and Poyang Lake fluctuate most in April, July, and November in 2017, 2019, and 2020, and the change trends of the two lakes are the same.

Mapping Paddy Rice with Satellite Remote Sensing: A Review

Paddy rice is a staple food of three billion people in the world. Timely and accurate estimation of the paddy rice planting area and paddy rice yield can provide valuable information for the government, planners and decision makers to formulate policies. This article reviews the existing paddy rice mapping methods presented in the literature since 2010, classifies these methods, and analyzes and summarizes the basic principles, advantages and disadvantages of these methods. According to the data sources used, the methods are divided into three categories: (I) Optical mapping methods based on remote sensing; (II) Mapping methods based on microwave remote sensing; and (III) Mapping methods based on the integration of optical and microwave remote sensing. We found that the optical remote sensing data sources are mainly MODIS, Landsat, and Sentinel-2, and the emergence of Sentinel-1 data has promoted research on radar mapping methods for paddy rice. Multisource data integration further enhances the accuracy of paddy rice mapping. The best methods are phenology algorithms, paddy rice mapping combined with machine learning, and multisource data integration. Innovative methods include the time series similarity method, threshold method combined with mathematical models, and object-oriented image classification. With the development of computer technology and the establishment of cloud computing platforms, opportunities are provided for obtaining large-scale high-resolution rice maps. Multisource data integration, paddy rice mapping under different planting systems and the connection with global changes are the focus of future development priorities.

Monitoring of Sugarcane Harvest in Brazil Based on Optical and SAR Data

The algorithms for determining sugarcane harvest dates are proposed; the algorithms allow the ability to monitor large areas and are based on the publicly available Synthetic Aperture Radar (SAR) and optical satellite data. Algorithm 1 uses the NDVI (Normalized Difference Vegetation Index) time series derived from Sentinel-2 data. Sharp and continuous decrease in the NDVI values is the main sign of sugarcane harvest. The NDVI time series allows the ability to determine most harvest dates. The best estimates of the sugarcane areas harvested per month have been obtained from March to August 2018 when cloudy pixel percentage is less than 45% of the image area. Algorithm 2 of the harvest monitoring uses the coherence time series derived from Sentinel-1 Single Look Complex (SLC) images and optical satellite data. Low coherence, demonstrating sharp growth upon the harvest completion, corresponds to the harvest period. The NDVI time series trends were used to refine the algorithm. It is supposed that the descending NDVI trend corresponds to harvest. The algorithms were used to identify the harvest dates and calculate the harvested areas of the reference sample of 574 sugarcane parcels with a total area of 3745 ha in the state of São Paulo, Brazil. The harvested areas identified by visual interpretation coincide with the optical-data algorithm (algorithm 1) by 97%; the coincidence with the algorithm based on SAR and optical data (algorithm 2) is 90%. The main practical applications of the algorithms are harvest monitoring and identification of the harvested fields to estimate the harvested area.

Mixture Optimization of Recycled Aggregate Concrete Using Hybrid Machine Learning Model

Recycled aggregate concrete (RAC) contributes to mitigating the depletion of natural aggregates, alleviating the carbon footprint of concrete construction, and averting the landfilling of colossal amounts of construction and demolition waste. However, complexities in the mixture optimization of RAC due to the variability of recycled aggregates and lack of accuracy in estimating its compressive strength require novel and sophisticated techniques. This paper aims at developing state-of-the-art machine learning models to predict the RAC compressive strength and optimize its mixture design. Results show that the developed models including Gaussian processes, deep learning, and gradient boosting regression achieved robust predictive performance, with the gradient boosting regression trees yielding highest prediction accuracy. Furthermore, a particle swarm optimization coupled with gradient boosting regression trees model was developed to optimize the mixture design of RAC for various compressive strength classes. The hybrid model achieved cost-saving RAC mixture designs with lower environmental footprint for different target compressive strength classes. The model could be further harvested to achieve sustainable concrete with optimal recycled aggregate content, least cost, and least environmental footprint.

Object Detection and Image Segmentation with Deep Learning on Earth Observation Data: A Review—Part II: Applications

In Earth observation (EO), large-scale land-surface dynamics are traditionally analyzed by investigating aggregated classes. The increase in data with a very high spatial resolution enables investigations on a fine-grained feature level which can help us to better understand the dynamics of land surfaces by taking object dynamics into account. To extract fine-grained features and objects, the most popular deep-learning model for image analysis is commonly used: the convolutional neural network (CNN). In this review, we provide a comprehensive overview of the impact of deep learning on EO applications by reviewing 429 studies on image segmentation and object detection with CNNs. We extensively examine the spatial distribution of study sites, employed sensors, used datasets and CNN architectures, and give a thorough overview of applications in EO which used CNNs. Our main finding is that CNNs are in an advanced transition phase from computer vision to EO. Upon this, we argue that in the near future, investigations which analyze object dynamics with CNNs will have a significant impact on EO research. With a focus on EO applications in this Part II, we complete the methodological review provided in Part I.

The Delineation and Grading of Actual Crop Production Units in Modern Smallholder Areas Using RS Data and Mask R-CNN

The extraction and evaluation of crop production units are important foundations for agricultural production and management in modern smallholder regions, which are very significant to the regulation and sustainable development of agriculture. Crop areas have been recognized efficiently and accurately via remote sensing (RS) and machine learning (ML), especially deep learning (DL), which are too rough for modern smallholder production. In this paper, a delimitation-grading method for actual crop production units (ACPUs) based on RS images was explored using a combination of a mask region-based convolutional neural network (Mask R-CNN), spatial analysis, comprehensive index evaluation, and cluster analysis. Da’an City, Jilin province, China, was chosen as the study region to satisfy the agro-production demands in modern smallholder areas. Firstly, the ACPUs were interpreted from perspectives such as production mode, spatial form, and actual productivity. Secondly, cultivated land plots (C-plots) were extracted by Mask R-CNN with high-resolution RS images, which were used to delineate contiguous cultivated land plots (CC-plots) on the basis of auxiliary data correction. Then, the refined delimitation-grading results of the ACPUs were obtained through comprehensive evaluation of spatial characteristics and real productivity clustering. For the conclusion, the effectiveness of the Mask R-CNN model in C-plot recognition (loss = 0.16, mean average precision (mAP) = 82.29%) and a reasonable distance threshold (20 m) for CC-plot delimiting were verified. The spatial features were evaluated with the scale-shape dimensions of nine specific indicators. Real productivities were clustered by the incorporation of two-step cluster and K-Means cluster. Furthermore, most of the ACPUs in the study area were of a reasonable scale and an appropriate shape, holding real productivities at a medium level or above. The proposed method in this paper can be adjusted according to the changes of the study area with flexibility to assist agro-supervision in many modern smallholder regions.

Synergistic Use of Multi-Temporal RADARSAT-2 and VENµS Data for Crop Classification Based on 1D Convolutional Neural Network

Annual crop inventory information is important for many agriculture applications and government statistics. The synergistic use of multi-temporal polarimetric synthetic aperture radar (SAR) and available multispectral remote sensing data can reduce the temporal gaps and provide the spectral and polarimetric information of the crops, which is effective for crop classification in areas with frequent cloud interference. The main objectives of this study are to develop a deep learning model to map agricultural areas using multi-temporal full polarimetric SAR and multi-spectral remote sensing data, and to evaluate the influence of different input features on the performance of deep learning methods in crop classification. In this study, a one-dimensional convolutional neural network (Conv1D) was proposed and tested on multi-temporal RADARSAT-2 and VENµS data for crop classification. Compared with the Multi-Layer Perceptron (MLP), Recurrent Neural Network (RNN) and non-deep learning methods including XGBoost, Random Forest (RF), and Support Vector Machina (SVM), the Conv1D performed the best when the multi-temporal RADARSAT-2 data (Pauli decomposition or coherency matrix) and VENµS multispectral data were fused by the Minimum Noise Fraction (MNF) transformation. The Pauli decomposition and coherency matrix gave similar overall accuracy (OA) for Conv1D when fused with the VENµS data by the MNF transformation (OA = 96.65 ± 1.03% and 96.72 ± 0.77%). The MNF transformation improved the OA and F-score for most classes when Conv1D was used. The results reveal that the coherency matrix has a great potential in crop classification and the MNF transformation of multi-temporal RADARSAT-2 and VENµS data can enhance the performance of Conv1D.