Deep Recurrent Neural Network for Agricultural Classification using multitemporal SAR Sentinel-1 for Camargue, France

Machine learning model for aberrant driving behaviour prediction using heart rate variability: a pilot study involving highway bus drivers

Objectives. Current approaches via physiological features detecting aberrant driving behaviour (ADB), including speeding, abrupt steering, hard braking and aggressive acceleration, are developing. This study proposes using machine learning approaches incorporating heart rate variability (HRV) parameters to predict ADB occurrence. Methods. Naturalistic driving data of 10 highway bus drivers in Taiwan from their daily routes were collected for 4 consecutive days. Their driving behaviours and physiological data during a driving task were determined using a navigation mobile application and heart rate watch. Participants' self-reported data on sleep, driving-related experience, open-source data on weather and the traffic congestion level were obtained. Five machine learning models - logistic regression, random forest, naive Bayes, support vector machine and gated recurrent unit (GRU) - were employed to predict ADBs. Results. Most drivers with ADB had low sleep efficiency (≤80%), with significantly higher scores in driver behaviour questionnaire subcategories of lapses and errors and in the Karolinska sleepiness scale than those without ADBs. Moreover, HRV parameters were significantly different between baseline and pre-ADB event measurements. GRU had the highest accuracy (81.16-84.22%). Conclusions. Sleep deficit may be related to the increased fatigue level and ADB occurrence predicted from HRV-based models among bus drivers.

Soil carbon content prediction using multi-source data feature fusion of deep learning based on spectral and hyperspectral images

Visible near-infrared reflectance spectroscopy (VNIR) and hyperspectral images (HSI) have their respective advantages in soil carbon content prediction, and the effective fusion of VNIR and HSI is of great significance for improving the prediction accuracy. But the contribution difference analysis of multiple features in the multi-source data is inadequate, and there is a lack of in-depth research on the contribution difference analysis of artificial feature and deep learning feature. In order to solve the problem, soil carbon content prediction methods based on VNIR and HSI multi-source data feature fusion are proposed. The multi-source data fusion network under the attention mechanism and the multi-source data fusion network with artificial feature are designed. For the multi-source data fusion network based on the attention mechanism, the information are fused through the attention mechanism according to the contribution difference of each feature. For the other network, artificial feature are introduced to fuse multi-source data. The results show that multi-source data fusion network based on the attention mechanism can improve the prediction accuracy of soil carbon content, and multi-source data fusion network combined with artificial feature has better prediction effect. Compared with two single-source data from the VNIR and HSI, the relative percent deviation of Neilu, Aoshan Bay and Jiaozhou Bay based on multi-source data fusion network combined with artificial feature are increased by 56.81% and 149.18%, 24.28% and 43.96%, 31.16% and 28.73% respectively. This study can effectively solve the problem of the deep fusion of multiple features in the soil carbon content prediction by VNIR and HSI, so as to improve the accuracy and stability of soil carbon content prediction, promote the application and development of soil carbon content prediction in spectral and hyperspectral image, and provide technical support for the study of carbon cycle and the carbon sink.

Combining Deep Learning and Multi-Source GIS Methods to Analyze Urban and Greening Changes

Although many authors have observed a degradation in greening cover alongside an increase in the built-up areas, resulting in a deterioration of the essential environmental services for the well-being of ecosystems and society, few studies have measured how greening developed in its full spatiotemporal configuration with urban development using innovative remote sensing (RS) technologies. Focusing on this issue, the authors propose an innovative methodology for the analysis of the urban and greening changes over time by integrating deep learning (DL) technologies to classify and segment the built-up area and the vegetation cover from satellite and aerial images and geographic information system (GIS) techniques. The core of the methodology is a trained and validated U-Net model, which was tested on an urban area in the municipality of Matera (Italy), analyzing the urban and greening changes from 2000 to 2020. The results demonstrate a very good level of accuracy of the U-Net model, a remarkable increment in the built-up area density (8.28%) and a decline in the vegetation cover density (5.13%). The obtained results demonstrate how the proposed method can be used to rapidly and accurately identify useful information about urban and greening spatiotemporal development using innovative RS technologies supporting sustainable development processes.

Identification of Typical Ecosystem Types by Integrating Active and Passive Time Series Data of the Guangdong-Hong Kong-Macao Greater Bay Area, China

The identification of ecosystem types is important in ecological environmental assessment. However, due to cloud and rain and complex land cover characteristics, commonly used ecosystem identification methods have always lacked accuracy in subtropical urban agglomerations. In this study, China's Guangdong-Hong Kong-Macao Greater Bay Area (GBA) was taken as a study area, and the Sentinel-1 and Sentinel-2 data were used as the fusion of active and passive remote sensing data with time series data to distinguish typical ecosystem types in subtropical urban agglomerations. Our results showed the following: (1) The importance of different features varies widely in different types of ecosystems. For grassland and arable land, two specific texture features (VV_dvar and VH_diss) are most important; in forest and mangrove areas, synthetic-aperture radar (SAR) data for the months of October and September are most important. (2) The use of active time series remote sensing data can significantly improve the classification accuracy by 3.33%, while passive time series remote sensing data improves by 4.76%. When they are integrated, accuracy is further improved, reaching a level of 84.29%. (3) Time series passive data (NDVI) serve best to distinguish grassland from arable land, while time series active data (SAR data) are best able to distinguish mangrove from forest. The integration of active and passive time series data also improves precision in distinguishing vegetation ecosystem types, such as forest, mangrove, arable land, and, especially, grassland, where the accuracy increased by 21.88%. By obtaining real-time and more accurate land cover type change information, this study could better serve regional change detection and ecosystem service function assessment at different scales, thereby supporting decision makers in urban agglomerations.

Mapping Paddy Rice in Complex Landscapes with Landsat Time Series Data and Superpixel-Based Deep Learning Method

The spatial pattern and temporal variation in paddy rice areas captured by remote sensing imagery provide an effective way of performing crop management and developing suitable agricultural policies. However, fragmented and scattered rice paddies due to undulating and varied topography, and the availability and quality of remote sensing images (e.g., frequent cloud coverage) pose significant challenges to accurate long-term rice mapping, especially for traditional pixel and phenological methods in subtropical monsoon regions. This study proposed a superpixel and deep-learning-based time series method to analyze Landsat time series data for paddy rice classification in complex landscape regions. First, a superpixel segmentation map was generated using a dynamic-time-warping-based simple non-iterative clustering algorithm with preprocessed spectral indices (SIs) time series data. Second, the SI images were overlaid onto the superpixel map to construct mean SIs time series for each superpixel. Third, a multivariate long short-term memory full convolution neural network (MLSTM-FCN) classifier was employed to learn time series features of rice paddies to produce accurate paddy rice maps. The method was evaluated using Landsat imagery from 2000 to 2020 in Cengong County, Guizhou Province, China. Results indicate that the superpixel MLSTM-FCN achieved a high performance with an overall accuracy varying from 0.9547 to 0.9721, which presents an 0.17–1.23% improvement compared to the random forest method. This study showed that combining spectral, spatial, and temporal features with deep learning methods can generate accurate paddy rice maps in complex landscape regions.

Deep Temporal Iterative Clustering for Satellite Image Time Series Land Cover Analysis

The extensive amount of Satellite Image Time Series (SITS) data brings new opportunities and challenges for land cover analysis. Many supervised machine learning methods have been applied in SITS, but the labeled SITS samples are time- and effort-consuming to acquire. It is necessary to analyze SITS data with an unsupervised learning method. In this paper, we propose a new unsupervised learning method named Deep Temporal Iterative Clustering (DTIC) to deal with SITS data. The proposed method jointly learns a neural network’s parameters and the resulting features’ cluster assignments, which uses a standard clustering algorithm, K-means, to iteratively cluster the features produced by the feature extraction network and then uses the subsequent assignments as supervision to update the network’s weights. We apply DTIC to the unsupervised training of neural networks on both SITS datasets. Experimental results demonstrate that DTIC outperforms the state-of-the-art K-means clustering algorithm, which proves that the proposed approach successfully provides a novel idea for unsupervised training of SITS data.

A Bibliometric and Visualized Analysis of Research Progress and Trends in Rice Remote Sensing over the Past 42 Years (1980–2021)

Rice is one of the most important food crops around the world. Remote sensing technology, as an effective and rapidly developing method, has been widely applied to precise rice management. To observe the current research status in the field of rice remote sensing (RRS), a bibliometric analysis was carried out based on 2680 papers of RRS published during 1980–2021, which were collected from the core collection of the Web of Science database. Quantitative analysis of the number of publications, top countries and institutions, popular keywords, etc. was conducted through the knowledge mapping software CiteSpace, and comprehensive discussions were carried out from the aspects of specific research objects, methods, spectral variables, and sensor platforms. The results revealed that an increasing number of countries and institutions have conducted research on RRS and a great number of articles have been published annually, among which, China, the United States of America, and Japan were the top three and the Chinese Academy of Sciences, Zhejiang University, and Nanjing Agricultural University were the first three research institutions with the largest publications. Abundant interest was paid to “reflectance”, followed by “vegetation index” and “yield” and the specific objects mainly focused on growth, yield, area, stress, and quality. From the perspective of spectral variables, reflectance, vegetation index, and back-scattering coefficient appeared the most frequently in the frontiers. In addition to satellite remote sensing data and empirical models, unmanned air vehicle (UAV) platforms and artificial intelligence models have gradually become hot topics. This study enriches the readers’ understanding and highlights the potential future research directions in RRS.

Rice Mapping in a Subtropical Hilly Region Based on Sentinel-1 Time Series Feature Analysis and the Dual Branch BiLSTM Model

Timely and accurate information on rice cultivation makes important contributions to the profound reform of the global food and agricultural system, and promotes the development of global sustainable agriculture. With all-day and all-weather observing ability, synthetic aperture radar (SAR) can monitor the distribution of rice in tropical and subtropical areas. To solve the problem of misclassification of rice with no marked signal during the flooding period in subtropical hilly areas, this paper proposes a new feature combination and dual branch bi-directional long short-term memory (DB-BiLSTM) model to achieve high-precision rice mapping using Sentinel-1 time series data. Based on field investigation data, the backscatter time series curves of the rice area were analyzed, and a characteristic index (VV − VH)/(VV + VH) (VV: vertical emission and vertical receipt of polarization, VH: vertical emission and horizontal receipt of polarization) for small areas of hilly land was proposed to effectively distinguish rice and non-rice crops with no marked flooding period. The DB-BiLSTM model was designed, ensuring the independent learning of multiple features and effectively combining the time series information of both (VV − VH)/(VV + VH) and VH features. The city of Shanwei, Guangdong Province, China, was selected as the study area. Experimental results showed that the overall accuracy of the rice mapping results was 97.29%, and the kappa coefficient reached 0.9424. Compared to other methods, the rice mapping results obtained by the proposed method maintained good integrity and had less misclassification, which demonstrated the proposed method’s practical value in accurate and effective rice mapping tasks.

The Classification Method Study of Crops Remote Sensing with Deep Learning, Machine Learning, and Google Earth Engine

The extraction and classification of crops is the core issue of agricultural remote sensing. The precise classification of crop types is of great significance to the monitoring and evaluation of crops planting area, growth, and yield. Based on the Google Earth Engine and Google Colab cloud platform, this study takes the typical agricultural oasis area of Xiangride Town, Qinghai Province, as an example. It compares traditional machine learning (random forest, RF), object-oriented classification (object-oriented, OO), and deep neural networks (DNN), which proposes a random forest combined with deep neural network (RF+DNN) classification framework. In this study, the spatial characteristics of band information, vegetation index, and polarization of main crops in the study area were constructed using Sentinel-1 and Sentinel-2 data. The temporal characteristics of crops phenology and growth state were analyzed using the curve curvature method, and the data were screened in time and space. By comparing and analyzing the accuracy of the four classification methods, the advantages of RF+DNN model and its application value in crops classification were illustrated. The results showed that for the crops in the study area during the period of good growth and development, a better crop classification result could be obtained using RF+DNN classification method, whose model accuracy, training, and predict time spent were better than that of using DNN alone. The overall accuracy and Kappa coefficient of classification were 0.98 and 0.97, respectively. It is also higher than the classification accuracy of random forest (OA = 0.87, Kappa = 0.82), object oriented (OA = 0.78, Kappa = 0.70) and deep neural network (OA = 0.93, Kappa = 0.90). The scalable and simple classification method proposed in this paper gives full play to the advantages of cloud platform in data and operation, and the traditional machine learning combined with deep learning can effectively improve the classification accuracy. Timely and accurate extraction of crop types at different spatial and temporal scales is of great significance for crops pattern change, crops yield estimation, and crops safety warning.

Parcel-Level Mapping of Horticultural Crop Orchards in Complex Mountain Areas Using VHR and Time-Series Images

Accurate and reliable farmland crop mapping is an important foundation for relevant departments to carry out agricultural management, crop planting structure adjustment and ecological assessment. The current crop identification work mainly focuses on conventional crops, and there are few studies on parcel-level mapping of horticultural crops in complex mountainous areas. Using Miaohou Town, China, as the research area, we developed a parcel-level method for the precise mapping of horticultural crops in complex mountainous areas using very-high-resolution (VHR) optical images and Sentinel-2 optical time-series images. First, based on the VHR images with a spatial resolution of 0.55 m, the complex mountainous areas were divided into subregions with their own independent characteristics according to a zoning and hierarchical strategy. The parcels in the different study areas were then divided into plain, greenhouse, slope and terrace parcels according to their corresponding parcel characteristics. The edge-based model RCF and texture-based model DABNet were subsequently used to extract the parcels according to the characteristics of different regions. Then, Sentinel-2 images were used to construct the time-series characteristics of different crops, and an LSTM algorithm was used to classify crop types. We then designed a parcel filling strategy to determine the categories of parcels based on the classification results of the time-series data, and accurate parcel-level mapping of a horticultural crop orchard in a complex mountainous area was finally achieved. Based on visual inspection, this method appears to effectively extract farmland parcels from VHR images of complex mountainous areas. The classification accuracy reached 93.01%, and the Kappa coefficient was 0.9015. This method thus serves as a methodological reference for parcel-level horticultural crop mapping and can be applied to the development of local precision agriculture.

Identification of Crop Type Based on C-AENN Using Time Series Sentinel-1A SAR Data

Crop type identification is the initial stage and an important part of the agricultural monitoring system. It is well known that synthetic aperture radar (SAR) Sentinel-1A imagery provides a reliable data source for crop type identification. However, a single-temporal SAR image does not contain enough features, and the unique physical characteristics of radar images are relatively lacking, which limits its potential in crop mapping. In addition, current methods may not be applicable for time -series SAR data. To address the above issues, a new crop type identification method was proposed. Specifically, a farmland mask was firstly generated by the object Markov random field (OMRF) model to remove the interference of non-farmland factors. Then, the features of the standard backscatter coefficient, Sigma-naught (σ0), and the normalized backscatter coefficient by the incident angle, Gamma-naught (γ0), were extracted for each type of crop, and the optimal feature combination was found from time -series SAR images by means of Jeffries-Matusita (J-M) distance analysis. Finally, to make efficient utilization of optimal multi-temporal feature combination, a new network, the convolutional-autoencoder neural network (C-AENN), was developed for the crop type identification task. In order to prove the effectiveness of the method, several classical machine learning methods such as support vector machine (SVM), random forest (RF), etc., and deep learning methods such as one dimensional convolutional neural network (1D-CNN) and stacked auto-encoder (SAE), etc., were used for comparison. In terms of quantitative assessment, the proposed method achieved the highest accuracy, with a macro-F1 score of 0.9825, an overall accuracy (OA) score of 0.9794, and a Kappa coefficient (Kappa) score of 0.9705. In terms of qualitative assessment, four typical regions were chosen for intuitive comparison with the sample maps, and the identification result covering the study area was compared with a contemporaneous optical image, which indicated the high accuracy of the proposed method. In short, this study enables the effective identification of crop types, which demonstrates the importance of multi-temporal radar images in feature combination and the necessity of deep learning networks to extract complex features.

Positive Unlabelled Learning for Satellite Images’Time Series Analysis: An Application to Cereal and Forest Mapping

Applications in which researchers aim to extract a single land type from remotely sensed data are quite common in practical scenarios: extract the urban footprint to make connections with socio-economic factors; map the forest extent to subsequently retrieve biophysical variables and detect a particular crop type to successively calibrate and deploy yield prediction models. In this scenario, the (positive) targeted class is well defined, while the negative class is difficult to describe. This one-class classification setting is also referred to as positive unlabelled learning (PUL) in the general field of machine learning. To deal with this challenging setting, when satellite image time series data are available, we propose a new framework named positive and unlabelled learning of satellite image time series (PUL-SITS). PUL-SITS involves two different stages: In the first one, a recurrent neural network autoencoder is trained to reconstruct only positive samples with the aim to higight reliable negative ones. In the second stage, both labelled and unlabelled samples are exploited in a semi-supervised manner to build the final binary classification model. To assess the quality of our approach, experiments were carried out on a real-world benchmark, namely Haute-Garonne, located in the southwest area of France. From this study site, we considered two different scenarios: a first one in which the process has the objective to map Cereals/Oilseeds cover versus the rest of the land cover classes and a second one in which the class of interest is the Forest land cover. The evaluation was carried out by comparing the proposed approach with recent competitors to deal with the considered positive and unlabelled learning scenarios.

Comparison of Long Short-Term Memory Networks and Random Forest for Sentinel-1 Time Series Based Large Scale Crop Classification

To ensure future food security, improved agricultural management approaches are required. For many of those applications, precise knowledge of the distribution of crop types is essential. Various machine and deep learning models have been used for automated crop classification using microwave remote sensing time series. However, the application of these approaches on a large spatial and temporal scale is barely investigated. In this study, the performance of two frequently used algorithms, Long Short-Term Memory (LSTM) networks and Random Forest (RF), for crop classification based on Sentinel-1 time series and meteorological data on a large spatial and temporal scale is assessed. For data from Austria, the Netherlands, and France and the years 2015–2019, scenarios with different spatial and temporal scales were defined. To quantify the complexity of these scenarios, the Fisher Discriminant measurement F1 (FDR1) was used. The results demonstrate that both classifiers achieve similar results for simple classification tasks with low FDR1 values. With increasing FDR1 values, however, LSTM networks outperform RF. This suggests that the ability of LSTM networks to learn long-term dependencies and identify the relation between radar time series and meteorological data becomes increasingly important for more complex applications. Thus, the study underlines the importance of deep learning models, including LSTM networks, for large-scale applications.

Crop Type Mapping from Optical and Radar Time Series Using Attention-Based Deep Learning

Crop maps are key inputs for crop inventory production and yield estimation and can inform the implementation of effective farm management practices. Producing these maps at detailed scales requires exhaustive field surveys that can be laborious, time-consuming, and expensive to replicate. With a growing archive of remote sensing data, there are enormous opportunities to exploit dense satellite image time series (SITS), temporal sequences of images over the same area. Generally, crop type mapping relies on single-sensor inputs and is solved with the help of traditional learning algorithms such as random forests or support vector machines. Nowadays, deep learning techniques have brought significant improvements by leveraging information in both spatial and temporal dimensions, which are relevant in crop studies. The concurrent availability of Sentinel-1 (synthetic aperture radar) and Sentinel-2 (optical) data offers a great opportunity to utilize them jointly; however, optimizing their synergy has been understudied with deep learning techniques. In this work, we analyze and compare three fusion strategies (input, layer, and decision levels) to identify the best strategy that optimizes optical-radar classification performance. They are applied to a recent architecture, notably, the pixel-set encoder–temporal attention encoder (PSE-TAE) developed specifically for object-based classification of SITS and based on self-attention mechanisms. Experiments are carried out in Brittany, in the northwest of France, with Sentinel-1 and Sentinel-2 time series. Input and layer-level fusion competitively achieved the best overall F-score surpassing decision-level fusion by 2%. On a per-class basis, decision-level fusion increased the accuracy of dominant classes, whereas layer-level fusion improves up to 13% for minority classes. Against single-sensor baseline, multi-sensor fusion strategies identified crop types more accurately: for example, input-level outperformed Sentinel-2 and Sentinel-1 by 3% and 9% in F-score, respectively. We have also conducted experiments that showed the importance of fusion for early time series classification and under high cloud cover condition.

Boost Precision Agriculture with Unmanned Aerial Vehicle Remote Sensing and Edge Intelligence: A Survey

In recent years unmanned aerial vehicles (UAVs) have emerged as a popular and cost-effective technology to capture high spatial and temporal resolution remote sensing (RS) images for a wide range of precision agriculture applications, which can help reduce costs and environmental impacts by providing detailed agricultural information to optimize field practices. Furthermore, deep learning (DL) has been successfully applied in agricultural applications such as weed detection, crop pest and disease detection, etc. as an intelligent tool. However, most DL-based methods place high computation, memory and network demands on resources. Cloud computing can increase processing efficiency with high scalability and low cost, but results in high latency and great pressure on the network bandwidth. The emerging of edge intelligence, although still in the early stages, provides a promising solution for artificial intelligence (AI) applications on intelligent edge devices at the edge of the network close to data sources. These devices are with built-in processors enabling onboard analytics or AI (e.g., UAVs and Internet of Things gateways). Therefore, in this paper, a comprehensive survey on the latest developments of precision agriculture with UAV RS and edge intelligence is conducted for the first time. The major insights observed are as follows: (a) in terms of UAV systems, small or light, fixed-wing or industrial rotor-wing UAVs are widely used in precision agriculture; (b) sensors on UAVs can provide multi-source datasets, and there are only a few public UAV dataset for intelligent precision agriculture, mainly from RGB sensors and a few from multispectral and hyperspectral sensors; (c) DL-based UAV RS methods can be categorized into classification, object detection and segmentation tasks, and convolutional neural network and recurrent neural network are the mostly common used network architectures; (d) cloud computing is a common solution to UAV RS data processing, while edge computing brings the computing close to data sources; (e) edge intelligence is the convergence of artificial intelligence and edge computing, in which model compression especially parameter pruning and quantization is the most important and widely used technique at present, and typical edge resources include central processing units, graphics processing units and field programmable gate arrays.

Monitoring environmental and climate goals for European agriculture: User perspectives on the optimization of the Copernicus evolution offer

A vicious cycle exists between agricultural production and climate change, where agriculture is both a driver and a victim of the changing climate. While new and ambitious environmental and climate change-oriented goals are being introduced in Europe, the monitoring of these objectives is often jeopardized by a lack of technological means and a reliance on heavy administrative procedures. In particular, remote sensing technologies have the potential to significantly improve the monitoring of such goals but the characteristics of such missions should take into consideration the needs of users to guarantee return on investments and effective policy implementation. This study aims at identifying gaps in the current offer of Copernicus products for the monitoring of the agricultural sector through the elicitation of stakeholder requirements. The methodology is applied to the case study of Italy while the approach is scalable at European level. The elicitation process associates user needs to the European and national legislative framework to create a policy-oriented demand of Copernicus Earth Observation services. Results show the limitations faced by environmental managers in relation to the use of Remote Sensing technologies and the shortcomings associated with a purely technology driven approach to the development of satellite missions. Through the introduction of this flexible and user centred approach instead, this paper provides a clear overview of agro-environmental user requirements and represents the basis for the definition of an integrated agricultural service.

Paddy Rice Mapping in Thailand Using Time-Series Sentinel-1 Data and Deep Learning Model

The elimination of hunger is the top concern for developing countries and is the key to maintain national stability and security. Paddy rice occupies an essential status in food supply, whose accurate monitoring is of great importance for human sustainable development. As one of the most important paddy rice production countries in the world, Thailand has a favorable hot and humid climate for paddy rice growing, but the growth patterns of paddy rice are too complicated to construct promising growth models for paddy rice discrimination. To solve this problem, this study proposes a large-scale paddy rice mapping scheme, which uses time-series Sentinel-1 data to generate a convincing annual paddy rice map of Thailand. The proposed method extracts temporal statistical features of the time-series SAR images to overcome the intra-class variability due to different management practices and modifies the U-Net model with the fully connected Conditional Random Field (CRF) to maintain the edge of the fields. In this study, 758 Sentinel-1 images that covered the whole country from the end of 2018 to 2019 were acquired to generate the annual paddy rice map. The accuracy, precision, and recall of the resultant paddy rice map reached 91%, 87%, and 95%, respectively. Compared to SVM classifier and the U-Net model based on feature selection strategy (FS-U-Net), the proposed scheme achieved the best overall performance, which demonstrated the capability of overcoming the complex cultivation conditions and accurately identifying the fragmented paddy rice fields in Thailand. This study provides a promising tool for large-scale paddy rice monitoring in tropical production regions and has great potential in the global sustainable development of food and environment management.

Gradient Boosting Estimation of the Leaf Area Index of Apple Orchards in UAV Remote Sensing

The leaf area index (LAI) is a key parameter for describing the canopy structure of apple trees. This index is also employed in evaluating the amount of pesticide sprayed per unit volume of apple trees. Hence, numerous manual and automatic methods have been explored for LAI estimation. In this work, the leaf area indices for different types of apple trees are obtained in terms of multispectral remote-sensing data collected with an unmanned aerial vehicle (UAV), along with simultaneous measurements of apple orchards. The proposed approach was tested on apple trees of the “Fuji”, “Golden Delicious”, and “Ruixue” types, which were planted in the Apple Experimental Station of the Northwest Agriculture and Forestry University in Baishui County, Shaanxi Province, China. Five vegetation indices of strong correlation with the apple leaf area index were selected and used to train models of support vector regression (SVR) and gradient-boosting decision trees (GBDT) for predicting the leaf area index of apple trees. The best model was selected based on the metrics of the coefficient of determination (R2) and the root-mean-square error (RMSE). The experimental results showed that the gradient-boosting decision tree model achieved the best performance with an R2 of 0.846, an RMSE of 0.356, and a spatial efficiency (SPAEF) of 0.57. This demonstrates the feasibility of our approach for fast and accurate remote-sensing-based estimation of the leaf area index of apple trees.

The Impact of Phenological Developments on Interferometric and Polarimetric Crop Signatures Derived from Sentinel-1: Examples from the DEMMIN Study Site (Germany)

This study explores the potential of Sentinel-1 Synthetic Aperture Radar (SAR) to identify phenological phases of wheat, sugar beet, and canola. Breakpoint and extreme value analyses were applied to a dense time series of interferometric (InSAR) and polarimetric (PolSAR) features recorded during the growing season of 2017 at the JECAM site DEMMIN (Germany). The analyses of breakpoints and extrema allowed for the distinction of vegetative and reproductive stages for wheat and canola. Certain phenological stages, measured in situ using the BBCH-scale, such as leaf development and rosette growth of sugar beet or stem elongation and ripening of wheat, were detectable by a combination of InSAR coherence, polarimetric Alpha and Entropy, and backscatter (VV/VH). Except for some fringe cases, the temporal difference between in situ observations and breakpoints or extrema ranged from zero to five days. Backscatter produced the signature that generated the most breakpoints and extrema. However, certain micro stadia, such as leaf development of BBCH 10 of sugar beet or flowering BBCH 69 of wheat, were only identifiable by the InSAR coherence and Alpha. Hence, it is concluded that combining PolSAR and InSAR features increases the number of detectable phenological events in the phenological cycles of crops.

Sentinel-1 Time Series for Crop Identification in the Framework of the Future CAP Monitoring

In this upcoming Common Agricultural Policy (CAP) reform, the use of satellite imagery is taking an increasing role for improving the Integrated Administration and Control System (IACS). Considering the operational aspect of the CAP monitoring process, the use of Sentinel-1 SAR (Synthetic Aperture Radar) images is highly relevant, especially in regions with a frequent cloud cover, such as Belgium. Indeed, SAR imagery does not depend on sunlight and is barely affected by the presence of clouds. Moreover, the SAR signal is particularly sensitive to the geometry and the water content of the target. Crop identification is often a pre-requisite to monitor agriculture at parcel level (ploughing, harvest, grassland mowing, intercropping, etc.) The main goal of this study is to assess the performances and constraints of a SAR-based crop classification in an operational large-scale application. The Random Forest object-oriented classification model is built on Sentinel-1 time series from January to August 2020 only. It can identify crops in the Walloon Region (south part of Belgium) with high performance: 93.4% of well-classified area, representing 88.4% of the parcels. Among the 48 crop groups, the six most represented ones get a F1-score higher or equal to 84%. Additionally, this research documents how the classification performance is affected by different parameters: the SAR orbit, the size of the training dataset, the use of different internal buffers on parcel polygons before signal extraction, the set of explanatory variables, and the period of the time series. In an operational context, this allows to choose the right balance between classification accuracy and model complexity. A key result is that using a training dataset containing only 3.2% of the total number of parcels allows to correctly classify 91.7% of the agricultural area. The impact of rain and snow is also discussed. Finally, this research analyses how the classification accuracy depends on some characteristics of the parcels like their shape or size. This allows to assess the relevance of the classification depending on those characteristics, as well as to identify a subset of parcels for which the global accuracy is higher.

An Operational Framework for Mapping Irrigated Areas at Plot Scale Using Sentinel-1 and Sentinel-2 Data

In this study, we present an operational methodology for mapping irrigated areas at plot scale, which overcomes the limitation of terrain data availability, using Sentinel-1 (S1) C-band SAR (synthetic-aperture radar) and Sentinel-2 (S2) optical time series. The method was performed over a study site located near Orléans city of north-central France for four years (2017 until 2020). First, training data of irrigated and non-irrigated plots were selected using predefined selection criteria to obtain sufficient samples of irrigated and non-irrigated plots each year. The training data selection criteria is based on two irrigation metrics; the first one is a SAR-based metric derived from the S1 time series and the second is an optical-based metric derived from the NDVI (normalized difference vegetation index) time series of the S2 data. Using the newly developed irrigation event detection model (IEDM) applied for all S1 time series in VV (Vertical-Vertical) and VH (Vertical-Horizontal) polarizations, an irrigation weight metric was calculated for each plot. Using the NDVI time series, the maximum NDVI value achieved in the crop cycle was considered as a second selection metric. By fixing threshold values for both metrics, a dataset of irrigated and non-irrigated samples was constructed each year. Later, a random forest classifier (RF) was built for each year in order to map the summer agricultural plots into irrigated/non-irrigated. The irrigation classification model uses the S1 and NDVI time series calculated over the selected training plots. Finally, the proposed irrigation classifier was validated using real in situ data collected each year. The results show that, using the proposed classification procedure, the overall accuracy for the irrigation classification reaches 84.3%, 93.0%, 81.8%, and 72.8% for the years 2020, 2019, 2018, and 2017, respectively. The comparison between our proposed classification approach and the RF classifier built directly from in situ data showed that our approach reaches an accuracy nearly similar to that obtained using in situ RF classifiers with a difference in overall accuracy not exceeding 6.2%. The analysis of the obtained classification accuracies of the proposed method with precipitation data revealed that years with higher rainfall amounts during the summer crop-growing season (irrigation period) had lower overall accuracy (72.8% for 2017) whereas years encountering a drier summer had very good accuracy (93.0% for 2019).

Mapping Smallholder Maize Farms Using Multi-Temporal Sentinel-1 Data in Support of the Sustainable Development Goals

Reducing food insecurity in developing countries is one of the crucial targets of the Sustainable Development Goals (SDGs). Smallholder farmers play a crucial role in combating food insecurity. However, local planning agencies and governments do not have adequate spatial information on smallholder farmers, and this affects the monitoring of the SDGs. This study utilized Sentinel-1 multi-temporal data to develop a framework for mapping smallholder maize farms and to estimate maize production area as a parameter for supporting the SDGs. We used Principal Component Analysis (PCA) to pixel fuse the multi-temporal data to only three components for each polarization (vertical transmit and vertical receive (VV), vertical transmit and horizontal receive (VH), and VV/VH), which explained more than 70% of the information. The Support Vector Machine (SVM) and Extreme Gradient Boosting (Xgboost) algorithms were used at model-level feature fusion to classify the data. The results show that the adopted strategy of two-stage image fusion was sufficient to map the distribution and estimate production areas for smallholder farms. An overall accuracy of more than 90% for both SVM and Xgboost algorithms was achieved. There was a 3% difference in production area estimation observed between the two algorithms. This framework can be used to generate spatial agricultural information in areas where agricultural survey data are limited and for areas that are affected by cloud coverage. We recommend the use of Sentinel-1 multi-temporal data in conjunction with machine learning algorithms to map smallholder maize farms to support the SDGs.

National Crop Mapping Using Sentinel-1 Time Series: A Knowledge-Based Descriptive Algorithm

National-level mapping of crop types is important to monitor food security, understand environmental conditions, inform optimal use of the landscape, and contribute to agricultural policy. Countries or economic regions currently and increasingly use satellite sensor data for classifying crops over large areas. However, most methods have been based on machine learning algorithms, with these often requiring large training datasets that are not always available and may be costly to produce or collect. Focusing on Wales (United Kingdom), the research demonstrates how the knowledge that the agricultural community has gathered together over past decades can be used to develop algorithms for mapping different crop types. Specifically, we aimed to develop an alternative method for consistent and accurate crop type mapping where cloud cover is quite persistent and without the need for extensive in situ/ground datasets. The classification approach is parcel-based and informed by concomitant analysis of knowledge-based crop growth stages and Sentinel-1 C-band SAR time series. For 2018, crop type classifications were generated nationally for Wales, with regional overall accuracies ranging between 85.8% and 90.6%. The method was particularly successful in distinguishing barley from wheat, which is a major source of error in other crop products available for Wales. This study demonstrates that crops can be accurately identified and mapped across a large area (i.e., Wales) using Sentinel-1 C-band data and by capitalizing on knowledge of crop growth stages. The developed algorithm is flexible and, compared to the other methods that allow crop mapping in Wales, the approach provided more consistent discrimination and lower variability in accuracies between classes and regions.

Using Time Series Sentinel-1 Images for Object-Oriented Crop Classification in Google Earth Engine

The purpose of this study was to evaluate the feasibility and applicability of object-oriented crop classification using Sentinel-1 images in the Google Earth Engine (GEE). In this study, two study areas (Keshan farm and Tongnan town) with different average plot sizes in Heilongjiang Province, China, were selected. The research time was two consecutive years (2018 and 2019), which were used to verify the robustness of the method. Sentinel-1 images of the crop growth period (May to September) in each study area were composited with three time intervals (10 d, 15 d and 30 d). Then, the composite images were segmented by simple noniterative clustering (SNIC) according to different sizes and finally, the training samples and processed images were input into a random forest classifier for crop classification. The results showed the following: (1) the overall accuracy of using the object-oriented classification method combined composite Sentinel-1 image represented a great improvement compared with the pixel-based classification method in areas with large average plots (increase by 10%), the applicable scope of the method depends on the plot size of the study area; (2) the shorter time interval of the composite Sentinel-1 image was, the higher the crop classification accuracy was; (3) the features with high importance of composite Sentinel-1 images with different time intervals were mainly distributed in July, August and September, which was mainly due to the large differences in crop growth in these months; and (4) the optimal segmentation size of crop classification was closely related to image resolution and plot size. Previous studies usually emphasize the advantages of object-oriented classification. Our research not only emphasizes the advantages of object-oriented classification but also analyzes the constraints of using object-oriented classification, which is very important for the follow-up research of crop classification using object-oriented and synthetic aperture radar (SAR).

Mapping Paddy Rice with Satellite Remote Sensing: A Review

Paddy rice is a staple food of three billion people in the world. Timely and accurate estimation of the paddy rice planting area and paddy rice yield can provide valuable information for the government, planners and decision makers to formulate policies. This article reviews the existing paddy rice mapping methods presented in the literature since 2010, classifies these methods, and analyzes and summarizes the basic principles, advantages and disadvantages of these methods. According to the data sources used, the methods are divided into three categories: (I) Optical mapping methods based on remote sensing; (II) Mapping methods based on microwave remote sensing; and (III) Mapping methods based on the integration of optical and microwave remote sensing. We found that the optical remote sensing data sources are mainly MODIS, Landsat, and Sentinel-2, and the emergence of Sentinel-1 data has promoted research on radar mapping methods for paddy rice. Multisource data integration further enhances the accuracy of paddy rice mapping. The best methods are phenology algorithms, paddy rice mapping combined with machine learning, and multisource data integration. Innovative methods include the time series similarity method, threshold method combined with mathematical models, and object-oriented image classification. With the development of computer technology and the establishment of cloud computing platforms, opportunities are provided for obtaining large-scale high-resolution rice maps. Multisource data integration, paddy rice mapping under different planting systems and the connection with global changes are the focus of future development priorities.

A Review of Plant Phenotypic Image Recognition Technology Based on Deep Learning

Plant phenotypic image recognition (PPIR) is an important branch of smart agriculture. In recent years, deep learning has achieved significant breakthroughs in image recognition. Consequently, PPIR technology that is based on deep learning is becoming increasingly popular. First, this paper introduces the development and application of PPIR technology, followed by its classification and analysis. Second, it presents the theory of four types of deep learning methods and their applications in PPIR. These methods include the convolutional neural network, deep belief network, recurrent neural network, and stacked autoencoder, and they are applied to identify plant species, diagnose plant diseases, etc. Finally, the difficulties and challenges of deep learning in PPIR are discussed.

An Adversarial Generative Network for Crop Classification from Remote Sensing Timeseries Images

Due to the increasing demand for the monitoring of crop conditions and food production, it is a challenging and meaningful task to identify crops from remote sensing images. The state-of the-art crop classification models are mostly built on supervised classification models such as support vector machines (SVM), convolutional neural networks (CNN), and long- and short-term memory neural networks (LSTM). Meanwhile, as an unsupervised generative model, the adversarial generative network (GAN) is rarely used to complete classification tasks for agricultural applications. In this work, we propose a new method that combines GAN, CNN, and LSTM models to classify crops of corn and soybeans from remote sensing time-series images, in which GAN’s discriminator was used as the final classifier. The method is feasible on the condition that the training samples are small, and it fully takes advantage of spectral, spatial, and phenology features of crops from satellite data. The classification experiments were conducted on crops of corn, soybeans, and others. To verify the effectiveness of the proposed method, comparisons with models of SVM, SegNet, CNN, LSTM, and different combinations were also conducted. The results show that our method achieved the best classification results, with the Kappa coefficient of 0.7933 and overall accuracy of 0.86. Experiments in other study areas also demonstrate the extensibility of the proposed method.

Extraction of Land Information, Future Landscape Changes and Seismic Hazard Assessment: A Case Study of Tabriz, Iran

Exact land cover inventory data should be extracted for future landscape prediction and seismic hazard assessment. This paper presents a comprehensive study towards the sustainable development of Tabriz City (NW Iran) including land cover change detection, future potential landscape, seismic hazard assessment and municipal performance evaluation. Landsat data using maximum likelihood (ML) and Markov chain algorithms were used to evaluate changes in land cover in the study area. The urbanization pattern taking place in the city was also studied via synthetic aperture radar (SAR) data of Sentinel-1 ground range detected (GRD) and single look complex (SLC). The age of buildings was extracted by using built-up areas of all classified maps. The logistic regression (LR) model was used for creating a seismic hazard assessment map. From the results, it can be concluded that the land cover (especially built-up areas) has seen considerable changes from 1989 to 2020. The overall accuracy (OA) values of the produced maps for the years 1989, 2005, 2011 and 2020 are 96%, 96%, 93% and 94%, respectively. The future potential landscape of the city showed that the land cover prediction by using the Markov chain model provided a promising finding. Four images of 1989, 2005, 2011 and 2020, were employed for built-up areas’ land information trends, from which it was indicated that most of the built-up areas had been constructed before 2011. The seismic hazard assessment map indicated that municipal zones of 1 and 9 were the least susceptible areas to an earthquake; conversely, municipal zones of 4, 6, 7 and 8 were located in the most susceptible regions to an earthquake in the future. More findings showed that municipal zones 1 and 4 demonstrated the best and worst performance among all zones, respectively.

Farmland Parcel Mapping in Mountain Areas Using Time-Series SAR Data and VHR Optical Images

Accurate, timely, and reliable farmland mapping is a prerequisite for agricultural management and environmental assessment in mountainous areas. However, in these areas, high spatial heterogeneity and diversified planting structures together generate various small farmland parcels with irregular shapes that are difficult to accurately delineate. In addition, the absence of optical data caused by the cloudy and rainy climate impedes the use of time-series optical data to distinguish farmland from other land use types. Automatic delineation of farmland parcels in mountain areas is still a very difficult task. This paper proposes an innovative precise farmland parcel extraction approach supported by very high resolution(VHR) optical image and time series synthetic aperture radar(SAR) data. Firstly, Google satellite imagery with a spatial resolution of 0.55 m was used for delineating the boundaries of ground parcel objects in mountainous areas by a hierarchical extraction scheme. This scheme divides farmland into four types based on the morphological features presented in optical imagery, and designs different extraction models to produce each farmland type, respectively. The potential farmland parcel distribution map is then obtained by the layered recombination of these four farmland types. Subsequently, the time profile of each parcel in this map was constructed by five radar variables from the Sentinel-1A dataset, and the time-series classification method was used to distinguish farmland parcels from other types. An experiment was carried out in the north of Guiyang City, Guizhou Province, Southwest China. The result shows that, the producer’s accuracy of farmland parcels obtained by the hierarchical scheme is increased by 7.39% to 96.38% compared with that without this scheme, and the time-series classification method produces an accuracy of 80.83% to further obtain the final overall accuracy of 96.05% for the farmland parcel maps, showing a good performance. In addition, through visual inspection, this method has a better suppression effect on background noise in mountainous areas, and the extracted farmland parcels are closer to the actual distribution of the ground farmland.

Combination of Landsat 8 OLI and Sentinel-1 SAR Time-Series Data for Mapping Paddy Fields in Parts of West and Central Java Provinces, Indonesia

The rise of Google Earth Engine, a cloud computing platform for spatial data, has unlocked seamless integration for multi-sensor and multi-temporal analysis, which is useful for the identification of land-cover classes based on their temporal characteristics. Our study aims to employ temporal patterns from monthly-median Sentinel-1 (S1) C-band synthetic aperture radar data and cloud-filled monthly spectral indices, i.e., Normalized Difference Vegetation Index (NDVI), Modified Normalized Difference Water Index (MNDWI), and Normalized Difference Built-up Index (NDBI), from Landsat 8 (L8) OLI for mapping rice cropland areas in the northern part of Central Java Province, Indonesia. The harmonic function was used to fill the cloud and cloud-masked values in the spectral indices from Landsat 8 data, and smile Random Forests (RF) and Classification And Regression Trees (CART) algorithms were used to map rice cropland areas using a combination of monthly S1 and monthly harmonic L8 spectral indices. An additional terrain variable, Terrain Roughness Index (TRI) from the SRTM dataset, was also included in the analysis. Our results demonstrated that RF models with 50 (RF50) and 80 (RF80) trees yielded better accuracy for mapping the extent of paddy fields, with user accuracies of 85.65% (RF50) and 85.75% (RF80), and producer accuracies of 91.63% (RF80) and 93.48% (RF50) (overall accuracies of 92.10% (RF80) and 92.47% (RF50)), respectively, while CART yielded a user accuracy of only 84.83% and a producer accuracy of 80.86%. The model variable importance in both RF50 and RF80 models showed that vertical transmit and horizontal receive (VH) polarization and harmonic-fitted NDVI were identified as the top five important variables, and the variables representing February, April, June, and December contributed more to the RF model. The detection of VH and NDVI as the top variables which contributed up to 51% of the Random Forest model indicated the importance of the multi-sensor combination for the identification of paddy fields.

Automatic Mapping of Rice Growth Stages Using the Integration of SENTINEL-2, MOD13Q1, and SENTINEL-1

Rice (Oryza sativa L.) is a staple food crop for more than half of the world’s population. Rice production is facing a myriad of problems, including water shortage, climate, and land-use change. Accurate maps of rice growth stages are critical for monitoring rice production and assessing its impacts on national and global food security. Rice growth stages are typically monitored by coarse-resolution satellite imagery. However, it is difficult to accurately map due to the occurrence of mixed pixels in fragmented and patchy rice fields, as well as cloud cover, particularly in tropical countries. To solve these problems, we developed an automated mapping workflow to produce near real-time multi-temporal maps of rice growth stages at a 10-m spatial resolution using multisource remote sensing data (Sentinel-2, MOD13Q1, and Sentinel-1). This study was investigated between 1 June and 29 September 2018 in two (wet and dry) areas of Java Island in Indonesia. First, we built prediction models based on Sentinel-2, and fusion of MOD13Q1/Sentinel-1 using the ground truth information. Second, we applied the prediction models on all images in area and time and separation between the non-rice planting class and rice planting class over the cropping pattern. Moreover, the model’s consistency on the multitemporal map with a 5–30-day lag was investigated. The result indicates that the Sentinel-2 based model classification gives a high overall accuracy of 90.6% and the fusion model MOD13Q1/Sentinel-1 shows 78.3%. The performance of multitemporal maps was consistent between time lags with an accuracy of 83.27–90.39% for Sentinel-2 and 84.15% for the integration of Sentinel-2/MOD13Q1/Sentinel-1. The results from this study show that it is possible to integrate multisource remote sensing for regular monitoring of rice phenology, thereby generating spatial information to support local-, national-, and regional-scale food security applications.

Simultaneous Prediction of Soil Properties Using Multi_CNN Model

Soil nutrient prediction based on near-infrared spectroscopy has become the main research direction for rapid acquisition of soil information. The development of deep learning has greatly improved the prediction accuracy of traditional modeling methods. In view of the low efficiency and low accuracy of current soil prediction models, this paper proposes a soil multi-attribute intelligent prediction method based on convolutional neural networks, by constructing a dual-stream convolutional neural network model Multi_CNN that combines one-dimensional convolution and two-dimensional convolution, the intelligent prediction of soil multi-attribute is realized. The model extracts the characteristics of soil attributes from spectral sequences and spectrograms respectively, and multiple attributes can be predicted simultaneously by feature fusion. The model is based on two different-scale soil near-infrared spectroscopy data sets for multi-attribute prediction. The experimental results show that the RP2 of the three attributes of Total Carbon, Total Nitrogen, and Alkaline Nitrogen on the small dataset are 0.94, 0.95, 0.87, respectively, and the RP2 of the attributes of Organic Carbon, Nitrogen, and Clay on the LUCAS dataset are, respectively, 0.95, 0.91, 0.83, And compared with traditional regression models and new prediction methods commonly used in soil nutrient prediction, the multi-task model proposed in this paper is more accurate.

Recent Advances of Hyperspectral Imaging Technology and Applications in Agriculture

Remote sensing is a useful tool for monitoring spatio-temporal variations of crop morphological and physiological status and supporting practices in precision farming. In comparison with multispectral imaging, hyperspectral imaging is a more advanced technique that is capable of acquiring a detailed spectral response of target features. Due to limited accessibility outside of the scientific community, hyperspectral images have not been widely used in precision agriculture. In recent years, different mini-sized and low-cost airborne hyperspectral sensors (e.g., Headwall Micro-Hyperspec, Cubert UHD 185-Firefly) have been developed, and advanced spaceborne hyperspectral sensors have also been or will be launched (e.g., PRISMA, DESIS, EnMAP, HyspIRI). Hyperspectral imaging is becoming more widely available to agricultural applications. Meanwhile, the acquisition, processing, and analysis of hyperspectral imagery still remain a challenging research topic (e.g., large data volume, high data dimensionality, and complex information analysis). It is hence beneficial to conduct a thorough and in-depth review of the hyperspectral imaging technology (e.g., different platforms and sensors), methods available for processing and analyzing hyperspectral information, and recent advances of hyperspectral imaging in agricultural applications. Publications over the past 30 years in hyperspectral imaging technology and applications in agriculture were thus reviewed. The imaging platforms and sensors, together with analytic methods used in the literature, were discussed. Performances of hyperspectral imaging for different applications (e.g., crop biophysical and biochemical properties’ mapping, soil characteristics, and crop classification) were also evaluated. This review is intended to assist agricultural researchers and practitioners to better understand the strengths and limitations of hyperspectral imaging to agricultural applications and promote the adoption of this valuable technology. Recommendations for future hyperspectral imaging research for precision agriculture are also presented.

Crop Mapping from Sentinel-1 Polarimetric Time-Series with a Deep Neural Network

Timely and accurate agricultural information is essential for food security assessment and agricultural management. Synthetic aperture radar (SAR) systems are increasingly available in crop mapping, as they provide all-weather imagery. In particular, the Sentinel-1 sensor provides dense time-series data, thus offering a unique opportunity for crop mapping. However, in most studies, the Sentinel-1 complex backscatter coefficient was used directly which limits the potential of the Sentinel-1 in crop mapping. Meanwhile, most of the existing methods may not be tailored for the task of crop classification in time-series polarimetric SAR data. To solve the above problem, we present a novel deep learning strategy in this research. To be specific, we collected Sentinel-1 time-series data in two study areas. The Sentinel-1 image covariance matrix is used as an input to maintain the integrity of polarimetric information. Then, a depthwise separable convolution recurrent neural network (DSCRNN) architecture is proposed to characterize crop types from multiple perspectives and achieve better classification results. The experimental results indicate that the proposed method achieves better accuracy in complex agricultural areas than other classical methods. Additionally, the variable importance provided by the random forest (RF) illustrated that the covariance vector has a far greater influence than the backscatter coefficient. Consequently, the strategy proposed in this research is effective and promising for crop mapping.

Multi-Temporal Unmanned Aerial Vehicle Remote Sensing for Vegetable Mapping Using an Attention-Based Recurrent Convolutional Neural Network

Vegetable mapping from remote sensing imagery is important for precision agricultural activities such as automated pesticide spraying. Multi-temporal unmanned aerial vehicle (UAV) data has the merits of both very high spatial resolution and useful phenological information, which shows great potential for accurate vegetable classification, especially under complex and fragmented agricultural landscapes. In this study, an attention-based recurrent convolutional neural network (ARCNN) has been proposed for accurate vegetable mapping from multi-temporal UAV red-green-blue (RGB) imagery. The proposed model firstly utilizes a multi-scale deformable CNN to learn and extract rich spatial features from UAV data. Afterwards, the extracted features are fed into an attention-based recurrent neural network (RNN), from which the sequential dependency between multi-temporal features could be established. Finally, the aggregated spatial-temporal features are used to predict the vegetable category. Experimental results show that the proposed ARCNN yields a high performance with an overall accuracy of 92.80%. When compared with mono-temporal classification, the incorporation of multi-temporal UAV imagery could significantly boost the accuracy by 24.49% on average, which justifies the hypothesis that the low spectral resolution of RGB imagery could be compensated by the inclusion of multi-temporal observations. In addition, the attention-based RNN in this study outperforms other feature fusion methods such as feature-stacking. The deformable convolution operation also yields higher classification accuracy than that of a standard convolution unit. Results demonstrate that the ARCNN could provide an effective way for extracting and aggregating discriminative spatial-temporal features for vegetable mapping from multi-temporal UAV RGB imagery.

Near Real-Time Irrigation Detection at Plot Scale Using Sentinel-1 Data

In the context of monitoring and assessment of water consumption in the agricultural sector, the objective of this study is to build an operational approach capable of detecting irrigation events at plot scale in a near real-time scenario using Sentinel-1 (S1) data. The proposed approach is a decision tree-based method relying on the change detection in the S1 backscattering coefficients at plot scale. First, the behavior of the S1 backscattering coefficients following irrigation events has been analyzed at plot scale over three study sites located in Montpellier (southeast France), Tarbes (southwest France), and Catalonia (northeast Spain). To eliminate the uncertainty between rainfall and irrigation, the S1 synthetic aperture radar (SAR) signal and the soil moisture estimations at grid scale (10 km × 10 km) have been used. Then, a tree-like approach has been constructed to detect irrigation events at each S1 date considering additional filters to reduce ambiguities due to vegetation development linked to the growth cycle of different crops types as well as the soil surface roughness. To enhance the detection of irrigation events, a filter using the normalized differential vegetation index (NDVI) obtained from Sentinel-2 optical images has been proposed. Over the three study sites, the proposed method was applied on all possible S1 acquisitions in ascending and descending modes. The results show that 84.8% of the irrigation events occurring over agricultural plots in Montpellier have been correctly detected using the proposed method. Over the Catalonian site, the use of the ascending and descending SAR acquisition modes shows that 90.2% of the non-irrigated plots encountered no detected irrigation events whereas 72.4% of the irrigated plots had one and more detected irrigation events. Results over Catalonia also show that the proposed method allows the discrimination between irrigated and non-irrigated plots with an overall accuracy of 85.9%. In Tarbes, the analysis shows that irrigation events could still be detected even in the presence of abundant rainfall events during the summer season where two and more irrigation events have been detected for 90% of the irrigated plots. The novelty of the proposed method resides in building an effective unsupervised tool for near real-time detection of irrigation events at plot scale independent of the studied geographical context.

Flood Proxy Mapping with Normalized Difference Sigma-Naught Index and Shannon’s Entropy

Rainfall-induced floods often cause significant loss of life as well as damage to infrastructure and crops. Synthetic Aperture Radar (SAR) Earth Observation Satellites (EOS) can be used to determine the extent of flooding over large geographical areas. Unlike optical sensors, SAR instruments are suitable for cloudy weather conditions, making them suitable for flood detection and mapping during extreme weather events. In this study, we explore the application of the Normalized Difference Sigma-Naught Index (NDSI) and Shannon’s entropy of NDSI (SNDSI) of Sentinel-1 data for open water flooding detection, based on automatic thresholding and Bayesian probability. The proposed methodology was tested using the floods in Sofala province, Mozambique, caused by cyclone Idai on March 14–19 of 2019. Results show that thresholding of the NDSI Vertical Transmit-Horizontal Receive (VH) can produce results with Overall Accuracy above 90%, and Kappa higher than 0.6. Considerable performance improvements were obtained by our thresholding method over the entropy of NDSI, yielding results with Kappa of 0.70–0.77. Additionally, it was found that Weibull distribution can properly describe the properties of flooded pixels within the histogram of SNDSI, which allows us to generate a flood probability raster using a Bayesian approach. The final per-pixel flooding probability is useful to indicate certainty in the classification results. The SNDSI Bayesian model produced an AUC (Area Under the Receiver Operating Characteristic Curve) of 0.93–0.97, with cross-polarized data yielding the most accurate results.

Landslide Image Captioning Method Based on Semantic Gate and Bi-Temporal LSTM

When a landslide happens, it is important to recognize the hazard-affected bodies surrounding the landslide for the risk assessment and emergency rescue. In order to realize the recognition, the spatial relationship between landslides and other geographic objects such as residence, roads and schools needs to be defined. Comparing with semantic segmentation and instance segmentation that can only recognize the geographic objects separately, image captioning can provide richer semantic information including the spatial relationship among these objects. However, the traditional image captioning methods based on RNNs have two main shortcomings: the errors in the prediction process are often accumulated and the location of attention is not always accurate which would lead to misjudgment of risk. To handle these problems, a landslide image interpretation network based on a semantic gate and a bi-temporal long-short term memory network (SG-BiTLSTM) is proposed in this paper. In the SG-BiTLSTM architecture, a U-Net is employed as an encoder to extract features of the images and generate the mask maps of the landslides and other geographic objects. The decoder of this structure consists of two interactive long-short term memory networks (LSTMs) to describe the spatial relationship among these geographic objects so that to further determine the role of the classified geographic objects for identifying the hazard-affected bodies. The purpose of this research is to judge the hazard-affected bodies of the landslide (i.e., buildings and roads) through the SG-BiTLSTM network to provide geographic information support for emergency service. The remote sensing data was taken by Worldview satellite after the Wenchuan earthquake happened in 2008. The experimental results demonstrate that SG-BiTLSTM network shows remarkable improvements on the recognition of landslide and hazard-affected bodies, compared with the traditional LSTM (the Baseline Model), the BLEU1 of the SG-BiTLSTM is improved by 5.89%, the matching rate between the mask maps and the focus matrix of the attention is improved by 42.81%. In conclusion, the SG-BiTLSTM network can recognize landslides and the hazard-affected bodies simultaneously to provide basic geographic information service for emergency decision-making.

Mapping Maize Fields by Using Multi-Temporal Sentinel-1A and Sentinel-2A Images in Makarfi, Northern Nigeria, Africa

A timely and accurate crop type mapping is very significant, and a prerequisite for agricultural regions and ensuring global food security. The combination of remotely sensed optical and radar datasets presents an opportunity for acquiring crop information at relative spatial resolution and temporal resolution adequately to capture the growth profiles of various crop species. In this paper, we employed Sentinel-1A (S-1) and Sentinel-2A (S-2) data acquired between the end of June and early September 2016, on a semi-arid area in northern Nigeria. A different set of (VV and VH) SAR and optical (SI and SB) images, illustrating crop phenological development stage, were employed as inputs to the two machines learning Random Forest (RF) and Support Vector Machine (SVM) algorithms to automatically map maize fields. Significant increases in overall classification were shown when the multi-temporal spectral indices (SI) and spectral band (SB) datasets were added with the different integration of SAR datasets (i.e., VV and VH). The best overall accuracy (OA) for maize (96.93%) was derived by using RF classification algorithms with SI-SB-SAR datasets, although the SI datasets for RF and SB datasets for SVM also produced high overall maize classification accuracies, of 97.04% and 97.44%. The outcomes indicate the robustness of the RF or SVM methods to produce high-resolution maps of maize for subsequent application from agronomists, policy planners, and the government, because such information is lacking in our study area.

Synergistic Use of Multi-Temporal RADARSAT-2 and VENµS Data for Crop Classification Based on 1D Convolutional Neural Network

Annual crop inventory information is important for many agriculture applications and government statistics. The synergistic use of multi-temporal polarimetric synthetic aperture radar (SAR) and available multispectral remote sensing data can reduce the temporal gaps and provide the spectral and polarimetric information of the crops, which is effective for crop classification in areas with frequent cloud interference. The main objectives of this study are to develop a deep learning model to map agricultural areas using multi-temporal full polarimetric SAR and multi-spectral remote sensing data, and to evaluate the influence of different input features on the performance of deep learning methods in crop classification. In this study, a one-dimensional convolutional neural network (Conv1D) was proposed and tested on multi-temporal RADARSAT-2 and VENµS data for crop classification. Compared with the Multi-Layer Perceptron (MLP), Recurrent Neural Network (RNN) and non-deep learning methods including XGBoost, Random Forest (RF), and Support Vector Machina (SVM), the Conv1D performed the best when the multi-temporal RADARSAT-2 data (Pauli decomposition or coherency matrix) and VENµS multispectral data were fused by the Minimum Noise Fraction (MNF) transformation. The Pauli decomposition and coherency matrix gave similar overall accuracy (OA) for Conv1D when fused with the VENµS data by the MNF transformation (OA = 96.65 ± 1.03% and 96.72 ± 0.77%). The MNF transformation improved the OA and F-score for most classes when Conv1D was used. The results reveal that the coherency matrix has a great potential in crop classification and the MNF transformation of multi-temporal RADARSAT-2 and VENµS data can enhance the performance of Conv1D.

Comparison of Machine Learning Methods Applied to SAR Images for Forest Classification in Mediterranean Areas

In this paper, multifrequency synthetic aperture radar (SAR) images from ALOS/PALSAR, ENVISAT/ASAR and Cosmo-SkyMed sensors were studied for forest classification in a test area in Central Italy (San Rossore), where detailed in-situ measurements were available. A preliminary discrimination of the main land cover classes and forest types was carried out by exploiting the synergy among L-, C- and X-bands and different polarizations. SAR data were preliminarily inspected to assess the capabilities of discriminating forest from non-forest and separating broadleaf from coniferous forests. The temporal average backscattering coefficient ( σ ¯ °) was computed for each sensor-polarization pair and labeled on a pixel basis according to the reference map. Several classification methods based on the machine learning framework were applied and validated considering different features, in order to highlight the contribution of bands and polarizations, as well as to assess the classifiers’ performance. The experimental results indicate that the different surface types are best identified by using all bands, followed by joint L- and X-bands. In the former case, the best overall average accuracy (83.1%) is achieved by random forest classification. Finally, the classification maps on class edges are discussed to highlight the misclassification errors.

Sea Clutter Amplitude Prediction Using a Long Short-Term Memory Neural Network

In the marine environment, shore-based radars play an important role in military surveillance and sensing. Sea clutter is one of the main factors affecting the performance of shore-based radar. Affected by marine environmental factors and radar parameters, the fluctuation law of sea clutter amplitude is very complicated. In the process of training a sea clutter amplitude prediction model, the traditional method updates the model parameters according to the current input data and the parameters in the current model, and cannot utilize the historical information of sea clutter amplitude. It is only possible to learn the short-term variation characteristics of the sea clutter. In order to learn the long-term variation law of sea clutter, a sea clutter prediction system based on the long short-term memory neural network is proposed. Based on sea clutter data collected by IPIX radar, UHF-band radar and S-band radar, the experimental results show that the mean square error of this prediction system is smaller than the traditional prediction methods. The sea clutter suppression signal is extracted by comparing the predicted sea clutter data with the original sea clutter data. The results show that the proposed sea clutter prediction system has a good effect on sea clutter suppression.

Evaluation of Three Deep Learning Models for Early Crop Classification Using Sentinel-1A Imagery Time Series—A Case Study in Zhanjiang, China

Timely and accurate estimation of the area and distribution of crops is vital for food security. Optical remote sensing has been a key technique for acquiring crop area and conditions on regional to global scales, but great challenges arise due to frequent cloudy days in southern China. This makes optical remote sensing images usually unavailable. Synthetic aperture radar (SAR) could bridge this gap since it is less affected by clouds. The recent availability of Sentinel-1A (S1A) SAR imagery with a 12-day revisit period at a high spatial resolution of about 10 m makes it possible to fully utilize phenological information to improve early crop classification. In deep learning methods, one-dimensional convolutional neural networks (1D CNNs), long short-term memory recurrent neural networks (LSTM RNNs), and gated recurrent unit RNNs (GRU RNNs) have been shown to efficiently extract temporal features for classification tasks. However, due to the complexity of training, these three deep learning methods have been less used in early crop classification. In this work, we attempted to combine them with an incremental classification method to avoid the need for training optimal architectures and hyper-parameters for data from each time series. First, we trained 1D CNNs, LSTM RNNs, and GRU RNNs based on the full images’ time series to attain three classifiers with optimal architectures and hyper-parameters. Then, starting at the first time point, we performed an incremental classification process to train each classifier using all of the previous data, and obtained a classification network with all parameter values (including the hyper-parameters) at each time point. Finally, test accuracies of each time point were assessed for each crop type to determine the optimal time series length. A case study was conducted in Suixi and Leizhou counties of Zhanjiang City, China. To verify the effectiveness of this method, we also implemented the classic random forest (RF) approach. The results were as follows: (i) 1D CNNs achieved the highest Kappa coefficient (0.942) of the four classifiers, and the highest value (0.934) in the GRU RNNs time series was attained earlier than with other classifiers; (ii) all three deep learning methods and the RF achieved F measures above 0.900 before the end of growth seasons of banana, eucalyptus, second-season paddy rice, and sugarcane; while, the 1D CNN classifier was the only one that could obtain an F-measure above 0.900 for pineapple before harvest. All results indicated the effectiveness of the solution combining the deep learning models with the incremental classification approach for early crop classification. This method is expected to provide new perspectives for early mapping of croplands in cloudy areas.

Cropland Classification Using Sentinel-1 Time Series: Methodological Performance and Prediction Uncertainty Assessment

Methods based on Sentinel-1 data were developed to monitor crops and fields to facilitate the distribution of subsidies. The objectives were to (1) develop a methodology to predict individual crop species or or management regimes; (2) investigate the earliest time point in the growing season when the species predictions are satisfactory; and (3) to present a method to assess the uncertainty of the predictions at an individual field level. Seventeen Sentinel-1 synthetic aperture radar (SAR) scenes (VV and VH polarizations) acquired in interferometric wide swath mode from 14 May through to 30 August 2017 in the same geometry, and selected based on the weather conditions, were used in the study. The improved k nearest neighbour estimation, ik-NN, with a genetic algorithm feature optimization was tailored for classification with optional Sentinel-1 data sets, species groupings, and thresholds for the minimum parcel area. The number of species groups varied from 7 to as large as 41. Multinomial logistic regression was tested as an optional method. The Overall Accuracies (OA) varied depending on the number of species included in the classification, and whether all or not field parcels were included. OA with nine species groups was 72% when all parcels were included, 81% when the parcels area threshold (for incorporating parcels into classification) was 0.5 ha, and around 90% when the threshold was 4 ha. The OA gradually increased when adding extra Sentinel-1 scenes up until the early August, and the initial scenes were acquired in early June or mid-May. After that, only minor improvements in the crop recognition accuracy were noted. The ik-NN method gave greater overall accuracies than the logistic regression analysis with all data combinations tested. The width of the 95% confidence intervals with ik-NN for the estimate of the probability of the species with the largest probability on an individual parcel varied depending on the species, the area threshold of the parcel and the number of the Sentinel-1 scenes used. The results ranged between 0.06–0.08 units (6–8% points) for the most common species when the Sentinel-1 scenes were between 1 June and 12 August. The results were well-received by the authorities and encourage further research to continue the study towards an operational method in which the space-borne SAR data are a part of the information chain.

Synchronous Response Analysis of Features for Remote Sensing Crop Classification Based on Optical and SAR Time-Series Data

Accurate crop classification is the basis of agricultural research, and remote sensing is the only effective measuring technique to classify crops over large areas. Optical remote sensing is effective in regions with good illumination; however, it usually fails to meet requirements for highly accurate crop classification in cloud-covered areas and rainy regions. Synthetic aperture radar (SAR) can achieve active data acquisition by transmitting signals; thus, it has strong resistance to cloud and rain interference. In this study, we designed an improved crop planting structure mapping framework for cloudy and rainy regions by combining optical data and SAR data, and we revealed the synchronous-response relationship of these two data types. First, we extracted geo-parcels from optical images with high spatial resolution. Second, we built a recurrent neural network (RNN)-based classifier suitable for remote sensing images on the geo-parcel scale. Third, we classified crops based on the two datasets and established the network. Fourth, we analyzed the synchronous response relationships of crops based on the results of the two classification schemes. This work is the basis for the application of remote sensing data for the fine mapping and growth monitoring of crop planting structures in cloudy and rainy areas in the future.

Combining Deep Learning and Prior Knowledge for Crop Mapping in Tropical Regions from Multitemporal SAR Image Sequences

Accurate crop type identification and crop area estimation from remote sensing data in tropical regions are still considered challenging tasks. The more favorable weather conditions, in comparison to the characteristic conditions of temperate regions, permit higher flexibility in land use, planning, and management, which implies complex crop dynamics. Moreover, the frequent cloud cover prevents the use of optical data during large periods of the year, making SAR data an attractive alternative for crop mapping in tropical regions. This paper evaluates the effectiveness of Deep Learning (DL) techniques for crop recognition from multi-date SAR images from tropical regions. Three DL strategies are investigated: autoencoders, convolutional neural networks, and fully-convolutional networks. The paper further proposes a post-classification technique to enforce prior knowledge about crop dynamics in the target area. Experiments conducted on a Sentinel-1 multitemporal sequence of a tropical region in Brazil reveal the pros and cons of the tested methods. In our experiments, the proposed crop dynamics model was able to correct up to 16.5% of classification errors and managed to improve the performance up to 3.2% and 8.7% in terms of overall accuracy and average F1-score, respectively.

Remote Sensing for the Quantification of Land Surface Dynamics in Large River Delta Regions—A Review

River deltas and estuaries belong to the most significant coastal landforms on our planet and are usually very densely populated. Nearly 600 million people live in river deltas, benefiting from the large variety of locational advantages and rich resources. Deltas are highly dynamic and vulnerable environments that are exposed to a wide range of natural and manmade threats. Sustainable management of river deltas therefore requires a holistic assessment of historic and recent ongoing changes and the dynamics in settlement sprawl, land cover and land use change, ecosystem development, as well as river and coastline geomorphology, all of which is difficult to achieve solely with traditional land-based surveying techniques. This review paper presents the potential of Earth Observation for analyses and quantification of land surface dynamics in the large river deltas globally, emphasizing the different geo-information products that can be derived from medium resolution, high resolution and highest resolution optical, multispectral, thermal and SAR data. Over 200 journal papers on remote sensing related studies for large river deltas and estuaries have been analyzed and categorized into thematic fields such as river course morphology, coastline changes, erosion and accretion processes, flood and inundation dynamics, regional land cover and land use dynamics, as well as the monitoring of compliance with respect to anthropogenic activity such as industry expansion-related habitat destruction. Additionally, our own exemplary analyses are interwoven into the review to visualize related delta work.

Crop Monitoring Using Sentinel-1 Data: A Case Study from The Netherlands

Agriculture is of huge economic significance in The Netherlands where the provision of real-time, reliable information on crop development is essential to support the transition towards precision agriculture. Optical imagery can provide invaluable insights into crop growth and development but is severely hampered by cloud cover. This case study in the Flevopolder illustrates the potential value of Sentinel-1 for monitoring five key crops in The Netherlands, namely sugar beet, potato, maize, wheat and English rye grass. Time series of radar backscatter from the European Space Agency’s Sentinel-1 Mission are analyzed and compared to ground measurements including phenological stage and height. Temporal variations in backscatter data reflect changes in water content and structure associated with phenological development. Emergence and closure dates are estimated from the backscatter time series and validated against a photo archive. Coherence data are compared to Normalized Difference Vegetation Index (NDVI) and ground data, illustrating that the sudden increase in coherence is a useful indicator of harvest. The results presented here demonstrate that Sentinel-1 data have significant potential value to monitor growth and development of key Dutch crops. Furthermore, the guaranteed availability of Sentinel-1 imagery in clouded conditions ensures the reliability of data to meet the monitoring needs of farmers, food producers and regulatory bodies.

Mapping Irrigated Areas Using Sentinel-1 Time Series in Catalonia, Spain

Mapping irrigated plots is essential for better water resource management. Today, the free and open access Sentinel-1 (S1) and Sentinel-2 (S2) data with high revisit time offers a powerful tool for irrigation mapping at plot scale. Up to date, few studies have used S1 and S2 data to provide approaches for mapping irrigated plots. This study proposes a method to map irrigated plots using S1 SAR (synthetic aperture radar) time series. First, a dense temporal series of S1 backscattering coefficients were obtained at plot scale in VV (Vertical-Vertical) and VH (Vertical-Horizontal) polarizations over a study site located in Catalonia, Spain. In order to remove the ambiguity between rainfall and irrigation events, the S1 signal obtained at plot scale was used conjointly to S1 signal obtained at a grid scale (10 km × 10 km). Later, two mathematical transformations, including the principal component analysis (PCA) and the wavelet transformation (WT), were applied to the several SAR temporal series obtained in both VV and VH polarization. Irrigated areas were then classified using the principal component (PC) dimensions and the WT coefficients in two different random forest (RF) classifiers. Another classification approach using one dimensional convolutional neural network (CNN) was also performed on the obtained S1 temporal series. The results derived from the RF classifiers with S1 data show high overall accuracy using the PC values (90.7%) and the WT coefficients (89.1%). By applying the CNN approach on SAR data, a significant overall accuracy of 94.1% was obtained. The potential of optical images to map irrigated areas by the mean of a normalized differential vegetation index (NDVI) temporal series was also tested in this study in both the RF and the CNN approaches. The overall accuracy obtained using the NDVI in RF classifier reached 89.5% while that in the CNN reached 91.6%. The combined use of optical and radar data slightly enhanced the classification in the RF classifier but did not significantly change the accuracy obtained in the CNN approach using S1 data.

DCN-Based Spatial Features for Improving Parcel-Based Crop Classification Using High-Resolution Optical Images and Multi-Temporal SAR Data

Spatial features retrieved from satellite data play an important role for improving crop classification. In this study, we proposed a deep-learning-based time-series analysis method to extract and organize spatial features to improve parcel-based crop classification using high-resolution optical images and multi-temporal synthetic aperture radar (SAR) data. Central to this method is the use of multiple deep convolutional networks (DCNs) to extract spatial features and to use the long short-term memory (LSTM) network to organize spatial features. First, a precise farmland parcel map was delineated from optical images. Second, hundreds of spatial features were retrieved using multiple DCNs from preprocessed SAR images and overlaid onto the parcel map to construct multivariate time-series of crop growth for parcels. Third, LSTM-based network structures for organizing these time-series features were constructed to produce a final parcel-based classification map. The method was applied to a dataset of high-resolution ZY-3 optical images and multi-temporal Sentinel-1A SAR data to classify crop types in the Hunan Province of China. The classification results, showing an improvement of greater than 5.0% in overall accuracy relative to methods without spatial features, demonstrated the effectiveness of the proposed method in extracting and organizing spatial features for improving parcel-based crop classification.