Improved Early Crop Type Identification By Joint Use of High Temporal Resolution SAR And Optical Image Time Series

Radar vegetation indices for monitoring surface vegetation: Developments, challenges, and trends

Monitoring surface vegetation is essential for environmental protection, disaster prevention, and carbon sequestration in forests. However, optical remote-sensing methods and their derivative technologies typically fail to fully meet this requirement due to constraints such as lighting and weather. Radar vegetation indices (RVIs), developed based on microwave remote-sensing data, describe the dielectric properties and morphological structure of vegetation and have been applied for vegetation monitoring at various scales. This technical review is the first to systematically summarize RVIs; it analyzes and discusses their principles, developments, categories and applications, and provides a comprehensive guide for their use. Additionally, the challenges faced by RVIs, as well as their applicability, were analyzed, and future improvements and development trends were carefully projected. The selection of RVIs must consider the type of data used, the terrain and location of the study area, and the major vegetation types. The effectiveness of RVIs applied to vegetation monitoring can be affected by various factors, including index performance, sensor type, study area, and data type and quality. These factors reduce the reliability and robustness of results, as well as guide the improvement direction of RVIs. The development of technologies, such as artificial intelligence, in remote sensing offers new possibilities for RVIs, enabling the removal of background scattering, improvement in interpretation accuracy, and reduction in application thresholds. Additionally, the development trends in high resolution, multi-polarization, multi-base, multi-dimensional, and networked synthetic aperture radar (SAR) and their satellite platforms offer data support for the next generation of RVIs. The rapid development of RVIs strongly supports the use of surface vegetation monitoring and terrestrial ecosystem research.

Use of Optical and Radar Imagery for Crop Type Classification in Africa: A Review

Multi-source remote sensing-derived information on crops contributes significantly to agricultural monitoring, assessment, and management. In Africa, some challenges (i.e., small-scale farming practices associated with diverse crop types and agricultural system complexity, and cloud coverage during the growing season) can imped agricultural monitoring using multi-source remote sensing. The combination of optical remote sensing and synthetic aperture radar (SAR) data has emerged as an opportune strategy for improving the precision and reliability of crop type mapping and monitoring. This work aims to conduct an extensive review of the challenges of agricultural monitoring and mapping in Africa in great detail as well as the current research progress of agricultural monitoring based on optical and Radar satellites. In this context optical data may provide high spatial resolution and detailed spectral information, which allows for the differentiation of different crop types based on their spectral signatures. However, synthetic aperture radar (SAR) satellites can provide important contributions given the ability of this technology to penetrate cloud cover, particularly in African tropical regions, as opposed to optical data. This review explores various combination techniques employed to integrate optical and SAR data for crop type classification and their applicability and limitations in the context of African countries. Furthermore, challenges are discussed in this review as well as and the limitations associated with optical and SAR data combination, such as the data availability, sensor compatibility, and the need for accurate ground truth data for model training and validation. This study also highlights the potential of advanced modelling (i.e., machine learning algorithms, such as support vector machines, random forests, and convolutional neural networks) in improving the accuracy and automation of crop type classification using combined data. Finally, this review concludes with future research directions and recommendations for utilizing optical and SAR data combination techniques in crop type classification for African agricultural systems. Furthermore, it emphasizes the importance of developing robust and scalable classification models that can accommodate the diversity of crop types, farming practices, and environmental conditions prevalent in Africa. Through the utilization of combined remote sensing technologies, informed decisions can be made to support sustainable agricultural practices, strengthen nutritional security, and contribute to the socioeconomic development of the continent.

A twenty-year dataset of high-resolution maize distribution in China

China is the world's second-largest maize producer, contributing 23% to global production and playing a crucial role in stabilizing the global maize supply. Therefore, accurately mapping the maize distribution in China is of great significance for regional and global food security and international cereals trade. However, it still lacks a long-term maize distribution dataset with fine spatial resolution, because the existing high spatial resolution satellite datasets suffer from data gaps caused by cloud cover, especially in humid and cloudy regions. This study aimed to produce a long-term, high-resolution maize distribution map for China (China Crop Dataset-Maize, CCD-Maize) identifying maize in 22 provinces and municipalities from 2001 to 2020. The map was produced using a high spatiotemporal resolution fused dataset and a phenology-based method called Time-Weighted Dynamic Time Warping. A validation based on 54,281 field survey samples with a 30-m resolution showed that the average user's accuracy and producer's accuracy of CCD-Maize were 77.32% and 80.98%, respectively, and the overall accuracy was 80.06% over all 22 provinces.

Cropland Mapping Using Sentinel-1 Data in the Southern Part of the Russian Far East

Crop identification is one of the most important tasks in digital farming. The use of remote sensing data makes it possible to clarify the boundaries of fields and identify fallow land. This study considered the possibility of using the seasonal variation in the Dual-polarization Radar Vegetation Index (DpRVI), which was calculated based on data acquired by the Sentinel-1B satellite between May and October 2021, as the main characteristic. Radar images of the Khabarovskiy District of the Khabarovsk Territory, as well as those of the Arkharinskiy, Ivanovskiy, and Oktyabrskiy districts in the Amur Region (Russian Far East), were obtained and processed. The identifiable classes were soybean and oat crops, as well as fallow land. Classification was carried out using the Support Vector Machines, Quadratic Discriminant Analysis (QDA), and Random Forest (RF) algorithms. The training (848 ha) and test (364 ha) samples were located in Khabarovskiy District. The best overall accuracy on the test set (82.0%) was achieved using RF. Classification accuracy at the field level was 79%. When using the QDA classifier on cropland in the Amur Region (2324 ha), the overall classification accuracy was 83.1% (F1 was 0.86 for soybean, 0.84 for fallow, and 0.79 for oat). Application of the Radar Vegetation Index (RVI) and VV/VH ratio enabled an overall classification accuracy in the Amur region of 74.9% and 74.6%, respectively. Thus, using DpRVI allowed us to achieve greater performance compared to other SAR data, and it can be used to identify crops in the south of the Far East and serve as the basis for the automatic classification of cropland.

Rapid and Automated Approach for Early Crop Mapping Using Sentinel-1 and Sentinel-2 on Google Earth Engine; A Case of a Highly Heterogeneous and Fragmented Agricultural Region

Accurate and rapid crop type mapping is critical for agricultural sustainability. The growing trend of cloud-based geospatial platforms provides rapid processing tools and cloud storage for remote sensing data. In particular, a variety of remote sensing applications have made use of publicly accessible data from the Sentinel missions of the European Space Agency (ESA). However, few studies have employed these data to evaluate the effectiveness of Sentinel-1, and Sentinel-2 spectral bands and Machine Learning (ML) techniques in challenging highly heterogeneous and fragmented agricultural landscapes using the Google Earth Engine (GEE) cloud computing platform. This work aims to map, accurately and early, the crop types in a highly heterogeneous and fragmented agricultural region of the Tadla Irrigated Perimeter (TIP) as a case study using the high spatiotemporal resolution of Sentinel-1, Sentinel-2, and a Random Forest (RF) classifier implemented on GEE. More specifically, five experiments were performed to assess the optical band reflectance values, vegetation indices, and SAR backscattering coefficients on the accuracy of crop classification. Besides, two scenarios were used to assess the monthly temporal windows on classification accuracy. The findings of this study show that the fusion of Sentinel-1 and Sentinel-2 data can accurately produce the early crop mapping of the studied area with an Overall Accuracy (OA) reaching 95.02%. The scenarios prove that the monthly time series perform better in terms of classification accuracy than single monthly windows images. Red-edge and shortwave infrared bands can improve the accuracy of crop classification by 1.72% when compared to only using traditional bands (i.e., visible and near-infrared bands). The inclusion of two common vegetation indices (The Normalized Vegetation Index (NDVI), the Enhanced Vegetation Index (EVI)) and Sentinel-1 backscattering coefficients to the crop classification enhanced the overall classification accuracy by 0.02% and 2.94%, respectively, compared to using the Sentinel-2 reflectance bands alone. The monthly windows analysis indicated that the improvement in the accuracy of crop classification is the greatest when the March images are accessible, with an OA higher than 80%.

An Innovative Fusion-Based Scenario for Improving Land Crop Mapping Accuracy

The accuracy of land crop maps obtained from satellite images depends on the type of feature selection algorithm and classifier. Each of these algorithms have different efficiency in different conditions; therefore, developing a suitable strategy for combining the capabilities of different algorithms in preparing a land crop map with higher accuracy can be very useful. The objective of this study was to develop a fusion-based framework for improving land crop mapping accuracy. First, the features were retrieved using the Sentinel 1, Sentinel 2, and Landsat-8 imagery. Then, training data and various feature selection algorithms including recursive feature elimination (RFE), random forest (RF), and Boruta were used for optimal feature selection. Various classifiers, including artificial neural network (ANN), support vector machine (SVM), and RF, were implemented to create maps of land crops relying on optimal features and training data. After that, in order to increase the result accuracy, maps of land crops derived from several scenarios were fused using a fusion-based voting strategy at the level of decision, and new maps of land crops and classification uncertainty maps were prepared. Subsequently, the performance of different scenarios was evaluated and compared. Among the feature selection algorithms, RF accuracy was higher than RFE and Boruta. Moreover, the efficiency of RF was higher than SVM and ANN. The overall accuracy of the voting scenario was higher than all other scenarios. The finding of this research demonstrated that combining the features' capabilities extracted from sensors in different spectral ranges, different feature selection algorithms, and classifiers improved the land crop classification accuracy.

Surface biophysical features fusion in remote sensing for improving land crop/cover classification accuracy

Preparing up-to-date land crop/cover maps is important to study in order to achieve food security. Therefore, the aim of this study was to evaluate the impact of surface biophysical features in the land crop/cover classification accuracy and introduce a new fusion-based method with more accurate results for land crop/cover classification. For this purpose, multi-temporal images from Sentinel 1 and 2, and an actual land crop map prepared by Agriculture and Agri-Food Canada (AAFC) in 2019 were used for 3 test sites in Ontario, Canada. Firstly, surface biophysical features maps were prepared based on spectral indices from Sentinel 2 including Normalized Difference Vegetation Index (NDVI), Index-based Built-up Index (IBI), Wetness, Albedo, and Brightness and co-polarization (VV) and cross-polarization (VH) from Sentinel 1 for different dates. Then, different scenarios were generated; these included single surface biophysical features as well as a combination of several surface biophysical features. Secondly, land crop/cover maps were prepared for each scenario based on the Random Forest (RF). In the third step, based on the voting strategy, classification maps from different scenarios were combined. Finally, the accuracy of the land crop/cover maps obtained from each of the scenario was evaluated. The results showed that the average overall accuracy of land crop/cover maps obtained from individual scenario (one feature) including NDVI, IBI, Wetness, Albedo, Brightness, VV and VH were 66%, 68%, 63%, 60%, 57%, 62% and 58%, respectively, which by the surface biophysical features fusion, the overall accuracy of land crop/cover maps increased to 83%. Also, by combining the classification results obtained from different scenarios based on voting strategy, the overall accuracy increased to 89%. The results of this study indicate that the feature level-based fusion of surface biophysical features and decision level based fusion of land crop/cover maps obtained from various scenarios increases the accuracy of land crop/cover classification.

Deep Temporal Iterative Clustering for Satellite Image Time Series Land Cover Analysis

The extensive amount of Satellite Image Time Series (SITS) data brings new opportunities and challenges for land cover analysis. Many supervised machine learning methods have been applied in SITS, but the labeled SITS samples are time- and effort-consuming to acquire. It is necessary to analyze SITS data with an unsupervised learning method. In this paper, we propose a new unsupervised learning method named Deep Temporal Iterative Clustering (DTIC) to deal with SITS data. The proposed method jointly learns a neural network’s parameters and the resulting features’ cluster assignments, which uses a standard clustering algorithm, K-means, to iteratively cluster the features produced by the feature extraction network and then uses the subsequent assignments as supervision to update the network’s weights. We apply DTIC to the unsupervised training of neural networks on both SITS datasets. Experimental results demonstrate that DTIC outperforms the state-of-the-art K-means clustering algorithm, which proves that the proposed approach successfully provides a novel idea for unsupervised training of SITS data.

Research on remote sensing classification of fruit trees based on Sentinel-2 multi-temporal imageries

Accurately obtaining the spatial distribution information of fruit tree planting is of great significance to the development of fruit tree growth monitoring, disease and pest control, and yield estimation. In this study, the Sentenel-2 multispectral remote sensing imageries of different months during the growth period of the fruit trees were used as the data source, and single month vegetation indices, accumulated monthly vegetation indices (∑VIs), and difference vegetation indices between adjacent months (∆VIs) were constructed as input variables. Four conventional vegetation indices of NDVI, PSRI, GNDVI, and RVI and four improved vegetation indices of NDVIre1, NDVIre2, NDVIre3, and NDVIre4 based on the red-edge band were selected to construct a decision tree classification model combined with machine learning technology. Through the analysis of vegetation indices under different treatments and different months, combined with the attribute of Feature_importances_, the vegetation indices of different periods with high contribution were selected as input features, and the Max_depth values of the decision tree model were determined by the hyperparameter learning curve. The results have shown that when the Max_depth value of the decision tree model of the vegetation indices under the three treatments was 6, 8, and 8, the model classification was the best. The accuracy of the three vegetation index processing models on the training set were 0.8936, 0.9153, and 0.8887, and the accuracy on the test set were 0.8355, 0.7611, and 0.7940, respectively. This method could be applied to remote sensing classification of fruit trees in a large area, and could provide effective technical means for monitoring fruit tree planting areas with medium and high resolution remote sensing imageries.

Mapping Crop Types of Germany by Combining Temporal Statistical Metrics of Sentinel-1 and Sentinel-2 Time Series with LPIS Data

Nationwide and consistent information on agricultural land use forms an important basis for sustainable land management maintaining food security, (agro)biodiversity, and soil fertility, especially as German agriculture has shown high vulnerability to climate change. Sentinel-1 and Sentinel-2 satellite data of the Copernicus program offer time series with temporal, spatial, radiometric, and spectral characteristics that have great potential for mapping and monitoring agricultural crops. This paper presents an approach which synergistically uses these multispectral and Synthetic Aperture Radar (SAR) time series for the classification of 17 crop classes at 10 m spatial resolution for Germany in the year 2018. Input data for the Random Forest (RF) classification are monthly statistics of Sentinel-1 and Sentinel-2 time series. This approach reduces the amount of input data and pre-processing steps while retaining phenological information, which is crucial for crop type discrimination. For training and validation, Land Parcel Identification System (LPIS) data were available covering 15 of the 16 German Federal States. An overall map accuracy of 75.5% was achieved, with class-specific F1-scores above 80% for winter wheat, maize, sugar beet, and rapeseed. By combining optical and SAR data, overall accuracies could be increased by 6% and 9%, respectively, compared to single sensor approaches. While no increase in overall accuracy could be achieved by stratifying the classification in natural landscape regions, the class-wise accuracies for all but the cereal classes could be improved, on average, by 7%. In comparison to census data, the crop areas could be approximated well with, on average, only 1% of deviation in class-specific acreages. Using this streamlined approach, similar accuracies for the most widespread crop types as well as for smaller permanent crop classes were reached as in other Germany-wide crop type studies, indicating its potential for repeated nationwide crop type mapping.

Two-Stepwise Hierarchical Adaptive Threshold Method for Automatic Rapeseed Mapping over Jiangsu Using Harmonized Landsat/Sentinel-2

Rapeseed distribution mapping is a crucial issue for food and oil security, entertainment, and tourism development. Previous studies have used various remote sensing approaches to map rapeseed. However, the time-consuming and labor-intensive sample data used in these supervised classification methods greatly limit the development of large-scale mapping in rapeseed studies. Regarding threshold methods, some empirical thresholding methods still need sample data to select the optimal threshold value, and their accuracies decrease when a fixed threshold is applied in complex and diverse environments. This study first developed the Normalized Difference Rapeseed Index (NDRI), defined as the difference in green and short-wave infrared bands divided by their sum, to find a suitable feature to distinguish rapeseed from other types of crops. Next, a two-stepwise hierarchical adaptive thresholding (THAT) algorithm requiring no training data was used to automatically extract rapeseed in Xinghua. Finally, two adaptive thresholding methods of the standalone Otsu and Otsu with Canny Edge Detection (OCED) were used to extract rapeseed across Jiangsu province. The results show that (1) NDRI can separate rapeseed from other vegetation well; (2) the OCED-THAT method can accurately map rapeseed in Jiangsu with an overall accuracy (OA) of 0.9559 and a Kappa coefficient of 0.8569, and it performed better than the Otsu-THAT method; (3) the OCED-THAT method had a lower but acceptable accuracy than the Random Forest method (OA = 0.9806 and Kappa = 0.9391). This study indicates that the THAT model is a promising automatic method for mapping rapeseed.

High-Resolution Mapping of Winter Cereals in Europe by Time Series Landsat and Sentinel Images for 2016–2020

Winter cereals, including wheat, rye, barley, and triticale, are important food crops, and it is crucial to identify the distribution of winter cereals for monitoring crop growth and predicting yield. The production and plating area of winter cereals in Europe both contribute 12.57% to the total global cereal production and plating area in 2020. However, the distribution maps of winter cereals with high spatial resolution are scarce in Europe. Here, we first used synthetic aperture radar (SAR) data from Sentinel-1 A/B, in the Interferometric Wide (IW) swath mode, to distinguish rapeseed and winter cereals; we then used a time-weighted dynamic time warping (TWDTW) method to discriminate winter cereals from other crops by comparing the similarity of seasonal changes in the Normalized Difference Vegetation Index (NDVI) from Landsat and Sentinel-2 images. We generated winter cereal maps for 2016–2020 that cover 32 European countries with 30 m spatial resolution. Validation using field samples obtained from the Google Earth Engine (GEE) platform show that the producer’s and user’s accuracies are 91% ± 7.8% and 89% ± 10.3%, respectively, averaged over 32 countries in Europe. The winter cereal map agrees well with agricultural census data for planted winter cereal areas at municipal and country levels, with the averaged coefficient of determination R2 as 0.77 ± 0.15 for 2016–2019. In addition, our method can identify the distribution of winter cereals two months before harvest, with an overall accuracy of 88.4%, indicating that TWDTW is an effective method for timely crop growth monitoring and identification at the continent level. The winter cereal maps in Europe are available via an open-data repository.

Performance and the Optimal Integration of Sentinel-1/2 Time-Series Features for Crop Classification in Northern Mongolia

Accurate and early crop-type maps are essential for agricultural policy development and food production assessment at regional and national levels. This study aims to produce a crop-type map with acceptable accuracy and spatial resolution in northern Mongolia by optimizing the combination of Sentinel-1 (S1) and Sentinel-2 (S2) images with the Google Earth Engine (GEE) environment. A total of three satellite data combination scenarios are set, including S1 alone, S2 alone, and the combination of S1 and S2. In order to avoid the impact of data gaps caused by clouds on crop classification, this study reconstructed the time series of S1 and S2 with a 10-day interval using the median composite method, linear moving interpolation, and Savitzky–Golay (SG) filter. Our results indicated that crop-type classification accuracy increased with the increase in data length to all three data combination scenarios. S2 alone has higher accuracy than S1 alone and the combination of S1 and S2. The crop-type map with the highest accuracy was generated using S2 data from 150 days of the year (DOY) (11 May) to 260 DOY (18 September). The OA and kappa were 0.93 and 0.78, respectively, and the F1-score for spring wheat and rapeseed were 0.96 and 0.80, respectively. The classification accuracy of the crop increased rapidly from 210 DOY (end of July) to 260 DOY (August to mid-September), and then it remained stable after 260 DOY. Based on our analysis, we filled the gap of the crop-type map with 10 m spatial resolution in northern Mongolia, revealing the best satellite combination and the best period for crop-type classification, which can benefit the achievement of sustainable development goals 2 (SDGs2).

Spatial Transferability of Random Forest Models for Crop Type Classification Using Sentinel-1 and Sentinel-2

Large-scale crop type mapping often requires prediction beyond the environmental settings of the training sites. Shifts in crop phenology, field characteristics, or ecological site conditions in the previously unseen area, may reduce the classification performance of machine learning classifiers that often overfit to the training sites. This study aims to assess the spatial transferability of Random Forest models for crop type classification across Germany. The effects of different input datasets, i.e., only optical, only Synthetic Aperture Radar (SAR), and optical-SAR data combination, and the impact of spatial feature selection were systematically tested to identify the optimal approach that shows the highest accuracy in the transfer region. The spatial feature selection, a feature selection approach combined with spatial cross-validation, should remove features that carry site-specific information in the training data, which in turn can reduce the accuracy of the classification model in previously unseen areas. Seven study sites distributed over Germany were analyzed using reference data for the major 11 crops grown in the year 2018. Sentinel-1 and Sentinel-2 data from October 2017 to October 2018 were used as input. The accuracy estimation was performed using the spatially independent sample sets. The results of the optical-SAR combination outperformed those of single sensors in the training sites (maximum F1-score–0.85), and likewise in the areas not covered by training data (maximum F1-score–0.79). Random forest models based on only SAR features showed the lowest accuracy losses when transferred to unseen regions (average F1loss–0.04). In contrast to using the entire feature set, spatial feature selection substantially reduces the number of input features while preserving good predictive performance on unseen sites. Altogether, applying spatial feature selection to a combination of optical-SAR features or using SAR-only features is beneficial for large-scale crop type classification where training data is not evenly distributed over the complete study region.

An Interannual Transfer Learning Approach for Crop Classification in the Hetao Irrigation District, China

Crop type classification is critical for crop production estimation and optimal water allocation. Crop type data are challenging to generate if crop reference data are lacking, especially for target years with reference data missed in collection. Is it possible to transfer a trained crop type classification model to retrace the historical spatial distribution of crop types? Taking the Hetao Irrigation District (HID) in China as the study area, this study first designed a 10 m crop type classification framework based on the Google Earth Engine (GEE) for crop type mapping in the current season. Then, its interannual transferability to accurately retrace historical crop distributions was tested. The framework used Sentinel-1/2 data as the satellite data source, combined percentile, and monthly composite approaches to generate classification metrics and employed a random forest classifier with 300 trees for crop classification. Based on the proposed framework, this study first developed a 10 m crop type map of the HID for 2020 with an overall accuracy (OA) of 0.89 and then obtained a 10 m crop type map of the HID for 2019 with an OA of 0.92 by transferring the trained model for 2020 without crop reference samples. The results indicated that the designed framework could effectively identify HID crop types and have good transferability to obtain historical crop type data with acceptable accuracy. Our results found that SWIR1, Green, and Red Edge2 were the top three reflectance bands for crop classification. The land surface water index (LSWI), normalized difference water index (NDWI), and enhanced vegetation index (EVI) were the top three vegetation indices for crop classification. April to August was the most suitable time window for crop type classification in the HID. Sentinel-1 information played a positive role in the interannual transfer of the trained model, increasing the OA from 90.73% with Sentinel 2 alone to 91.58% with Sentinel-1 and Sentinel-2 together.

Designing a European-Wide Crop Type Mapping Approach Based on Machine Learning Algorithms Using LUCAS Field Survey and Sentinel-2 Data

One of the most challenging aspects of obtaining detailed and accurate land-use and land-cover (LULC) maps is the availability of representative field data for training and validation. In this manuscript, we evaluate the use of the Eurostat Land Use and Coverage Area frame Survey (LUCAS) 2018 data to generate a detailed LULC map with 19 crop type classes and two broad categories for woodland and shrubland, and grassland. The field data were used in combination with Copernicus Sentinel-2 (S2) satellite data covering Europe. First, spatially and temporally consistent S2 image composites of (1) spectral reflectances, (2) a selection of spectral indices, and (3) several bio-geophysical indicators were created for the year 2018. From the large number of features, the most important were selected for classification using two machine-learning algorithms (support vector machine and random forest). Results indicated that the 19 crop type classes and the two broad categories could be classified with an overall accuracy (OA) of 77.6%, using independent data for validation. Our analysis of three methods to select optimum training data showed that by selecting the most spectrally different pixels for training data, the best OA could be achieved, and this already using only 11% of the total training data. Comparing our results to a similar study using Sentinel-1 (S1) data indicated that S2 can achieve slightly better results, although the spatial coverage was slightly reduced due to gaps in S2 data. Further analysis is ongoing to leverage synergies between optical and microwave data.

Density Estimates as Representations of Agricultural Fields for Remote Sensing-Based Monitoring of Tillage and Vegetation Cover

We consider the use of remote sensing for large-scale monitoring of agricultural land use, focusing on classification of tillage and vegetation cover for individual field parcels across large spatial areas. From the perspective of remote sensing and modelling, field parcels are challenging as objects of interest due to highly varying shape and size but relatively uniform pixel content and texture. To model such areas we need representations that can be reliably estimated already for small parcels and that are invariant to the size of the parcel. We propose representing the parcels using density estimates of remote imaging pixels and provide a computational pipeline that combines the representation with arbitrary supervised learning algorithms, while allowing easy integration of multiple imaging sources. We demonstrate the method in the task of the automatic monitoring of autumn tillage method and vegetation cover of Finnish crop fields, based on the integrated analysis of intensity of Synthetic Aperture Radar (SAR) polarity bands of the Sentinel-1 satellite and spectral indices calculated from Sentinel-2 multispectral image data. We use a collection of 127,757 field parcels monitored in April 2018 and annotated to six tillage method and vegetation cover classes, reaching 70% classification accuracy for test parcels when using both SAR and multispectral data. Besides this task, the method could also directly be applied for other agricultural monitoring tasks, such as crop yield prediction.

Land Cover Changes after the Massive Rohingya Refugee Influx in Bangladesh: Neo-Classic Unsupervised Approach

The Rohingya refugee influx to Bangladesh in 2017 was a historical incident; the number of refugees was so massive that significant impacts to local communities was inevitable. The Bangladesh government provided land in a preserved area for constructing makeshift camps for the refugees. Previous studies have revealed the land cover changes and impacts of the refugee influx around campsites, especially with regard to local forest resources. Our aim is to establish a convenient approach of providing up-to-date information to monitor holistic local situations. We employed a classic unsupervised technique—a combination of k-means clustering and maximum likelihood estimation—with the latest rich time-series satellite images of Sentinal-1 and Sentinal-2. A combination of VV and normalized difference water index (NDWI) images was successful in identifying built-up/disturbed areas, and a combination of VH and NDWI images was successful in differentiating wetland/saltpan, agriculture /open field, degraded forest/bush, and forest areas. By doing this, we provided annual land cover classification maps for the entire Teknaf peninsula for the pre- and post-influx periods with both fair quality and without prior training data. Our analyses revealed that on-going impacts were still observed by May 2021. As a simple estimation of the intervention consequence, the built-up/disturbed areas increased 6825 ha (compared with the 2015–17 period). However, while the impacts on the original forest were not found to be significant, the degraded forest/bush areas were largely degraded by 4606 ha. These cultivated lands would be used for agricultural activities. This is in line with the reported farmers’ increased income, despite local people with other occupations that are all equally facing the decreases in income. The convenience of our unsupervised classification approach would help keep accumulating a time-series land cover classification, which is important in monitoring impacts on local communities.

Synergy of Sentinel-1 and Sentinel-2 Imagery for Early Seasonal Agricultural Crop Mapping

The exploitation of the unprecedented capacity of Sentinel-1 (S1) and Sentinel-2 (S2) data offers new opportunities for crop mapping. In the framework of the SenSAgri project, this work studies the synergy of very high-resolution Sentinel time series to produce accurate early seasonal binary cropland mask and crop type map products. A crop classification processing chain is proposed to address the following: (1) high dimensionality challenges arising from the explosive growth in available satellite observations and (2) the scarcity of training data. The two-fold methodology is based on an S1-S2 classification system combining the so-called soft output predictions of two individually trained classifiers. The performances of the SenSAgri processing chain were assessed over three European test sites characterized by different agricultural systems. A large number of highly diverse and independent data sets were used for validation experiments. The agreement between independent classification algorithms of the Sentinel data was confirmed through different experiments. The presented results assess the interest of decision-level fusion strategies, such as the product of experts. Accurate crop map products were obtained over different countries in the early season with limited training data. The results highlight the benefit of fusion for early crop mapping and the interest of detecting cropland areas before the identification of crop types.

The impact of conflict-driven cropland abandonment on food insecurity in South Sudan revealed using satellite remote sensing

Armed conflicts often hinder food security through cropland abandonment and restrict the collection of on-the-ground information required for targeted relief distribution. Satellite remote sensing provides a means for gathering information about disruptions during armed conflicts and assessing the food security status in conflict zones. Using ~7,500 multisource satellite images, we implemented a data-driven approach that showed a reduction in cultivated croplands in war-ravaged South Sudan by 16% from 2016 to 2018. Propensity score matching revealed a statistical relationship between cropland abandonment and armed conflicts that contributed to drastic decreases in food supply. Our analysis shows that the abandoned croplands could have supported at least a quarter of the population in the southern states of South Sudan and demonstrates that remote sensing can play a crucial role in the assessment of cropland abandonment in food-insecure regions, thereby improving the basis for timely aid provision.

An integrated method for optimized identification of effective natural inhibitors against SARS-CoV-2 3CLpro

The current severe situation of coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has not been reversed and posed great threats to global health. Therefore, there is an urgent need to find out effective antiviral drugs. The 3-chymotrypsin-like protease (3CLpro) in SARS-CoV-2 serve as a promising anti-virus target due to its essential role in the regulation of virus reproduction. Here, we report an improved integrated approach to identify effective 3CLpro inhibitors from effective Chinese herbal formulas. With this approach, we identified the 5 natural products (NPs) including narcissoside, kaempferol-3-O-gentiobioside, rutin, vicenin-2 and isoschaftoside as potential anti-SARS-CoV-2 candidates. Subsequent molecular dynamics simulation additionally revealed that these molecules can be tightly bound to 3CLpro and confirmed effectiveness against COVID-19. Moreover, kaempferol-3-o-gentiobioside, vicenin-2 and isoschaftoside were first reported to have SARS-CoV-2 3CLpro inhibitory activity. In summary, this optimized integrated strategy for drug screening can be utilized in the discovery of antiviral drugs to achieve rapid acquisition of drugs with specific effects on antiviral targets.

Crop Type Mapping from Optical and Radar Time Series Using Attention-Based Deep Learning

Crop maps are key inputs for crop inventory production and yield estimation and can inform the implementation of effective farm management practices. Producing these maps at detailed scales requires exhaustive field surveys that can be laborious, time-consuming, and expensive to replicate. With a growing archive of remote sensing data, there are enormous opportunities to exploit dense satellite image time series (SITS), temporal sequences of images over the same area. Generally, crop type mapping relies on single-sensor inputs and is solved with the help of traditional learning algorithms such as random forests or support vector machines. Nowadays, deep learning techniques have brought significant improvements by leveraging information in both spatial and temporal dimensions, which are relevant in crop studies. The concurrent availability of Sentinel-1 (synthetic aperture radar) and Sentinel-2 (optical) data offers a great opportunity to utilize them jointly; however, optimizing their synergy has been understudied with deep learning techniques. In this work, we analyze and compare three fusion strategies (input, layer, and decision levels) to identify the best strategy that optimizes optical-radar classification performance. They are applied to a recent architecture, notably, the pixel-set encoder–temporal attention encoder (PSE-TAE) developed specifically for object-based classification of SITS and based on self-attention mechanisms. Experiments are carried out in Brittany, in the northwest of France, with Sentinel-1 and Sentinel-2 time series. Input and layer-level fusion competitively achieved the best overall F-score surpassing decision-level fusion by 2%. On a per-class basis, decision-level fusion increased the accuracy of dominant classes, whereas layer-level fusion improves up to 13% for minority classes. Against single-sensor baseline, multi-sensor fusion strategies identified crop types more accurately: for example, input-level outperformed Sentinel-2 and Sentinel-1 by 3% and 9% in F-score, respectively. We have also conducted experiments that showed the importance of fusion for early time series classification and under high cloud cover condition.

Integration of Sentinel 1 and Sentinel 2 Satellite Images for Crop Mapping

Crop identification is key to global food security. Due to the large scale of crop estimation, the science of remote sensing was able to do well in this field. The purpose of this study is to study the shortcomings and strengths of combined radar data and optical images to identify the type of crops in Tarom region (Iran). For this purpose, Sentinel 1 and Sentinel 2 images were used to create a map in the study area. The Sentinel 1 data came from Google Earth Engine’s (GEE) Level-1 Ground Range Detected (GRD) Interferometric Wide Swath (IW) product. Sentinel 1 radar observations were projected onto a standard 10-m grid in GRD output. The Sen2Cor method was used to mask for clouds and cloud shadows, and the Sentinel 2 Level-1C data was sourced from the Copernicus Open Access Hub. To estimate the purpose of classification, stochastic forest classification method was used to predict classification accuracy. Using seven types of crops, the classification map of the 2020 growth season in Tarom was prepared using 10-day Sentinel 2 smooth mosaic NDVI and 12-day Sentinel 1 back mosaic. Kappa coefficient of 0.75 and a maximum accuracy of 85% were reported in this study. To achieve maximum classification accuracy, it is recommended to use a combination of radar and optical data, as this combination increases the chances of examining the details compared to the single-sensor classification method and achieves more reliable information.

A Review of Crop Water Stress Assessment Using Remote Sensing

Currently, the world is facing high competition and market risks in improving yield, crop illness, and crop water stress. This could potentially be addressed by technological advancements in the form of precision systems, improvements in production, and through ensuring the sustainability of development. In this context, remote-sensing systems are fully equipped to address the complex and technical assessment of crop production, security, and crop water stress in an easy and efficient way. They provide simple and timely solutions for a diverse set of ecological zones. This critical review highlights novel methods for evaluating crop water stress and its correlation with certain measurable parameters, investigated using remote-sensing systems. Through an examination of previous literature, technologies, and data, we review the application of remote-sensing systems in the analysis of crop water stress. Initially, the study presents the relationship of relative water content (RWC) with equivalent water thickness (EWT) and soil moisture crop water stress. Evapotranspiration and sun-induced chlorophyll fluorescence are then analyzed in relation to crop water stress using remote sensing. Finally, the study presents various remote-sensing technologies used to detect crop water stress, including optical sensing systems, thermometric sensing systems, land-surface temperature-sensing systems, multispectral (spaceborne and airborne) sensing systems, hyperspectral sensing systems, and the LiDAR sensing system. The study also presents the future prospects of remote-sensing systems in analyzing crop water stress and how they could be further improved.

Mapping Maize Area in Heterogeneous Agricultural Landscape with Multi-Temporal Sentinel-1 and Sentinel-2 Images Based on Random Forest

Accurate estimation of crop area is essential to adjusting the regional crop planting structure and the rational planning of water resources. However, it is quite challenging to map crops accurately by high-resolution remote sensing images because of the ecological gradient and ecological convergence between crops and non-crops. The purpose of this study is to explore the combining application of high-resolution multi-temporal Sentinel-1 (S1) radar backscatter and Sentinel-2 (S2) optical reflectance images for maize mapping in highly complex and heterogeneous landscapes in the middle reaches of Heihe River, northwest China. We proposed a new two-step method of vegetation extraction and followed by maize extraction, that is, extract the vegetation-covered areas first to reduce the inter-class variance by using a Random Forest (RF) classifier based on S2 data, and then extract the maize distribution in the vegetation area by using another RF classifier based on S1 and/or S2 data. The results demonstrate that the vegetation extraction classifier successfully identified vegetation-covered regions with an overall accuracy above 96% in the study area, and the accuracy of the maize extraction classifier constructed by the combined multi-temporal S1 and S2 images is significantly improved compared with that S1 (alone) or S2 (alone), with an overall accuracy of 87.63%, F1_Score of 0.86, and Kappa coefficient of 0.75. In addition, with the introduction of multi-temporal S1 and/or S2 images in crop growing season, the constructed RF model is more beneficial to maize mapping.

Sentinel-1 and 2 Time-Series for Vegetation Mapping Using Random Forest Classification: A Case Study of Northern Croatia

Land-cover (LC) mapping in a morphologically heterogeneous landscape area is a challenging task since various LC classes (e.g., crop types in agricultural areas) are spectrally similar. Most research is still mostly relying on optical satellite imagery for these tasks, whereas synthetic aperture radar (SAR) imagery is often neglected. Therefore, this research assessed the classification accuracy using the recent Sentinel-1 (S1) SAR and Sentinel-2 (S2) time-series data for LC mapping, especially vegetation classes. Additionally, ancillary data, such as texture features, spectral indices from S1 and S2, respectively, as well as digital elevation model (DEM), were used in different classification scenarios. Random Forest (RF) was used for classification tasks using a proposed hybrid reference dataset derived from European Land Use and Coverage Area Frame Survey (LUCAS), CORINE, and Land Parcel Identification Systems (LPIS) LC database. Based on the RF variable selection using Mean Decrease Accuracy (MDA), the combination of S1 and S2 data yielded the highest overall accuracy (OA) of 91.78%, with a total disagreement of 8.22%. The most pertinent features for vegetation mapping were GLCM Mean and Variance for S1, NDVI, along with Red and SWIR band for S2, whereas the digital elevation model produced major classification enhancement as an input feature. The results of this study demonstrated that the aforementioned approach (i.e., RF using a hybrid reference dataset) is well-suited for vegetation mapping using Sentinel imagery, which can be applied for large-scale LC classifications.

Delineating Smallholder Maize Farms from Sentinel-1 Coupled with Sentinel-2 Data Using Machine Learning

Rural communities rely on smallholder maize farms for subsistence agriculture, the main driver of local economic activity and food security. However, their planted area estimates are unknown in most developing countries. This study explores the use of Sentinel-1 and Sentinel-2 data to map smallholder maize farms. The random forest (RF), support vector (SVM) machine learning algorithms and model stacking (ST) were applied. Results show that the classification of combined Sentinel-1 and Sentinel-2 data improved the RF, SVM and ST algorithms by 24.2%, 8.7%, and 9.1%, respectively, compared to the classification of Sentinel-1 data individually. Similarities in the estimated areas (7001.35 ± 1.2 ha for RF, 7926.03 ± 0.7 ha for SVM and 7099.59 ± 0.8 ha for ST) show that machine learning can estimate smallholder maize areas with high accuracies. The study concludes that the single-date Sentinel-1 data were insufficient to map smallholder maize farms. However, single-date Sentinel-1 combined with Sentinel-2 data were sufficient in mapping smallholder farms. These results can be used to support the generation and validation of national crop statistics, thus contributing to food security.

Potential of Hybrid CNN-RF Model for Early Crop Mapping with Limited Input Data

When sufficient time-series images and training data are unavailable for crop classification, features extracted from convolutional neural network (CNN)-based representative learning may not provide useful information to discriminate crops with similar spectral characteristics, leading to poor classification accuracy. In particular, limited input data are the main obstacles to obtain reliable classification results for early crop mapping. This study investigates the potential of a hybrid classification approach, i.e., CNN-random forest (CNN-RF), in the context of early crop mapping, that combines the automatic feature extraction capability of CNN with the superior discrimination capability of an RF classifier. Two experiments on incremental crop classification with unmanned aerial vehicle images were conducted to compare the performance of CNN-RF with that of CNN and RF with respect to the length of the time-series and training data sizes. When sufficient time-series images and training data were used for the classification, the accuracy of CNN-RF was slightly higher or comparable with that of CNN. In contrast, when fewer images and the smallest training data were used at the early crop growth stage, CNN-RF was substantially beneficial and the overall accuracy increased by maximum 6.7%p and 4.6%p in the two study areas, respectively, compared to CNN. This is attributed to its ability to discriminate crops from features with insufficient information using a more sophisticated classifier. The experimental results demonstrate that CNN-RF is an effective classifier for early crop mapping when only limited input images and training samples are available.

Outlier Detection at the Parcel-Level in Wheat and Rapeseed Crops Using Multispectral and SAR Time Series

This paper studies the detection of anomalous crop development at the parcel-level based on an unsupervised outlier detection technique. The experimental validation is conducted on rapeseed and wheat parcels located in Beauce (France). The proposed methodology consists of four sequential steps: (1) preprocessing of synthetic aperture radar (SAR) and multispectral images acquired using Sentinel-1 and Sentinel-2 satellites, (2) extraction of SAR and multispectral pixel-level features, (3) computation of parcel-level features using zonal statistics and (4) outlier detection. The different types of anomalies that can affect the studied crops are analyzed and described. The different factors that can influence the outlier detection results are investigated with a particular attention devoted to the synergy between Sentinel-1 and Sentinel-2 data. Overall, the best performance is obtained when using jointly a selection of Sentinel-1 and Sentinel-2 features with the isolation forest algorithm. The selected features are co-polarized (VV) and cross-polarized (VH) backscattering coefficients for Sentinel-1 and five Vegetation Indexes for Sentinel-2 (among us, the Normalized Difference Vegetation Index and two variants of the Normalized Difference Water). When using these features with an outlier ratio of 10%, the percentage of detected true positives (i.e., crop anomalies) is equal to 94.1% for rapeseed parcels and 95.5% for wheat parcels.

Mapping Seasonal Agricultural Land Use Types Using Deep Learning on Sentinel-2 Image Time Series

The size and location of agricultural fields that are in active use and the type of use during the growing season are among the vital information that is needed for the careful planning and forecasting of agricultural production at national and regional scales. In areas where such data are not readily available, an independent seasonal monitoring method is needed. Remote sensing is a widely used tool to map land use types, although there are some limitations that can partly be circumvented by using, among others, multiple observations, careful feature selection and appropriate analysis methods. Here, we used Sentinel-2 satellite image time series (SITS) over the land area of Norway to map three agricultural land use classes: cereal crops, fodder crops (grass) and unused areas. The Multilayer Perceptron (MLP) and two variants of the Convolutional Neural Network (CNN), are implemented on SITS data of four different temporal resolutions. These enabled us to compare twelve model-dataset combinations to identify the model-dataset combination that results in the most accurate predictions. The CNN is implemented in the spectral and temporal dimensions instead of the conventional spatial dimension. Rather than using existing deep learning architectures, an autotuning procedure is implemented so that the model hyperparameters are empirically optimized during the training. The results obtained on held-out test data show that up to 94% overall accuracy and 90% Cohen’s Kappa can be obtained when the 2D CNN is applied on the SITS data with a temporal resolution of 7 days. This is closely followed by the 1D CNN on the same dataset. However, the latter performs better than the former in predicting data outside the training set. It is further observed that cereal is predicted with the highest accuracy, followed by grass. Predicting the unused areas has been found to be difficult as there is no distinct surface condition that is common for all unused areas.

Improving the Accuracy of Multiple Algorithms for Crop Classification by Integrating Sentinel-1 Observations with Sentinel-2 Data

The availability of an unprecedented amount of open remote sensing data, such as Sentinel-1 and -2 data within the Copernicus program, has boosted the idea of combining the use of optical and radar data to improve the accuracy of agricultural applications such as crop classification. Sentinel-1’s Synthetic Aperture Radar (SAR) provides co- and cross-polarized backscatter, which offers the opportunity to monitor agricultural crops using radar at high spatial and temporal resolution. In this study, we assessed the potential of integrating Sentinel-1 information (VV and VH backscatter and their ratio VH/VV with Sentinel-2A data (NDVI) to perform crop classification and to define which are the most important input data that provide the most accurate classification results. Further, we examined the temporal dynamics of remote sensing data for cereal, horticultural, and industrial crops, perennials, deciduous trees, and legumes. To select the best SAR input feature, we tried two approaches, one based on classification with only SAR features and one based on integrating SAR with optical data. In total, nine scenarios were tested. Furthermore, we evaluated the performance of 22 nonparametric classifiers on which most of these algorithms had not been tested before with SAR data. The results revealed that the best performing scenario was the one integrating VH and VV with normalized difference vegetation index (NDVI) and cubic support vector machine (SVM) (the kernel function of the classifier is cubic) as the classifier with the highest accuracy among all those tested.

Comparative Analysis of Edge Information and Polarization on SAR-to-Optical Translation Based on Conditional Generative Adversarial Networks

To accurately describe dynamic vegetation changes, high temporal and spectral resolution data are urgently required. Optical images contain rich spectral information but are limited by poor weather conditions and cloud contamination. Conversely, synthetic-aperture radar (SAR) is effective under all weather conditions but contains insufficient spectral information to recognize certain vegetation changes. Conditional adversarial networks (cGANs) can be adopted to transform SAR images (Sentinel-1) into optical images (Landsat8), which exploits the advantages of both optical and SAR images. As the features of SAR and optical remote sensing data play a decisive role in the translation process, this study explores the quantitative impact of edge information and polarization (VV, VH, VV&VH) on the peak signal-to-noise ratio, structural similarity index measure, correlation coefficient (r), and root mean squared error. The addition of edge information improves the structural similarity between generated and real images. Moreover, using the VH and VV&VH polarization modes as the input provides the cGANs with more effective information and results in better image quality. The optimal polarization mode with the addition of edge information is VV&VH, whereas that without edge information is VV. Near-infrared and short-wave infrared bands in the generated image exhibit higher accuracy (r > 0.8) than visible light bands. The conclusions of this study could serve as an important reference for selecting cGANs input features, and as a potential reference for the applications of cGANs to the SAR-to-optical translation of other multi-source remote sensing data.

Crop Classification Using Multi-Temporal Sentinel-2 Data in the Shiyang River Basin of China

Timely and accurate crop classification is of enormous significance for agriculture management. The Shiyang River Basin, an inland river basin, is one of the most prominent water resource shortage regions with intensive agriculture activities in northwestern China. However, a free crop map with high spatial resolution is not available in the Shiyang River Basin. The European Space Agency (ESA) satellite Sentinel-2 has multi-spectral bands ranging in the visible-red edge-near infrared-shortwave infrared (VIS-RE-NIR-SWIR) spectrum. Understanding the impact of spectral-temporal information on crop classification is helpful for users to select optimized spectral bands combinations and temporal window in crop mapping when using Sentinel-2 data. In this study, multi-temporal Sentinel-2 data acquired in the growing season in 2019 were applied to the random forest algorithm to generate the crop classification map at 10 m spatial resolution for the Shiyang River Basin. Four experiments with different combinations of feature sets were carried out to explore which Sentinel-2 information was more effective for higher crop classification accuracy. The results showed that the augment of multi-spectral and multi-temporal information of Sentinel-2 improved the accuracy of crop classification remarkably, and the improvement was firmly related to strategies of feature selections. Compared with other bands, red-edge band 1 (RE-1) and shortwave-infrared band 1 (SWIR-1) of Sentinel-2 showed a higher competence in crop classification. The combined application of images in the early, middle and late crop growth stage is significant for achieving optimal performance. A relatively accurate classification (overall accuracy = 0.94) was obtained by utilizing the pivotal spectral bands and dates of image. In addition, a crop map with a satisfied accuracy (overall accuracy > 0.9) could be generated as early as late July. This study gave an inspiration in selecting targeted spectral bands and period of images for acquiring more accurate and timelier crop map. The proposed method could be transferred to other arid areas with similar agriculture structure and crop phenology.

Recent Applications of Landsat 8/OLI and Sentinel-2/MSI for Land Use and Land Cover Mapping: A Systematic Review

Recent applications of Landsat 8 Operational Land Imager (L8/OLI) and Sentinel-2 MultiSpectral Instrument (S2/MSI) data for acquiring information about land use and land cover (LULC) provide a new perspective in remote sensing data analysis. Jointly, these sources permit researchers to improve operational classification and change detection, guiding better reasoning about landscape and intrinsic processes, as deforestation and agricultural expansion. However, the results of their applications have not yet been synthesized in order to provide coherent guidance on the effect of their applications in different classification processes, as well as to identify promising approaches and issues which affect classification performance. In this systematic review, we present trends, potentialities, challenges, actual gaps, and future possibilities for the use of L8/OLI and S2/MSI for LULC mapping and change detection. In particular, we highlight the possibility of using medium-resolution (Landsat-like, 10–30 m) time series and multispectral optical data provided by the harmonization between these sensors and data cube architectures for analysis-ready data that are permeated by publicizations, open data policies, and open science principles. We also reinforce the potential for exploring more spectral bands combinations, especially by using the three Red-edge and the two Near Infrared and Shortwave Infrared bands of S2/MSI, to calculate vegetation indices more sensitive to phenological variations that were less frequently applied for a long time, but have turned on since the S2/MSI mission. Summarizing peer-reviewed papers can guide the scientific community to the use of L8/OLI and S2/MSI data, which enable detailed knowledge on LULC mapping and change detection in different landscapes, especially in agricultural and natural vegetation scenarios.

Multispectral high resolution sensor fusion for smoothing and gap-filling in the cloud

Remote sensing optical sensors onboard operational satellites cannot have high spectral, spatial and temporal resolutions simultaneously. In addition, clouds and aerosols can adversely affect the signal contaminating the land surface observations. We present a HIghly Scalable Temporal Adaptive Reflectance Fusion Model (HISTARFM) algorithm to combine multispectral images of different sensors to reduce noise and produce monthly gap free high resolution (30 m) observations over land. Our approach uses images from the Landsat (30 m spatial resolution and 16 day revisit cycle) and the MODIS missions, both from Terra and Aqua platforms (500 m spatial resolution and daily revisit cycle). We implement a bias-aware Kalman filter method in the Google Earth Engine (GEE) platform to obtain fused images at the Landsat spatial-resolution. The added bias correction in the Kalman filter estimates accounts for the fact that both model and observation errors are temporally auto-correlated and may have a non-zero mean. This approach also enables reliable estimation of the uncertainty associated with the final reflectance estimates, allowing for error propagation analyses in higher level remote sensing products. Quantitative and qualitative evaluations of the generated products through comparison with other state-of-the-art methods confirm the validity of the approach, and open the door to operational applications at enhanced spatio-temporal resolutions at broad continental scales.

Crop Type Classification Using Fusion of Sentinel-1 and Sentinel-2 Data: Assessing the Impact of Feature Selection, Optical Data Availability, and Parcel Sizes on the Accuracies

Crop type classification using Earth Observation (EO) data is challenging, particularly for crop types with similar phenological growth stages. In this regard, the synergy of optical and Synthetic-Aperture Radar (SAR) data enables a broad representation of biophysical and structural information on target objects, enhancing crop type mapping. However, the fusion of multi-sensor dense time-series data often comes with the challenge of high dimensional feature space. In this study, we (1) evaluate how the usage of only optical, only SAR, and their fusion affect the classification accuracy; (2) identify the combination of which time-steps and feature-sets lead to peak accuracy; (3) analyze misclassifications based on the parcel size, optical data availability, and crops’ temporal profiles. Two fusion approaches were considered and compared in this study: feature stacking and decision fusion. To distinguish the most relevant feature subsets time- and variable-wise, grouped forward feature selection (gFFS) was used. gFFS allows focusing analysis and interpretation on feature sets of interest like spectral bands, vegetation indices (VIs), or data sensing time rather than on single features. This feature selection strategy leads to better interpretability of results while substantially reducing computational expenses. The results showed that, in contrast to most other studies, SAR datasets outperform optical datasets. Similar to most other studies, the optical-SAR combination outperformed single sensor predictions. No significant difference was recorded between feature stacking and decision fusion. Random Forest (RF) appears to be robust to high feature space dimensionality. The feature selection did not improve the accuracies even for the optical-SAR feature stack with 320 features. Nevertheless, the combination of RF feature importance and time- and variable-wise gFFS rankings in one visualization enhances interpretability and understanding of the features’ relevance for specific classification tasks. For example, by enabling the identification of features that have high RF feature importance values but are, in their information content, correlated with other features. This study contributes to the growing domain of interpretable machine learning.

Crop Mapping from Sentinel-1 Polarimetric Time-Series with a Deep Neural Network

Timely and accurate agricultural information is essential for food security assessment and agricultural management. Synthetic aperture radar (SAR) systems are increasingly available in crop mapping, as they provide all-weather imagery. In particular, the Sentinel-1 sensor provides dense time-series data, thus offering a unique opportunity for crop mapping. However, in most studies, the Sentinel-1 complex backscatter coefficient was used directly which limits the potential of the Sentinel-1 in crop mapping. Meanwhile, most of the existing methods may not be tailored for the task of crop classification in time-series polarimetric SAR data. To solve the above problem, we present a novel deep learning strategy in this research. To be specific, we collected Sentinel-1 time-series data in two study areas. The Sentinel-1 image covariance matrix is used as an input to maintain the integrity of polarimetric information. Then, a depthwise separable convolution recurrent neural network (DSCRNN) architecture is proposed to characterize crop types from multiple perspectives and achieve better classification results. The experimental results indicate that the proposed method achieves better accuracy in complex agricultural areas than other classical methods. Additionally, the variable importance provided by the random forest (RF) illustrated that the covariance vector has a far greater influence than the backscatter coefficient. Consequently, the strategy proposed in this research is effective and promising for crop mapping.

Assessing Spatial Limits of Sentinel-2 Data on Arable Crops in the Context of Checks by Monitoring

The availability of large amounts of Sentinel-2 data has been a trigger for its increasing exploitation in various types of applications. It is, therefore, of importance to understand the limits above which these data still guarantee a meaningful outcome. This paper proposes a new method to quantify and specify restrictions of the Sentinel-2 imagery in the context of checks by monitoring, a newly introduced control approach within the European Common Agriculture Policy framework. The method consists of a comparison of normalized difference vegetation index (NDVI) time series constructed from data of different spatial resolution to estimate the performance and limits of the coarser one. Using similarity assessment of Sentinel-2 (10 m pixel size) and PlanetScope (3 m pixel size) NDVI time series, it was estimated that for 10% out of 867 fields less than 0.5 ha in size, Sentinel-2 data did not provide reliable evidence of the activity or state of the agriculture field over a given timeframe. Statistical analysis revealed that the number of clean or full pixels and the proportion of pixels lost after an application of a 5-m (1/2 pixel) negative buffer are the geospatial parameters of the field that have the highest influence on the ability of the Sentinel-2 data to qualify the field’s state in time. We specified the following limiting criteria: at least 8 full pixels inside a border and less than 60% of pixels lost. It was concluded that compliance with the criteria still assures a high level of extracted information reliability. Our research proved the promising potential, which was higher than anticipated, of Sentinel-2 data for the continuous state assessment of small fields. The method could be applied to other sensors and indicators.

Comparative Assessment of Machine Learning Methods for Urban Vegetation Mapping Using Multitemporal Sentinel-1 Imagery

Mapping of green vegetation in urban areas using remote sensing techniques can be used as a tool for integrated spatial planning to deal with urban challenges. In this context, multitemporal (MT) synthetic aperture radar (SAR) data have not been equally investigated, as compared to optical satellite data. This research compared various machine learning methods using single-date and MT Sentinel-1 (S1) imagery. The research was focused on vegetation mapping in urban areas across Europe. Urban vegetation was classified using six classifiers—random forests (RF), support vector machine (SVM), extreme gradient boosting (XGB), multi-layer perceptron (MLP), AdaBoost.M1 (AB), and extreme learning machine (ELM). Whereas, SVM showed the best performance in the single-date image analysis, the MLP classifier yielded the highest overall accuracy in the MT classification scenario. Mean overall accuracy (OA) values for all machine learning methods increased from 57% to 77% with speckle filtering. Using MT SAR data, i.e., three and five S1 imagery, an additional increase in the OA of 8.59% and 13.66% occurred, respectively. Additionally, using three and five S1 imagery for classification, the F1 measure for forest and low vegetation land-cover class exceeded 90%. This research allowed us to confirm the possibility of MT C-band SAR imagery for urban vegetation mapping.

Assessment of Multi-Date Sentinel-1 Polarizations and GLCM Texture Features Capacity for Onion and Sunflower Classification in an Irrigated Valley: An Object Level Approach

The objective of this work is to evaluate the capacity of the C-band Synthetic Aperture Radar (SAR) time series imagery, acquired by the European satellite Sentinel-1 (S1), for the agriculture crop classification and its reliability to differentiate onion from sunflower, among others. The work then focused on classifying land cover in intensively cultivated agricultural regions. The study was developed in the Bonaerense Valley of the Colorado River (BVCR), Buenos Aires Province in Argentina, backed up by the field truth of 1634 field samples. In addition to the onion and sunflower crops, there are other crops present in the study area such as cereals, alfalfa, potatoes and maize, which are considered as the image background in the classification process. The field samples database was used for training and supporting a supervised classification with two machine learning algorithms—Random Forest (RF) and Support Vector Machine (SVM)—obtaining high levels of accuracy in each case. Different S1 SAR time-series features were used to assess the performance of S1 crop classification in terms of polarization VH+VV, Grey Level Co-occurrence Matrix (GLCM) image texture and a combination of both. The analysis of SAR data and their features was carried out at OBIA lot level (Object Based Image Analysis) showing an optimal strategy to counteract the effect of the residual and inherent speckle noise of the radar signal. In the process of differentiating onion and sunflower crops, the analysis of the VH+VV stack with the SVM algorithm delivered the best statistical classification results in terms of Overall Accuracy (OA) and Kappa Index, (Kp) when other crops (image background) were not considered (OA = 95.35%, Kp = 0.89). Certainly, the GLCM texture analysis derived from the S1 SAR images is a valuable source of information for obtaining very good classification results. When differentiating sunflower from onion considering also other crops present in the BVCR, the GLCM stack proved to be the most suitable dataset analyzed in this work (OA = 89.98%, Kp = 0.66 for SVM algorithm). This working methodology is applicable to other irrigated valleys in Argentina dedicated to intensive crops. There are also variables inherent to each lot, soil, crop and agricultural producer that differ according to the study area and that should be considered for each case in the future.

Assessing the Accuracy of Multiple Classification Algorithms for Crop Classification Using Landsat-8 and Sentinel-2 Data

The launch of Sentinel-2A and B satellites has boosted the development of many applications that could benefit from the fine resolution of the supplied information, both in time and in space. Crop classification is a necessary task for efficient land management. We evaluated the benefits of combining Landsat-8 and Sentinel-2A information for irrigated crop classification. We also assessed the robustness and efficiency of 22 nonparametric classification algorithms for classifying irrigated crops in a semiarid region in the southeast of Spain. A parcel-based approach was proposed calculating the mean normalized difference vegetation index (NDVI) of each plot and the standard deviation to generate a calibration-testing set of data. More than 2000 visited plots for 12 different crops along the study site were utilized as ground truth. Ensemble classifiers were the most robust algorithms but not the most efficient because of their low prediction rate. Nearest neighbor methods and support vector machines have the best balance between robustness and efficiency as methods for classification. Although the F1 score is close to 90%, some misclassifications were found for spring crops (e.g., barley, wheat and peas). However, crops with quite similar cycles could be differentiated, such as purple garlic and white garlic, showing the powerfulness of the developed tool.

Systematic Mapping Study on Remote Sensing in Agriculture

The area of remote sensing techniques in agriculture has reached a significant degree of development and maturity, with numerous journals, conferences, and organizations specialized in it. Moreover, many review papers are available in the literature. The present work describes a literature review that adopts the form of a systematic mapping study, following a formal methodology. Eight mapping questions were defined, analyzing the main types of research, techniques, platforms, topics, and spectral information. A predefined search string was applied in the Scopus database, obtaining 1590 candidate papers. Afterwards, the most relevant 106 papers were selected, considering those with more than six citations per year. These are analyzed in more detail, answering the mapping questions for each paper. In this way, the current trends and new opportunities are discovered. As a result, increasing interest in the area has been observed since 2000; the most frequently addressed problems are those related to parameter estimation, growth vigor, and water usage, using classification techniques, that are mostly applied on RGB and hyperspectral images, captured from drones and satellites. A general recommendation that emerges from this study is to build on existing resources, such as agricultural image datasets, public satellite imagery, and deep learning toolkits.

Crop Mapping Using Random Forest and Particle Swarm Optimization based on Multi-Temporal Sentinel-2

Timely and accurate information on crop mapping and monitoring is necessary for agricultural resources management. Accordingly, the applicability of the proposed classification-feature selection ensemble procedure with different feature sets for crop mapping is investigated. Here, we produced various feature sets including spectral bands, spectral indices, variation of spectral index, texture, and combinations of features to map different types of crops. By using various feature sets and the random forest (RF) classifier, the crop maps were created. In aiming to determine the most relevant and distinctive features, the particle swarm optimization (PSO) and RF-variable importance measure feature selection methods were examined. The classification-feature selection ensemble procedure was adapted to combine the outputs of different feature sets from the better feature selection method using majority votes. Multi-temporal Sentinel-2 data has been used in Ghale-Nou county of Tehran, Iran. The performance of RF was efficient in crop mapping especially by spectral bands and texture in combination with other feature sets. Our results showed that the PSO-based feature selection leads to a more accurate classification than the RF-variable importance measure, in almost all feature sets for all crop types. The RF classifier-PSO ensemble procedure for crop mapping outperformed the RF classifier in each feature set with regard to the class-wise and overall accuracies (OA) (of about 2.7–7.4% increases in OA and 0.48–3.68% (silage maize), 0–1.61% (rice), 2.82–15.43% (alfalfa), and 10.96–41.13% (vegetables) improvement in F-scores for all feature sets). The proposed method could mainly be useful to differentiate between heterogeneous crop fields (e.g., vegetables in this study) due to their more obtained omission/commission errors reduction.

Mapping Maize Fields by Using Multi-Temporal Sentinel-1A and Sentinel-2A Images in Makarfi, Northern Nigeria, Africa

A timely and accurate crop type mapping is very significant, and a prerequisite for agricultural regions and ensuring global food security. The combination of remotely sensed optical and radar datasets presents an opportunity for acquiring crop information at relative spatial resolution and temporal resolution adequately to capture the growth profiles of various crop species. In this paper, we employed Sentinel-1A (S-1) and Sentinel-2A (S-2) data acquired between the end of June and early September 2016, on a semi-arid area in northern Nigeria. A different set of (VV and VH) SAR and optical (SI and SB) images, illustrating crop phenological development stage, were employed as inputs to the two machines learning Random Forest (RF) and Support Vector Machine (SVM) algorithms to automatically map maize fields. Significant increases in overall classification were shown when the multi-temporal spectral indices (SI) and spectral band (SB) datasets were added with the different integration of SAR datasets (i.e., VV and VH). The best overall accuracy (OA) for maize (96.93%) was derived by using RF classification algorithms with SI-SB-SAR datasets, although the SI datasets for RF and SB datasets for SVM also produced high overall maize classification accuracies, of 97.04% and 97.44%. The outcomes indicate the robustness of the RF or SVM methods to produce high-resolution maps of maize for subsequent application from agronomists, policy planners, and the government, because such information is lacking in our study area.

Synergistic Use of Multi-Temporal RADARSAT-2 and VENµS Data for Crop Classification Based on 1D Convolutional Neural Network

Annual crop inventory information is important for many agriculture applications and government statistics. The synergistic use of multi-temporal polarimetric synthetic aperture radar (SAR) and available multispectral remote sensing data can reduce the temporal gaps and provide the spectral and polarimetric information of the crops, which is effective for crop classification in areas with frequent cloud interference. The main objectives of this study are to develop a deep learning model to map agricultural areas using multi-temporal full polarimetric SAR and multi-spectral remote sensing data, and to evaluate the influence of different input features on the performance of deep learning methods in crop classification. In this study, a one-dimensional convolutional neural network (Conv1D) was proposed and tested on multi-temporal RADARSAT-2 and VENµS data for crop classification. Compared with the Multi-Layer Perceptron (MLP), Recurrent Neural Network (RNN) and non-deep learning methods including XGBoost, Random Forest (RF), and Support Vector Machina (SVM), the Conv1D performed the best when the multi-temporal RADARSAT-2 data (Pauli decomposition or coherency matrix) and VENµS multispectral data were fused by the Minimum Noise Fraction (MNF) transformation. The Pauli decomposition and coherency matrix gave similar overall accuracy (OA) for Conv1D when fused with the VENµS data by the MNF transformation (OA = 96.65 ± 1.03% and 96.72 ± 0.77%). The MNF transformation improved the OA and F-score for most classes when Conv1D was used. The results reveal that the coherency matrix has a great potential in crop classification and the MNF transformation of multi-temporal RADARSAT-2 and VENµS data can enhance the performance of Conv1D.

Evaluation of Three Deep Learning Models for Early Crop Classification Using Sentinel-1A Imagery Time Series—A Case Study in Zhanjiang, China

Timely and accurate estimation of the area and distribution of crops is vital for food security. Optical remote sensing has been a key technique for acquiring crop area and conditions on regional to global scales, but great challenges arise due to frequent cloudy days in southern China. This makes optical remote sensing images usually unavailable. Synthetic aperture radar (SAR) could bridge this gap since it is less affected by clouds. The recent availability of Sentinel-1A (S1A) SAR imagery with a 12-day revisit period at a high spatial resolution of about 10 m makes it possible to fully utilize phenological information to improve early crop classification. In deep learning methods, one-dimensional convolutional neural networks (1D CNNs), long short-term memory recurrent neural networks (LSTM RNNs), and gated recurrent unit RNNs (GRU RNNs) have been shown to efficiently extract temporal features for classification tasks. However, due to the complexity of training, these three deep learning methods have been less used in early crop classification. In this work, we attempted to combine them with an incremental classification method to avoid the need for training optimal architectures and hyper-parameters for data from each time series. First, we trained 1D CNNs, LSTM RNNs, and GRU RNNs based on the full images’ time series to attain three classifiers with optimal architectures and hyper-parameters. Then, starting at the first time point, we performed an incremental classification process to train each classifier using all of the previous data, and obtained a classification network with all parameter values (including the hyper-parameters) at each time point. Finally, test accuracies of each time point were assessed for each crop type to determine the optimal time series length. A case study was conducted in Suixi and Leizhou counties of Zhanjiang City, China. To verify the effectiveness of this method, we also implemented the classic random forest (RF) approach. The results were as follows: (i) 1D CNNs achieved the highest Kappa coefficient (0.942) of the four classifiers, and the highest value (0.934) in the GRU RNNs time series was attained earlier than with other classifiers; (ii) all three deep learning methods and the RF achieved F measures above 0.900 before the end of growth seasons of banana, eucalyptus, second-season paddy rice, and sugarcane; while, the 1D CNN classifier was the only one that could obtain an F-measure above 0.900 for pineapple before harvest. All results indicated the effectiveness of the solution combining the deep learning models with the incremental classification approach for early crop classification. This method is expected to provide new perspectives for early mapping of croplands in cloudy areas.

Sentinel-1 Data for Winter Wheat Phenology Monitoring and Mapping

The ability of Synthetic Aperture Radar (SAR) Sentinel-1 data to detect the main wheat phenological phases was investigated in the Bekaa plain of Lebanon. Accordingly, the temporal variation of Sentinel-1 (S1) signal was analyzed as a function of the phenological phases’ dates observed in situ (germination; heading and soft dough), and harvesting. Results showed that S1 data, unlike the Normalized Difference Vegetation Index (NDVI) data, were able to estimate the dates of theses phenological phases due to significant variations in S1 temporal series at the dates of germination, heading, soft dough, and harvesting. Particularly, the ratio VV/VH at low incidence angle (32–34°) was able to detect the germination and harvesting dates. VV polarization at low incidence angle (32–34°) was able to detect the heading phase, while VH polarization at high incidence angle (43–45°) was better than that at low incidence angle (32–34°), in detecting the soft dough phase. An automated approach for main wheat phenological phases’ determination was then developed on the western part of the Bekaa plain. This approach modelled the S1 SAR temporal series by smoothing and fitting the temporal series with Gaussian functions (up to three Gaussians) allowing thus to automatically detect the main wheat phenological phases from the sum of these Gaussians. To test its robustness, the automated method was applied on the northern part of the Bekaa plain, in which winter wheat is harvested usually earlier because of the different weather conditions. The Root Mean Square Error (RMSE) of the estimation of the phenological phases’ dates was 2.9 days for germination, 5.5 days for heading, 5.1 days soft dough, 3.0 days for West Bekaa’s harvesting, and 4.5 days for North Bekaa’s harvesting. In addition, a slight underestimation was observed for germination and heading of West Bekaa (−0.2 and −1.1 days, respectively) while an overestimation was observed for soft dough of West Bekaa and harvesting for both West and North Bekaa (3.1, 0.6, and 3.6 days, respectively). These results are encouraging, and thus prove that S1 data are powerful as a tool for crop monitoring, to serve enhanced crop management and production handling.

Combining Deep Learning and Prior Knowledge for Crop Mapping in Tropical Regions from Multitemporal SAR Image Sequences

Accurate crop type identification and crop area estimation from remote sensing data in tropical regions are still considered challenging tasks. The more favorable weather conditions, in comparison to the characteristic conditions of temperate regions, permit higher flexibility in land use, planning, and management, which implies complex crop dynamics. Moreover, the frequent cloud cover prevents the use of optical data during large periods of the year, making SAR data an attractive alternative for crop mapping in tropical regions. This paper evaluates the effectiveness of Deep Learning (DL) techniques for crop recognition from multi-date SAR images from tropical regions. Three DL strategies are investigated: autoencoders, convolutional neural networks, and fully-convolutional networks. The paper further proposes a post-classification technique to enforce prior knowledge about crop dynamics in the target area. Experiments conducted on a Sentinel-1 multitemporal sequence of a tropical region in Brazil reveal the pros and cons of the tested methods. In our experiments, the proposed crop dynamics model was able to correct up to 16.5% of classification errors and managed to improve the performance up to 3.2% and 8.7% in terms of overall accuracy and average F1-score, respectively.