Deep Convolutional Neural Network for Mapping Smallholder Agriculture Using High Spatial Resolution Satellite Image

Mapping 10-m harvested area in the major winter wheat-producing regions of China from 2018 to 2022

Winter wheat constitutes approximately 20% of China's total cereal production. However, calculations of total production based on multiplying the planted area by the yield have tended to produce overestimates. In this study, we generated sample points from existing winter wheat maps and obtained samples for different years using a temporal migration method. Random forest classifiers were then constructed using optimized features extracted from spectral and phenological characteristics and elevation information. Maps of the harvested and planted areas of winter wheat in Chinese eight provinces from 2018 to 2022 were then produced. The resulting maps of the harvested areas achieved an overall accuracy of 95.06% verified by the sample points, and the correlation coefficient between the CROPGRIDS dataset is about 0.77. The harvested area was found to be about 13% smaller than the planted area, which can primarily be attributed to meteorological hazards. This study represents the first attempt to map the winter wheat harvested area at 10-m resolution in China, and it should improve the accuracy of yield estimation.

SCANet: Securing the Weights With Superparamagnetic-MTJ Crossbar Array Networks

Deep neural networks (DNNs) form a critical infrastructure supporting various systems, spanning from the iPhone neural engine to imaging satellites and drones. The design of these neural cores is often proprietary or a military secret. Nevertheless, they remain vulnerable to model replication attacks that seek to reverse engineer the network's synaptic weights. In this article, we propose SCANet (Superparamagnetic-MTJ Crossbar Array Networks), a novel defense mechanism against such model stealing attacks by utilizing the innate stochasticity in superparamagnets. When used as the synapse in DNNs, superparamagnetic magnetic tunnel junctions (s-MTJs) are shown to be significantly more secure than prior memristor-based solutions. The thermally induced telegraphic switching in the s-MTJs is robust and uncontrollable, thus thwarting the attackers from obtaining sensitive data from the network. Using a mixture of both superparamagnetic and conventional MTJs in the neural network (NN), the designer can optimize the time period between the weight updation and the power consumed by the system. Furthermore, we propose a modified NN architecture that can prevent replication attacks while minimizing power consumption. We investigate the effect of the number of layers in the deep network and the number of neurons in each layer on the sharpness of accuracy degradation when the network is under attack. We also explore the efficacy of SCANet in real-time scenarios, using a case study on object detection.

Effects of Driving Factors on Forest Aboveground Biomass (AGB) in China’s Loess Plateau by Using Spatial Regression Models

Forests are the main body of carbon sequestration in terrestrial ecosystems and forest aboveground biomass (AGB) is an important manifestation of forest carbon sequestration. Reasonable and accurate quantification of the relationship between AGB and its driving factors is of great importance for increasing the biomass and function of forests. Remote sensing observations and field measurements can be used to estimate AGB in large areas. To explore the applicability of the panel data models in AGB and its driving factors, we compared the results of panel data models (spatial error model and spatial lag model) with those of geographically weighted regression (GWR) and ordinary least squares (OLS) to quantify the relationship between AGB and its driving factors. Furthermore, we estimated the tree height, diameter at breast height, canopy cover (CC) and species diversity index (Shannon–Wiener index) of Robinia pseudoacacia plantations in Changwu on the Loess Plateau using field data and remote sensing images by a random forest model and estimated soil organic carbon (SOC) contents using laboratory data by ordinary kriging (OK) interpolation. We estimated AGB using the already estimated tree height and diameter at breast height combined with the allometric growth equation. In this study, we estimated SOC contents by OK interpolation, and the accuracy R2 values for each soil layer were greater than 0.81. We estimated diameter at breast height (DBH), CC, SW and tree height (TH) using the random forest, and the accuracy R2 values were 0.85, 0.82, 0.76 and 0.68, respectively. We estimated AGB with random forest and the allometric growth equation and found that the average AGB was 55.80 t/ha. The OLS results showed that the residuals of the OLS regression exhibited obvious spatial correlations and rejected OLS applications. GWR, SEM and SLM were used for spatial regression analysis, and SEM was the best model for explaining the relationship between AGB and its driving factors. We also found that AGB was significantly positively correlated with CC, SW, and 0–60 cm SOC content (p < 0.05) and significantly negatively correlated with slope aspect (p < 0.01). This study provides a new idea for studying the relationship between AGB and its driving factors and provides a basis for practical forest management, increasing biomass, and giving full play to the role of carbon sequestration.

Cropping Patterns of Annual Crops: A Remote Sensing Review

Cropping patterns are defined as the sequence and spatial arrangement of annual crops on a piece of land. Knowledge of cropping patterns is crucial for crop production and land-use intensity. While cropping patterns are related to crop production and land use intensity, they are rarely reported in agricultural statistics, especially those relating to small farms in developing countries. Remote sensing has enabled mapping cropping patterns by monitoring crops’ spatial and temporal dynamics. In this paper, we reviewed remote sensing studies of single, sequential and intercropping patterns of annual crops practiced at local and regional scales. A total of 90 studies were selected from 753 publications based on their cropping pattern types and relevance to the scope of this review. The review found that despite the increase in single cropping pattern studies due to the Sentinel missions, studies on intercropping patterns are rare, suggesting that mapping intercropping is still challenging. More so, microwave remote sensing for mapping intercropping has not been fully explored. Given the complexities in mapping intercropping, our review highlights how less frequently used vegetation indices (VIs) that benefit from red-edge and SWIR spectral bands may improve intercropping mapping.

Invited review: Toward a common language in data-driven mastitis detection research

Sensor technologies for mastitis detection have resulted in the collection and availability of a large amount of data. As a result, scientific publications reporting mastitis detection research have become less driven by approaches based on biological assumptions and more by data-driven modeling. Most of these approaches try to predict mastitis events from (combinations of) raw sensor data to which a wide variety of methods are applied originating from machine learning and classical statistical approaches. However, an even wider variety in terminologies is used by researchers for methods that are similar in nature. This makes it difficult for readers from other disciplines to understand the specific methods that are used and how these differ from each other. The aim of this paper was to provide a framework (filtering, transformation, and classification) for describing the different methods applied in sensor data-based clinical mastitis detection research and use this framework to review and categorize the approaches and underlying methods described in the scientific literature on mastitis detection. We identified 40 scientific publications between 1992 and 2020 that applied methods to detect clinical mastitis from sensor data. Based on these publications, we developed and used the framework and categorized these scientific publications into the 2 data processing techniques of filtering and transformation. These data processing techniques make raw data more amendable to be used for the third step in our framework, that of classification, which is used to distinguish between healthy and nonhealthy (mastitis) cows. Most publications (n = 34) used filtering or transformation, or a combination of these 2, for data processing before classification, whereas the remaining publications (n = 6) classified the observations directly from raw data. Concerning classification, applying a simple threshold was the most used method (n = 19 publications). Our work identified that within approaches several different methods and terminologies for similar methods were used. Not all publications provided a clear description of the method used, and therefore it seemed that different methods were used between publications, whereas in fact just a different terminology was used, or the other way around. This paper is intended to serve as a reference for people from various research disciplines who need to collaborate and communicate efficiently about the topic of sensor-based mastitis detection and the methods used in this context. The framework used in this paper can support future research to correctly classify approaches and methods, which can improve the understanding of scientific publication. We encourage future research on sensor-based animal disease detection, including that of mastitis detection, to use a more coherent terminology for methods, and clearly state which technique (e.g., filtering) and approach (e.g., moving average) are used. This paper, therefore, can serve as a starting point and further stimulates the interdisciplinary cooperation in sensor-based mastitis research.

Application of Hyperspectral Imaging and Deep Learning for Robust Prediction of Sugar and pH Levels in Wine Grape Berries

Remote sensing technology, such as hyperspectral imaging, in combination with machine learning algorithms, has emerged as a viable tool for rapid and nondestructive assessment of wine grape ripeness. However, the differences in terroir, together with the climatic variations and the variability exhibited by different grape varieties, have a considerable impact on the grape ripening stages within a vintage and between vintages and, consequently, on the robustness of the predictive models. To address this challenge, we present a novel one-dimensional convolutional neural network architecture-based model for the prediction of sugar content and pH, using reflectance hyperspectral data from different vintages. We aimed to evaluate the model’s generalization capacity for different varieties and for a different vintage not employed in the training process, using independent test sets. A transfer learning mechanism, based on the proposed convolutional neural network, was also used to evaluate improvements in the model’s generalization. Overall, the results for generalization ability showed a very good performance with RMSEP values of 1.118 °Brix and 1.085 °Brix for sugar content and 0.199 and 0.183 for pH, for test sets using different varieties and a different vintage, respectively, improving and updating the current state of the art.

Potential of Hybrid CNN-RF Model for Early Crop Mapping with Limited Input Data

When sufficient time-series images and training data are unavailable for crop classification, features extracted from convolutional neural network (CNN)-based representative learning may not provide useful information to discriminate crops with similar spectral characteristics, leading to poor classification accuracy. In particular, limited input data are the main obstacles to obtain reliable classification results for early crop mapping. This study investigates the potential of a hybrid classification approach, i.e., CNN-random forest (CNN-RF), in the context of early crop mapping, that combines the automatic feature extraction capability of CNN with the superior discrimination capability of an RF classifier. Two experiments on incremental crop classification with unmanned aerial vehicle images were conducted to compare the performance of CNN-RF with that of CNN and RF with respect to the length of the time-series and training data sizes. When sufficient time-series images and training data were used for the classification, the accuracy of CNN-RF was slightly higher or comparable with that of CNN. In contrast, when fewer images and the smallest training data were used at the early crop growth stage, CNN-RF was substantially beneficial and the overall accuracy increased by maximum 6.7%p and 4.6%p in the two study areas, respectively, compared to CNN. This is attributed to its ability to discriminate crops from features with insufficient information using a more sophisticated classifier. The experimental results demonstrate that CNN-RF is an effective classifier for early crop mapping when only limited input images and training samples are available.

3DeepM: An Ad Hoc Architecture Based on Deep Learning Methods for Multispectral Image Classification

Current predefined architectures for deep learning are computationally very heavy and use tens of millions of parameters. Thus, computational costs may be prohibitive for many experimental or technological setups. We developed an ad hoc architecture for the classification of multispectral images using deep learning techniques. The architecture, called 3DeepM, is composed of 3D filter banks especially designed for the extraction of spatial-spectral features in multichannel images. The new architecture has been tested on a sample of 12210 multispectral images of seedless table grape varieties: Autumn Royal, Crimson Seedless, Itum4, Itum5 and Itum9. 3DeepM was able to classify 100% of the images and obtained the best overall results in terms of accuracy, number of classes, number of parameters and training time compared to similar work. In addition, this paper presents a flexible and reconfigurable computer vision system designed for the acquisition of multispectral images in the range of 400 nm to 1000 nm. The vision system enabled the creation of the first dataset consisting of 12210 37-channel multispectral images (12 VIS + 25 IR) of five seedless table grape varieties that have been used to validate the 3DeepM architecture. Compared to predefined classification architectures such as AlexNet, ResNet or ad hoc architectures with a very high number of parameters, 3DeepM shows the best classification performance despite using 130-fold fewer parameters than the architecture to which it was compared. 3DeepM can be used in a multitude of applications that use multispectral images, such as remote sensing or medical diagnosis. In addition, the small number of parameters of 3DeepM make it ideal for application in online classification systems aboard autonomous robots or unmanned vehicles.

A Comparison of Machine Learning Approaches to Improve Free Topography Data for Flood Modelling

Given the high financial and institutional cost of collecting and processing accurate topography data, many large-scale flood hazard assessments continue to rely instead on freely-available global Digital Elevation Models, despite the significant vertical biases known to affect them. To predict (and thereby reduce) these biases, we apply a fully-convolutional neural network (FCN), a form of artificial neural network originally developed for image segmentation which is capable of learning from multi-variate spatial patterns at different scales. We assess its potential by training such a model on a wide variety of remote-sensed input data (primarily multi-spectral imagery), using high-resolution, LiDAR-derived Digital Terrain Models published by the New Zealand government as the reference topography data. In parallel, two more widely used machine learning models are also trained, in order to provide benchmarks against which the novel FCN may be assessed. We find that the FCN outperforms the other models (reducing root mean square error in the testing dataset by 71%), likely due to its ability to learn from spatial patterns at multiple scales, rather than only a pixel-by-pixel basis. Significantly for flood hazard modelling applications, corrections were found to be especially effective along rivers and their floodplains. However, our results also suggest that models are likely to be biased towards the land cover and relief conditions most prevalent in their training data, with further work required to assess the importance of limiting training data inputs to those most representative of the intended application area(s).

Himawari-8 Aerosol Optical Depth (AOD) Retrieval Using a Deep Neural Network Trained Using AERONET Observations

Spectral aerosol optical depth (AOD) estimation from satellite-measured top of atmosphere (TOA) reflectances is challenging because of the complicated TOA-AOD relationship and a nexus of land surface and atmospheric state variations. This task is usually undertaken using a physical model to provide a first estimate of the TOA reflectances which are then optimized by comparison with the satellite data. Recently developed deep neural network (DNN) models provide a powerful tool to represent the complicated relationship statistically. This study presents a methodology based on DNN to estimate AOD using Himawari-8 Advanced Himawari Imager (AHI) TOA observations. A year (2017) of AHI TOA observations over the Himawari-8 full disk collocated in space and time with Aerosol Robotic Network (AERONET) AOD data were used to derive a total of 14,154 training and validation samples. The TOA reflectance in all six AHI solar bands, three TOA reflectance ratios derived based on the dark-target assumptions, sun-sensor geometry, and auxiliary data are used as predictors to estimate AOD at 500 nm. The DNN AOD is validated by separating training and validation samples using random k-fold cross-validation and using AERONET site-specific leave-one-station-out validation, and is compared with a random forest regression estimator and Japan Meteorological Agency (JMA) AOD. The DNN AOD shows high accuracy: (1) RMSE = 0.094, R2 = 0.915 for k-fold cross-validation, and (2) RMSE = 0.172, R2 = 0.730 for leave-one-station-out validation. The k-fold cross-validation overestimates the DNN accuracy as the training and validation samples may come from the same AHI pixel location. The leave-one-station-out validation reflects the accuracy for large-area applications where there are no training samples for the pixel location to be estimated. The DNN AOD has better accuracy than the random forest AOD and JMA AOD. In addition, the contribution of the dark-target derived TOA ratio predictors is examined and confirmed, and the sensitivity to the DNN structure is discussed.

A Rapid and Adaptive Alignment under Mooring Condition Using Adaptive EKF and CNN-Based Learning

Alignment of the inertial navigation system (INS) in the mooring environment should take into account the movements of the waves or wind. The alignment of the INS is performed through an extended Kalman filter (EKF) using zero velocity as a measurement. However, in the mooring condition, this is not perfect stationary, thus the measurement error covariance matrix should be adjusted. In addition, if the measurement error covariance matrix is fixed to one value, the alignment time may take longer or the performance may be reduced depending on the change in mooring conditions. To solve this problem, we propose an alignment method using adaptive Kalman filter and convolution neural network (CNN)-based learning. The proposed method was verified for the superiority of alignment time and accuracy through Monte Carlo simulation in a mooring environment.

Convolutional Neural Networks for Image-Based Corn Kernel Detection and Counting

Precise in-season corn grain yield estimates enable farmers to make real-time accurate harvest and grain marketing decisions minimizing possible losses of profitability. A well developed corn ear can have up to 800 kernels, but manually counting the kernels on an ear of corn is labor-intensive, time consuming and prone to human error. From an algorithmic perspective, the detection of the kernels from a single corn ear image is challenging due to the large number of kernels at different angles and very small distance among the kernels. In this paper, we propose a kernel detection and counting method based on a sliding window approach. The proposed method detects and counts all corn kernels in a single corn ear image taken in uncontrolled lighting conditions. The sliding window approach uses a convolutional neural network (CNN) for kernel detection. Then, a non-maximum suppression (NMS) is applied to remove overlapping detections. Finally, windows that are classified as kernel are passed to another CNN regression model for finding the ( x , y ) coordinates of the center of kernel image patches. Our experiments indicate that the proposed method can successfully detect the corn kernels with a low detection error and is also able to detect kernels on a batch of corn ears positioned at different angles.

Improved Winter Wheat Spatial Distribution Extraction from High-Resolution Remote Sensing Imagery Using Semantic Features and Statistical Analysis

Improving the accuracy of edge pixel classification is an important aspect of using convolutional neural networks (CNNs) to extract winter wheat spatial distribution information from remote sensing imagery. In this study, we established a method using prior knowledge obtained from statistical analysis to refine CNN classification results, named post-processing CNN (PP-CNN). First, we used an improved RefineNet model to roughly segment remote sensing imagery in order to obtain the initial winter wheat area and the category probability vector for each pixel. Second, we used manual labels as references and performed statistical analysis on the class probability vectors to determine the filtering conditions and select the pixels that required optimization. Third, based on the prior knowledge that winter wheat pixels were internally similar in color, texture, and other aspects, but different from other neighboring land-use types, the filtered pixels were post-processed to improve the classification accuracy. We used 63 Gaofen-2 images obtained from 2017 to 2019 of a representative Chinese winter wheat region (Feicheng, Shandong Province) to create the dataset and employed RefineNet and SegNet as standard CNN and conditional random field (CRF) as post-process methods, respectively, to conduct comparison experiments. PP-CNN’s accuracy (94.4%), precision (93.9%), and recall (94.4%) were clearly superior, demonstrating its advantages for the improved refinement of edge areas during image classification.