Deep-Learning-Based Multispectral Image Reconstruction from Single Natural Color RGB Image—Enhancing UAV-Based Phenotyping

Classification of soybean seeds based on RGB reconstruction of hyperspectral images

Soyabean is an incredibly significant component of Chinese agricultural product, and categorizing soyabean seeds allows for a better understanding of the features, attributes, and applications of many species of soyabean. This enables farmers to choose appropriate seeds for sowing in order to increase production and quality. As a result, this thesis provides a method for classifying soybean seeds that uses hyperspectral RGB picture reconstruction. Firstly, hyperspectral images of seven varieties of soybean, H1, H2, H3, H4, H5, H6 and H7, were collected by hyperspectral imager, and by using the principle of the three base colours, the R, G and B bands which have more characteristic information are selected to reconstruct the images with different texture and colour characteristics to generate a new dataset for seed segmentation, and finally, a comparison is made with the classification effect of the seven models. The experimental results in ResNet34 show that the classification accuracy of the dataset before and after RGB reconstruction increases from 88.87% to 91.75%, demonstrating that RGB image reconstruction can strengthen image features; ResNet18, ResNet34, ResNet50, ResNet101, CBAM-ResNet34, SENet-ResNet34, and SENet-ResNet34-DCN models have classification accuracies of 72.25%, 91.75%, 89%, 88.48%, 92.28%, 92.80%, and 94.24%, respectively.SENet-ResNet34-DCN achieves the greatest classification accuracy results, with a model loss of roughly 0.3. The proposed SENet-ResNet34-DCN model is the most effective at classifying soybean seeds. By classifying and optimally selecting seed varieties, agricultural production can become more scientific, efficient, and sustainable, resulting in higher returns for farmers and contributing to global food security and sustainable development.

Charge-Coupled Frequency Response Multispectral Inversion Network-Based Detection Method of Oil Contamination on Airport Runway

Aircraft failures can result in the leakage of fuel, hydraulic oil, or other lubricants onto the runway during landing or taxiing. Damage to fuel tanks or oil lines during hard landings or accidents can also contribute to these spills. Further, improper maintenance or operational errors may leave oil traces on the runway before take-off or after landing. Identifying oil spills in airport runway videos is crucial to flight safety and accident investigation. Advanced image processing techniques can overcome the limitations of conventional RGB-based detection, which struggles to differentiate between oil spills and sewage due to similar coloration; given that oil and sewage have distinct spectral absorption patterns, precise detection can be performed based on multispectral images. In this study, we developed a method for spectrally enhancing RGB images of oil spills on airport runways to generate HSI images, facilitating oil spill detection in conventional RGB imagery. To this end, we employed the MST++ spectral reconstruction network model to effectively reconstruct RGB images into multispectral images, yielding improved accuracy in oil detection compared with other models. Additionally, we utilized the Fast R-CNN oil spill detection model, resulting in a 5% increase in Intersection over Union (IOU) for HSI images. Moreover, compared with RGB images, this approach significantly enhanced detection accuracy and completeness by 25.3% and 26.5%, respectively. These findings clearly demonstrate the superior precision and accuracy of HSI images based on spectral reconstruction in oil spill detection compared with traditional RGB images. With the spectral reconstruction technique, we can effectively make use of the spectral information inherent in oil spills, thereby enhancing detection accuracy. Future research could delve deeper into optimization techniques and conduct extensive validation in real airport environments. In conclusion, this spectral reconstruction-based technique for detecting oil spills on airport runways offers a novel and efficient approach that upholds both efficacy and accuracy. Its wide-scale implementation in airport operations holds great potential for improving aviation safety and environmental protection.

Are unmanned aerial vehicle-based hyperspectral imaging and machine learning advancing crop science?

The past few years have seen increased interest in unmanned aerial vehicle (UAV)-based hyperspectral imaging (HSI) and machine learning (ML) in agricultural research, concomitant with an increase in published research on these topics. We provide an updated review, written for agriculturalists, highlighting the benefits in the retrieval of biophysical parameters of crops via UAVs relative to less sophisticated options. We reviewed >70 recent papers and found few consistent results between similar studies. Owing to their high complexity and cost, especially when applied to crops of low value, the benefits of most of the research reviewed are difficult to explain. Future effort will be necessary to distill research findings into lower-cost options for end-users.

Learning-Based Near-Infrared Band Simulation with Applications on Large-Scale Landcover Classification

Multispectral sensors are important instruments for Earth observation. In remote sensing applications, the near-infrared (NIR) band, together with the visible spectrum (RGB), provide abundant information about ground objects. However, the NIR band is typically not available on low-cost camera systems, which presents challenges for the vegetation extraction. To this end, this paper presents a conditional generative adversarial network (cGAN) method to simulate the NIR band from RGB bands of Sentinel-2 multispectral data. We adapt a robust loss function and a structural similarity index loss (SSIM) in addition to the GAN loss to improve the model performance. With 45,529 multi-seasonal test images across the globe, the simulated NIR band had a mean absolute error of 0.02378 and an SSIM of 89.98%. A rule-based landcover classification using the simulated normalized difference vegetation index (NDVI) achieved a Jaccard score of 89.50%. The evaluation metrics demonstrated the versatility of the learning-based paradigm in remote sensing applications. Our simulation approach is flexible and can be easily adapted to other spectral bands.

Deep-agriNet: a lightweight attention-based encoder-decoder framework for crop identification using multispectral images

The field of computer vision has shown great potential for the identification of crops at large scales based on multispectral images. However, the challenge in designing crop identification networks lies in striking a balance between accuracy and a lightweight framework. Furthermore, there is a lack of accurate recognition methods for non-large-scale crops. In this paper, we propose an improved encoder-decoder framework based on DeepLab v3+ to accurately identify crops with different planting patterns. The network employs ShuffleNet v2 as the backbone to extract features at multiple levels. The decoder module integrates a convolutional block attention mechanism that combines both channel and spatial attention mechanisms to fuse attention features across the channel and spatial dimensions. We establish two datasets, DS1 and DS2, where DS1 is obtained from areas with large-scale crop planting, and DS2 is obtained from areas with scattered crop planting. On DS1, the improved network achieves a mean intersection over union (mIoU) of 0.972, overall accuracy (OA) of 0.981, and recall of 0.980, indicating a significant improvement of 7.0%, 5.0%, and 5.7%, respectively, compared to the original DeepLab v3+. On DS2, the improved network improves the mIoU, OA, and recall by 5.4%, 3.9%, and 4.4%, respectively. Notably, the number of parameters and giga floating-point operations (GFLOPs) required by the proposed Deep-agriNet is significantly smaller than that of DeepLab v3+ and other classic networks. Our findings demonstrate that Deep-agriNet performs better in identifying crops with different planting scales, and can serve as an effective tool for crop identification in various regions and countries.

Comparison of Artificial Neural Network and Polynomial Approximation Models for Reflectance Spectra Reconstruction

Knowledge of surface reflection of an object is essential in many technological fields, including graphics and cultural heritage. Compared to direct multi- or hyper-spectral capturing approaches, commercial RGB cameras allow for a high resolution and fast acquisition, so the idea of mapping this information into a reflectance spectrum (RS) is promising. This study compared two modelling approaches based on a training set of RGB-reflectance pairs, one implementing artificial neural networks (ANN) and the other one using multivariate polynomial approximation (PA). The effect of various parameters was investigated: the ANN learning algorithm-standard backpropagation (BP) or Levenberg-Marquardt (LM), the number of hidden layers (HLs) and neurons, the degree of multivariate polynomials in PA, the number of inputs, and the training set size on both models. In the two-layer ANN with significantly fewer inputs than outputs, a better MSE performance was found where the number of neurons in the first HL was smaller than in the second one. For ANNs with one and two HLs with the same number of neurons in the first layer, the RS reconstruction performance depends on the choice of BP or LM learning algorithm. RS reconstruction methods based on ANN and PA are comparable, but the ANN models' better fine-tuning capabilities enable, under realistic constraints, finding ANNs that outperform PA models. A profiling approach was proposed to determine the initial number of neurons in HLs-the search centre of ANN models for different training set sizes.