Spectrum- and RGB-D-Based Image Fusion for the Prediction of Nitrogen Accumulation in Wheat

A Spatiotemporal Fusion Method Based on Multiscale Feature Extraction and Spatial Channel Attention Mechanism

Remote sensing satellite images with a high spatial and temporal resolution play a crucial role in Earth science applications. However, due to technology and cost constraints, it is difficult for a single satellite to achieve both a high spatial resolution and high temporal resolution. The spatiotemporal fusion method is a cost-effective solution for generating a dense temporal data resolution with a high spatial resolution. In recent years, spatiotemporal image fusion based on deep learning has received wide attention. In this article, a spatiotemporal fusion method based on multiscale feature extraction and a spatial channel attention mechanism is proposed. Firstly, the method uses a multiscale mechanism to fully utilize the structural features in the images. Then a novel attention mechanism is used to capture both spatial and channel information; finally, the rich features and spatial and channel information are used to fuse the images. Experimental results obtained from two datasets show that the proposed method outperforms existing fusion methods in both subjective and objective evaluations.

Estimation of Plant Height and Aboveground Biomass of Toona sinensis under Drought Stress Using RGB-D Imaging

Rapid and accurate plant growth and biomass estimation is essential for formulating and implementing targeted forest cultivation measures. In this study, RGB-D imaging technology was used to obtain the RGB and depth imaging data for a Toona sinensis seedling canopy to estimate plant growth and aboveground biomass (AGB). Three hundred T. sinensis seedlings from 20 varieties were planted under five different drought stress treatments. The U-Net model was applied first to achieve highly accurate segmentation of plants from complex backgrounds. Simple linear regression (SLR) was used for plant height prediction, and the other three models, including multivariate linear (ML), random forest (RF) and multilayer perceptron (MLP) regression, were applied to predict the AGB and compared for optimal model selection. The results showed that the SLR model yields promising and reliable results for the prediction of plant height, with R2 and RMSE values of 0.72 and 1.89 cm, respectively. All three regression methods perform well in the prediction of AGB estimation. MLP yields the highest accuracy in predicting dry and fresh aboveground biomass compared to the other two regression models, with R2 values of 0.77 and 0.83, respectively. The combination of Gray, Green minus red (GMR) and Excess green index (ExG) was identified as the key predictor by RReliefF for predicting dry AGB. GMR was the most important in predicting fresh AGB. This study demonstrated that the merits of RGB-D and machine learning models are effective phenotyping techniques for plant height and AGB prediction, and can be used to assist dynamic responses to drought stress for breeding selection.

Multi-Modal Deep Learning for Weeds Detection in Wheat Field Based on RGB-D Images

Single-modal images carry limited information for features representation, and RGB images fail to detect grass weeds in wheat fields because of their similarity to wheat in shape. We propose a framework based on multi-modal information fusion for accurate detection of weeds in wheat fields in a natural environment, overcoming the limitation of single modality in weeds detection. Firstly, we recode the single-channel depth image into a new three-channel image like the structure of RGB image, which is suitable for feature extraction of convolutional neural network (CNN). Secondly, the multi-scale object detection is realized by fusing the feature maps output by different convolutional layers. The three-channel network structure is designed to take into account the independence of RGB and depth information, respectively, and the complementarity of multi-modal information, and the integrated learning is carried out by weight allocation at the decision level to realize the effective fusion of multi-modal information. The experimental results show that compared with the weed detection method based on RGB image, the accuracy of our method is significantly improved. Experiments with integrated learning shows that mean average precision (mAP) of 36.1% for grass weeds and 42.9% for broad-leaf weeds, and the overall detection precision, as indicated by intersection over ground truth (IoG), is 89.3%, with weights of RGB and depth images at α = 0.4 and β = 0.3. The results suggest that our methods can accurately detect the dominant species of weeds in wheat fields, and that multi-modal fusion can effectively improve object detection performance.