1
|
Busov ID, Genaev MA, Komyshev EG, Koval VS, Zykova TE, Glagoleva AY, Afonnikov DA. A pipeline for processing hyperspectral images, with a case of melanin-containing barley grains as an example. Vavilovskii Zhurnal Genet Selektsii 2024; 28:443-455. [PMID: 39040972 PMCID: PMC11260993 DOI: 10.18699/vjgb-24-50] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Revised: 03/22/2024] [Accepted: 03/26/2024] [Indexed: 07/24/2024] Open
Abstract
Analysis of hyperspectral images is of great interest in plant studies. Nowadays, this analysis is used more and more widely, so the development of hyperspectral image processing methods is an urgent task. This paper presents a hyperspectral image processing pipeline that includes: preprocessing, basic statistical analysis, visualization of a multichannel hyperspectral image, and solving classification and clustering problems using machine learning methods. The current version of the package implements the following methods: construction of a confidence interval of an arbitrary level for the difference of sample averages; verification of the similarity of intensity distributions of spectral lines for two sets of hyperspectral images on the basis of the Mann-Whitney U-criterion and Pearson's criterion of agreement; visualization in two-dimensional space using dimensionality reduction methods PCA, ISOMAP and UMAP; classification using linear or ridge regression, random forest and catboost; clustering of samples using the EM-algorithm. The software pipeline is implemented in Python using the Pandas, NumPy, OpenCV, SciPy, Sklearn, Umap, CatBoost and Plotly libraries. The source code is available at: https://github.com/igor2704/Hyperspectral_images. The pipeline was applied to identify melanin pigment in the shell of barley grains based on hyperspectral data. Visualization based on PCA, UMAP and ISOMAP methods, as well as the use of clustering algorithms, showed that a linear separation of grain samples with and without pigmentation could be performed with high accuracy based on hyperspectral data. The analysis revealed statistically significant differences in the distribution of median intensities for samples of images of grains with and without pigmentation. Thus, it was demonstrated that hyperspectral images can be used to determine the presence or absence of melanin in barley grains with great accuracy. The flexible and convenient tool created in this work will significantly increase the efficiency of hyperspectral image analysis.
Collapse
Affiliation(s)
- I D Busov
- Institute of Cytology and Genetics of the Siberian Branch of the Russian Academy of Sciences, Novosibirsk, Russia Novosibirsk State University, Novosibirsk, Russia
| | - M A Genaev
- Institute of Cytology and Genetics of the Siberian Branch of the Russian Academy of Sciences, Novosibirsk, Russia Novosibirsk State University, Novosibirsk, Russia
| | - E G Komyshev
- Institute of Cytology and Genetics of the Siberian Branch of the Russian Academy of Sciences, Novosibirsk, Russia
| | - V S Koval
- Institute of Cytology and Genetics of the Siberian Branch of the Russian Academy of Sciences, Novosibirsk, Russia
| | - T E Zykova
- Institute of Cytology and Genetics of the Siberian Branch of the Russian Academy of Sciences, Novosibirsk, Russia Novosibirsk State University, Novosibirsk, Russia
| | - A Y Glagoleva
- Institute of Cytology and Genetics of the Siberian Branch of the Russian Academy of Sciences, Novosibirsk, Russia
| | - D A Afonnikov
- Institute of Cytology and Genetics of the Siberian Branch of the Russian Academy of Sciences, Novosibirsk, Russia Novosibirsk State University, Novosibirsk, Russia
| |
Collapse
|
2
|
Jeong SW, Lyu JI, Jeong H, Baek J, Moon JK, Lee C, Choi MG, Kim KH, Park YI. SUnSeT: spectral unmixing of hyperspectral images for phenotyping soybean seed traits. PLANT CELL REPORTS 2024; 43:164. [PMID: 38852113 PMCID: PMC11162974 DOI: 10.1007/s00299-024-03249-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2024] [Accepted: 05/06/2024] [Indexed: 06/10/2024]
Abstract
KEY MESSAGE Hyperspectral features enable accurate classification of soybean seeds using linear discriminant analysis and GWAS for novel seed trait genes. Evaluating crop seed traits such as size, shape, and color is crucial for assessing seed quality and improving agricultural productivity. The introduction of the SUnSet toolbox, which employs hyperspectral sensor-derived image analysis, addresses this necessity. In a validation test involving 420 seed accessions from the Korean Soybean Core Collections, the pixel purity index algorithm identified seed- specific hyperspectral endmembers to facilitate segmentation. Various metrics extracted from ventral and lateral side images facilitated the categorization of seeds into three size groups and four shape groups. Additionally, quantitative RGB triplets representing seven seed coat colors, averaged reflectance spectra, and pigment indices were acquired. Machine learning models, trained on a dataset comprising 420 accession seeds and 199 predictors encompassing seed size, shape, and reflectance spectra, achieved accuracy rates of 95.8% for linear discriminant analysis model. Furthermore, a genome-wide association study utilizing hyperspectral features uncovered associations between seed traits and genes governing seed pigmentation and shapes. This comprehensive approach underscores the effectiveness of SUnSet in advancing precision agriculture through meticulous seed trait analysis.
Collapse
Affiliation(s)
- Seok Won Jeong
- Biological Sciences, Chungnam National University, 99 Daehagro, Youseong, Daejon, 34134, Korea
| | - Jae Il Lyu
- Gene Engineering Division, National Institute of Agricultural Sciences, 370 Nongsaengmyeongro, Jeonju, Jeollabuk-do, 54874, Korea
| | - HwangWeon Jeong
- Gene Engineering Division, National Institute of Agricultural Sciences, 370 Nongsaengmyeongro, Jeonju, Jeollabuk-do, 54874, Korea
| | - Jeongho Baek
- Gene Engineering Division, National Institute of Agricultural Sciences, 370 Nongsaengmyeongro, Jeonju, Jeollabuk-do, 54874, Korea
| | - Jung-Kyung Moon
- Crop Foundation Research Division, National Institute of Crop Sciences, 181 Hyeoksinro, Wanju, Jeollabuk-do, 55365, Korea
| | - Chaewon Lee
- Crop Cultivation and Environment Research Division, National Institute of Crop Sciences, 54 Seohoro, Suwon, Kyounggi-do, 16613, Korea
| | - Myoung-Goo Choi
- Wheat Research Team, National Institute of Crop Sciences, RDA, 181 Hyeoksinro, Wanju, Jeollabuk-do, 55365, Korea
| | - Kyoung-Hwan Kim
- Gene Engineering Division, National Institute of Agricultural Sciences, 370 Nongsaengmyeongro, Jeonju, Jeollabuk-do, 54874, Korea
| | - Youn-Il Park
- Biological Sciences, Chungnam National University, 99 Daehagro, Youseong, Daejon, 34134, Korea.
| |
Collapse
|
3
|
Li X, Feng X, Fang H, Yang N, Yang G, Yu Z, Shen J, Geng W, He Y. Classification of multi-year and multi-variety pumpkin seeds using hyperspectral imaging technology and three-dimensional convolutional neural network. PLANT METHODS 2023; 19:82. [PMID: 37563698 PMCID: PMC10413611 DOI: 10.1186/s13007-023-01057-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Accepted: 07/17/2023] [Indexed: 08/12/2023]
Abstract
BACKGROUND Pumpkin seeds are major oil crops with high nutritional value and high oil content. The collection and identification of different pumpkin germplasm resources play a significant role in the realization of precision breeding and variety improvement. In this research, we collected 75 species of pumpkin from the Zhejiang Province of China. 35,927 near-infrared hyperspectral images of 75 types of pumpkin seeds were used as the research object. RESULTS To realize the rapid classification of pumpkin seed varieties, position attention embedded three-dimensional convolutional neural network (PA-3DCNN) was designed based on hyperspectral image technology. The experimental results showed that PA-3DCNN had the best classification effect than other classical machine learning technology. The classification accuracy of 99.14% and 95.20% were severally reached on the training and test sets. We also demonstrated that the PA-3DCNN model performed well in next year's classification with fine-tuning and met with 94.8% accuracy. CONCLUSIONS The model performance improved by introducing double convolution and pooling structure and position attention module. Meanwhile, the generalization performance of the model was verified, which can be adopted for the classification of pumpkin seeds in multiple years. This study provided a new strategy and a feasible technical approach for identifying germplasm resources of pumpkin seeds.
Collapse
Affiliation(s)
- Xiyao Li
- College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, 310058, China
| | - Xuping Feng
- The Rural Development Academy, Zhejiang University, Hangzhou, 310058, China
| | - Hui Fang
- College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, 310058, China
| | - Ningyuan Yang
- College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, 310058, China
| | - Guofeng Yang
- College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, 310058, China
| | - Zeyu Yu
- College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, 310058, China
| | - Jia Shen
- Institute of Vegetables, Zhejiang Academy of Agricultural Sciences, Hangzhou, 310000, China.
| | - Wei Geng
- Institute of Vegetables, Zhejiang Academy of Agricultural Sciences, Hangzhou, 310000, China.
| | - Yong He
- College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, 310058, China.
| |
Collapse
|
4
|
Díaz-Martínez V, Orozco-Sandoval J, Manian V, Dhatt BK, Walia H. A Deep Learning Framework for Processing and Classification of Hyperspectral Rice Seed Images Grown under High Day and Night Temperatures. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23094370. [PMID: 37177572 PMCID: PMC10181662 DOI: 10.3390/s23094370] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Revised: 04/26/2023] [Accepted: 04/26/2023] [Indexed: 05/15/2023]
Abstract
A framework combining two powerful tools of hyperspectral imaging and deep learning for the processing and classification of hyperspectral images (HSI) of rice seeds is presented. A seed-based approach that trains a three-dimensional convolutional neural network (3D-CNN) using the full seed spectral hypercube for classifying the seed images from high day and high night temperatures, both including a control group, is developed. A pixel-based seed classification approach is implemented using a deep neural network (DNN). The seed and pixel-based deep learning architectures are validated and tested using hyperspectral images from five different rice seed treatments with six different high temperature exposure durations during day, night, and both day and night. A stand-alone application with Graphical User Interfaces (GUI) for calibrating, preprocessing, and classification of hyperspectral rice seed images is presented. The software application can be used for training two deep learning architectures for the classification of any type of hyperspectral seed images. The average overall classification accuracy of 91.33% and 89.50% is obtained for seed-based classification using 3D-CNN for five different treatments at each exposure duration and six different high temperature exposure durations for each treatment, respectively. The DNN gives an average accuracy of 94.83% and 91% for five different treatments at each exposure duration and six different high temperature exposure durations for each treatment, respectively. The accuracies obtained are higher than those presented in the literature for hyperspectral rice seed image classification. The HSI analysis presented here is on the Kitaake cultivar, which can be extended to study the temperature tolerance of other rice cultivars.
Collapse
Affiliation(s)
| | | | - Vidya Manian
- University of Puerto Rico, Mayagüez, PR 00681, USA
| | | | | |
Collapse
|
5
|
Orozco J, Manian V, Alfaro E, Walia H, Dhatt BK. Graph Convolutional Network Using Adaptive Neighborhood Laplacian Matrix for Hyperspectral Images with Application to Rice Seed Image Classification. SENSORS (BASEL, SWITZERLAND) 2023; 23:3515. [PMID: 37050573 PMCID: PMC10099153 DOI: 10.3390/s23073515] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/25/2023] [Revised: 03/21/2023] [Accepted: 03/25/2023] [Indexed: 06/19/2023]
Abstract
Graph convolutional neural network architectures combine feature extraction and convolutional layers for hyperspectral image classification. An adaptive neighborhood aggregation method based on statistical variance integrating the spatial information along with the spectral signature of the pixels is proposed for improving graph convolutional network classification of hyperspectral images. The spatial-spectral information is integrated into the adjacency matrix and processed by a single-layer graph convolutional network. The algorithm employs an adaptive neighborhood selection criteria conditioned by the class it belongs to. Compared to fixed window-based feature extraction, this method proves effective in capturing the spectral and spatial features with variable pixel neighborhood sizes. The experimental results from the Indian Pines, Houston University, and Botswana Hyperion hyperspectral image datasets show that the proposed AN-GCN can significantly improve classification accuracy. For example, the overall accuracy for Houston University data increases from 81.71% (MiniGCN) to 97.88% (AN-GCN). Furthermore, the AN-GCN can classify hyperspectral images of rice seeds exposed to high day and night temperatures, proving its efficacy in discriminating the seeds under increased ambient temperature treatments.
Collapse
Affiliation(s)
- Jairo Orozco
- University of Puerto Rico at Mayaguez, Mayagüez, PR 00681, USA
| | - Vidya Manian
- University of Puerto Rico at Mayaguez, Mayagüez, PR 00681, USA
| | | | | | | |
Collapse
|
6
|
Xu P, Sun W, Xu K, Zhang Y, Tan Q, Qing Y, Yang R. Identification of Defective Maize Seeds Using Hyperspectral Imaging Combined with Deep Learning. Foods 2022; 12:foods12010144. [PMID: 36613360 PMCID: PMC9818215 DOI: 10.3390/foods12010144] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 12/22/2022] [Accepted: 12/23/2022] [Indexed: 12/29/2022] Open
Abstract
Seed quality affects crop yield and the quality of agricultural products, and traditional identification methods are time-consuming, complex, and irreversibly destructive. This study aims to establish a fast, non-destructive, and effective approach for defect detection in maize seeds based on hyperspectral imaging (HSI) technology combined with deep learning. Raw spectra collected from maize seeds (200 each healthy and worm-eaten) were pre-processed using detrending (DE) and multiple scattering correction (MSC) to highlight the spectral differences between samples. A convolutional neural network architecture (CNN-FES) based on a feature selection mechanism was proposed according to the importance of wavelength in the target classification task. The results show that the subset of 24 feature wavelengths selected by the proposed CNN-FES can capture important feature information in the spectral data more effectively than the conventional successive projections algorithm (SPA) and competitive adaptive reweighted sampling (CARS) algorithms. In addition, a convolutional neural network architecture (CNN-ATM) based on an attentional classification mechanism was designed for one-dimensional spectral data classification and compared with three commonly used machine learning methods, linear discriminant analysis (LDA), random forest (RF), and support vector machine (SVM). The results show that the classification performance of the designed CNN-ATM on the full wavelength does not differ much from the above three methods, and the classification accuracy is above 90% on both the training and test sets. Meanwhile, the accuracy, sensitivity, and specificity of CNN-ATM based on feature wavelength modeling can reach up to 97.50%, 98.28%, and 96.77% at the highest, respectively. The study shows that hyperspectral imaging-based defect detection of maize seed is feasible and effective, and the proposed method has great potential for the processing and analysis of complex hyperspectral data.
Collapse
Affiliation(s)
- Peng Xu
- College of Information and Communication Engineering, Hainan University, Haikou 570228, China
| | - Wenbin Sun
- College of Information and Communication Engineering, Hainan University, Haikou 570228, China
| | - Kang Xu
- College of Information and Communication Engineering, Hainan University, Haikou 570228, China
| | - Yunpeng Zhang
- College of Mechanical and Electrical Engineering, Hainan University, Haikou 570228, China
| | - Qian Tan
- College of Mechanical and Electrical Engineering, Hainan University, Haikou 570228, China
| | - Yiren Qing
- College of Mechanical and Electrical Engineering, Hainan University, Haikou 570228, China
| | - Ranbing Yang
- College of Mechanical and Electrical Engineering, Hainan University, Haikou 570228, China
- Correspondence: ; Tel.: +86-0898-66267576
| |
Collapse
|
7
|
Chandran AKN, Sandhu J, Irvin L, Paul P, Dhatt BK, Hussain W, Gao T, Staswick P, Yu H, Morota G, Walia H. Rice Chalky Grain 5 regulates natural variation for grain quality under heat stress. FRONTIERS IN PLANT SCIENCE 2022; 13:1026472. [PMID: 36304400 PMCID: PMC9593041 DOI: 10.3389/fpls.2022.1026472] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Accepted: 09/21/2022] [Indexed: 06/16/2023]
Abstract
Heat stress occurring during rice (Oryza sativa) grain development reduces grain quality, which often manifests as increased grain chalkiness. Although the impact of heat stress on grain yield is well-studied, the genetic basis of rice grain quality under heat stress is less explored as quantifying grain quality is less tractable than grain yield. To address this, we used an image-based colorimetric assay (Red, R; and Green, G) for genome-wide association analysis to identify genetic loci underlying the phenotypic variation in rice grains exposed to heat stress. We found the R to G pixel ratio (RG) derived from mature grain images to be effective in distinguishing chalky grains from translucent grains derived from control (28/24°C) and heat stressed (36/32°C) plants. Our analysis yielded a novel gene, rice Chalky Grain 5 (OsCG5) that regulates natural variation for grain chalkiness under heat stress. OsCG5 encodes a grain-specific, expressed protein of unknown function. Accessions with lower transcript abundance of OsCG5 exhibit higher chalkiness, which correlates with higher RG values under stress. These findings are supported by increased chalkiness of OsCG5 knock-out (KO) mutants relative to wildtype (WT) under heat stress. Grains from plants overexpressing OsCG5 are less chalky than KOs but comparable to WT under heat stress. Compared to WT and OE, KO mutants exhibit greater heat sensitivity for grain size and weight relative to controls. Collectively, these results show that the natural variation at OsCG5 may contribute towards rice grain quality under heat stress.
Collapse
Affiliation(s)
| | - Jaspreet Sandhu
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, United States
| | - Larissa Irvin
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, United States
| | - Puneet Paul
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, United States
| | - Balpreet K. Dhatt
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, United States
| | - Waseem Hussain
- Rice Breeding Innovation Platform, International Rice Research Institute (IRRI), Los Banos, Philippines
| | - Tian Gao
- Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, NE, United States
| | - Paul Staswick
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, United States
| | - Hongfeng Yu
- Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, NE, United States
| | - Gota Morota
- Department of Animal and Poultry Sciences, Virginia Polytechnic Institute and State University, Blacksburg, VA, United States
| | - Harkamal Walia
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, United States
| |
Collapse
|