Variety Identification of Single Rice Seed Using Hyperspectral Imaging Combined with Convolutional Neural Network

Research on variety identification of common bean seeds based on hyperspectral and deep learning

Accurate, fast and non-destructive identification of varieties of common bean seeds is important for the cultivation and efficient utilization of common beans. This study is based on hyperspectral and deep learning to identify the varieties of common bean seeds non-destructively. In this study, the average spectrum of 3078 hyperspectral images from 500 varieties was obtained after image segmentation and sensitive region extraction, and the Synthetic Minority Over-sampling Technique (SMOTE) was used to achieve the equilibrium of the samples of various varieties. A one-dimensional convolutional neural network model (IResCNN) incorporating Inception module and residual structure was proposed to identify seed varieties, and Support Vector Machine (SVM), K-Nearest Neighbor (KNN), VGG19, AlexNet, ResNet50 were established to compare the identification effect. After analyzing the effects of multiple spectral preprocessing methods on the model, the study selected Savitzky-Golay smoothing correction (SG) for spectral preprocessing and extracted 66 characteristic wavelengths using Successive Projections Algorithm (SPA) as inputs to the discriminative model. Ultimately, the IResCNN model achieved the highest accuracy of 93.06 % on the test set, indicating that hyperspectral technology can accurately identify bean varieties, and the study provides a correct method of thinking for the non-destructive classification of multi-species small-sample bean varieties.

Automation and Optimization of Food Process Using CNN and Six-Axis Robotic Arm

The Food Process Robot Intelligent System (FPRIS) integrates a 3D-printed six-axis robotic arm with Artificial Intelligence (AI) and Computer Vision (CV) to optimize and automate the coffee roasting process. As an application of FPRIS coffee roasting, this system uses a Convolutional Neural Network (CNN) to classify coffee beans inside the roaster and control the roaster in real time, avoiding obstacles and empty spaces. This study demonstrates FPRIS's capability to precisely control the Degree of Roasting (DoR) by combining gas and image sensor data to assess coffee bean quality. A comparative analysis between the Preliminary Coffee Sample (PCS) and Validation Coffee Sample (VCS) revealed that increasing roast intensity resulted in consistent trends for both samples, including an increase in weight loss and Gas sensor Initial Difference (GID) and a decrease in Sum of Pixel Grayscale Values (SPGVs). This study underscores the potential of FPRIS to enhance precision and efficiency in coffee roasting. Future studies will expand on these findings by testing FPRIS across various food processes, potentially establishing a universal automation system for the food industry.

Machine learning-based non-destructive terahertz detection of seed quality in peanut

Rapid identification of peanut seed quality is crucial for public health. In this study, we present a terahertz wave imaging system using a convolutional neural network (CNN) machine learning approach. Terahertz waves are capable of penetrating the seed shell to identify the quality of peanuts without causing any damage to the seeds. The specificity of seed quality on terahertz wave images is investigated, and the image characteristics of five different qualities are summarized. Terahertz wave images are digitized and used for training and testing of convolutional neural networks, resulting in a high model accuracy of 98.7% in quality identification. The trained THz-CNNs system can accurately identify standard, mildewed, defective, dried and germinated seeds, with an average detection time of 2.2 s. This process does not require any sample preparation steps such as concentration or culture. Our method swiftly and accurately assesses shelled seed quality non-destructively.

Paddy seed viability prediction based on feature fusion of color and hyperspectral image with multivariate analysis

Seed viability is essential to have a homogeneous plant population. The seed industry cannot adopt traditional procedures for seed viability evaluation since they are destructive, time-consuming, and need chemicals. This study aimed to investigate the potential of combining hyperspectral and color image features to differentiate viable and non-viable paddy seeds. The hyperspectral and color image of the 355 paddy seeds was captured and later used to examine their viability. An image processing algorithm was developed to extract features from color images of paddy seeds and investigated significant differences in the retrieved feature data using variance analysis. The spectra were extracted from the selected region of interest (ROI) of the hyperspectral paddy seed image and averaged. In the next step, the partial least square discrimination analysis (PLS-DA) model was developed to distinguish viable and non-viable paddy seeds. Initially, the PLS-DA model was developed using spectral data with different preprocessing techniques, and the result obtained an accuracy of 88.9 % in the calibration set and 86.1 % in the prediction set using Savitzky-Golay 2nd derivative preprocessed spectra. With the fusion of spectral and significant color image features, the model's accuracy improved to 93.3 % and 90.9 % in the calibration and prediction sets, respectively. Results also showed that the fusion of selected color image features with Savitzky-Golay 2nd derivative preprocessed spectra could achieve higher F1-score, recall, and precision values. The visualization map for the viable and non-viable paddy seeds was also developed utilizing the most effective predictive model. The results demonstrate the possibility of using the fusion of the hyperspectral and color image features to sort seeds according to viability, which may be applied in developing an online seed sorting method.

Study on the Determination of Flavor Value of Rice Based on Grid Iterative Search Swarm Optimization Support Vector Machine Model and Hyperspectral Imaging

In the field of rice processing and cultivation, it is crucial to adopt efficient, rapid and user-friendly techniques to detect the flavor values of various rice varieties. The conventional methods for flavor value assessment mainly rely on chemical analysis and technical evaluation, which not only deplete the rice resources but also incur significant time and labor costs. In this study, hyperspectral imaging technology was utilized in combination with an improved Particle Swarm Optimization Support Vector Machine (PSO-SVM) algorithm, i.e., the Grid Iterative Search Particle Swarm Optimization Support Vector Machine (GISPSO-SVM) algorithm, introducing a new non-destructive technique to determine the flavor value of rice. The method captures the hyperspectral feature data of different rice varieties through image acquisition, preprocessing and feature extraction, and then uses these features to train a model using an optimized machine learning algorithm. The results show that the introduction of GIS algorithms in a PSO-optimized SVM is very effective and can improve the parameter finding ability. In terms of flavor value prediction accuracy, the Principal Component Analysis (PCA) combined with the GISPSO-SVM algorithm achieved 96% accuracy, which was higher than the 93% of the Competitive Adaptive Weighted Sampling (CARS) algorithm. And the introduction of the GIS algorithm in different feature selection can improve the accuracy to different degrees. This novel approach helps to evaluate the flavor values of new rice varieties non-destructively and provides a new perspective for future rice flavor value detection methods.

Grain Protein Content Phenotyping in Rice via Hyperspectral Imaging Technology and a Genome-Wide Association Study

Efficient and accurate acquisition of the rice grain protein content (GPC) is important for selecting high-quality rice varieties, and remote sensing technology is an attractive potential method for this task. However, the majority of multispectral sensors are poor predictors of GPC due to their broad spectral bands. Hyperspectral technology provides a new analytical technology for bridging the gap between phenomics and genomics. However, the small size of typical datasets is a constraint for model construction for estimating GPC, limiting their accuracy and reducing their ability to generalize to a wide range of varieties. In this study, we used hyperspectral data of rice grains from 515 japonica varieties and deep convolution generative adversarial networks (DCGANs) to generate simulated data to improve the model accuracy. Features sensitive to GPC were extracted after applying a continuous wavelet transform (CWT), and the estimated GPC model was constructed by partial least squares regression (PLSR). Finally, a genome-wide association study (GWAS) was applied to the measured and generated datasets to detect GPC loci. The results demonstrated that the simulated GPC values generated after 8,000 epochs were closest to the measured values. The wavelet feature (WF1743, 2), obtained from the data with the addition of 200 simulated samples, exhibited the highest GPC estimation accuracy (R 2 = 0.58 and RRMSE = 6.70%). The GWAS analysis showed that the estimated values based on the simulated data detected the same loci as the measured values, including the OsmtSSB1L gene related to grain storage protein. This study provides a new technique for the efficient genetic study of phenotypic traits in rice based on hyperspectral technology.

SUnSeT: spectral unmixing of hyperspectral images for phenotyping soybean seed traits

KEY MESSAGE

Hyperspectral features enable accurate classification of soybean seeds using linear discriminant analysis and GWAS for novel seed trait genes. Evaluating crop seed traits such as size, shape, and color is crucial for assessing seed quality and improving agricultural productivity. The introduction of the SUnSet toolbox, which employs hyperspectral sensor-derived image analysis, addresses this necessity. In a validation test involving 420 seed accessions from the Korean Soybean Core Collections, the pixel purity index algorithm identified seed- specific hyperspectral endmembers to facilitate segmentation. Various metrics extracted from ventral and lateral side images facilitated the categorization of seeds into three size groups and four shape groups. Additionally, quantitative RGB triplets representing seven seed coat colors, averaged reflectance spectra, and pigment indices were acquired. Machine learning models, trained on a dataset comprising 420 accession seeds and 199 predictors encompassing seed size, shape, and reflectance spectra, achieved accuracy rates of 95.8% for linear discriminant analysis model. Furthermore, a genome-wide association study utilizing hyperspectral features uncovered associations between seed traits and genes governing seed pigmentation and shapes. This comprehensive approach underscores the effectiveness of SUnSet in advancing precision agriculture through meticulous seed trait analysis.

EfficientMaize: A Lightweight Dataset for Maize Classification on Resource-Constrained Devices

Hyperspectral imaging, combined with deep learning techniques, has been employed to classify maize. However, the implementation of these automated methods often requires substantial processing and computing resources, presenting a significant challenge for deployment on embedded devices due to high GPU power consumption. Access to Ghanaian local maize data for such classification tasks is also extremely difficult in Ghana. To address these challenges, this research aims to create a simple dataset comprising three distinct types of local maize seeds in Ghana. The goal is to facilitate the development of an efficient maize classification tool that minimizes computational costs and reduces human involvement in the process of grading seeds for marketing and production. The dataset is presented in two parts: raw images, consisting of 4,846 images, are categorized into bad and good. Specifically, 2,211 images belong to the bad class, while 2,635 belong to the good class. Augmented images consist of 28,910 images, with 13,250 representing bad data and 15,660 representing good data. All images have been validated by experts from Heritage Seeds Ghana and are freely available for use within the research community.

Hyperspectral Imaging and Machine Learning as a Nondestructive Method for Proso Millet Seed Detection and Classification

Millet is a small-seeded cereal crop with big potential. There are many different cultivars of proso millet (Panicum miliaceum L.) with different characteristics, bringing forth the issue of sorting which are important for growers, processors, and consumers. Current methods of grain cultivar detection and classification are subjective, destructive, and time-consuming. Therefore, there is a need to develop nondestructive methods for sorting the cultivars of proso millet. In this study, the feasibility of using near-infrared (NIR) hyperspectral imaging (900-1700 nm) to discriminate between different cultivars of proso millet seeds was evaluated. A total of 5000 proso millet seeds were randomly obtained and investigated from the ten most popular cultivars in the United States, namely Cerise, Cope, Earlybird, Huntsman, Minco, Plateau, Rise, Snowbird, Sunrise, and Sunup. To reduce the large dimensionality of the hyperspectral imaging, principal component analysis (PCA) was applied, and the first two principal components were used as spectral features for building the classification models because they had the largest variance. The classification performance showed prediction accuracy rates as high as 99% for classifying the different cultivars of proso millet using a Gradient tree boosting ensemble machine learning algorithm. Moreover, the classification was successfully performed using only 15 and 5 selected spectral features (wavelengths), with an accuracy of 98.14% and 97.6%, respectively. The overall results indicate that NIR hyperspectral imaging could be used as a rapid and nondestructive method for the classification of proso millet seeds.

Non-destructive identification of Pseudostellaria heterophylla from different geographical origins by Vis/NIR and SWIR hyperspectral imaging techniques

The composition of Pseudostellaria heterophylla (Tai-Zi-Shen, TZS) is greatly influenced by the growing area of the plants, making it significant to distinguish the origins of TZS. However, traditional methods for TZS origin identification are time-consuming, laborious, and destructive. To address this, two or three TZS accessions were selected from four different regions of China, with each of these resources including distinct quality grades of TZS samples. The visible near-infrared (Vis/NIR) and short-wave infrared (SWIR) hyperspectral information from these samples were then collected. Fast and high-precision methods to identify the origins of TZS were developed by combining various preprocessing algorithms, feature band extraction algorithms (CARS and SPA), traditional two-stage machine learning classifiers (PLS-DA, SVM, and RF), and an end-to-end deep learning classifier (DCNN). Specifically, SWIR hyperspectral information outperformed Vis/NIR hyperspectral information in detecting geographic origins of TZS. The SPA algorithm proved particularly effective in extracting SWIR information that was highly correlated with the origins of TZS. The corresponding FD-SPA-SVM model reduced the number of bands by 77.2% and improved the model accuracy from 97.6% to 98.1% compared to the full-band FD-SVM model. Overall, two sets of fast and high-precision models, SWIR-FD-SPA-SVM and SWIR-FD-DCNN, were established, achieving accuracies of 98.1% and 98.7% respectively. This work provides a potentially efficient alternative for rapidly detecting the origins of TZS during actual production.

Systematic meta-analysis of computer-aided detection to detect early esophageal cancer using hyperspectral imaging

One of the leading causes of cancer deaths is esophageal cancer (EC) because identifying it in early stage is challenging. Computer-aided diagnosis (CAD) could detect the early stages of EC have been developed in recent years. Therefore, in this study, complete meta-analysis of selected studies that only uses hyperspectral imaging to detect EC is evaluated in terms of their diagnostic test accuracy (DTA). Eight studies are chosen based on the Quadas-2 tool results for systematic DTA analysis, and each of the methods developed in these studies is classified based on the nationality of the data, artificial intelligence, the type of image, the type of cancer detected, and the year of publishing. Deeks' funnel plot, forest plot, and accuracy charts were made. The methods studied in these articles show the automatic diagnosis of EC has a high accuracy, but external validation, which is a prerequisite for real-time clinical applications, is lacking.

Feasibility Study of Combining Hyperspectral Imaging with Deep Learning for Chestnut-Quality Detection

The rapid detection of chestnut quality is a critical aspect of chestnut processing. However, traditional imaging methods pose a challenge for chestnut-quality detection due to the absence of visible epidermis symptoms. This study aims to develop a quick and efficient detection method using hyperspectral imaging (HSI, 935-1720 nm) and deep learning modeling for qualitative and quantitative identification of chestnut quality. Firstly, we used principal component analysis (PCA) to visualize the qualitative analysis of chestnut quality, followed by the application of three pre-processing methods to the spectra. To compare the accuracy of different models for chestnut-quality detection, traditional machine learning models and deep learning models were constructed. Results showed that deep learning models were more accurate, with FD-LSTM achieving the highest accuracy of 99.72%. Moreover, the study identified important wavelengths for chestnut-quality detection at around 1000, 1400 and 1600 nm, to improve the efficiency of the model. The FD-UVE-CNN model achieved the highest accuracy of 97.33% after incorporating the important wavelength identification process. By using the important wavelengths as input for the deep learning network model, recognition time decreased on average by 39 s. After a comprehensive analysis, FD-UVE-CNN was deter-mined to be the most effective model for chestnut-quality detection. This study suggests that deep learning combined with HSI has potential for chestnut-quality detection, and the results are encouraging.

A Deep Learning Framework for Processing and Classification of Hyperspectral Rice Seed Images Grown under High Day and Night Temperatures

A framework combining two powerful tools of hyperspectral imaging and deep learning for the processing and classification of hyperspectral images (HSI) of rice seeds is presented. A seed-based approach that trains a three-dimensional convolutional neural network (3D-CNN) using the full seed spectral hypercube for classifying the seed images from high day and high night temperatures, both including a control group, is developed. A pixel-based seed classification approach is implemented using a deep neural network (DNN). The seed and pixel-based deep learning architectures are validated and tested using hyperspectral images from five different rice seed treatments with six different high temperature exposure durations during day, night, and both day and night. A stand-alone application with Graphical User Interfaces (GUI) for calibrating, preprocessing, and classification of hyperspectral rice seed images is presented. The software application can be used for training two deep learning architectures for the classification of any type of hyperspectral seed images. The average overall classification accuracy of 91.33% and 89.50% is obtained for seed-based classification using 3D-CNN for five different treatments at each exposure duration and six different high temperature exposure durations for each treatment, respectively. The DNN gives an average accuracy of 94.83% and 91% for five different treatments at each exposure duration and six different high temperature exposure durations for each treatment, respectively. The accuracies obtained are higher than those presented in the literature for hyperspectral rice seed image classification. The HSI analysis presented here is on the Kitaake cultivar, which can be extended to study the temperature tolerance of other rice cultivars.

Vis-NIR Spectroscopy Combined with GAN Data Augmentation for Predicting Soil Nutrients in Degraded Alpine Meadows on the Qinghai-Tibet Plateau

Soil nutrients play vital roles in vegetation growth and are a key indicator of land degradation. Accurate, rapid, and non-destructive measurement of the soil nutrient content is important for ecological conservation, degradation monitoring, and precision farming. Currently, visible and near-infrared (Vis-NIR) spectroscopy allows for rapid and non-destructive monitoring of soil nutrients. However, the performance of Vis-NIR inversion models is extremely dependent on the number of samples. Limited samples may lead to low prediction accuracy of the models. Therefore, modeling and prediction based on a small sample size remain a challenge. This study proposes a method for the simultaneous augmentation of soil spectral and nutrient data (total nitrogen (TN), soil organic matter (SOM), total potassium oxide (TK₂O), and total phosphorus pentoxide (TP₂O₅)) using a generative adversarial network (GAN). The sample augmentation range and the level of accuracy improvement were also analyzed. First, 42 soil samples were collected from the pika disturbance area on the QTP. The collected soils were measured in the laboratory for Vis-NIR and TN, SOM, TK₂O, and TP₂O₅ data. A GAN was then used to augment the soil spectral and nutrient data simultaneously. Finally, the effect of adding different numbers of generative samples to the training set on the predictive performance of a convolutional neural network (CNN) was analyzed and compared with another data augmentation method (extended multiplicative signal augmentation, EMSA). The results showed that a GAN can generate data very similar to real data and with better diversity. A total of 15, 30, 60, 120, and 240 generative samples (GAN and EMSA) were randomly selected from 300 generative samples to be included in the real data to train the CNN model. The model performance first improved and then deteriorated, and the GAN was more effective than EMSA. Further shortening the interval for adding GAN data revealed that the optimal ranges were 30-40, 50-60, 30-35, and 25-35 for TK₂O, TN, TP₂O₅, and SOM, respectively, and the validation set accuracy was maximized in these ranges. Therefore, the above method can compensate to some extent for insufficient samples in the hyperspectral prediction of soil nutrients, and can quickly and accurately estimate the content of soil TK₂O, TN, TP₂O₅, and SOM.

A Deep Learning Image System for Classifying High Oleic Sunflower Seed Varieties

Sunflower seeds, one of the main oilseeds produced around the world, are widely used in the food industry. Mixtures of seed varieties can occur throughout the supply chain. Intermediaries and the food industry need to identify the varieties to produce high-quality products. Considering that high oleic oilseed varieties are similar, a computer-based system to classify varieties could be useful to the food industry. The objective of our study is to examine the capacity of deep learning (DL) algorithms to classify sunflower seeds. An image acquisition system, with controlled lighting and a Nikon camera in a fixed position, was constructed to take photos of 6000 seeds of six sunflower seed varieties. Images were used to create datasets for training, validation, and testing of the system. A CNN AlexNet model was implemented to perform variety classification, specifically classifying from two to six varieties. The classification model reached an accuracy value of 100% for two classes and 89.5% for the six classes. These values can be considered acceptable, because the varieties classified are very similar, and they can hardly be classified with the naked eye. This result proves that DL algorithms can be useful for classifying high oleic sunflower seeds.

Discrimination of Pericarpium Citri Reticulatae in different years using Terahertz Time-Domain spectroscopy combined with convolutional neural network

Pericarpium Citri Reticulatae (PCR) in longer storage years possess higher medicinal values, but their differentiation is difficult due to similar morphological characteristics. Therefore, this study investigated the feasibility of using terahertz time-domain spectroscopy (THz-TDS) combined with a convolutional neural network (CNN) to identify PCR samples stored from 1 to 20 years. The absorption coefficient and refractive index spectra in the range of 0.2-1.5 THz were acquired. Partial least squares discriminant analysis, random forest, least squares support vector machines, and CNN were used to establish discriminant models, showing better performance of the CNN model than the others. In addition, the output data points of the CNN intermediate layer were visualized, illustrating gradual changes in these points from overlapping to clear separation. Overall, THz-TDS combined with CNN models could realize rapid identification of different year PCRs, thus providing an efficient alternative method for PCR quality inspection.

Spectral-Based Classification of Genetically Differentiated Groups in Spring Wheat Grown under Contrasting Environments

The global concern about the gap between food production and consumption has intensified the research on the genetics, ecophysiology, and breeding of cereal crops. In this sense, several genetic studies have been conducted to assess the effectiveness and sustainability of collections of germplasm accessions of major crops. In this study, a spectral-based classification approach for the assignment of wheat cultivars to genetically differentiated subpopulations (genetic structure) was carried out using a panel of 316 spring bread cultivars grown in two environments with different water regimes (rainfed and fully irrigated). For that, different machine-learning models were trained with foliar spectral and genetic information to assign the wheat cultivars to subpopulations. The results revealed that, in general, the hyperparameters ReLU (as the activation function), adam (as the optimizer), and a size batch of 10 give neural network models better accuracy. Genetically differentiated groups showed smaller differences in mean wavelengths under rainfed than under full irrigation, which coincided with a reduction in clustering accuracy in neural network models. The comparison of models indicated that the Convolutional Neural Network (CNN) was significantly more accurate in classifying individuals into their respective subpopulations, with 92 and 93% of correct individual assignments in water-limited and fully irrigated environments, respectively, whereas 92% (full irrigation) and 78% (rainfed) of cultivars were correctly assigned to their respective classes by the multilayer perceptron method and partial least squares discriminant analysis, respectively. Notably, CNN did not show significant differences between both environments, which indicates stability in the prediction independent of the different water regimes. It is concluded that foliar spectral variation can be used to accurately infer the belonging of a cultivar to its respective genetically differentiated group, even considering radically different environments, which is highly desirable in the context of crop genetic resources management.

Development and comparison of classification models on VIS-NIR hyperspectral imaging spectra for qualitative detection of the Staphylococcus aureus in fresh chicken breast

Chicken is at risk of contaminated foodborne pathogens in the production process. Timely and nondestructive detection of foodborne pathogens in chicken is essential for food security. The study aims to explore the feasibility of developing efficient classification models for qualitative detection of Staphylococcus aureus in chicken breast using the hyperspectral imaging technique. Principal component analysis was used to process the full spectral information and three wavelength selection methods (competitive adaptive reweighted sampling, genetic algorithm, and successive projections algorithm) were applied to extract effective wavelengths. These methods were combined with the support vector machine algorithm to develop conventional classification models, respectively. In addition, a convolutional neural network model based on deep learning was designed and trained for comparison. The performance of the convolutional neural network model was significantly better than that of conventional classification models. The overall accuracy for chicken sample classifications was improved from 83.88% to 91.38%. The results demonstrated that deep learning can effectively extract spectral features and promote the application of hyperspectral imaging in foodborne pathogens detection of chicken products.

Rice Labeling according to Grain Quality Features Using Laser-Induced Breakdown Spectroscopy

Rice is an important source of nutrition and energy consumed around the world. Thus, quality inspection is crucial for protecting consumers and increasing the rice's value in the productive chain. Currently, methods for rice labeling depending on grain quality features are based on image and/or visual inspection. These methods have shown subjectivity and inefficiency for large-scale analyses. Laser-induced breakdown spectroscopy (LIBS) is an analytical technique showing attractive features due to how quick the analysis can be carried out and its capability of providing spectra that are true fingerprints of the sample's elemental composition. In this work, LIBS performance was evaluated for labeling rice according to grain quality features. The LIBS spectra of samples with their grain quality numerically described as Type 1, 2, and 3 were measured. Several spectral processing methods were evaluated when modeling a k-nearest neighbors (k-NN) classifier. Variable selection was also carried out by principal component analysis (PCA), and then the optimal k-value was selected. The best result was obtained by applying spectrum smoothing followed by normalization by using the first fifteen principal components (PCs) as input variables and k = 9. Under these conditions, the method showed excellent performance, achieving sample classification with 94% overall prediction accuracy. The sensitivities ranged from 90 to 100%, and specificities were in the range of 92-100%. The proposed method has remarkable characteristics, e.g., analytical speed and analysis guided by chemical responses; therefore, the method is not susceptible to subjectivity errors.

Neural network in food analytics

Neural network (i.e. deep learning, NN)-based data analysis techniques have been listed as a pivotal opportunity to protect the integrity and safety of the global food supply chain and forecast $11.2 billion in agriculture markets. As a general-purpose data analytic tool, NN has been applied in several areas of food science, such as food recognition, food supply chain security and omics analysis, and so on. Therefore, given the rapid emergence of NN applications in food safety, this review aims to provide a comprehensive overview of the NN application in food analysis for the first time, focusing on domain-specific applications in food analysis by introducing fundamental methodology, reviewing recent and notable progress, and discussing challenges and potential pitfalls. NN demonstrated that it has a bright future through effective collaboration between food specialist and the broader community in the food field, for example, superiority in food recognition, sensory evaluation, pattern recognition of spectroscopy and chromatography. However, major challenges impeded NN extension including void in the food scientist-friendly interface software package, incomprehensible model behavior, multi-source heterogeneous data, and so on. The breakthrough from other fields proved NN has the potential to offer a revolution in the immediate future.

Hyperspectral remote sensing to assess weed competitiveness in maize farmland ecosystems

Weed competition causes serious economic losses to maize production. Timely and accurate assessment of pressure from competition is crucial for ecological weed management. In this work, we apply hyperspectral remote sensing (HRS) technology to conduct a competitive experiment in Harbin, China, in 2021, with 5-leaf maize as the study target. A weed competition assessment method that combines comprehensive competition indices (CCI) and deep learning is proposed. For the comprehensive competition assessment, the relationship between different weed competitive pressures (Levels 1-5) and changes in the structural and physiological information of maize was analyzed. The accumulative/transient competition indices CCI-A and CCI-T were designed for accurate quantification. The results showed that parameters such as plant height, stalk thickness and nutrient elements of maize decreased with increasing competition level. Parameters, such as stomatal conductance and transpiration rate, showed a fluctuating change of increasing and then decreasing with increasing competition level. Compared with the traditional relative competitive intensity (RCI), the standard deviation of CCI is 0.303 and 0.499. The dispersion effect of CCI is better and more suitable for quantifying the competition response. HRS images combined with 3D-CNN model were then applied to reveal the spectral response to different weed competition pressures (Levels 1-5) and to make early predictions of weed competition. The first-order derivative showed that the spectral reflectance exhibited significant differences at 520-525 nm peak, 570-655 nm trough, and near 700 nm red edge. For hyperspectral spatial-spectral features, the 3D-CNN model is proposed for prediction of competing indices CCI. In addition, the VIP method is used to select the characteristic wavelengths. The 3D-CNN model achieves a prediction accuracy of RMSE = 0.106 and 0.152 using 13 feature bands, which can accurately quantify the subtle changes in competition indices. Overall, this study shows that the combination of CCI and deep learning can provide a multivariate and comprehensive assessment of weed competition pressure.

Convolutional neural network for apple bruise detection based on hyperspectral

The timely detection of apple bruises caused by collision and squeeze is of great significance to reduce the economic losses of the apple industry. This study proposed a spectral analysis model (SpectralCNN) based on a one-dimensional convolutional neural network to detect apple bruises. The influences of six spectral preprocessing methods on the SpectralCNN model were firstly analyzed in this paper. Compared with traditional chemometric models, the SpectralCNN model had a better accuracy, which was demonstrated not depend on the spectral preprocessing method by experiment results. Then, 20 characteristic wavelengths could be extracted by successive projection algorithm. The SpectralCNN model could achieve an accuracy of 95.79% on the test set of characteristic wavelengths, indicating that the extracted characteristic wavelengths contain most of the features of bruised and healthy pixels.

Deep Learning Combined with Hyperspectral Imaging Technology for Variety Discrimination of Fritillaria thunbergii

Traditional Chinese herbal medicine (TCHM) plays an essential role in the international pharmaceutical industry due to its rich resources and unique curative properties. The flowers, stems, and leaves of Fritillaria contain a wide range of phytochemical compounds, including flavonoids, essential oils, saponins, and alkaloids, which may be useful for medicinal purposes. Fritillaria thunbergii Miq. Bulbs are commonly used in traditional Chinese medicine as expectorants and antitussives. In this paper, a feasibility study is presented that examines the use of hyperspectral imaging integrated with convolutional neural networks (CNN) to distinguish twelve (12) Fritillaria varieties (n = 360). The performance of support vector machines (SVM) and partial least squares-discriminant analysis (PLS-DA) was compared with that of convolutional neural network (CNN). Principal component analysis (PCA) was used to assess the presence of cluster trends in the spectral data. To optimize the performance of the models, cross-validation was used. Among all the discriminant models, CNN was the most accurate with 98.88%, 88.89% in training and test sets, followed by PLS-DA and SVM with 92.59%, 81.94% and 99.65%, 79.17%, respectively. The results obtained in the present study revealed that application of HSI in conjunction with the deep learning technique can be used for classification of Fritillaria thunbergii varieties rapidly and non-destructively.

End-to-End Fusion of Hyperspectral and Chlorophyll Fluorescence Imaging to Identify Rice Stresses

Herbicides and heavy metals are hazardous substances of environmental pollution, resulting in plant stress and harming humans and animals. Identification of stress types can help trace stress sources, manage plant growth, and improve stress-resistant breeding. In this research, hyperspectral imaging (HSI) and chlorophyll fluorescence imaging (Chl-FI) were adopted to identify the rice plants under two types of herbicide stresses (butachlor (DCA) and quinclorac (ELK)) and two types of heavy metal stresses (cadmium (Cd) and copper (Cu)). Visible/near-infrared spectra of leaves (L-VIS/NIR) and stems (S-VIS/NIR) extracted from HSI and chlorophyll fluorescence kinetic curves of leaves (L-Chl-FKC) and stems (S-Chl-FKC) extracted from Chl-FI were fused to establish the models to detect the stress of the hazardous substances. Novel end-to-end deep fusion models were proposed for low-level, middle-level, and high-level information fusion to improve identification accuracy. Results showed that the high-level fusion-based convolutional neural network (CNN) models reached the highest detection accuracy (97.7%), outperforming the models using a single data source (<94.7%). Furthermore, the proposed end-to-end deep fusion models required a much simpler training procedure than the conventional two-stage deep learning fusion. This research provided an efficient alternative for plant stress phenotyping, including identifying plant stresses caused by hazardous substances of environmental pollution.

Characterisation of Pineapple Cultivars under Different Storage Conditions Using Infrared Thermal Imaging Coupled with Machine Learning Algorithms

The non-invasive ability of infrared thermal imaging has gained interest in various food classification and recognition tasks. In this work, infrared thermal imaging was used to distinguish different pineapple cultivars, i.e., MD2, Morris, and Josapine, which were subjected to different storage temperatures, i.e., 5, 10, and 25 °C and a relative humidity of 85% to 90%. A total of 14 features from the thermal images were obtained to determine the variation in terms of image parameters among the different pineapple cultivars. Principal component analysis was applied for feature reduction in order to prevent any effect of significant difference between the selected features. Several types of machine learning algorithms were compared, including linear discriminant analysis, quadratic discriminant analysis, support vector machine, k-nearest neighbour, decision tree, and naïve Bayes, to obtain the best performance for the classification of pineapple cultivars. The results showed that support vector machine achieved the best performance from the combination of optimal image parameters with the highest classification rate of 100%. The ability of infrared thermal imaging coupled with machine learning approaches can be potentially used to distinguish pineapple cultivars, which could enhance the grading and sorting processes of the fruit.

A Neural Network-Based Spectral Approach for the Assignment of Individual Trees to Genetically Differentiated Subpopulations

Studying population structure has made an essential contribution to understanding evolutionary processes and demographic history in forest ecology research. This inference process basically involves the identification of common genetic variants among individuals, then grouping the similar individuals into subpopulations. In this study, a spectral-based classification of genetically differentiated groups was carried out using a provenance–progeny trial of Eucalyptus cladocalyx. First, the genetic structure was inferred through a Bayesian analysis using single-nucleotide polymorphisms (SNPs). Then, different machine learning models were trained with foliar spectral information to assign individual trees to subpopulations. The results revealed that spectral-based classification using the multilayer perceptron method was very successful at classifying individuals into their respective subpopulations (with an average of 87% of correct individual assignments), whereas 85% and 81% of individuals were assigned to their respective classes correctly by convolutional neural network and partial least squares discriminant analysis, respectively. Notably, 93% of individual trees were assigned correctly to the class with the smallest size using the spectral data-based multi-layer perceptron classification method. In conclusion, spectral data, along with neural network models, are able to discriminate and assign individuals to a given subpopulation, which could facilitate the implementation and application of population structure studies on a large scale.

Deep Learning and Hyperspectral Images Based Tomato Soluble Solids Content and Firmness Estimation

Cherry tomato (Solanum lycopersicum) is popular with consumers over the world due to its special flavor. Soluble solids content (SSC) and firmness are two key metrics for evaluating the product qualities. In this work, we develop non-destructive testing techniques for SSC and fruit firmness based on hyperspectral images and the corresponding deep learning regression model. Hyperspectral reflectance images of over 200 tomato fruits are derived with the spectrum ranging from 400 to 1,000 nm. The acquired hyperspectral images are corrected and the spectral information are extracted. A novel one-dimensional (1D) convolutional ResNet (Con1dResNet) based regression model is proposed and compared with the state of art techniques. Experimental results show that, with a relatively large number of samples our technique is 26.4% better than state of art technique for SSC and 33.7% for firmness. The results of this study indicate the application potential of hyperspectral imaging technique in the SSC and firmness detection, which provides a new option for non-destructive testing of cherry tomato fruit quality in the future.

Advances in infrared spectroscopy and hyperspectral imaging combined with artificial intelligence for the detection of cereals quality

Cereals provide humans with essential nutrients, and its quality assessment has attracted widespread attention. Infrared (IR) spectroscopy (IRS) and hyperspectral imaging (HSI), as powerful nondestructive testing technologies, are widely used in the quality monitoring of food and agricultural products. Artificial intelligence (AI) plays a crucial role in data mining, especially in recent years, a new generation of AI represented by deep learning (DL) has made breakthroughs in analyzing spectral data of food and agricultural products. The combination of IRS/HSI and AI further promotes the development of quality evaluation of cereals. This paper comprehensively reviews the advances of IRS and HSI combined with AI in the detection of cereals quality. The aim is to present a complete review topic as it touches the background knowledge, instrumentation, spectral data processing (including preprocessing, feature extraction and modeling), spectral interpretation, etc. To suit this goal, principles of IRS and HSI, as well as basic concepts related to AI are first introduced, followed by a critical evaluation of representative reports integrating IRS and HSI with AI. Finally, the advantages, challenges and future trends of IRS and HSI combined with AI are further discussed, so as to provide constructive suggestions and guidance for researchers.

Multiclass Skin Lesion Classification Using a Novel Lightweight Deep Learning Framework for Smart Healthcare

Skin lesion classification has recently attracted significant attention. Regularly, physicians take much time to analyze the skin lesions because of the high similarity between these skin lesions. An automated classification system using deep learning can assist physicians in detecting the skin lesion type and enhance the patient’s health. The skin lesion classification has become a hot research area with the evolution of deep learning architecture. In this study, we propose a novel method using a new segmentation approach and wide-ShuffleNet for skin lesion classification. First, we calculate the entropy-based weighting and first-order cumulative moment (EW-FCM) of the skin image. These values are used to separate the lesion from the background. Then, we input the segmentation result into a new deep learning structure wide-ShuffleNet and determine the skin lesion type. We evaluated the proposed method on two large datasets: HAM10000 and ISIC2019. Based on our numerical results, EW-FCM and wide-ShuffleNet achieve more accuracy than state-of-the-art approaches. Additionally, the proposed method is superior lightweight and suitable with a small system like a mobile healthcare system.

A novel feature based algorithm for soil type classification

Agriculture is the backbone of Bangladesh’s economy and it is one of the largest employment sectors. In Bangladesh, the population is increasing rapidly and at the same time, the total cultivable land is decreasing significantly. To ensure maximum crop production using the limited land resources, it is essential to identify and select the appropriate type of soil because different crops need different soil types. Currently, there are two types of methods available to determine the soil type, namely chemical and image analysis. Although the first one is accurate, it is expensive and time consuming. On the other hand, image based soil classification is much cheaper and faster but its accuracy level is low. In this study, we present a novel feature based algorithm that combines quartile histogram oriented gradients (Q-HOG), most frequent $$\varphi $$ φ -Pixels and a new feature selection method for classifying soil types. We have used four machine learning algorithms and evaluated the performance with different sets of features. We have also compared our work with two prominent and recent works on image-based soil classification systems. The experimental results show that the performance of our proposed method in terms of four standard evaluation metrics, namely accuracy, precision, F1_score, and recall scores are higher than the existing image-based soil classification systems.

Identification of Rice Seed Varieties Based on Near-Infrared Hyperspectral Imaging Technology Combined with Deep Learning

Rice is one of the most important food crops in the world, and rice seed varieties are related to the yield and quality of rice. This study used near-infrared (NIR) hyperspectral technology with conventional machine learning methods (support vector machine (SVM), logistic regression (LR), and random forest (RF)) and deep learning methods (LeNet, GoogLeNet, and residual network (ResNet)) to establish variety identification models for five common types of rice seeds. Among the deep learning methods, the classification accuracies of most models were higher than 95%. This study further used the deep learning methods to establish variety identification models for 10 varieties of rice seeds without considering their types. Among them, the ResNet model had the best classification results. The classification accuracy on the test set was 86.08%. This study used the saliency map method to visualize each convolutional neural network (CNN) model to find the band region that contributed the most to the data. The results showed that the bands with the largest data contribution were mainly concentrated at approximately 1300-1400 nm and secondarily concentrated at approximately 1050-1250 nm. The overall results showed that NIR hyperspectral imaging technology combined with deep learning could effectively distinguish rice seeds of different varieties. This method provided an effective way to identify rice seed varieties in a quick and nondestructive manner.

Near-infrared hyperspectral imaging technology combined with deep convolutional generative adversarial network to predict oil content of single maize kernel

Rapidly and non-destructively predicting the oil content of single maize kernel is crucial for food industry. However, obtaining a large number of oil content reference values of maize kernels is time-consuming and expensive, and the limited data set also leads to low generalization ability of the model. Here, hyperspectral imaging technology and deep convolutional generative adversarial network (DCGAN) were combined to predict the oil content of single maize kernel. DCGAN was used to simultaneously expand their spectral data and oil content data. After many iterations, fake data that was very similar to the experimental data was generated. Partial least squares regression (PLSR) and support vector regression (SVR) models were established respectively, and their performance was compared before and after data augmentation. The results showed that this method not only improved the performance of two regression models, but also solved the problem of requiring a large amount of training data.

Identification of micro- and nanoplastics released from medical masks using hyperspectral imaging and deep learning

In this study, dark-field microscopy-based hyperspectral imaging augmented with deep learning data analysis was applied for effective visualisation, detection and identification of microplastics released from polypropylene medical masks.

HyperSeed: An End-to-End Method to Process Hyperspectral Images of Seeds

High-throughput, nondestructive, and precise measurement of seeds is critical for the evaluation of seed quality and the improvement of agricultural productions. To this end, we have developed a novel end-to-end platform named HyperSeed to provide hyperspectral information for seeds. As a test case, the hyperspectral images of rice seeds are obtained from a high-performance line-scan image spectrograph covering the spectral range from 600 to 1700 nm. The acquired images are processed via a graphical user interface (GUI)-based open-source software for background removal and seed segmentation. The output is generated in the form of a hyperspectral cube and curve for each seed. In our experiment, we presented the visual results of seed segmentation on different seed species. Moreover, we conducted a classification of seeds raised in heat stress and control environments using both traditional machine learning models and neural network models. The results show that the proposed 3D convolutional neural network (3D CNN) model has the highest accuracy, which is 97.5% in seed-based classification and 94.21% in pixel-based classification, compared to 80.0% in seed-based classification and 85.67% in seed-based classification from the support vector machine (SVM) model. Moreover, our pipeline enables systematic analysis of spectral curves and identification of wavelengths of biological interest.

Performance of Various Deep-Learning Networks in the Seed Classification Problem

We report the results of an in-depth study of 15 variants of five different Convolutional Neural Network (CNN) architectures for the classification of seeds of seven different grass species that possess symmetry properties. The performance metrics of the nets are investigated in relation to the computational load and the number of parameters. The results indicate that the relation between the accuracy performance and operation count or number of parameters is linear in the same family of nets but that there is no relation between the two when comparing different CNN architectures. Using default pre-trained weights of the CNNs was found to increase the classification accuracy by ≈3% compared with training from scratch. The best performing CNN was found to be DenseNet201 with a 99.42% test accuracy for the highest resolution image set.

Rapid and Accurate Varieties Classification of Different Crop Seeds Under Sample-Limited Condition Based on Hyperspectral Imaging and Deep Transfer Learning

Rapid varieties classification of crop seeds is significant for breeders to screen out seeds with specific traits and market regulators to detect seed purity. However, collecting high-quality, large-scale samples takes high costs in some cases, making it difficult to build an accurate classification model. This study aimed to explore a rapid and accurate method for varieties classification of different crop seeds under the sample-limited condition based on hyperspectral imaging (HSI) and deep transfer learning. Three deep neural networks with typical structures were designed based on a sample-rich Pea dataset. Obtained the highest accuracy of 99.57%, VGG-MODEL was transferred to classify four target datasets (rice, oat, wheat, and cotton) with limited samples. Accuracies of the deep transferred model achieved 95, 99, 80.8, and 83.86% on the four datasets, respectively. Using training sets with different sizes, the deep transferred model could always obtain higher performance than other traditional methods. The visualization of the deep features and classification results confirmed the portability of the shared features of seed spectra, providing an interpreted method for rapid and accurate varieties classification of crop seeds. The overall results showed great superiority of HSI combined with deep transfer learning for seed detection under sample-limited condition. This study provided a new idea for facilitating a crop germplasm screening process under the scenario of sample scarcity and the detection of other qualities of crop seeds under sample-limited condition based on HSI.