SMOTE-based method for balanced spectral nondestructive testing of moldy apple core

Applying virtual sample generation and ensemble modeling for improving the spectral diagnosis of cancer

Cancer diagnosis plays a key role in facilitating treatment and improving survival rates of patients. The combination of near-infrared (NIR) spectroscopy with data-driven algorithms offers a rapid and cost-effective approach for such a task. Due to the limitations of objective cases, the number of tumor samples is usually smaller, and the resulting dataset exhibit the issues of class imbalance, which has a more serious impact on the performance of diagnostic models. To deal with class imbalance and improve the sensitivity, this work investigates the feasibility of NIR spectroscopy combined with virtual sample generation (VSG) as well as ensemble strategy for developing diagnostic models. Based on preliminary experiment, several learning algorithms such as discriminant analysis (DA) and partial least square-discriminant analysis (PLS-DA) are screened out as algorithms for constructing prediction models. Three algorithms of VSG including synthetic minority oversampling technique (SMOTE), Borderline-SMOTE and adaptive synthetic sampling (ADASYN) are used for experiment. A fixed sample subset composed of 27 cancer samples and 54 normal samples are hold out as the test set. Three training sets containing 5, 10, 25 minority class samples and 54 majority class samples are used for model development. The experimental result indicates that overall, with PLS-DA algorithm, all VSG approaches can significantly improve the sensitivity of cancer diagnosis for all cases of training sets with different minority samples, but ADASYN performs the best. It reveals that the integration of NIR, PLS-DA, and ADASYN is a promising tool package for developing diagnosis methods.

Improving prediction model robustness with virtual sample construction for near-infrared spectra analysis

In a qualitative analysis of near-infrared spectroscopy (NIRS), when the samples to be analyzed are difficult to obtain or there are few counterexamples, the robustness of the models is poor, resulting in the decline of the generalization ability of the models. In this case, the effective method is to construct virtual samples to achieve the balance of categories. In this contribution, three virtual spectrum construction strategies including Synthetic Minority Oversampling Technique (SMOTE), Adaptive Synthetic Sampling (ADASYN), and Deep Convolutional Generative Adversarial Network (DCGAN) were explored to deal with the problem of insufficient or imbalanced sample numbers in NIRS analysis. The strategies were tested with the melamine and Yali pears two spectral datasets. The PLS-DA and Correct Recognition Rate (CRR) were used for discriminant model construction and accuracy evaluation, respectively. The results show that SMOTE, ADASYN, and DCGAN processing strategies can all improve the global CRR (CRRglob). The SMOTE and ADASYN can improve the CRR for majority class sample (CRRmaj), but the CRR for minority class sample (CRRmin) has decreased. For the DCGAN method, the CRRglob, CRRmaj, and CRRmin were all improved. The standard deviation of the results of the multiple parallel calculations demonstrates the robustness of DCGAN generation method. Therefore, the DCGAN method has good reliability and practicability, and can increase the robustness and generalization ability of the NIRS model.

Detecting Starch-Head and Mildewed Fruit in Dried Hami Jujubes Using Visible/Near-Infrared Spectroscopy Combined with MRSA-SVM and Oversampling

Dried Hami jujube has great commercial and nutritional value. Starch-head and mildewed fruit are defective jujubes that pose a threat to consumer health. A novel method for detecting starch-head and mildewed fruit in dried Hami jujubes with visible/near-infrared spectroscopy was proposed. For this, the diffuse reflectance spectra in the range of 400–1100 nm of dried Hami jujubes were obtained. Borderline synthetic minority oversampling technology (BL-SMOTE) was applied to solve the problem of imbalanced sample distribution, and its effectiveness was demonstrated compared to other methods. Then, the feature variables selected by competitive adaptive reweighted sampling (CARS) were used as the input to establish the support vector machine (SVM) classification model. The parameters of SVM were optimized by the modified reptile search algorithm (MRSA). In MRSA, Tent chaotic mapping and the Gaussian random walk strategy were used to improve the optimization ability of the original reptile search algorithm (RSA). The final results showed that the MRSA-SVM method combined with BL-SMOTE had the best classification performance, and the detection accuracy reached 97.22%. In addition, the recall, precision, F₁ and kappa coefficient outperform other models. Furthermore, this study provided a valuable reference for the detection of defective fruit in other fruits.