1
|
Qing S, Qiu Z, Wang W, Wang F, Jin X, Ji J, Zhao L, Shi Y. Improved YOLO-FastestV2 wheat spike detection model based on a multi-stage attention mechanism with a LightFPN detection head. FRONTIERS IN PLANT SCIENCE 2024; 15:1411510. [PMID: 38962247 PMCID: PMC11219818 DOI: 10.3389/fpls.2024.1411510] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/03/2024] [Accepted: 05/31/2024] [Indexed: 07/05/2024]
Abstract
The number of wheat spikes has an important influence on wheat yield, and the rapid and accurate detection of wheat spike numbers is of great significance for wheat yield estimation and food security. Computer vision and machine learning have been widely studied as potential alternatives to human detection. However, models with high accuracy are computationally intensive and time consuming, and lightweight models tend to have lower precision. To address these concerns, YOLO-FastestV2 was selected as the base model for the comprehensive study and analysis of wheat sheaf detection. In this study, we constructed a wheat target detection dataset comprising 11,451 images and 496,974 bounding boxes. The dataset for this study was constructed based on the Global Wheat Detection Dataset and the Wheat Sheaf Detection Dataset, which was published by PP Flying Paddle. We selected three attention mechanisms, Large Separable Kernel Attention (LSKA), Efficient Channel Attention (ECA), and Efficient Multi-Scale Attention (EMA), to enhance the feature extraction capability of the backbone network and improve the accuracy of the underlying model. First, the attention mechanism was added after the base and output phases of the backbone network. Second, the attention mechanism that further improved the model accuracy after the base and output phases was selected to construct the model with a two-phase added attention mechanism. On the other hand, we constructed SimLightFPN to improve the model accuracy by introducing SimConv to improve the LightFPN module. The results of the study showed that the YOLO-FastestV2-SimLightFPN-ECA-EMA hybrid model, which incorporates the ECA attention mechanism in the base stage and introduces the EMA attention mechanism and the combination of SimLightFPN modules in the output stage, has the best overall performance. The accuracy of the model was P=83.91%, R=78.35%, AP= 81.52%, and F1 = 81.03%, and it ranked first in the GPI (0.84) in the overall evaluation. The research examines the deployment of wheat ear detection and counting models on devices with constrained resources, delivering novel solutions for the evolution of agricultural automation and precision agriculture.
Collapse
Affiliation(s)
- Shunhao Qing
- College of Agricultural Equipment Engineering, Henan University of Science and Technology, Luoyang, Henan, China
| | - Zhaomei Qiu
- College of Agricultural Equipment Engineering, Henan University of Science and Technology, Luoyang, Henan, China
| | - Weili Wang
- College of Agricultural Equipment Engineering, Henan University of Science and Technology, Luoyang, Henan, China
| | - Fei Wang
- College of Agricultural Equipment Engineering, Henan University of Science and Technology, Luoyang, Henan, China
| | - Xin Jin
- College of Agricultural Equipment Engineering, Henan University of Science and Technology, Luoyang, Henan, China
- Science and Technology Innovation Center for Completed Set Equipment, Longmen Laboratory, Luoyang, China
| | - Jiangtao Ji
- College of Agricultural Equipment Engineering, Henan University of Science and Technology, Luoyang, Henan, China
| | - Long Zhao
- College of Agricultural Equipment Engineering, Henan University of Science and Technology, Luoyang, Henan, China
| | - Yi Shi
- College of Agricultural Equipment Engineering, Henan University of Science and Technology, Luoyang, Henan, China
| |
Collapse
|
2
|
Zhang J, Zou W, Hu N, Zhang B, Wang J. S-Net: an S-shaped network for nodule detection in 3D CT images. Phys Med Biol 2024; 69:075013. [PMID: 38382097 DOI: 10.1088/1361-6560/ad2b96] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2023] [Accepted: 02/21/2024] [Indexed: 02/23/2024]
Abstract
Objective. Accurate and automatic detection of pulmonary nodules is critical for early lung cancer diagnosis, and promising progress has been achieved in developing effective deep models for nodule detection. However, most existing nodule detection methods merely focus on integrating elaborately designed feature extraction modules into the backbone of the detection network to extract rich nodule features while ignore disadvantages of the structure of detection network itself. This study aims to address these disadvantages and develop a deep learning-based algorithm for pulmonary nodule detection to improve the accuracy of early lung cancer diagnosis.Approach. In this paper, an S-shaped network called S-Net is developed with the U-shaped network as backbone, where an information fusion branch is used to propagate lower-level details and positional information critical for nodule detection to higher-level feature maps, head shared scale adaptive detection strategy is utilized to capture information from different scales for better detecting nodules with different shapes and sizes and the feature decoupling detection head is used to allow the classification and regression branches to focus on the information required for their respective tasks. A hybrid loss function is utilized to fully exploit the interplay between the classification and regression branches.Main results. The proposed S-Net network with ResSENet and other three U-shaped backbones from SANet, OSAF-YOLOv3 and MSANet (R+SC+ECA) models achieve average CPM scores of 0.914, 0.915, 0.917 and 0.923 on the LUNA16 dataset, which are significantly higher than those achieved with other existing state-of-the-art models.Significance. The experimental results demonstrate that our proposed method effectively improves nodule detection performance, which implies potential applications of the proposed method in clinical practice.
Collapse
Affiliation(s)
- JingYu Zhang
- School of Electronic and Information Engineering, Soochow University, Suzhou 215006, People's Republic of China
| | - Wei Zou
- School of Electronic and Information Engineering, Soochow University, Suzhou 215006, People's Republic of China
| | - Nan Hu
- School of Electronic and Information Engineering, Soochow University, Suzhou 215006, People's Republic of China
| | - Bin Zhang
- Department of Nuclear Medicine, the First Affiliated Hospital of Soochow University, Suzhou 215006, People's Republic of China
| | - Jiajun Wang
- School of Electronic and Information Engineering, Soochow University, Suzhou 215006, People's Republic of China
| |
Collapse
|
3
|
Ma L, Li G, Feng X, Fan Q, Liu L. TiCNet: Transformer in Convolutional Neural Network for Pulmonary Nodule Detection on CT Images. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:196-208. [PMID: 38343213 DOI: 10.1007/s10278-023-00904-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/22/2023] [Revised: 07/19/2023] [Accepted: 08/10/2023] [Indexed: 03/02/2024]
Abstract
Lung cancer is the leading cause of cancer death. Since lung cancer appears as nodules in the early stage, detecting the pulmonary nodules in an early phase could enhance the treatment efficiency and improve the survival rate of patients. The development of computer-aided analysis technology has made it possible to automatically detect lung nodules in Computed Tomography (CT) screening. In this paper, we propose a novel detection network, TiCNet. It is attempted to embed a transformer module in the 3D Convolutional Neural Network (CNN) for pulmonary nodule detection on CT images. First, we integrate the transformer and CNN in an end-to-end structure to capture both the short- and long-range dependency to provide rich information on the characteristics of nodules. Second, we design the attention block and multi-scale skip pathways for improving the detection of small nodules. Last, we develop a two-head detector to guarantee high sensitivity and specificity. Experimental results on the LUNA16 dataset and PN9 dataset showed that our proposed TiCNet achieved superior performance compared with existing lung nodule detection methods. Moreover, the effectiveness of each module has been proven. The proposed TiCNet model is an effective tool for pulmonary nodule detection. Validation revealed that this model exhibited excellent performance, suggesting its potential usefulness to support lung cancer screening.
Collapse
Affiliation(s)
- Ling Ma
- College of Software, Nankai University, Tianjin, China
| | - Gen Li
- College of Software, Nankai University, Tianjin, China
| | - Xingyu Feng
- College of Software, Nankai University, Tianjin, China
| | - Qiliang Fan
- College of Software, Nankai University, Tianjin, China
| | - Lizhi Liu
- Department of Radiology, Sun Yat-Sen University Cancer Center, Guangdong, China.
| |
Collapse
|
4
|
Wu R, Liang C, Zhang J, Tan Q, Huang H. Multi-kernel driven 3D convolutional neural network for automated detection of lung nodules in chest CT scans. BIOMEDICAL OPTICS EXPRESS 2024; 15:1195-1218. [PMID: 38404310 PMCID: PMC10890889 DOI: 10.1364/boe.504875] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 12/27/2023] [Accepted: 12/28/2023] [Indexed: 02/27/2024]
Abstract
The accurate position detection of lung nodules is crucial in early chest computed tomography (CT)-based lung cancer screening, which helps to improve the survival rate of patients. Deep learning methodologies have shown impressive feature extraction ability in the CT image analysis task, but it is still a challenge to develop a robust nodule detection model due to the salient morphological heterogeneity of nodules and complex surrounding environment. In this study, a multi-kernel driven 3D convolutional neural network (MK-3DCNN) is proposed for computerized nodule detection in CT scans. In the MK-3DCNN, a residual learning-based encoder-decoder architecture is introduced to employ the multi-layer features of the deep model. Considering the various nodule sizes and shapes, a multi-kernel joint learning block is developed to capture 3D multi-scale spatial information of nodule CT images, and this is conducive to improving nodule detection performance. Furthermore, a multi-mode mixed pooling strategy is designed to replace the conventional single-mode pooling manner, and it reasonably integrates the max pooling, average pooling, and center cropping pooling operations to obtain more comprehensive nodule descriptions from complicated CT images. Experimental results on the public dataset LUNA16 illustrate that the proposed MK-3DCNN method achieves more competitive nodule detection performance compared to some state-of-the-art algorithms. The results on our constructed clinical dataset CQUCH-LND indicate that the MK-3DCNN has a good prospect in clinical practice.
Collapse
Affiliation(s)
- Ruoyu Wu
- Key Laboratory of Optoelectronic Technology and Systems of the Education Ministry of China, Chongqing University, Chongqing 400044, China
| | - Changyu Liang
- Department of Radiology, Chongqing University Cancer Hospital & Chongqing Cancer Institute & Chongqing Cancer Hospital, Chongqing 400030, China
| | - Jiuquan Zhang
- Department of Radiology, Chongqing University Cancer Hospital & Chongqing Cancer Institute & Chongqing Cancer Hospital, Chongqing 400030, China
| | - QiJuan Tan
- Department of Radiology, Chongqing University Cancer Hospital & Chongqing Cancer Institute & Chongqing Cancer Hospital, Chongqing 400030, China
| | - Hong Huang
- Key Laboratory of Optoelectronic Technology and Systems of the Education Ministry of China, Chongqing University, Chongqing 400044, China
| |
Collapse
|
5
|
Yang F, Li X, Duan H, Xu F, Huang Y, Zhang X, Long Y, Zheng Y. MRL-Seg: Overcoming Imbalance in Medical Image Segmentation With Multi-Step Reinforcement Learning. IEEE J Biomed Health Inform 2024; 28:858-869. [PMID: 38032774 DOI: 10.1109/jbhi.2023.3336726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2023]
Abstract
Medical image segmentation is a critical task for clinical diagnosis and research. However, dealing with highly imbalanced data remains a significant challenge in this domain, where the region of interest (ROI) may exhibit substantial variations across different slices. This presents a significant hurdle to medical image segmentation, as conventional segmentation methods may either overlook the minority class or overly emphasize the majority class, ultimately leading to a decrease in the overall generalization ability of the segmentation results. To overcome this, we propose a novel approach based on multi-step reinforcement learning, which integrates prior knowledge of medical images and pixel-wise segmentation difficulty into the reward function. Our method treats each pixel as an individual agent, utilizing diverse actions to evaluate its relevance for segmentation. To validate the effectiveness of our approach, we conduct experiments on four imbalanced medical datasets, and the results show that our approach surpasses other state-of-the-art methods in highly imbalanced scenarios. These findings hold substantial implications for clinical diagnosis and research.
Collapse
|
6
|
Zhou R, Yao J, Hong Q, Zheng Y, Zheng L. DAMS-Net: Dual attention and multi-scale information fusion network for 12-lead ECG classification. Methods 2023; 220:134-141. [PMID: 37967757 DOI: 10.1016/j.ymeth.2023.10.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Revised: 10/08/2023] [Accepted: 10/23/2023] [Indexed: 11/17/2023] Open
Abstract
Automated 12-lead electrocardiographic (ECG) classification algorithms play an important role in the diagnosis of clinical arrhythmias. Current methods that perform well in the field of automatic ECG classification are usually based on Convolutional Neural Networks (CNN) or Transformer. However, due to the intrinsic locality of convolution operations, CNN can't extract long-dependence between series. On the other side, the Transformer design includes a built-in global self-attention mechanism, but it doesn't pay enough attention to local features. In this paper, we propose DAMS-Net, which combines the advantages of Transformer and CNN, introducing a spatial attention module and a channel attention module using a CNN-Transformer hybrid encoder to adaptively focus on the significant features of global and local parts between space and channels. In addition, our proposal fuses multi-scale information to capture high and low-level semantic information by skip-connections. We evaluate our method on the 2018 Physiological Electrical Signaling Challenge dataset, and our proposal achieves a precision rate of 83.6%, a recall rate of 84.7%, and an F1-score of 0.839. The classification performance is superior to all current single-model methods evaluated in this dataset. The experimental results demonstrate the promising application of our proposed method in 12-lead ECG automatic classification tasks.
Collapse
Affiliation(s)
- Rongzhou Zhou
- Center for Digital Media Computing, School of Film, Xiamen University, Xiamen, Fujian, 361005, China.
| | - Junfeng Yao
- Center for Digital Media Computing, School of Film, Xiamen University, Xiamen, Fujian, 361005, China; School of Information, Xiamen University, Xiamen, Fujian, 361005, China; Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, China.
| | - Qingqi Hong
- Center for Digital Media Computing, School of Film, Xiamen University, Xiamen, Fujian, 361005, China.
| | - Yuan Zheng
- School of Information, Xiamen University, Xiamen, Fujian, 361005, China.
| | - Liling Zheng
- Department of Cardiovascular Surgery, First Hospital of Quanzhou Affiliated to Fujian Medical University, Quanzhou, Fujian, 36200, China.
| |
Collapse
|
7
|
Xiao H, Liu Q, Li L. MFMANet: Multi-feature Multi-attention Network for efficient subtype classification on non-small cell lung cancer CT images. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104768] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/19/2023]
|
8
|
Iqbal S, N. Qureshi A, Li J, Mahmood T. On the Analyses of Medical Images Using Traditional Machine Learning Techniques and Convolutional Neural Networks. ARCHIVES OF COMPUTATIONAL METHODS IN ENGINEERING : STATE OF THE ART REVIEWS 2023; 30:3173-3233. [PMID: 37260910 PMCID: PMC10071480 DOI: 10.1007/s11831-023-09899-9] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/01/2022] [Accepted: 02/19/2023] [Indexed: 06/02/2023]
Abstract
Convolutional neural network (CNN) has shown dissuasive accomplishment on different areas especially Object Detection, Segmentation, Reconstruction (2D and 3D), Information Retrieval, Medical Image Registration, Multi-lingual translation, Local language Processing, Anomaly Detection on video and Speech Recognition. CNN is a special type of Neural Network, which has compelling and effective learning ability to learn features at several steps during augmentation of the data. Recently, different interesting and inspiring ideas of Deep Learning (DL) such as different activation functions, hyperparameter optimization, regularization, momentum and loss functions has improved the performance, operation and execution of CNN Different internal architecture innovation of CNN and different representational style of CNN has significantly improved the performance. This survey focuses on internal taxonomy of deep learning, different models of vonvolutional neural network, especially depth and width of models and in addition CNN components, applications and current challenges of deep learning.
Collapse
Affiliation(s)
- Saeed Iqbal
- Department of Computer Science, Faculty of Information Technology & Computer Science, University of Central Punjab, Lahore, Punjab 54000 Pakistan
- Faculty of Information Technology, Beijing University of Technology, Beijing, 100124 Beijing China
| | - Adnan N. Qureshi
- Department of Computer Science, Faculty of Information Technology & Computer Science, University of Central Punjab, Lahore, Punjab 54000 Pakistan
| | - Jianqiang Li
- Faculty of Information Technology, Beijing University of Technology, Beijing, 100124 Beijing China
- Beijing Engineering Research Center for IoT Software and Systems, Beijing University of Technology, Beijing, 100124 Beijing China
| | - Tariq Mahmood
- Artificial Intelligence and Data Analytics (AIDA) Lab, College of Computer & Information Sciences (CCIS), Prince Sultan University, Riyadh, 11586 Kingdom of Saudi Arabia
| |
Collapse
|