1
|
Pan L, Yan X, Zheng Y, Huang L, Zhang Z, Fu R, Zheng B, Zheng S. Automatic pulmonary artery-vein separation in CT images using a twin-pipe network and topology reconstruction. PeerJ Comput Sci 2023; 9:e1537. [PMID: 37810355 PMCID: PMC10557495 DOI: 10.7717/peerj-cs.1537] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Accepted: 07/24/2023] [Indexed: 10/10/2023]
Abstract
Background With the wide application of CT scanning, the separation of pulmonary arteries and veins (A/V) based on CT images plays an important role for assisting surgeons in preoperative planning of lung cancer surgery. However, distinguishing between arteries and veins in chest CT images remains challenging due to the complex structure and the presence of their similarities. Methods We proposed a novel method for automatically separating pulmonary arteries and veins based on vessel topology information and a twin-pipe deep learning network. First, vessel tree topology is constructed by combining scale-space particles and multi-stencils fast marching (MSFM) methods to ensure the continuity and authenticity of the topology. Second, a twin-pipe network is designed to learn the multiscale differences between arteries and veins and the characteristics of the small arteries that closely accompany bronchi. Finally, we designed a topology optimizer that considers interbranch and intrabranch topological relationships to optimize the results of arteries and veins classification. Results The proposed approach is validated on the public dataset CARVE14 and our private dataset. Compared with ground truth, the proposed method achieves an average accuracy of 90.1% on the CARVE14 dataset, and 96.2% on our local dataset. Conclusions The method can effectively separate pulmonary arteries and veins and has good generalization for chest CT images from different devices, as well as enhanced and noncontrast CT image sequences from the same device.
Collapse
Affiliation(s)
- Lin Pan
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian, China
| | - Xiaochao Yan
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian, China
| | - Yaoyong Zheng
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian, China
| | - Liqin Huang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian, China
| | - Zhen Zhang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian, China
| | - Rongda Fu
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian, China
| | - Bin Zheng
- Key Laboratory of Cardio-Thoracic Surgery, Fujian Medical University, Fuzhou, Fujian, China
| | - Shaohua Zheng
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian, China
| |
Collapse
|
2
|
Bjekic M, Lazovic A, K V, Bacanin N, Zivkovic M, Kvascev G, Nikolic B. Wall segmentation in 2D images using convolutional neural networks. PeerJ Comput Sci 2023; 9:e1565. [PMID: 37810356 PMCID: PMC10557507 DOI: 10.7717/peerj-cs.1565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Accepted: 08/13/2023] [Indexed: 10/10/2023]
Abstract
Wall segmentation is a special case of semantic segmentation, and the task is to classify each pixel into one of two classes: wall and no-wall. The segmentation model returns a mask showing where objects like windows and furniture are located, as well as walls. This article proposes the module's structure for semantic segmentation of walls in 2D images, which can effectively address the problem of wall segmentation. The proposed model achieved higher accuracy and faster execution than other solutions. An encoder-decoder architecture of the segmentation module was used. Dilated ResNet50/101 network was used as an encoder, representing ResNet50/101 network in which dilated convolutional layers replaced the last convolutional layers. The ADE20K dataset subset containing only interior images, was used for model training, while only its subset was used for model evaluation. Three different approaches to model training were analyzed in the research. On the validation dataset, the best approach based on the proposed structure with the ResNet101 network resulted in an average accuracy at the pixel level of 92.13% and an intersection over union (IoU) of 72.58%. Moreover, all proposed approaches can be applied to recognize other objects in the image to solve specific tasks.
Collapse
Affiliation(s)
| | | | - Venkatachalam K
- Department of Applied Cybernetics, University of Hradec Králové, Faculty of Science, Hradec Králové, Czech Republic
| | - Nebojsa Bacanin
- Department of Informatics and Computing, Singidunum University, Belgrade, Serbia
| | - Miodrag Zivkovic
- Department of Informatics and Computing, Singidunum University, Belgrade, Serbia
| | - Goran Kvascev
- University of Belgrade, School of Electrical Engineering, Belgrade, Serbia
| | - Bosko Nikolic
- University of Belgrade, School of Electrical Engineering, Belgrade, Serbia
| |
Collapse
|
3
|
Liu S, Wei J, Liu G, Zhou B. Image classification model based on large kernel attention mechanism and relative position self-attention mechanism. PeerJ Comput Sci 2023; 9:e1344. [PMID: 37346614 PMCID: PMC10280586 DOI: 10.7717/peerj-cs.1344] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Accepted: 03/21/2023] [Indexed: 06/23/2023]
Abstract
The Transformer has achieved great success in many computer vision tasks. With the in-depth exploration of it, researchers have found that Transformers can better obtain long-range features than convolutional neural networks (CNN). However, there will be a deterioration of local feature details when the Transformer extracts local features. Although CNN is adept at capturing the local feature details, it cannot easily obtain the global representation of features. In order to solve the above problems effectively, this paper proposes a hybrid model consisting of CNN and Transformer inspired by Visual Attention Net (VAN) and CoAtNet. This model optimizes its shortcomings in the difficulty of capturing the global representation of features by introducing Large Kernel Attention (LKA) in CNN while using the Transformer blocks with relative position self-attention variant to alleviate the problem of detail deterioration in local features of the Transformer. Our model effectively combines the advantages of the above two structures to obtain the details of local features more accurately and capture the relationship between features far apart more efficiently on a large receptive field. Our experiments show that in the image classification task without additional training data, the proposed model in this paper can achieve excellent results on the cifar10 dataset, the cifar100 dataset, and the birds400 dataset (a public dataset on the Kaggle platform) with fewer model parameters. Among them, SE_LKACAT achieved a Top-1 accuracy of 98.01% on the cifar10 dataset with only 7.5M parameters.
Collapse
Affiliation(s)
- Siqi Liu
- College of Information Engineering, Sichuan Agricultural University, Ya’an, Sichuan, China
| | - Jiangshu Wei
- College of Information Engineering, Sichuan Agricultural University, Ya’an, Sichuan, China
| | - Gang Liu
- College of Information Engineering, Sichuan Agricultural University, Ya’an, Sichuan, China
| | - Bei Zhou
- College of Information Engineering, Sichuan Agricultural University, Ya’an, Sichuan, China
| |
Collapse
|
4
|
Arellano-Verdejo J, Santos-Romero M, Lazcano-Hernandez HE. Use of semantic segmentation for mapping Sargassum on beaches. PeerJ 2022; 10:e13537. [PMID: 35702255 PMCID: PMC9188770 DOI: 10.7717/peerj.13537] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2021] [Accepted: 05/13/2022] [Indexed: 01/17/2023] Open
Abstract
The unusual arrival of Sargassum on Caribbean beaches is an emerging problem that has generated numerous challenges. The monitoring, visualization, and estimation of Sargassum coverage on the beaches remain a constant complication. This study proposes a new mapping methodology to estimate Sargassum coverage on the beaches. Semantic segmentation of geotagged photographs allows the generation of accurate maps showing the percent coverage of Sargassum. The first dataset of segmented Sargassum images was built for this study and used to train the proposed model. The results demonstrate that the currently proposed method has an accuracy of 91%, improving on the results reported in the state-of-the-art method where data was also collected through a crowdsourcing scheme, in which only information on the presence and absence of Sargassum is displayed.
Collapse
Affiliation(s)
- Javier Arellano-Verdejo
- Department of Observation and Study of the Earth, Atmosphere and Ocean, El Colegio de la Frontera Sur, Chetumal, Quintana Roo, Mexico
| | | | - Hugo E. Lazcano-Hernandez
- Department of Observation and Study of the Earth, Atmosphere and Ocean, CONACYT-El Colegio de la Frontera Sur, Chetumal, Quintana Roo, Mexico
| |
Collapse
|
5
|
Pan L, Chen K, Zheng Z, Zhao Y, Yang P, Li Z, Wu S. Aging of Chinese bony orbit: automatic calculation based on UNet++ and connected component analysis. Surg Radiol Anat 2022; 44:749-758. [PMID: 35384466 PMCID: PMC8985397 DOI: 10.1007/s00276-022-02933-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Accepted: 03/25/2022] [Indexed: 11/26/2022]
Abstract
Purpose Current research on the aging of bony orbit is usually done manually, which is inefficient and has a large error. In this paper, automatic segmentation of bony orbit based on deep learning and automatic calculation of the parameters of the segmented orbital contour (area and height of bony orbit) are presented. Methods The craniofacial CT scanning data of 595 Chinese were used to carry out three-dimensional reconstruction and output the craniofacial images. The orbital contour images are obtained automatically by UNet++ segmentation network, and then the bony orbital area and height were calculated automatically by connected component analysis. Results The automatic segmentation method has an Intersection of Union of 95.41% in craniofacial CT images. During the aging, the bony orbital area of males increased with age, while that of females decreased, and the area in male was larger than that in female (P < 0.05). The distance from equal points 10 and 40–90 to the supraorbital rim was significantly larger (P < 0.05). Except for the equal point 90, the distance from equal points to the inferior orbital rim was obviously larger (P < 0.05). In the females, the distance from equal points 50–70 to inferior orbital rim was significantly lower (P < 0.05). Conclusion The method proposed here can automatically and accurately study image dataset of large-scale bony orbital CT imaging. UNet++ can achieve high-precision segmentation of bony orbital contours. The bony orbital area of Chinese changes with aging, and the bony orbital height changes different between males and females, which may be caused by the different position and degree of orbital bone resorption of males and females in the process of aging.
Collapse
Affiliation(s)
- Lei Pan
- Plastic Surgery Center, Department of Plastic Surgery, Zhejiang Provincial People's Hospital, Affiliated People's Hospital, Hangzhou Medical College, Hangzhou, Zhejiang, China
| | - Kunjian Chen
- School of Electronics and Information, Hangzhou Dianzi University, Hangzhou, 310018, Zhejiang, China
| | - Zepei Zheng
- College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou, Zhejiang, China
| | - Ye Zhao
- Plastic Surgery Center, Department of Plastic Surgery, Zhejiang Provincial People's Hospital, Affiliated People's Hospital, Hangzhou Medical College, Hangzhou, Zhejiang, China
| | - Panfeng Yang
- Department of Radiology, Zhejiang Provincial People's Hospital, Affiliated People's Hospital, Hangzhou Medical College, Hangzhou, Zhejiang, China
| | - Zhu Li
- School of Electronics and Information, Hangzhou Dianzi University, Hangzhou, 310018, Zhejiang, China.
| | - Sufan Wu
- Plastic Surgery Center, Department of Plastic Surgery, Zhejiang Provincial People's Hospital, Affiliated People's Hospital, Hangzhou Medical College, Hangzhou, Zhejiang, China
| |
Collapse
|