1
|
Tanthanathewin R, Wongrattanapipat W, Khaing TT, Aimmanee P. Automatic exudate and aneurysm segmentation in OCT images using UNET++ and hyperreflective-foci feature based bagged tree ensemble. PLoS One 2024; 19:e0304146. [PMID: 38787844 PMCID: PMC11125471 DOI: 10.1371/journal.pone.0304146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Accepted: 05/07/2024] [Indexed: 05/26/2024] Open
Abstract
Diabetic retinopathy's signs, such as exudates (EXs) and aneurysms (ANs), initially develop from under the retinal surface detectable from optical coherence tomography (OCT) images. Detecting these signs helps ophthalmologists diagnose DR sooner. Detecting and segmenting exudates (EXs) and aneurysms (ANs) in medical images is challenging due to their small size, similarity to other hyperreflective regions, noise presence, and low background contrast. Furthermore, the scarcity of public OCT images featuring these abnormalities has limited the number of studies related to the automatic segmentation of EXs and ANs, and the reported performance of such studies has not been satisfactory. This work proposes an efficient algorithm that can automatically segment these anomalies by improving key steps in the process. The potential area where these hyper-reflective EXs and ANs occur was scoped by our method using a deep-learning U-Net++ program. From this area, the candidates for EX-AN were segmented using the adaptive thresholding method. Nine features based on appearances, locations, and shadow markers were extracted from these candidates. They were trained and tested using bagged tree ensemble classifiers to obtain only EX-AN blobs. The proposed method was tested on a collection of a public dataset comprising 80 images with hand-drawn ground truths. The experimental results showed that our method could segment EX-AN blobs with average recall, precision, and F1-measure as 87.9%, 86.1%, and 87.0%, respectively. Its F1-measure drastically outperformed two comparative methods, binary thresholding and watershed (BT-WS) and adaptive thresholding with shadow tracking (AT-ST), by 78.0% and 82.1%, respectively.
Collapse
Affiliation(s)
- Rinrada Tanthanathewin
- School of Information, Computer and Communication Technology, Sirindhorn International Institute of Technology, Thammasat University, Meung, Patumthani, Thailand
| | - Warissaporn Wongrattanapipat
- School of Information, Computer and Communication Technology, Sirindhorn International Institute of Technology, Thammasat University, Meung, Patumthani, Thailand
| | - Tin Tin Khaing
- School of Information, Computer and Communication Technology, Sirindhorn International Institute of Technology, Thammasat University, Meung, Patumthani, Thailand
| | - Pakinee Aimmanee
- School of Information, Computer and Communication Technology, Sirindhorn International Institute of Technology, Thammasat University, Meung, Patumthani, Thailand
| |
Collapse
|
2
|
Zhang T, Wei Q, Li Z, Meng W, Zhang M, Zhang Z. Segmentation of paracentral acute middle maculopathy lesions in spectral-domain optical coherence tomography images through weakly supervised deep convolutional networks. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107632. [PMID: 37329802 DOI: 10.1016/j.cmpb.2023.107632] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Revised: 05/23/2023] [Accepted: 05/28/2023] [Indexed: 06/19/2023]
Abstract
BACKGROUND AND OBJECTIVES Spectral-domain optical coherence tomography (SD-OCT) is a valuable tool for non-invasive imaging of the retina, allowing the discovery and visualization of localized lesions, the presence of which is associated with eye diseases. The present study introduces X-Net, a weakly supervised deep-learning framework for automated segmentation of paracentral acute middle maculopathy (PAMM) lesions in retinal SD-OCT images. Despite recent advances in the development of automatic methods for clinical analysis of OCT scans, there remains a scarcity of studies focusing on the automated detection of small retinal focal lesions. Additionally, most existing solutions depend on supervised learning, which can be time-consuming and require extensive image labeling, whereas X-Net offers a solution to these challenges. As far as we can determine, no prior study has addressed the segmentation of PAMM lesions in SD-OCT images. METHODS This study leverages 133 SD-OCT retinal images, each containing instances of paracentral acute middle maculopathy lesions. A team of eye experts annotated the PAMM lesions in these images using bounding boxes. Then, labeled data were used to train a U-Net that performs pre-segmentation, producing region labels of pixel-level accuracy. To attain a highly-accurate final segmentation, we introduced X-Net, a novel neural network made up of a master and a slave U-Net. During training, it takes the expert annotated, and pixel-level pre-segment annotated images and employs sophisticated strategies to ensure the highest segmentation accuracy. RESULTS The proposed method was rigorously evaluated on clinical retinal images excluded from training and achieved an accuracy of 99% with a high level of similarity between the automatic segmentation and expert annotation, as demonstrated by a mean Intersection-over-Union of 0.8. Alternative methods were tested on the same data. Single-stage neural networks proved insufficient for achieving satisfactory results, confirming that more advanced solutions, such as the proposed method, are necessary. We also found that X-Net using Attention U-net for both the pre-segmentation and X-Net arms for the final segmentation shows comparable performance to the proposed method, suggesting that the proposed approach remains a viable solution even when implemented with variants of the classic U-Net. CONCLUSIONS The proposed method exhibits reasonably high performance, validated through quantitative and qualitative evaluations. Medical eye specialists have also verified its validity and accuracy. Thus, it could be a viable tool in the clinical assessment of the retina. Additionally, the demonstrated approach for annotating the training set has proven to be effective in reducing the expert workload.
Collapse
Affiliation(s)
- Tianqiao Zhang
- School of Life and Environmental Sciences, Guilin University of Electronic Technology, Guilin, China
| | - Qiaoqian Wei
- School of Life and Environmental Sciences, Guilin University of Electronic Technology, Guilin, China
| | - Zhenzhen Li
- School of Information Engineering, Nanchang Institute of Technology, Nanchang, China
| | - Wenjing Meng
- Department of Library Services, Guilin University of Electronic Technology, Guilin, China
| | - Mengjiao Zhang
- School of Life and Environmental Sciences, Guilin University of Electronic Technology, Guilin, China
| | - Zhengwei Zhang
- Department of Ophthalmology, Jiangnan University Medical Center, Wuxi, China; Department of Ophthalmology, Wuxi No.2 People's Hospital, Affiliated Wuxi Clinical College of Nantong University, Wuxi, China.
| |
Collapse
|
3
|
Wei J, Yu S, Du Y, Liu K, Xu Y, Xu X. Automatic Segmentation of Hyperreflective Foci in OCT Images Based on Lightweight DBR Network. J Digit Imaging 2023; 36:1148-1157. [PMID: 36749455 PMCID: PMC10287852 DOI: 10.1007/s10278-023-00786-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2022] [Revised: 01/14/2023] [Accepted: 01/18/2023] [Indexed: 02/08/2023] Open
Abstract
Hyperreflective foci (HF) reflects inflammatory responses for fundus diseases such as diabetic macular edema (DME), retina vein occlusion (RVO), and central serous chorioretinopathy (CSC). Shown as high contrast and reflectivity in optical coherence tomography (OCT) images, automatic segmentation of HF in OCT images is helpful for the prognosis of fundus diseases. Previous traditional methods were time-consuming and required high computing power. Hence, we proposed a lightweight network to segment HF (with a speed of 57 ms per OCT image, at least 150 ms faster than other methods). Our framework consists of two stages: an NLM filter and patch-based split to preprocess images and a lightweight DBR neural network to segment HF automatically. Experimental results from 3000 OCT images of 300 patients (100 DME,100 RVO, and 100 CSC) revealed that our method achieved HF segmentation successfully. The DBR network had the area under curves dice similarity coefficient (DSC) of 83.65%, 76.43%, and 82.20% in segmenting HF in DME, RVO, and CSC on the test cohort respectively. Our DBR network achieves at least 5% higher DSC than previous methods. HF in DME was more easily segmented compared with the other two types. In addition, our DBR network is universally applicable to clinical practice with the ability to segment HF in a wide range of fundus diseases.
Collapse
Affiliation(s)
- Jin Wei
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, National Clinical Research Center for Eye Diseases, Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai Engineering Center for Precise Diagnosis and Treatment of Eye Diseases, Shanghai, 200080, China
- Shanghai Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China
| | - Suqin Yu
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, National Clinical Research Center for Eye Diseases, Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai Engineering Center for Precise Diagnosis and Treatment of Eye Diseases, Shanghai, 200080, China
| | - Yuchen Du
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, National Clinical Research Center for Eye Diseases, Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai Engineering Center for Precise Diagnosis and Treatment of Eye Diseases, Shanghai, 200080, China
- Department of Automation, Shanghai Jiao Tong University, Shanghai, 200240, China
| | - Kun Liu
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, National Clinical Research Center for Eye Diseases, Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai Engineering Center for Precise Diagnosis and Treatment of Eye Diseases, Shanghai, 200080, China
| | - Yupeng Xu
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, National Clinical Research Center for Eye Diseases, Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai Engineering Center for Precise Diagnosis and Treatment of Eye Diseases, Shanghai, 200080, China.
| | - Xun Xu
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, National Clinical Research Center for Eye Diseases, Shanghai Key Laboratory of Ocular Fundus Diseases, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai Engineering Center for Precise Diagnosis and Treatment of Eye Diseases, Shanghai, 200080, China
| |
Collapse
|
4
|
Tan X, Chen X, Meng Q, Shi F, Xiang D, Chen Z, Pan L, Zhu W. OCT 2Former: A retinal OCT-angiography vessel segmentation transformer. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 233:107454. [PMID: 36921468 DOI: 10.1016/j.cmpb.2023.107454] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 01/25/2023] [Accepted: 02/27/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Retinal vessel segmentation plays an important role in the automatic retinal disease screening and diagnosis. How to segment thin vessels and maintain the connectivity of vessels are the key challenges of the retinal vessel segmentation task. Optical coherence tomography angiography (OCTA) is a noninvasive imaging technique that can reveal high-resolution retinal vessels. Aiming at make full use of its characteristic of high resolution, a new end-to-end transformer based network named as OCT2Former (OCT-a Transformer) is proposed to segment retinal vessel accurately in OCTA images. METHODS The proposed OCT2Former is based on encoder-decoder structure, which mainly includes dynamic transformer encoder and lightweight decoder. Dynamic transformer encoder consists of dynamic token aggregation transformer and auxiliary convolution branch, in which the multi-head dynamic token aggregation attention based dynamic token aggregation transformer is designed to capture the global retinal vessel context information from the first layer throughout the network and the auxiliary convolution branch is proposed to compensate for the lack of inductive bias of the transformer and assist in the efficient feature extraction. A convolution based lightweight decoder is proposed to decode features efficiently and reduce the complexity of the proposed OCT2Former. RESULTS The proposed OCT2Former is validated on three publicly available datasets i.e. OCTA-SS, ROSE-1, OCTA-500 (subset OCTA-6M and OCTA-3M). The Jaccard indexes of the proposed OCT2Former on these datasets are 0.8344, 0.7855, 0.8099 and 0.8513, respectively, outperforming the best convolution based network 1.43, 1.32, 0.75 and 1.46%, respectively. CONCLUSION The experimental results have demonstrated that the proposed OCT2Former can achieve competitive performance on retinal OCTA vessel segmentation tasks.
Collapse
Affiliation(s)
- Xiao Tan
- MIPAV Lab, the School of Electronic and Information Engineering, Soochow University, Jiangsu, China
| | - Xinjian Chen
- MIPAV Lab, the School of Electronic and Information Engineering, Soochow University, Jiangsu, China; The State Key Laboratory of Radiation Medicine and Protection, Soochow University, Jiangsu, China
| | - Qingquan Meng
- MIPAV Lab, the School of Electronic and Information Engineering, Soochow University, Jiangsu, China
| | - Fei Shi
- MIPAV Lab, the School of Electronic and Information Engineering, Soochow University, Jiangsu, China
| | - Dehui Xiang
- MIPAV Lab, the School of Electronic and Information Engineering, Soochow University, Jiangsu, China
| | - Zhongyue Chen
- MIPAV Lab, the School of Electronic and Information Engineering, Soochow University, Jiangsu, China
| | - Lingjiao Pan
- School of Electrical and Information Engineering, Jiangsu University of Technology, Jiangsu, China
| | - Weifang Zhu
- MIPAV Lab, the School of Electronic and Information Engineering, Soochow University, Jiangsu, China.
| |
Collapse
|
5
|
A deep network embedded with rough fuzzy discretization for OCT fundus image segmentation. Sci Rep 2023; 13:328. [PMID: 36609585 PMCID: PMC9822971 DOI: 10.1038/s41598-023-27479-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Accepted: 01/03/2023] [Indexed: 01/08/2023] Open
Abstract
The noise and redundant information are the main reasons for the performance bottleneck of medical image segmentation algorithms based on the deep learning. To this end, we propose a deep network embedded with rough fuzzy discretization (RFDDN) for OCT fundus image segmentation. Firstly, we establish the information decision table of OCT fundus image segmentation, and regard each category of segmentation region as a fuzzy set. Then, we use the fuzzy c-means clustering to get the membership degrees of pixels to each segmentation region. According to membership functions and the equivalence relation generated by the brightness attribute, we design the individual fitness function based on the rough fuzzy set, and use a genetic algorithm to search for the best breakpoints to discretize the features of OCT fundus images. Finally, we take the feature discretization based on the rough fuzzy set as the pre-module of the deep neural network, and introduce the deep supervised attention mechanism to obtain the important multi-scale information. We compare RFDDN with U-Net, ReLayNet, CE-Net, MultiResUNet, and ISCLNet on the two groups of 3D retinal OCT data. RFDDN is superior to the other five methods on all evaluation indicators. The results obtained by ISCLNet are the second only inferior to those obtained by RFDDN. DSC, sensitivity, and specificity of RFDDN are evenly 3.3%, 2.6%, and 7.1% higher than those of ISCLNet, respectively. HD95 and ASD of RFDDN are evenly 6.6% and 19.7% lower than those of ISCLNet, respectively. The experimental results show that our method can effectively eliminate the noise and redundant information in Oct fundus images, and greatly improve the accuracy of OCT fundus image segmentation while taking into account the interpretability and computational efficiency.
Collapse
|
6
|
Zhan B, Song E, Liu H, Gong Z, Ma G, Hung CC. CFNet: A medical image segmentation method using the multi-view attention mechanism and adaptive fusion strategy. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2022]
|
7
|
Goel S, Sethi A, Pfau M, Munro M, Chan RVP, Lim JI, Hallak J, Alam M. Automated Region of Interest Selection Improves Deep Learning-Based Segmentation of Hyper-Reflective Foci in Optical Coherence Tomography Images. J Clin Med 2022; 11:jcm11247404. [PMID: 36556019 PMCID: PMC9784409 DOI: 10.3390/jcm11247404] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 11/30/2022] [Accepted: 12/11/2022] [Indexed: 12/15/2022] Open
Abstract
Hyperreflective foci (HRF) have been associated with retinal disease progression and demonstrated as a negative prognostic biomarker for visual function. Automated segmentation of HRF in retinal optical coherence tomography (OCT) scans can be beneficial to identify the formation and movement of the HRF biomarker as a retinal disease progresses and can serve as the first step in understanding the nature and severity of the disease. In this paper, we propose a fully automated deep neural network based HRF segmentation model in OCT images. We enhance the model's performance by using a patch-based strategy that increases the model's compute on the HRF pixels. The patch-based strategy is evaluated against state of the art HRF segmentation pipelines on clinical retinal image data. Our results shows that the patch-based approach demonstrates a high precision score and intersection over union (IOU) using a ResNet34 segmentation model with Binary Cross Entropy loss function. The HRF segmentation pipeline can be used for analyzing HRF biomarkers for different retinopathies.
Collapse
Affiliation(s)
- Sarang Goel
- Texas Academy of Mathematics and Science, Denton, TX 76203, USA
| | - Abhishek Sethi
- Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - Maximilian Pfau
- Institute of Molecular and Clinical Ophthalmology Basel, 4031 Basel, Switzerland
| | - Monique Munro
- Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - Robison Vernon Paul Chan
- Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - Jennifer I. Lim
- Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - Joelle Hallak
- Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, IL 60612, USA
| | - Minhaj Alam
- Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC 28223, USA
- Correspondence:
| |
Collapse
|
8
|
Tao Z, Dang H, Shi Y, Wang W, Wang X, Ren S. Local and Context-Attention Adaptive LCA-Net for Thyroid Nodule Segmentation in Ultrasound Images. SENSORS (BASEL, SWITZERLAND) 2022; 22:5984. [PMID: 36015742 PMCID: PMC9413141 DOI: 10.3390/s22165984] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 08/05/2022] [Accepted: 08/09/2022] [Indexed: 06/15/2023]
Abstract
The thyroid nodule segmentation of ultrasound images is a critical step for the early diagnosis of thyroid cancers in clinics. Due to the weak edge of ultrasound images and the complexity of thyroid tissue structure, it is still challenging to accurately segment the delicate contour of thyroid nodules. A local and context-attention adaptive network (LCA-Net) for thyroid nodule segmentation is proposed to address these shortcomings, which leverages both local feature information from convolution neural networks and global context information from transformers. Firstly, since most existing thyroid nodule segmentation models are skilled at local detail features and lose some context information, we propose a transformers-based context-attention module to capture more global associative information for the network and perceive the edge information of the nodule contour. Secondly, a backbone module with 7×1, 1×7 convolutions and the activation function Mish is designed, which enlarges the receptive field and extracts more feature details. Furthermore, a nodule adaptive convolution (NAC) module is introduced to adaptively deal with thyroid nodules of different sizes and positions, thereby improving the generalization performance of the model. Simultaneously, an optimized loss function is proposed to solve the pixels class imbalance problem in segmentation. The proposed LCA-Net, validated on the public TN-SCUI2020 and TN3K datasets, achieves Dice scores of 90.26% and 82.08% and PA scores of 98.87% and 96.97%, respectively, which outperforms other state-of-the-art thyroid nodule segmentation models. This paper demonstrates the superiority of the proposed LCA-Net for thyroid nodule segmentation, which possesses strong generalization performance and promising segmentation accuracy. Consequently, the proposed model has wide application prospects for thyroid nodule diagnosis in clinics.
Collapse
Affiliation(s)
- Zhen Tao
- School of Information and Electronics, Beijing Institute of Technology, 5 South Zhongguancun Street, Haidian District, Beijing 100081, China
| | - Hua Dang
- School of Information and Electronics, Beijing Institute of Technology, 5 South Zhongguancun Street, Haidian District, Beijing 100081, China
| | - Yueting Shi
- School of Information and Electronics, Beijing Institute of Technology, 5 South Zhongguancun Street, Haidian District, Beijing 100081, China
- Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing 314019, China
| | - Weijiang Wang
- School of Information and Electronics, Beijing Institute of Technology, 5 South Zhongguancun Street, Haidian District, Beijing 100081, China
- Beijing Institute of Technology, Chongqing Center for Microelectronics and Microsystems, Chongqing 401332, China
| | - Xiaohua Wang
- School of Information and Electronics, Beijing Institute of Technology, 5 South Zhongguancun Street, Haidian District, Beijing 100081, China
- Beijing Institute of Technology, Chongqing Center for Microelectronics and Microsystems, Chongqing 401332, China
| | - Shiwei Ren
- School of Information and Electronics, Beijing Institute of Technology, 5 South Zhongguancun Street, Haidian District, Beijing 100081, China
- Beijing Institute of Technology, Chongqing Center for Microelectronics and Microsystems, Chongqing 401332, China
| |
Collapse
|