1
|
Laouarem A, Kara-Mohamed C, Bourennane EB, Hamdi-Cherif A. HTC-retina: A hybrid retinal diseases classification model using transformer-Convolutional Neural Network from optical coherence tomography images. Comput Biol Med 2024; 178:108726. [PMID: 38878400 DOI: 10.1016/j.compbiomed.2024.108726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2023] [Revised: 06/04/2024] [Accepted: 06/07/2024] [Indexed: 07/24/2024]
Abstract
Retinal diseases are among nowadays major public health issues, deservedly needing advanced computer-aided diagnosis. We propose a hybrid model for multi label classification, whereby seven retinal diseases are automatically classified from Optical Coherence Tomography (OCT) images. We show that, by combining the strengths of Convolutional Neural Networks (CNNs) and Visual Transformers (ViTs), we can produce a more powerful type of model for medical image classification, especially when considering local lesion information such as retinal diseases. CNNs are indeed proved to be efficient at parameter utilization and provide the ability to extract local features and multi-scale feature maps through convolutional operations. On the other hand, ViT's self-attention procedure allows processing long-range and global dependencies within an image. The paper clearly shows that the hybridization of these complementary capabilities (CNNs-ViTs) presents a high image processing potential that is more robust and efficient. The proposed model adopts a hierarchical CNN module called Convolutional Patch and Token Embedding (CPTE) instead of employing a direct tokenization approach using the raw input OCT image in the transformer. The CPTE module's role is to incorporate an inductive bias, to reduce the reliance on large-scale datasets, and to address the low-level feature extraction challenges of the ViT. In addition, considering the importance of local lesion information in OCT images, the model relies on a parallel module called Residual Depthwise-Pointwise ConvNet (RDP-ConvNet) for extracting high-level features. RDP-ConvNet utilizes depthwise and pointwise convolution layers within a residual network architecture. The overall performance of the HTC-Retina model was evaluated on three datasets: the OCT-2017, OCT-C8, and OCT-2014 ; outperforming previous established models, achieving accuracy rates of 99.40%, 97.00%, and 99.77%, respectively ; and sensitivity rates of 99.41%, 97.00%, and 99.77%, respectively. Notably, the model showed high performance while maintaining computational efficiency.
Collapse
Affiliation(s)
- Ayoub Laouarem
- Department of Computer Science, University of Ferhat Abbas 1, Setif, Algeria.
| | - Chafia Kara-Mohamed
- Department of Computer Science, University of Ferhat Abbas 1, Setif, Algeria
| | | | | |
Collapse
|
2
|
Deng L, Zou Y, Yang X, Wang J, Huang S. L2NLF: a novel linear-to-nonlinear framework for multi-modal medical image registration. Biomed Eng Lett 2024; 14:497-509. [PMID: 38645595 PMCID: PMC11026354 DOI: 10.1007/s13534-023-00344-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 10/29/2023] [Accepted: 12/11/2023] [Indexed: 04/23/2024] Open
Abstract
In recent years, deep learning has ushered in significant development in medical image registration, and the method of non-rigid registration using deep neural networks to generate a deformation field has higher accuracy. However, unlike monomodal medical image registration, multimodal medical image registration is a more complex and challenging task. This paper proposes a new linear-to-nonlinear framework (L2NLF) for multimodal medical image registration. The first linear stage is essentially image conversion, which can reduce the difference between two images without changing the authenticity of medical images, thus transforming multimodal registration into monomodal registration. The second nonlinear stage is essentially unsupervised deformable registration based on the deep neural network. In this paper, a brand-new registration network, CrossMorph, is designed, a deep neural network similar to the U-net structure. As the backbone of the encoder, the volume CrossFormer block can better extract local and global information. Booster promotes the reduction of more deep features and shallow features. The qualitative and quantitative experimental results on T1 and T2 data of 240 patients' brains show that L2NLF can achieve excellent registration effect in the image conversion part with very low computation, and it will not change the authenticity of the converted image at all. Compared with the current state-of-the-art registration method, CrossMorph can effectively reduce average surface distance, improve dice score, and improve the deformation field's smoothness. The proposed methods have potential value in clinical application.
Collapse
Affiliation(s)
- Liwei Deng
- Heilongjiang Provincial Key Laboratory of Complex Intelligent System and Integration, School of Automation, Harbin University of Science and Technology, Harbin, 150080 China
| | - Yanchao Zou
- Heilongjiang Provincial Key Laboratory of Complex Intelligent System and Integration, School of Automation, Harbin University of Science and Technology, Harbin, 150080 China
| | - Xin Yang
- Department of Radiation Oncology, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou, 510060 Guangdong China
| | - Jing Wang
- Institute for Brain Research and Rehabilitation, South China Normal University, Zhongshan Avenue, Guangzhou, 510631 China
| | - Sijuan Huang
- Department of Radiation Oncology, Sun Yat-Sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangdong Key Laboratory of Nasopharyngeal Carcinoma Diagnosis and Therapy, Guangzhou, 510060 Guangdong China
| |
Collapse
|
3
|
Peng J, Lu J, Zhuo J, Li P. Multi-Scale-Denoising Residual Convolutional Network for Retinal Disease Classification Using OCT. SENSORS (BASEL, SWITZERLAND) 2023; 24:150. [PMID: 38203011 PMCID: PMC10781341 DOI: 10.3390/s24010150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Revised: 12/13/2023] [Accepted: 12/15/2023] [Indexed: 01/12/2024]
Abstract
Macular pathologies can cause significant vision loss. Optical coherence tomography (OCT) images of the retina can assist ophthalmologists in diagnosing macular diseases. Traditional deep learning networks for retinal disease classification cannot extract discriminative features under strong noise conditions in OCT images. To address this issue, we propose a multi-scale-denoising residual convolutional network (MS-DRCN) for classifying retinal diseases. Specifically, the MS-DRCN includes a soft-denoising block (SDB), a multi-scale context block (MCB), and a feature fusion block (FFB). The SDB can determine the threshold for soft thresholding automatically, which removes speckle noise features efficiently. The MCB is designed to capture multi-scale context information and strengthen extracted features. The FFB is dedicated to integrating high-resolution and low-resolution features to precisely identify variable lesion areas. Our approach achieved classification accuracies of 96.4% and 96.5% on the OCT2017 and OCT-C4 public datasets, respectively, outperforming other classification methods. To evaluate the robustness of our method, we introduced Gaussian noise and speckle noise with varying PSNRs into the test set of the OCT2017 dataset. The results of our anti-noise experiments demonstrate that our approach exhibits superior robustness compared with other methods, yielding accuracy improvements ranging from 0.6% to 2.9% when compared with ResNet under various PSNR noise conditions.
Collapse
Affiliation(s)
- Jinbo Peng
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
| | - Jinling Lu
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
- Britton Chance Center for Biomedical Photonics and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Junjie Zhuo
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
| | - Pengcheng Li
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
- Britton Chance Center for Biomedical Photonics and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China
| |
Collapse
|
4
|
Arslan S, Kaya MK, Tasci B, Kaya S, Tasci G, Ozsoy F, Dogan S, Tuncer T. Attention TurkerNeXt: Investigations into Bipolar Disorder Detection Using OCT Images. Diagnostics (Basel) 2023; 13:3422. [PMID: 37998558 PMCID: PMC10669998 DOI: 10.3390/diagnostics13223422] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2023] [Revised: 11/04/2023] [Accepted: 11/08/2023] [Indexed: 11/25/2023] Open
Abstract
Background and Aim: In the era of deep learning, numerous models have emerged in the literature and various application domains. Transformer architectures, particularly, have gained popularity in deep learning, with diverse transformer-based computer vision algorithms. Attention convolutional neural networks (CNNs) have been introduced to enhance image classification capabilities. In this context, we propose a novel attention convolutional model with the primary objective of detecting bipolar disorder using optical coherence tomography (OCT) images. Materials and Methods: To facilitate our study, we curated a unique OCT image dataset, initially comprising two distinct cases. For the development of an automated OCT image detection system, we introduce a new attention convolutional neural network named "TurkerNeXt". This proposed Attention TurkerNeXt encompasses four key modules: (i) the patchify stem block, (ii) the Attention TurkerNeXt block, (iii) the patchify downsampling block, and (iv) the output block. In line with the swin transformer, we employed a patchify operation in this study. The design of the attention block, Attention TurkerNeXt, draws inspiration from ConvNeXt, with an added shortcut operation to mitigate the vanishing gradient problem. The overall architecture is influenced by ResNet18. Results: The dataset comprises two distinctive cases: (i) top to bottom and (ii) left to right. Each case contains 987 training and 328 test images. Our newly proposed Attention TurkerNeXt achieved 100% test and validation accuracies for both cases. Conclusions: We curated a novel OCT dataset and introduced a new CNN, named TurkerNeXt in this research. Based on the research findings and classification results, our proposed TurkerNeXt model demonstrated excellent classification performance. This investigation distinctly underscores the potential of OCT images as a biomarker for bipolar disorder.
Collapse
Affiliation(s)
| | | | - Burak Tasci
- Vocational School of Technical Sciences, Firat University, 23119 Elazig, Turkey
| | - Suheda Kaya
- Department of Psychiatry, Elazig Fethi Sekin City Hospital, 23100 Elazig, Turkey; (S.K.); (G.T.)
| | - Gulay Tasci
- Department of Psychiatry, Elazig Fethi Sekin City Hospital, 23100 Elazig, Turkey; (S.K.); (G.T.)
| | - Filiz Ozsoy
- Department of Psychiatry, School of Medicine, Tokat Gaziosmanpasa University, 60100 Tokat, Turkey;
| | - Sengul Dogan
- Department of Digital Forensics Engineering, College of Technology, Firat University, 23119 Elazig, Turkey; (S.D.); (T.T.)
| | - Turker Tuncer
- Department of Digital Forensics Engineering, College of Technology, Firat University, 23119 Elazig, Turkey; (S.D.); (T.T.)
| |
Collapse
|
5
|
Diao S, Su J, Yang C, Zhu W, Xiang D, Chen X, Peng Q, Shi F. Classification and segmentation of OCT images for age-related macular degeneration based on dual guidance networks. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104810] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/13/2023]
|