1
|
Huang J, Zhang X, Jin R, Xu T, Jin Z, Shen M, Lv F, Chen J, Liu J. Wavelet-based selection-and-recalibration network for Parkinson's disease screening in OCT images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 256:108368. [PMID: 39154408 DOI: 10.1016/j.cmpb.2024.108368] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Revised: 07/30/2024] [Accepted: 08/07/2024] [Indexed: 08/20/2024]
Abstract
BACKGROUND AND OBJECTIVE Parkinson's disease (PD) is one of the most prevalent neurodegenerative brain diseases worldwide. Therefore, accurate PD screening is crucial for early clinical intervention and treatment. Recent clinical research indicates that changes in pathology, such as the texture and thickness of the retinal layers, can serve as biomarkers for clinical PD diagnosis based on optical coherence tomography (OCT) images. However, the pathological manifestations of PD in the retinal layers are subtle compared to the more salient lesions associated with retinal diseases. METHODS Inspired by textural edge feature extraction in frequency domain learning, we aim to explore a potential approach to enhance the distinction between the feature distributions in retinal layers of PD cases and healthy controls. In this paper, we introduce a simple yet novel wavelet-based selection and recalibration module to effectively enhance the feature representations of the deep neural network by aggregating the unique clinical properties, such as the retinal layers in each frequency band. We combine this module with the residual block to form a deep network named Wavelet-based Selection and Recalibration Network (WaveSRNet) for automatic PD screening. RESULTS The extensive experiments on a clinical PD-OCT dataset and two publicly available datasets demonstrate that our approach outperforms state-of-the-art methods. Visualization analysis and ablation studies are conducted to enhance the explainability of WaveSRNet in the decision-making process. CONCLUSIONS Our results suggest the potential role of the retina as an assessment tool for PD. Visual analysis shows that PD-related elements include not only certain retinal layers but also the location of the fovea in OCT images.
Collapse
Affiliation(s)
- Jingqi Huang
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China
| | - Xiaoqing Zhang
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China; Center for High Performance Computing and Shenzhen Key Laboratory of Intelligent Bioinformatics, Shenzhen institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China
| | - Richu Jin
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China
| | - Tao Xu
- The State Key Laboratory of Ophthalmology, Optometry and Vision Science, Wenzhou Medical University, Wenzhou, Zhejiang, China; The Oujiang Laboratory; The Affiliated Eye Hospital, Wenzhou Medical University, 270 Xueyuan Road, Wenzhou, Zhejiang, China
| | - Zi Jin
- National Engineering Research Center of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Meixiao Shen
- National Engineering Research Center of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Fan Lv
- The Oujiang Laboratory; The Affiliated Eye Hospital, Wenzhou Medical University, 270 Xueyuan Road, Wenzhou, Zhejiang, China; National Engineering Research Center of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Jiangfan Chen
- The State Key Laboratory of Ophthalmology, Optometry and Vision Science, Wenzhou Medical University, Wenzhou, Zhejiang, China; The Oujiang Laboratory; The Affiliated Eye Hospital, Wenzhou Medical University, 270 Xueyuan Road, Wenzhou, Zhejiang, China
| | - Jiang Liu
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China; The State Key Laboratory of Ophthalmology, Optometry and Vision Science, Wenzhou Medical University, Wenzhou, Zhejiang, China; Singapore Eye Research Institute, 169856, Singapore.
| |
Collapse
|
2
|
Haghayegh F, Norouziazad A, Haghani E, Feygin AA, Rahimi RH, Ghavamabadi HA, Sadighbayan D, Madhoun F, Papagelis M, Felfeli T, Salahandish R. Revolutionary Point-of-Care Wearable Diagnostics for Early Disease Detection and Biomarker Discovery through Intelligent Technologies. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2024:e2400595. [PMID: 38958517 DOI: 10.1002/advs.202400595] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 06/19/2024] [Indexed: 07/04/2024]
Abstract
Early-stage disease detection, particularly in Point-Of-Care (POC) wearable formats, assumes pivotal role in advancing healthcare services and precision-medicine. Public benefits of early detection extend beyond cost-effectively promoting healthcare outcomes, to also include reducing the risk of comorbid diseases. Technological advancements enabling POC biomarker recognition empower discovery of new markers for various health conditions. Integration of POC wearables for biomarker detection with intelligent frameworks represents ground-breaking innovations enabling automation of operations, conducting advanced large-scale data analysis, generating predictive models, and facilitating remote and guided clinical decision-making. These advancements substantially alleviate socioeconomic burdens, creating a paradigm shift in diagnostics, and revolutionizing medical assessments and technology development. This review explores critical topics and recent progress in development of 1) POC systems and wearable solutions for early disease detection and physiological monitoring, as well as 2) discussing current trends in adoption of smart technologies within clinical settings and in developing biological assays, and ultimately 3) exploring utilities of POC systems and smart platforms for biomarker discovery. Additionally, the review explores technology translation from research labs to broader applications. It also addresses associated risks, biases, and challenges of widespread Artificial Intelligence (AI) integration in diagnostics systems, while systematically outlining potential prospects, current challenges, and opportunities.
Collapse
Affiliation(s)
- Fatemeh Haghayegh
- Laboratory of Advanced Biotechnologies for Health Assessments (Lab-HA), Biomedical Engineering Program, Lassonde School of Engineering, York University, Toronto, M3J 1P3, Canada
- Department of Electrical Engineering and Computer Science (EECS), Lassonde School of Engineering, York University, Toronto, ON, M3J 1P3, Canada
| | - Alireza Norouziazad
- Laboratory of Advanced Biotechnologies for Health Assessments (Lab-HA), Biomedical Engineering Program, Lassonde School of Engineering, York University, Toronto, M3J 1P3, Canada
- Department of Electrical Engineering and Computer Science (EECS), Lassonde School of Engineering, York University, Toronto, ON, M3J 1P3, Canada
| | - Elnaz Haghani
- Laboratory of Advanced Biotechnologies for Health Assessments (Lab-HA), Biomedical Engineering Program, Lassonde School of Engineering, York University, Toronto, M3J 1P3, Canada
- Department of Electrical Engineering and Computer Science (EECS), Lassonde School of Engineering, York University, Toronto, ON, M3J 1P3, Canada
| | - Ariel Avraham Feygin
- Laboratory of Advanced Biotechnologies for Health Assessments (Lab-HA), Biomedical Engineering Program, Lassonde School of Engineering, York University, Toronto, M3J 1P3, Canada
- Department of Electrical Engineering and Computer Science (EECS), Lassonde School of Engineering, York University, Toronto, ON, M3J 1P3, Canada
| | - Reza Hamed Rahimi
- Laboratory of Advanced Biotechnologies for Health Assessments (Lab-HA), Biomedical Engineering Program, Lassonde School of Engineering, York University, Toronto, M3J 1P3, Canada
- Department of Electrical Engineering and Computer Science (EECS), Lassonde School of Engineering, York University, Toronto, ON, M3J 1P3, Canada
| | - Hamidreza Akbari Ghavamabadi
- Laboratory of Advanced Biotechnologies for Health Assessments (Lab-HA), Biomedical Engineering Program, Lassonde School of Engineering, York University, Toronto, M3J 1P3, Canada
- Department of Electrical Engineering and Computer Science (EECS), Lassonde School of Engineering, York University, Toronto, ON, M3J 1P3, Canada
| | - Deniz Sadighbayan
- Department of Biology, Faculty of Science, York University, Toronto, ON, M3J 1P3, Canada
| | - Faress Madhoun
- Laboratory of Advanced Biotechnologies for Health Assessments (Lab-HA), Biomedical Engineering Program, Lassonde School of Engineering, York University, Toronto, M3J 1P3, Canada
- Department of Electrical Engineering and Computer Science (EECS), Lassonde School of Engineering, York University, Toronto, ON, M3J 1P3, Canada
| | - Manos Papagelis
- Department of Electrical Engineering and Computer Science (EECS), Lassonde School of Engineering, York University, Toronto, ON, M3J 1P3, Canada
| | - Tina Felfeli
- Department of Ophthalmology and Vision Sciences, University of Toronto, Ontario, M5T 3A9, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Ontario, M5T 3M6, Canada
| | - Razieh Salahandish
- Laboratory of Advanced Biotechnologies for Health Assessments (Lab-HA), Biomedical Engineering Program, Lassonde School of Engineering, York University, Toronto, M3J 1P3, Canada
- Department of Electrical Engineering and Computer Science (EECS), Lassonde School of Engineering, York University, Toronto, ON, M3J 1P3, Canada
| |
Collapse
|
3
|
Sendecki A, Ledwoń D, Nycz J, Wąsowska A, Boguszewska-Chachulska A, Mitas AW, Wylęgała E, Teper S. A deep learning approach to explore the association of age-related macular degeneration polygenic risk score with retinal optical coherence tomography: A preliminary study. Acta Ophthalmol 2024. [PMID: 38761033 DOI: 10.1111/aos.16710] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Accepted: 05/06/2024] [Indexed: 05/20/2024]
Abstract
PURPOSE Age-related macular degeneration (AMD) is a complex eye disorder affecting millions worldwide. This article uses deep learning techniques to investigate the relationship between AMD, genetics and optical coherence tomography (OCT) scans. METHODS The cohort consisted of 332 patients, of which 235 were diagnosed with AMD and 97 were controls with no signs of AMD. The genome-wide association studies summary statistics utilized to establish the polygenic risk score (PRS) in relation to AMD were derived from the GERA European study. A PRS estimation based on OCT volumes for both eyes was performed using a proprietary convolutional neural network (CNN) model supported by machine learning models. The method's performance was assessed using numerical evaluation metrics, and the Grad-CAM technique was used to evaluate the results by visualizing the features learned by the model. RESULTS The best results were obtained with the CNN and the Extra Tree regressor (MAE = 0.55, MSE = 0.49, RMSE = 0.70, R2 = 0.34). Extending the feature vector with additional information on AMD diagnosis, age and smoking history improved the results slightly, with mainly AMD diagnosis used by the model (MAE = 0.54, MSE = 0.44, RMSE = 0.66, R2 = 0.42). Grad-CAM heatmap evaluation showed that the model decisions rely on retinal morphology factors relevant to AMD diagnosis. CONCLUSION The developed method allows an efficient PRS estimation from OCT images. A new technique for analysing the association of OCT images with PRS of AMD, using a deep learning approach, may provide an opportunity to discover new associations between genotype-based AMD risk and retinal morphology.
Collapse
Affiliation(s)
- Adam Sendecki
- Chair and Clinical Department of Ophthalmology, Faculty of Medical Sciences in Zabrze, Medical University of Silesia, Katowice, Poland
| | - Daniel Ledwoń
- Faculty of Biomedical Engineering, Silesian University of Technology, Zabrze, Poland
| | - Julia Nycz
- Institute of Biomedical Engineering and Informatics, Technische Universität Ilmenau, Ilmenau, Germany
| | - Anna Wąsowska
- Chair and Clinical Department of Ophthalmology, Faculty of Medical Sciences in Zabrze, Medical University of Silesia, Katowice, Poland
- Genomed S.A., Warszawa, Poland
| | | | - Andrzej W Mitas
- Faculty of Biomedical Engineering, Silesian University of Technology, Zabrze, Poland
| | - Edward Wylęgała
- Chair and Clinical Department of Ophthalmology, Faculty of Medical Sciences in Zabrze, Medical University of Silesia, Katowice, Poland
| | - Sławomir Teper
- Chair and Clinical Department of Ophthalmology, Faculty of Medical Sciences in Zabrze, Medical University of Silesia, Katowice, Poland
| |
Collapse
|
4
|
Cheng Z, Wang S, Gao Y, Zhu Z, Yan C. Invariant Content Representation for Generalizable Medical Image Segmentation. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01088-9. [PMID: 38758420 DOI: 10.1007/s10278-024-01088-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Revised: 01/20/2024] [Accepted: 02/09/2024] [Indexed: 05/18/2024]
Abstract
Domain generalization (DG) for medical image segmentation due to privacy preservation prefers learning from a single-source domain and expects good robustness on unseen target domains. To achieve this goal, previous methods mainly use data augmentation to expand the distribution of samples and learn invariant content from them. However, most of these methods commonly perform global augmentation, leading to limited augmented sample diversity. In addition, the style of the augmented image is more scattered than the source domain, which may cause the model to overfit the style of the source domain. To address the above issues, we propose an invariant content representation network (ICRN) to enhance the learning of invariant content and suppress the learning of variability styles. Specifically, we first design a gamma correction-based local style augmentation (LSA) to expand the distribution of samples by augmenting foreground and background styles, respectively. Then, based on the augmented samples, we introduce invariant content learning (ICL) to learn generalizable invariant content from both augmented and source-domain samples. Finally, we design domain-specific batch normalization (DSBN) based style adversarial learning (SAL) to suppress the learning of preferences for source-domain styles. Experimental results show that our proposed method improves by 8.74% and 11.33% in overall dice coefficient (Dice) and reduces 15.88 mm and 3.87 mm in overall average surface distance (ASD) on two publicly available cross-domain datasets, Fundus and Prostate, compared to the state-of-the-art DG methods. The code is available at https://github.com/ZMC-IIIM/ICRN-DG .
Collapse
Affiliation(s)
- Zhiming Cheng
- School of Automation, Hangzhou Dianzi University, Hangzhou, 310018, China
| | - Shuai Wang
- School of Cyberspace, Hangzhou Dianzi University, Hangzhou, 310018, China.
- Suzhou Research Institute of Shandong University, SuZhou, 215123, China.
| | - Yuhan Gao
- School of Automation, Hangzhou Dianzi University, Hangzhou, 310018, China
- Lishui Institute of Hangzhou Dianzi Universitu, Lishui, 323010, China
| | - Zunjie Zhu
- Lishui Institute of Hangzhou Dianzi Universitu, Lishui, 323010, China
- School of Communication, Engineering, Hangzhou Dianzi Universitu, Hangzhou, 310018, China
| | - Chenggang Yan
- School of Communication, Engineering, Hangzhou Dianzi Universitu, Hangzhou, 310018, China
| |
Collapse
|
5
|
Henriques J, Amaro AM, Piedade AP. Biomimicking Atherosclerotic Vessels: A Relevant and (Yet) Sub-Explored Topic. Biomimetics (Basel) 2024; 9:135. [PMID: 38534820 DOI: 10.3390/biomimetics9030135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2023] [Revised: 02/20/2024] [Accepted: 02/21/2024] [Indexed: 03/28/2024] Open
Abstract
Atherosclerosis represents the etiologic source of several cardiovascular events, including myocardial infarction, cerebrovascular accidents, and peripheral artery disease, which remain the leading cause of mortality in the world. Numerous strategies are being delineated to revert the non-optimal projections of the World Health Organization, by both designing new diagnostic and therapeutic approaches or improving the interventional procedures performed by physicians. Deeply understanding the pathological process of atherosclerosis is, therefore, mandatory to accomplish improved results in these trials. Due to their availability, reproducibility, low expensiveness, and rapid production, biomimicking physical models are preferred over animal experimentation because they can overcome some limitations, mainly related to replicability and ethical issues. Their capability to represent any atherosclerotic stage and/or plaque type makes them valuable tools to investigate hemodynamical, pharmacodynamical, and biomechanical behaviors, as well as to optimize imaging systems and, thus, obtain meaningful prospects to improve the efficacy and effectiveness of treatment on a patient-specific basis. However, the broadness of possible applications in which these biomodels can be used is associated with a wide range of tissue-mimicking materials that are selected depending on the final purpose of the model and, consequently, prioritizing some materials' properties over others. This review aims to summarize the progress in fabricating biomimicking atherosclerotic models, mainly focusing on using materials according to the intended application.
Collapse
Affiliation(s)
- Joana Henriques
- University of Coimbra, CEMMPRE, ARISE, Department of Mechanical Engineering, 3030-788 Coimbra, Portugal
| | - Ana M Amaro
- University of Coimbra, CEMMPRE, ARISE, Department of Mechanical Engineering, 3030-788 Coimbra, Portugal
| | - Ana P Piedade
- University of Coimbra, CEMMPRE, ARISE, Department of Mechanical Engineering, 3030-788 Coimbra, Portugal
| |
Collapse
|
6
|
Wang YZ, Juroch K, Birch DG. Deep Learning-Assisted Measurements of Photoreceptor Ellipsoid Zone Area and Outer Segment Volume as Biomarkers for Retinitis Pigmentosa. Bioengineering (Basel) 2023; 10:1394. [PMID: 38135984 PMCID: PMC10740805 DOI: 10.3390/bioengineering10121394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 11/13/2023] [Accepted: 11/29/2023] [Indexed: 12/24/2023] Open
Abstract
The manual segmentation of retinal layers from OCT scan images is time-consuming and costly. The deep learning approach has potential for the automatic delineation of retinal layers to significantly reduce the burden of human graders. In this study, we compared deep learning model (DLM) segmentation with manual correction (DLM-MC) to conventional manual grading (MG) for the measurements of the photoreceptor ellipsoid zone (EZ) area and outer segment (OS) volume in retinitis pigmentosa (RP) to assess whether DLM-MC can be a new gold standard for retinal layer segmentation and for the measurement of retinal layer metrics. Ninety-six high-speed 9 mm 31-line volume scans obtained from 48 patients with RPGR-associated XLRP were selected based on the following criteria: the presence of an EZ band within the scan limit and a detectable EZ in at least three B-scans in a volume scan. All the B-scan images in each volume scan were manually segmented for the EZ and proximal retinal pigment epithelium (pRPE) by two experienced human graders to serve as the ground truth for comparison. The test volume scans were also segmented by a DLM and then manually corrected for EZ and pRPE by the same two graders to obtain DLM-MC segmentation. The EZ area and OS volume were determined by interpolating the discrete two-dimensional B-scan EZ-pRPE layer over the scan area. Dice similarity, Bland-Altman analysis, correlation, and linear regression analyses were conducted to assess the agreement between DLM-MC and MG for the EZ area and OS volume measurements. For the EZ area, the overall mean dice score (SD) between DLM-MC and MG was 0.8524 (0.0821), which was comparable to 0.8417 (0.1111) between two MGs. For the EZ area > 1 mm2, the average dice score increased to 0.8799 (0.0614). When comparing DLM-MC to MG, the Bland-Altman plots revealed a mean difference (SE) of 0.0132 (0.0953) mm2 and a coefficient of repeatability (CoR) of 1.8303 mm2 for the EZ area and a mean difference (SE) of 0.0080 (0.0020) mm3 and a CoR of 0.0381 mm3 for the OS volume. The correlation coefficients (95% CI) were 0.9928 (0.9892-0.9952) and 0.9938 (0.9906-0.9958) for the EZ area and OS volume, respectively. The linear regression slopes (95% CI) were 0.9598 (0.9399-0.9797) and 1.0104 (0.9909-1.0298), respectively. The results from this study suggest that the manual correction of deep learning model segmentation can generate EZ area and OS volume measurements in excellent agreement with those of conventional manual grading in RP. Because DLM-MC is more efficient for retinal layer segmentation from OCT scan images, it has the potential to reduce the burden of human graders in obtaining quantitative measurements of biomarkers for assessing disease progression and treatment outcomes in RP.
Collapse
Affiliation(s)
- Yi-Zhong Wang
- Retina Foundation of the Southwest, 9600 North Central Expressway, Suite 200, Dallas, TX 75231, USA; (K.J.); (D.G.B.)
- Department of Ophthalmology, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX 75390, USA
| | - Katherine Juroch
- Retina Foundation of the Southwest, 9600 North Central Expressway, Suite 200, Dallas, TX 75231, USA; (K.J.); (D.G.B.)
| | - David Geoffrey Birch
- Retina Foundation of the Southwest, 9600 North Central Expressway, Suite 200, Dallas, TX 75231, USA; (K.J.); (D.G.B.)
- Department of Ophthalmology, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX 75390, USA
| |
Collapse
|
7
|
Darooei R, Nazari M, Kafieh R, Rabbani H. Optimal Deep Learning Architecture for Automated Segmentation of Cysts in OCT Images Using X-Let Transforms. Diagnostics (Basel) 2023; 13:1994. [PMID: 37370889 DOI: 10.3390/diagnostics13121994] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2023] [Revised: 05/22/2023] [Accepted: 06/02/2023] [Indexed: 06/29/2023] Open
Abstract
The retina is a thin, light-sensitive membrane with a multilayered structure found in the back of the eyeball. There are many types of retinal disorders. The two most prevalent retinal illnesses are Age-Related Macular Degeneration (AMD) and Diabetic Macular Edema (DME). Optical Coherence Tomography (OCT) is a vital retinal imaging technology. X-lets (such as curvelet, DTCWT, contourlet, etc.) have several benefits in image processing and analysis. They can capture both local and non-local features of an image simultaneously. The aim of this paper is to propose an optimal deep learning architecture based on sparse basis functions for the automated segmentation of cystic areas in OCT images. Different X-let transforms were used to produce different network inputs, including curvelet, Dual-Tree Complex Wavelet Transform (DTCWT), circlet, and contourlet. Additionally, three different combinations of these transforms are suggested to achieve more accurate segmentation results. Various metrics, including Dice coefficient, sensitivity, false positive ratio, Jaccard index, and qualitative results, were evaluated to find the optimal networks and combinations of the X-let's sub-bands. The proposed network was tested on both original and noisy datasets. The results show the following facts: (1) contourlet achieves the optimal results between different combinations; (2) the five-channel decomposition using high-pass sub-bands of contourlet transform achieves the best performance; and (3) the five-channel decomposition using high-pass sub-bands formations out-performs the state-of-the-art methods, especially in the noisy dataset. The proposed method has the potential to improve the accuracy and speed of the segmentation process in clinical settings, facilitating the diagnosis and treatment of retinal diseases.
Collapse
Affiliation(s)
- Reza Darooei
- Medical Image and Signal Processing Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan 8174673461, Iran
- Department of Bioelectrics and Biomedical Engineering, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan 8174673461, Iran
| | - Milad Nazari
- Department of Molecular Biology and Genetics, Aarhus University, 8200 Aarhus, Denmark
- The Danish Research Institute of Translational Neuroscience (DANDRITE), Aarhus University, 8200 Aarhus, Denmark
| | - Rahele Kafieh
- Medical Image and Signal Processing Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan 8174673461, Iran
- Department of Engineering, Durham University, South Road, Durham DH1 3RW, UK
| | - Hossein Rabbani
- Medical Image and Signal Processing Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan 8174673461, Iran
- Department of Bioelectrics and Biomedical Engineering, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan 8174673461, Iran
| |
Collapse
|
8
|
Shi P, Qiu J, Abaxi SMD, Wei H, Lo FPW, Yuan W. Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation. Diagnostics (Basel) 2023; 13:diagnostics13111947. [PMID: 37296799 DOI: 10.3390/diagnostics13111947] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 05/26/2023] [Accepted: 05/31/2023] [Indexed: 06/12/2023] Open
Abstract
Medical image analysis plays an important role in clinical diagnosis. In this paper, we examine the recent Segment Anything Model (SAM) on medical images, and report both quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks, covering various imaging modalities, such as optical coherence tomography (OCT), magnetic resonance imaging (MRI), and computed tomography (CT), as well as different applications including dermatology, ophthalmology, and radiology. Those benchmarks are representative and commonly used in model development. Our experimental results indicate that while SAM presents remarkable segmentation performance on images from the general domain, its zero-shot segmentation ability remains restricted for out-of-distribution images, e.g., medical images. In addition, SAM exhibits inconsistent zero-shot segmentation performance across different unseen medical domains. For certain structured targets, e.g., blood vessels, the zero-shot segmentation of SAM completely failed. In contrast, a simple fine-tuning of it with a small amount of data could lead to remarkable improvement of the segmentation quality, showing the great potential and feasibility of using fine-tuned SAM to achieve accurate medical image segmentation for a precision diagnostics. Our study indicates the versatility of generalist vision foundation models on medical imaging, and their great potential to achieve desired performance through fine-turning and eventually address the challenges associated with accessing large and diverse medical datasets in support of clinical diagnostics.
Collapse
Affiliation(s)
- Peilun Shi
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| | - Jianing Qiu
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
- Department of Computing, Imperial College London, London SW7 2AZ, UK
| | - Sai Mu Dalike Abaxi
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| | - Hao Wei
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| | - Frank P-W Lo
- Hamlyn Centre, Department of Surgery and Cancer, Imperial College London, London SW7 2AZ, UK
| | - Wu Yuan
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| |
Collapse
|
9
|
Buyukcelik ON, Lapierre-Landry M, Kolluru C, Upadhye AR, Marshall DP, Pelot NA, Ludwig KA, Gustafson KJ, Wilson DL, Jenkins MW, Shoffstall AJ. Deep-learning segmentation of fascicles from microCT of the human vagus nerve. Front Neurosci 2023; 17:1169187. [PMID: 37332862 PMCID: PMC10275336 DOI: 10.3389/fnins.2023.1169187] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2023] [Accepted: 04/12/2023] [Indexed: 06/20/2023] Open
Abstract
Introduction MicroCT of the three-dimensional fascicular organization of the human vagus nerve provides essential data to inform basic anatomy as well as the development and optimization of neuromodulation therapies. To process the images into usable formats for subsequent analysis and computational modeling, the fascicles must be segmented. Prior segmentations were completed manually due to the complex nature of the images, including variable contrast between tissue types and staining artifacts. Methods Here, we developed a U-Net convolutional neural network (CNN) to automate segmentation of fascicles in microCT of human vagus nerve. Results The U-Net segmentation of ~500 images spanning one cervical vagus nerve was completed in 24 s, versus ~40 h for manual segmentation, i.e., nearly four orders of magnitude faster. The automated segmentations had a Dice coefficient of 0.87, a measure of pixel-wise accuracy, thus suggesting a rapid and accurate segmentation. While Dice coefficients are a commonly used metric to assess segmentation performance, we also adapted a metric to assess fascicle-wise detection accuracy, which showed that our network accurately detects the majority of fascicles, but may under-detect smaller fascicles. Discussion This network and the associated performance metrics set a benchmark, using a standard U-Net CNN, for the application of deep-learning algorithms to segment fascicles from microCT images. The process may be further optimized by refining tissue staining methods, modifying network architecture, and expanding the ground-truth training data. The resulting three-dimensional segmentations of the human vagus nerve will provide unprecedented accuracy to define nerve morphology in computational models for the analysis and design of neuromodulation therapies.
Collapse
Affiliation(s)
- Ozge N. Buyukcelik
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
- Advanced Platform Technologies Center, Louis Stokes Cleveland VA Medical Center, Cleveland, OH, United States
| | - Maryse Lapierre-Landry
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Chaitanya Kolluru
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Aniruddha R. Upadhye
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
- Advanced Platform Technologies Center, Louis Stokes Cleveland VA Medical Center, Cleveland, OH, United States
| | - Daniel P. Marshall
- Department of Biomedical Engineering, Duke University, Durham, NC, United States
| | - Nicole A. Pelot
- Department of Biomedical Engineering, Duke University, Durham, NC, United States
| | - Kip A. Ludwig
- Department of Biomedical Engineering, University of Wisconsin Madison, Madison, WI, United States
- Department of Neurological Surgery, University of Wisconsin Madison, Madison, WI, United States
- Wisconsin Institute for Translational Neuroengineering, Madison, WI, United States
| | - Kenneth J. Gustafson
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
- Functional Electrical Stimulation Center, Louis Stokes Cleveland VA Medical Center, Cleveland, OH, United States
| | - David L. Wilson
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Michael W. Jenkins
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
- Department of Pediatrics, School of Medicine, Case Western Reserve University, Cleveland, OH, United States
| | - Andrew J. Shoffstall
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
- Advanced Platform Technologies Center, Louis Stokes Cleveland VA Medical Center, Cleveland, OH, United States
| |
Collapse
|
10
|
Dehghani S, Sommersperger M, Zhang P, Martin-Gomez A, Busam B, Gehlbach P, Navab N, Nasseri MA, Iordachita I. Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing. IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION : ICRA : [PROCEEDINGS]. IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION 2023; 2023:4724-4731. [PMID: 38125032 PMCID: PMC10732544 DOI: 10.1109/icra48891.2023.10160372] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/23/2023]
Abstract
In the last decade, various robotic platforms have been introduced that could support delicate retinal surgeries. Concurrently, to provide semantic understanding of the surgical area, recent advances have enabled microscope-integrated intraoperative Optical Coherent Tomography (iOCT) with high-resolution 3D imaging at near video rate. The combination of robotics and semantic understanding enables task autonomy in robotic retinal surgery, such as for subretinal injection. This procedure requires precise needle insertion for best treatment outcomes. However, merging robotic systems with iOCT introduces new challenges. These include, but are not limited to high demands on data processing rates and dynamic registration of these systems during the procedure. In this work, we propose a framework for autonomous robotic navigation for subretinal injection, based on intelligent real-time processing of iOCT volumes. Our method consists of an instrument pose estimation method, an online registration between the robotic and the iOCT system, and trajectory planning tailored for navigation to an injection target. We also introduce intelligent virtual B-scans, a volume slicing approach for rapid instrument pose estimation, which is enabled by Convolutional Neural Networks (CNNs). Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method. Finally, we discuss identified challenges in this work and suggest potential solutions to further the development of such systems.
Collapse
Affiliation(s)
- Shervin Dehghani
- Department of Computer Science, Technische Universität München, München 85748 Germany
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA
| | - Michael Sommersperger
- Department of Computer Science, Technische Universität München, München 85748 Germany
| | - Peiyao Zhang
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA
| | - Alejandro Martin-Gomez
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA
| | - Benjamin Busam
- Department of Computer Science, Technische Universität München, München 85748 Germany
| | - Peter Gehlbach
- Wilmer Eye Institute, Johns Hopkins Hospital, Baltimore, MD, USA
| | - Nassir Navab
- Computer Aided Medical Procedures & Augmented Reality, Technical University of Munich, 85748 Munich, Germany, and an adjunct professor at the Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - M. Ali Nasseri
- Department of Computer Science, Technische Universität München, München 85748 Germany
- Augenklinik und Poliklinik, Klinikum rechts der Isar der Technische Universität München, München 81675 Germany
| | - Iulian Iordachita
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
11
|
Zhang W, Sun X, Zhou L, Xie X, Zhao W, Liang Z, Zhuang P. Dual-branch collaborative learning network for crop disease identification. FRONTIERS IN PLANT SCIENCE 2023; 14:1117478. [PMID: 36844059 PMCID: PMC9950499 DOI: 10.3389/fpls.2023.1117478] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Accepted: 01/20/2023] [Indexed: 06/18/2023]
Abstract
Crop diseases seriously affect the quality, yield, and food security of crops. redBesides, traditional manual monitoring methods can no longer meet intelligent agriculture's efficiency and accuracy requirements. Recently, deep learning methods have been rapidly developed in computer vision. To cope with these issues, we propose a dual-branch collaborative learning network for crop disease identification, called DBCLNet. Concretely, we propose a dual-branch collaborative module using convolutional kernels of different scales to extract global and local features of images, which can effectively utilize both global and local features. Meanwhile, we embed a channel attention mechanism in each branch module to refine the global and local features. Whereafter, we cascade multiple dual-branch collaborative modules to design a feature cascade module, which further learns features at more abstract levels via the multi-layer cascade design strategy. Extensive experiments on the Plant Village dataset demonstrated the best classification performance of our DBCLNet method compared to the state-of-the-art methods for the identification of 38 categories of crop diseases. Besides, the Accuracy, Precision, Recall, and F-score of our DBCLNet for the identification of 38 categories of crop diseases are 99.89%, 99.97%, 99.67%, and 99.79%, respectively. 811.
Collapse
Affiliation(s)
- Weidong Zhang
- School of Information Engineering, Henan Institute of Science and Technology, Xinxiang, China
| | - Xuewei Sun
- School of Information Engineering, Henan Institute of Science and Technology, Xinxiang, China
| | - Ling Zhou
- School of Information Engineering, Henan Institute of Science and Technology, Xinxiang, China
| | - Xiwang Xie
- School of Information Science and Technology, Dalian Maritime University, Dalian, Liaoning, China
| | - Wenyi Zhao
- School of Artificial Intelligence, Beijing University of Posts and Telecommunications (BUPT), Beijing, China
| | - Zheng Liang
- Internet Academy, Anhui University, Hefei, Anhui, China
| | - Peixian Zhuang
- School of Information Engineering, Henan Institute of Science and Technology, Xinxiang, China
- School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, China
| |
Collapse
|
12
|
CTS-Net: A Segmentation Network for Glaucoma Optical Coherence Tomography Retinal Layer Images. Bioengineering (Basel) 2023; 10:bioengineering10020230. [PMID: 36829724 PMCID: PMC9952657 DOI: 10.3390/bioengineering10020230] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2022] [Revised: 01/29/2023] [Accepted: 02/04/2023] [Indexed: 02/11/2023] Open
Abstract
Optical Coherence Tomography (OCT) technology is essential to obtain glaucoma diagnostic data non-invasively and rapidly. Early diagnosis of glaucoma can be achieved by analyzing the thickness and shape of retinal layers. Accurate retinal layer segmentation assists ophthalmologists in improving the efficiency of disease diagnosis. Deep learning technology is one of the most effective methods for processing OCT retinal layer images, which can segment different retinal layers and effectively obtain the topological structure of the boundary. This paper proposes a neural network method for retinal layer segmentation based on the CSWin Transformer (CTS-Net), which can achieve pixel-level segmentation and obtain smooth boundaries. A Dice loss function based on boundary areas (BADice Loss) is proposed to make CTS-Net learn more features of edge regions and improve the accuracy of boundary segmentation. We applied the model to the publicly available dataset of glaucoma retina, and the test results showed that mean absolute distance (MAD), root mean square error (RMSE), and dice-similarity coefficient (DSC) metrics were 1.79 pixels, 2.15 pixels, and 92.79%, respectively, which are better than those of the compared model. In the cross-validation experiment, the ranges of MAD, RMSE, and DSC are 0.05 pixels, 0.03 pixels, and 0.33%, respectively, with a slight difference, which further verifies the generalization ability of CTS-Net.
Collapse
|
13
|
Spatial-contextual variational autoencoder with attention correction for anomaly detection in retinal OCT images. Comput Biol Med 2023; 152:106328. [PMID: 36462369 DOI: 10.1016/j.compbiomed.2022.106328] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 10/23/2022] [Accepted: 11/14/2022] [Indexed: 11/18/2022]
Abstract
Anomaly detection refers to leveraging only normal data to train a model for identifying unseen abnormal cases, which is extensively studied in various fields. Most previous methods are based on reconstruction models, and use anomaly score calculated by the reconstruction error as the metric to tackle anomaly detection. However, these methods just employ single constraint on latent space to construct reconstruction model, resulting in limited performance in anomaly detection. To address this problem, we propose a Spatial-Contextual Variational Autoencoder with Attention Correction for anomaly detection in retinal OCT images. Specifically, we first propose a self-supervised segmentation network to extract retinal regions, which can effectively eliminate interference of background regions. Next, by introducing both multi-dimensional and one-dimensional latent space, our proposed framework can then learn the spatial and contextual manifolds of normal images, which is conducive to enlarging the difference between reconstruction errors of normal images and those of abnormal ones. Furthermore, an ablation-based method is proposed to localize anomalous regions by computing the importance of feature maps, which is used to correct anomaly score calculated by reconstruction error. Finally, a novel anomaly score is constructed to separate the abnormal images from the normal ones. Extensive experiments on two retinal OCT datasets are conducted to evaluate our proposed method, and the experimental results demonstrate the effectiveness of our approach.
Collapse
|
14
|
Multi-layer segmentation of retina OCT images via advanced U-net architecture. Neurocomputing 2023. [DOI: 10.1016/j.neucom.2022.10.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|