1
|
Wang N, Hu L, Walsh AJ. Evaluation of Cellpose segmentation with sequential thresholding for instance segmentation of cytoplasms within autofluorescence images. Comput Biol Med 2024; 179:108846. [PMID: 38976959 DOI: 10.1016/j.compbiomed.2024.108846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Revised: 07/01/2024] [Accepted: 07/02/2024] [Indexed: 07/10/2024]
Abstract
BACKGROUND Autofluorescence imaging of the coenzyme, reduced nicotinamide adenine dinucleotide (phosphate) (NAD(P)H), provides a label-free technique to assess cellular metabolism. Because NAD(P)H is localized in the cytosol and mitochondria, instance segmentation of cell cytoplasms from NAD(P)H images allows quantification of metabolism with cellular resolution. However, accurate cytoplasmic segmentation of autofluorescence images is difficult due to irregular cell shapes and cell clusters. METHOD Here, a cytoplasm segmentation method is presented and tested. First, autofluorescence images are segmented into cells via either hand-segmentation or Cellpose, a deep learning-based segmentation method. Then, a cytoplasmic post-processing algorithm (CPPA) is applied for cytoplasmic segmentation. CPPA uses a binarized segmentation image to remove non-segmented pixels from the NAD(P)H image and then applies an intensity-based threshold to identify nuclei regions. Errors at cell edges are removed using a distance transform algorithm. The nucleus mask is then subtracted from the cell segmented image to yield the cytoplasm mask image. CPPA was tested on five NAD(P)H images of three different cell samples, quiescent T cells, activated T cells, and MCF7 cells. RESULTS Using POSEA, an evaluation method tailored for instance segmentation, the CPPA yielded F-measure values of 0.89, 0.87, and 0.94 for quiescent T cells, activated T cells, and MCF7 cells, respectively, for cytoplasm identification of hand-segmented cells. CPPA achieved F-measure values of 0.84, 0.74, and 0.72 for Cellpose segmented cells. CONCLUSION These results exceed the F-measure value of a comparative cell segmentation method (CellProfiler, ∼0.50-0.60) and support the use of artificial intelligence and post-processing techniques for accurate segmentation of autofluorescence images for single-cell metabolic analyses.
Collapse
Affiliation(s)
- Nianchao Wang
- Texas A&M University, 3120 TAMU, College Station, 77840, United States
| | - Linghao Hu
- Texas A&M University, 3120 TAMU, College Station, 77840, United States
| | - Alex J Walsh
- Texas A&M University, 3120 TAMU, College Station, 77840, United States.
| |
Collapse
|
2
|
Spaide T, Jiang J, Patil J, Anegondi N, Steffen V, Kawczynski MG, Newton EM, Rabe C, Gao SS, Lee AY, Holz FG, Sadda S, Schmitz-Valckenberg S, Ferrara D. Geographic Atrophy Segmentation Using Multimodal Deep Learning. Transl Vis Sci Technol 2023; 12:10. [PMID: 37428131 DOI: 10.1167/tvst.12.7.10] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/11/2023] Open
Abstract
Purpose To examine deep learning (DL)-based methods for accurate segmentation of geographic atrophy (GA) lesions using fundus autofluorescence (FAF) and near-infrared (NIR) images. Methods This retrospective analysis utilized imaging data from study eyes of patients enrolled in Proxima A and B (NCT02479386; NCT02399072) natural history studies of GA. Two multimodal DL networks (UNet and YNet) were used to automatically segment GA lesions on FAF; segmentation accuracy was compared with annotations by experienced graders. The training data set comprised 940 image pairs (FAF and NIR) from 183 patients in Proxima B; the test data set comprised 497 image pairs from 154 patients in Proxima A. Dice coefficient scores, Bland-Altman plots, and Pearson correlation coefficient (r) were used to assess performance. Results On the test set, Dice scores for the DL network to grader comparison ranged from 0.89 to 0.92 for screening visit; Dice score between graders was 0.94. GA lesion area correlations (r) for YNet versus grader, UNet versus grader, and between graders were 0.981, 0.959, and 0.995, respectively. Longitudinal GA lesion area enlargement correlations (r) for screening to 12 months (n = 53) were lower (0.741, 0.622, and 0.890, respectively) compared with the cross-sectional results at screening. Longitudinal correlations (r) from screening to 6 months (n = 77) were even lower (0.294, 0.248, and 0.686, respectively). Conclusions Multimodal DL networks to segment GA lesions can produce accurate results comparable with expert graders. Translational Relevance DL-based tools may support efficient and individualized assessment of patients with GA in clinical research and practice.
Collapse
Affiliation(s)
- Theodore Spaide
- Roche Personalized Healthcare, Genentech, Inc., South San Francisco, CA, USA
| | - Jiaxiang Jiang
- Clinical Imaging Group, Genentech, Inc., South San Francisco, CA, USA
- Department of Electrical and Computer Engineering, University of California, Santa Barbara, Santa Barbara, CA, USA
| | - Jasmine Patil
- Clinical Imaging Group, Genentech, Inc., South San Francisco, CA, USA
| | - Neha Anegondi
- Roche Personalized Healthcare, Genentech, Inc., South San Francisco, CA, USA
- Clinical Imaging Group, Genentech, Inc., South San Francisco, CA, USA
| | - Verena Steffen
- Roche Personalized Healthcare, Genentech, Inc., South San Francisco, CA, USA
- Biostatistics, Genentech, Inc., South San Francisco, CA, USA
| | | | - Elizabeth M Newton
- Roche Personalized Healthcare, Genentech, Inc., South San Francisco, CA, USA
| | - Christina Rabe
- Roche Personalized Healthcare, Genentech, Inc., South San Francisco, CA, USA
- Biostatistics, Genentech, Inc., South San Francisco, CA, USA
| | - Simon S Gao
- Roche Personalized Healthcare, Genentech, Inc., South San Francisco, CA, USA
- Clinical Imaging Group, Genentech, Inc., South San Francisco, CA, USA
| | - Aaron Y Lee
- Department of Ophthalmology, University of Washington, School of Medicine, Seattle, WA, USA
| | - Frank G Holz
- Department of Ophthalmology and GRADE Reading Center, University of Bonn, Bonn, Germany
| | - SriniVas Sadda
- Doheny Eye Institute, Los Angeles, CA, USA
- Department of Ophthalmology, David Geffen School of Medicine at University of California, Los Angeles, Los Angeles, CA, USA
| | - Steffen Schmitz-Valckenberg
- Department of Ophthalmology and GRADE Reading Center, University of Bonn, Bonn, Germany
- John A. Moran Eye Center, University of Utah, Salt Lake City, UT, USA
| | - Daniela Ferrara
- Roche Personalized Healthcare, Genentech, Inc., South San Francisco, CA, USA
| |
Collapse
|
3
|
Wei W, Anantharanjit R, Patel RP, Cordeiro MF. Detection of macular atrophy in age-related macular degeneration aided by artificial intelligence. Expert Rev Mol Diagn 2023:1-10. [PMID: 37144908 DOI: 10.1080/14737159.2023.2208751] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/06/2023]
Abstract
INTRODUCTION Age-related macular degeneration (AMD) is a leading cause of irreversible visual impairment worldwide. The endpoint of AMD, both in its dry or wet form, is macular atrophy (MA) which is characterized by the permanent loss of the RPE and overlying photoreceptors either in dry AMD or in wet AMD. A recognized unmet need in AMD is the early detection of MA development. AREAS COVERED Artificial Intelligence (AI) has demonstrated great impact in detection of retinal diseases, especially with its robust ability to analyze big data afforded by ophthalmic imaging modalities, such as color fundus photography (CFP), fundus autofluorescence (FAF), near-infrared reflectance (NIR), and optical coherence tomography (OCT). Among these, OCT has been shown to have great promise in identifying early MA using the new criteria in 2018. EXPERT OPINION There are few studies in which AI-OCT methods have been used to identify MA; however, results are very promising when compared to other imaging modalities. In this paper, we review the development and advances of ophthalmic imaging modalities and their combination with AI technology to detect MA in AMD. In addition, we emphasize the application of AI-OCT as an objective, cost-effective tool for the early detection and monitoring of the progression of MA in AMD.
Collapse
Affiliation(s)
- Wei Wei
- Department of Ophthalmology, Ningbo Medical Center Lihuili Hospital, Ningbo, China
- Department of Surgery & Cancer, Imperial College London, London, UK
- Imperial College Ophthalmology Research Group (ICORG), Imperial College Ophthalmology Research Group, London, UK
| | - Rajeevan Anantharanjit
- Imperial College Ophthalmology Research Group (ICORG), Imperial College Ophthalmology Research Group, London, UK
- Western Eye Hospital, Imperial College Healthcare NHS trust, London, UK
| | - Radhika Pooja Patel
- Imperial College Ophthalmology Research Group (ICORG), Imperial College Ophthalmology Research Group, London, UK
- Western Eye Hospital, Imperial College Healthcare NHS trust, London, UK
| | - Maria Francesca Cordeiro
- Department of Surgery & Cancer, Imperial College London, London, UK
- Imperial College Ophthalmology Research Group (ICORG), Imperial College Ophthalmology Research Group, London, UK
- Western Eye Hospital, Imperial College Healthcare NHS trust, London, UK
| |
Collapse
|
4
|
Wang N, Hu L, Walsh AJ. POSEA: A novel algorithm to evaluate the performance of multi-object instance image segmentation. PLoS One 2023; 18:e0283692. [PMID: 36989326 PMCID: PMC10057750 DOI: 10.1371/journal.pone.0283692] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Accepted: 03/14/2023] [Indexed: 03/30/2023] Open
Abstract
Many techniques and software packages have been developed to segment individual cells within microscopy images, necessitating a robust method to evaluate images segmented into a large number of unique objects. Currently, segmented images are often compared with ground-truth images at a pixel level; however, this standard pixel-level approach fails to compute errors due to pixels incorrectly assigned to adjacent objects. Here, we define a per-object segmentation evaluation algorithm (POSEA) that calculates segmentation accuracy metrics for each segmented object relative to a ground truth segmented image. To demonstrate the performance of POSEA, precision, recall, and f-measure metrics are computed and compared with the standard pixel-level evaluation for simulated images and segmented fluorescence microscopy images of three different cell samples. POSEA yields lower accuracy metrics than the standard pixel-level evaluation due to correct accounting of misclassified pixels of adjacent objects. Therefore, POSEA provides accurate evaluation metrics for objects with pixels incorrectly assigned to adjacent objects and is robust for use across a variety of applications that require evaluation of the segmentation of unique adjacent objects.
Collapse
Affiliation(s)
- Nianchao Wang
- Texas A&M University, TAMU, College Station, Texas, United States of America
| | - Linghao Hu
- Texas A&M University, TAMU, College Station, Texas, United States of America
| | - Alex J Walsh
- Texas A&M University, TAMU, College Station, Texas, United States of America
| |
Collapse
|
5
|
Automated segmentation and feature discovery of age-related macular degeneration and Stargardt disease via self-attended neural networks. Sci Rep 2022; 12:14565. [PMID: 36028647 PMCID: PMC9418226 DOI: 10.1038/s41598-022-18785-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Accepted: 08/18/2022] [Indexed: 11/09/2022] Open
Abstract
Age-related macular degeneration (AMD) and Stargardt disease are the leading causes of blindness for the elderly and young adults respectively. Geographic atrophy (GA) of AMD and Stargardt atrophy are their end-stage outcomes. Efficient methods for segmentation and quantification of these atrophic lesions are critical for clinical research. In this study, we developed a deep convolutional neural network (CNN) with a trainable self-attended mechanism for accurate GA and Stargardt atrophy segmentation. Compared with traditional post-hoc attention mechanisms which can only visualize CNN features, our self-attended mechanism is embedded in a fully convolutional network and directly involved in training the CNN to actively attend key features for enhanced algorithm performance. We applied the self-attended CNN on the segmentation of AMD and Stargardt atrophic lesions on fundus autofluorescence (FAF) images. Compared with a preexisting regular fully convolutional network (the U-Net), our self-attended CNN achieved 10.6% higher Dice coefficient and 17% higher IoU (intersection over union) for AMD GA segmentation, and a 22% higher Dice coefficient and a 32% higher IoU for Stargardt atrophy segmentation. With longitudinal image data having over a longer time, the developed self-attended mechanism can also be applied on the visual discovery of early AMD and Stargardt features.
Collapse
|
6
|
Rossant F, Paques M. Normalization of series of fundus images to monitor the geographic atrophy growth in dry age-related macular degeneration. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 208:106234. [PMID: 34229997 DOI: 10.1016/j.cmpb.2021.106234] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/28/2020] [Accepted: 06/05/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVE Age-related macular degeneration (ARMD) is a degenerative disease that affects the retina, and the leading cause of visual loss. In its dry form, the pathology is characterized by the progressive, centrifugal expansion of retinal lesions, called geographic atrophy (GA). In infrared eye fundus images, the GA appears as localized bright areas and its growth can be observed in series of images acquired at regular time intervals. However, illumination distortions between the images make impossible the direct comparison of intensities in order to study the GA progress. Here, we propose a new method to compensate for illumination distortion between images. METHODS We process all images of the series so that any two images have comparable gray levels. Our approach relies on an illumination/reflectance model. We first estimate the pixel-wise illumination ratio between any two images of the series, in a recursive way; then we correct each image against all the others, based on those estimates. The algorithm is applied on a sliding temporal window to cope with large changes in reflectance. We also propose morphological processing to suppress illumination artefacts. RESULTS The corrected illumination function is homogeneous in the series, enabling the direct comparison of grey-levels intensities in each pixel, and so the detection of the GA growth between any two images. To demonstrate that, we present numerous experiments performed on a dataset of 18 series (328 images), manually segmented by an ophthalmologist. First, we show that the normalization preprocessing dramatically increases the contrast of the GA growth areas. Secondly, we apply segmentation algorithms derived from Otsu's thresholding to detect automatically the GA total growth and the GA progress between consecutive images. We demonstrate qualitatively and quantitatively that these algorithms, although fully automatic, unsupervised and basic, already lead to interesting segmentation results when applied to the normalized images. Colored maps representing the GA evolution can be derived from the segmentations. CONCLUSION To our knowledge, the proposed method is the first one which corrects automatically and jointly the illumination inhomogeneity in a series of fundus images, regardless of the number of images, the size, shape and progression of lesion areas. This algorithm greatly facilitates the visual interpretation by the medical expert. It opens up the possibility of treating automatically each series as a whole (not just in pairs of images) to model the GA growth.
Collapse
Affiliation(s)
- Florence Rossant
- ISEP, Institut Supérieur d'Electronique de Paris, 10 rue de Vanves, 92130 Issy-les-Moulineaux, France.
| | - Michel Paques
- Clinical Investigation Center 1423, Quinze-Vingts Hospital, 28 rue de Charenton, 75012 Paris, France
| |
Collapse
|
7
|
Royer C, Sublime J, Rossant F, Paques M. Unsupervised Approaches for the Segmentation of Dry ARMD Lesions in Eye Fundus cSLO Images. J Imaging 2021; 7:143. [PMID: 34460779 PMCID: PMC8404939 DOI: 10.3390/jimaging7080143] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Revised: 08/09/2021] [Accepted: 08/10/2021] [Indexed: 11/17/2022] Open
Abstract
Age-related macular degeneration (ARMD), a major cause of sight impairment for elderly people, is still not well understood despite intensive research. Measuring the size of the lesions in the fundus is the main biomarker of the severity of the disease and as such is widely used in clinical trials yet only relies on manual segmentation. Artificial intelligence, in particular automatic image analysis based on neural networks, has a major role to play in better understanding the disease, by analyzing the intrinsic optical properties of dry ARMD lesions from patient images. In this paper, we propose a comparison of automatic segmentation methods (classical computer vision method, machine learning method and deep learning method) in an unsupervised context applied on cSLO IR images. Among the methods compared, we propose an adaptation of a fully convolutional network, called W-net, as an efficient method for the segmentation of ARMD lesions. Unlike supervised segmentation methods, our algorithm does not require annotated data which are very difficult to obtain in this application. Our method was tested on a dataset of 328 images and has shown to reach higher quality results than other compared unsupervised methods with a F1 score of 0.87, while having a more stable model, even though in some specific cases, texture/edges-based methods can produce relevant results.
Collapse
Affiliation(s)
- Clément Royer
- ISEP-School of Digital Engineers, 92130 Issy-Les-Moulineaux, France
| | - Jérémie Sublime
- ISEP-School of Digital Engineers, 92130 Issy-Les-Moulineaux, France
- LIPN-CNRS UMR 7030, LaMSN-Université Sorbonne Paris Nord, 93210 St Denis, France
| | - Florence Rossant
- ISEP-School of Digital Engineers, 92130 Issy-Les-Moulineaux, France
| | - Michel Paques
- Clinical Imaging Center 1423, Quinze-Vingts Hospital, INSERM-DGOS Clinical Investigation Center, 75012 Paris, France
| |
Collapse
|
8
|
Arslan J, Samarasinghe G, Sowmya A, Benke KK, Hodgson LAB, Guymer RH, Baird PN. Deep Learning Applied to Automated Segmentation of Geographic Atrophy in Fundus Autofluorescence Images. Transl Vis Sci Technol 2021; 10:2. [PMID: 34228106 PMCID: PMC8267211 DOI: 10.1167/tvst.10.8.2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2020] [Accepted: 05/23/2021] [Indexed: 11/02/2022] Open
Abstract
Purpose This study describes the development of a deep learning algorithm based on the U-Net architecture for automated segmentation of geographic atrophy (GA) lesions in fundus autofluorescence (FAF) images. Methods Image preprocessing and normalization by modified adaptive histogram equalization were used for image standardization to improve effectiveness of deep learning. A U-Net-based deep learning algorithm was developed and trained and tested by fivefold cross-validation using FAF images from clinical datasets. The following metrics were used for evaluating the performance for lesion segmentation in GA: dice similarity coefficient (DSC), DSC loss, sensitivity, specificity, mean absolute error (MAE), accuracy, recall, and precision. Results In total, 702 FAF images from 51 patients were analyzed. After fivefold cross-validation for lesion segmentation, the average training and validation scores were found for the most important metric, DSC (0.9874 and 0.9779), for accuracy (0.9912 and 0.9815), for sensitivity (0.9955 and 0.9928), and for specificity (0.8686 and 0.7261). Scores for testing were all similar to the validation scores. The algorithm segmented GA lesions six times more quickly than human performance. Conclusions The deep learning algorithm can be implemented using clinical data with a very high level of performance for lesion segmentation. Automation of diagnostics for GA assessment has the potential to provide savings with respect to patient visit duration, operational cost and measurement reliability in routine GA assessments. Translational Relevance A deep learning algorithm based on the U-Net architecture and image preprocessing appears to be suitable for automated segmentation of GA lesions on clinical data, producing fast and accurate results.
Collapse
Affiliation(s)
- Janan Arslan
- Centre for Eye Research Australia, University of Melbourne, Royal Victorian Eye & Ear Hospital, East Melbourne, Victoria, Australia
- Department of Surgery, Ophthalmology, University of Melbourne, Parkville, Victoria, Australia
| | - Gihan Samarasinghe
- School of Computer Science and Engineering, University of New South Wales, Kensington, New South Wales, Australia
| | - Arcot Sowmya
- School of Computer Science and Engineering, University of New South Wales, Kensington, New South Wales, Australia
| | - Kurt K. Benke
- School of Engineering, University of Melbourne, Parkville, Victoria, Australia
- Centre for AgriBioscience, AgriBio, Bundoora, Victoria, Australia
| | - Lauren A. B. Hodgson
- Centre for Eye Research Australia, University of Melbourne, Royal Victorian Eye & Ear Hospital, East Melbourne, Victoria, Australia
| | - Robyn H. Guymer
- Centre for Eye Research Australia, University of Melbourne, Royal Victorian Eye & Ear Hospital, East Melbourne, Victoria, Australia
- Department of Surgery, Ophthalmology, University of Melbourne, Parkville, Victoria, Australia
| | - Paul N. Baird
- Department of Surgery, Ophthalmology, University of Melbourne, Parkville, Victoria, Australia
| |
Collapse
|
9
|
Ocular Imaging for Enhancing the Understanding, Assessment, and Management of Age-Related Macular Degeneration. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2021; 1256:33-66. [PMID: 33847997 DOI: 10.1007/978-3-030-66014-7_2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Age-related macular degeneration (AMD) is a progressive neuro-retinal disease and the leading cause of central vision loss among elderly individuals in the developed countries. Modern ocular imaging technologies constitute an essential component of the evaluation of these patients and have contributed extensively to our understanding of the disease. A challenge with any review of ocular imaging technologies is the rapid pace of progress and evolution of these instruments. Nonetheless, for proper and optimal use of these technologies, it is essential for the user to understand the technical principles underlying the imaging modality and their role in assessing the disease in various settings. Indeed, AMD, like many other retinal diseases, benefits from a multimodal imaging approach to optimally characterize the disease. In this chapter, we will review the various imaging technologies currently used in the assessment and management of AMD.
Collapse
|
10
|
Miere A, Capuano V, Kessler A, Zambrowski O, Jung C, Colantuono D, Pallone C, Semoun O, Petit E, Souied E. Deep learning-based classification of retinal atrophy using fundus autofluorescence imaging. Comput Biol Med 2020; 130:104198. [PMID: 33383315 DOI: 10.1016/j.compbiomed.2020.104198] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2020] [Revised: 12/14/2020] [Accepted: 12/19/2020] [Indexed: 01/20/2023]
Abstract
PURPOSE To automatically classify retinal atrophy according to its etiology, using fundus autofluorescence (FAF) images, using a deep learning model. METHODS In this study, FAF images of patients with advanced dry age-related macular degeneration (AMD), also called geographic atrophy (GA), and genetically confirmed inherited retinal diseases (IRDs) in late atrophic stages [Stargardt disease (STGD1) and Pseudo-Stargardt Pattern Dystrophy (PSPD)] were included. The FAF images were used to train a multi-layer deep convolutional neural network (CNN) to differentiate on FAF between atrophy in the context of AMD (GA) and atrophy secondary to IRDs. Three-hundred fourteen FAF images were included, of which 110 images were of GA eyes and 204 were eyes with genetically confirmed STGD1 or PSPD. In the first approach, the CNN was trained and validated with 251 FAF images. Established augmentation techniques were used and an Adam optimizer was used for training. For the subsequent testing, the built classifiers were then tested with 63 untrained FAF images. The visualization method was integrated gradient visualization. In the second approach, 10-fold cross-validation was used to determine the model's performance. RESULTS In the first approach, the best performance of the model was obtained using 10 epochs, with an accuracy of 0.92 and an area under the curve for Receiver Operating Characteristic (AUC-ROC) of 0.981. Mean accuracy was 87.30 ± 2.96. In the second approach, a mean accuracy of 0.79 ± 0.06 was obtained. CONCLUSION This study describes the use of a deep learning-based algorithm to automatically classify atrophy on FAF imaging according to its etiology. Accurate differential diagnosis between GA and late-onset IRDs masquerading as GA on FAF can be performed with good accuracy and AUC-ROC values.
Collapse
Affiliation(s)
- Alexandra Miere
- Department of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, France; Laboratory of Images, Signals and Intelligent Systems (LISSI), (EA N° 3956), University Paris-Est, Créteil, France.
| | - Vittorio Capuano
- Department of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, France
| | | | - Olivia Zambrowski
- Department of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, France
| | - Camille Jung
- Clinical Research Center, Centre Hospitalier Intercommunal de Créteil, Créteil, France
| | - Donato Colantuono
- Department of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, France
| | - Carlotta Pallone
- Department of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, France
| | - Oudy Semoun
- Department of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, France
| | - Eric Petit
- Laboratory of Images, Signals and Intelligent Systems (LISSI), (EA N° 3956), University Paris-Est, Créteil, France
| | - Eric Souied
- Department of Ophthalmology, Centre Hospitalier Intercommunal de Créteil, Créteil, France
| |
Collapse
|
11
|
Arslan J, Samarasinghe G, Benke KK, Sowmya A, Wu Z, Guymer RH, Baird PN. Artificial Intelligence Algorithms for Analysis of Geographic Atrophy: A Review and Evaluation. Transl Vis Sci Technol 2020; 9:57. [PMID: 33173613 PMCID: PMC7594588 DOI: 10.1167/tvst.9.2.57] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2020] [Accepted: 09/28/2020] [Indexed: 12/28/2022] Open
Abstract
Purpose The purpose of this study was to summarize and evaluate artificial intelligence (AI) algorithms used in geographic atrophy (GA) diagnostic processes (e.g. isolating lesions or disease progression). Methods The search strategy and selection of publications were both conducted in accordance with the Preferred of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. PubMed and Web of Science were used to extract literary data. The algorithms were summarized by objective, performance, and scope of coverage of GA diagnosis (e.g. lesion automation and GA progression). Results Twenty-seven studies were identified for this review. A total of 18 publications focused on lesion segmentation only, 2 were designed to detect and classify GA, 2 were designed to predict future overall GA progression, 3 focused on prediction of future spatial GA progression, and 2 focused on prediction of visual function in GA. GA-related algorithms reported sensitivities from 0.47 to 0.98, specificities from 0.73 to 0.99, accuracies from 0.42 to 0.995, and Dice coefficients from 0.66 to 0.89. Conclusions Current GA-AI publications have a predominant focus on lesion segmentation and a minor focus on classification and progression analysis. AI could be applied to other facets of GA diagnoses, such as understanding the role of hyperfluorescent areas in GA. Using AI for GA has several advantages, including improved diagnostic accuracy and faster processing speeds. Translational Relevance AI can be used to quantify GA lesions and therefore allows one to impute visual function and quality-of-life. However, there is a need for the development of reliable and objective models and software to predict the rate of GA progression and to quantify improvements due to interventions.
Collapse
Affiliation(s)
- Janan Arslan
- Centre for Eye Research Australia, University of Melbourne, Royal Victorian Eye and Ear Hospital, East Melbourne, Victoria, Australia
- Department of Surgery, Ophthalmology, University of Melbourne, Victoria, Australia
| | - Gihan Samarasinghe
- School of Computer Science and Engineering, University of New South Wales, Kensington, New South Wales, Australia
| | - Kurt K. Benke
- School of Engineering, University of Melbourne, Parkville, Victoria, Australia
- Centre for AgriBioscience, AgriBio, Bundoora, Victoria, Australia
| | - Arcot Sowmya
- School of Computer Science and Engineering, University of New South Wales, Kensington, New South Wales, Australia
| | - Zhichao Wu
- Centre for Eye Research Australia, University of Melbourne, Royal Victorian Eye and Ear Hospital, East Melbourne, Victoria, Australia
| | - Robyn H. Guymer
- Centre for Eye Research Australia, University of Melbourne, Royal Victorian Eye and Ear Hospital, East Melbourne, Victoria, Australia
- Department of Surgery, Ophthalmology, University of Melbourne, Victoria, Australia
| | - Paul N. Baird
- Department of Surgery, Ophthalmology, University of Melbourne, Victoria, Australia
| |
Collapse
|
12
|
Quantitative Assessment of the Severity of Diabetic Retinopathy. Am J Ophthalmol 2020; 218:342-352. [PMID: 32446737 DOI: 10.1016/j.ajo.2020.05.021] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2019] [Revised: 05/13/2020] [Accepted: 05/14/2020] [Indexed: 12/28/2022]
Abstract
PURPOSE To determine whether a quantitative approach to assessment of the severity of diabetic retinopathy (DR) lesions on ultrawide field (UWF) images can provide new parameters to predict progression to proliferative diabetic retinopathy (PDR). METHODS One hundred forty six eyes from 73 participants with DR and 4 years of follow-up data were included in this post hoc analysis, which was based on a cohort of 100 diabetic patients enrolled in a previously published prospective, comparative study of UWF imaging at the Joslin Diabetes Center. Diabetic Retinopathy Severity Score level was determined at baseline and 4-year follow-up visits using mydriatic 7-standard field Early Treatment Diabetic Retinopathy Study (ETDRS) photographs. All individual DR lesions (hemorrhage [H], microaneurysm [ma], cotton wool spot [CWS], intraretinal microvascular abnormality [IRMA]) were manually segmented on stereographic projected UWF. For each lesion type, the frequency/number, surface area, and distances from the optic nerve head (ONH) were computed. These quantitative parameters were compared between eyes that progressed to PDR in 4 years and eyes that did not progress. Univariable and multivariable logistic regression analyses were performed to identify parameters that were associated with an increased risk for progression to PDR. RESULTS A total of 146 eyes of 73 subjects were included in the final analysis. The mean age of the study cohort was 53.1 years, and 42 (56.8%) subjects were female. The number and surface area of H/ma's and CWSs were significantly (P ≤ .05) higher in eyes that progressed to PDR compared with eyes that did not progress by 4 years. Similarly, H/ma's and CWSs were located further away from the ONH (ie, more peripheral) in eyes that progressed (P < .05). DR lesion parameters that conferred a statistically significant increased risk for proliferative diabetic retinopathy in the multivariate model included hemorrhage area (odds ratio [OR], 2.63; 95% confidence interval [CI], 1.25-5.53), and greater distance of hemorrhages from the ONH (OR, 1.24; 95% CI, 0.97-1.59). CONCLUSIONS Quantitative analysis of DR lesions on UWF images identifies new risk parameters for progression to PDR including the surface area of hemorrhages and the distance of hemorrhages from the ONH. Although these risk factors will need to be confirmed in larger, prospective studies, they highlight the potential for quantitative lesion analysis to inform the design of a more precise and complete staging system for diabetic retinopathy severity in the future. NOTE: Publication of this article is sponsored by the American Ophthalmological Society.
Collapse
|
13
|
Ometto G, Montesano G, Sadeghi Afgeh S, Lazaridis G, Liu X, Keane PA, Crabb DP, Denniston AK. Merging Information From Infrared and Autofluorescence Fundus Images for Monitoring of Chorioretinal Atrophic Lesions. Transl Vis Sci Technol 2020; 9:38. [PMID: 32908801 PMCID: PMC7453042 DOI: 10.1167/tvst.9.9.38] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Accepted: 08/06/2020] [Indexed: 11/24/2022] Open
Abstract
Purpose To develop a method for automated detection and progression analysis of chorioretinal atrophic lesions using the combined information of standard infrared (IR) and autofluorescence (AF) fundus images. Methods Eighteen eyes (from 16 subjects) with punctate inner choroidopathy were analyzed. Macular IR and blue AF images were acquired in all eyes with a Spectralis HRA+OCT device (Heidelberg Engineering, Heidelberg, Germany). Two clinical experts manually segmented chorioretinal lesions on the AF image. AF images were aligned to the corresponding IR. Two random forest models were trained to classify pixels of lesions, one based on the AF image only, the other based on the aligned IR-AF. The models were validated using a leave-one-out cross-validation and were tested against the manual segmentation to compare their performance. A time series from one eye was identified and used to evaluate the method based on the IR-AF in a case study. Results The method based on the AF images correctly classified 95% of the pixels (i.e., in vs. out of the lesion) with a Dice's coefficient of 0.80. The method based on the combined IR-AF correctly classified 96% of the pixels with a Dice's coefficient of 0.84. Conclusions The automated segmentation of chorioretinal lesions using IR and AF shows closer alignment to manual segmentation than the same method based on AF only. Merging information from multimodal images improves the automatic and objective segmentation of chorioretinal lesions even when based on a small dataset. Translational Relevance Merged information from multimodal images improves segmentation performance of chorioretinal lesions.
Collapse
Affiliation(s)
- Giovanni Ometto
- Division of Optometry and Visual Sciences, School of Health Sciences, City, University of London, London, UK.,Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Giovanni Montesano
- Division of Optometry and Visual Sciences, School of Health Sciences, City, University of London, London, UK.,Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | | | - Georgios Lazaridis
- Moorfields Eye Hospital NHS Foundation Trust, London, UK.,Centre for Medical Image Computing, University College London, London, UK
| | - Xiaoxuan Liu
- Department of Ophthalmology, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.,Academic Unit of Ophthalmology, Institute of Inflammation & Ageing, College of Medical and Dental Sciences, University of Birmingham, Edgbaston, Birmingham, UK.,Health Data Research UK, London, UK
| | - Pearse A Keane
- Moorfields Eye Hospital NHS Foundation Trust, London, UK.,Health Data Research UK, London, UK.,NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, UK
| | - David P Crabb
- Division of Optometry and Visual Sciences, School of Health Sciences, City, University of London, London, UK
| | - Alastair K Denniston
- Department of Ophthalmology, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.,Academic Unit of Ophthalmology, Institute of Inflammation & Ageing, College of Medical and Dental Sciences, University of Birmingham, Edgbaston, Birmingham, UK.,Health Data Research UK, London, UK.,NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, UK
| |
Collapse
|
14
|
Dupont G, Kalinicheva E, Sublime J, Rossant F, Pâques M. Analyzing Age-Related Macular Degeneration Progression in Patients with Geographic Atrophy Using Joint Autoencoders for Unsupervised Change Detection. J Imaging 2020; 6:57. [PMID: 34460650 PMCID: PMC8321155 DOI: 10.3390/jimaging6070057] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2020] [Revised: 06/20/2020] [Accepted: 06/23/2020] [Indexed: 11/16/2022] Open
Abstract
Age-Related Macular Degeneration (ARMD) is a progressive eye disease that slowly causes patients to go blind. For several years now, it has been an important research field to try to understand how the disease progresses and find effective medical treatments. Researchers have been mostly interested in studying the evolution of the lesions using different techniques ranging from manual annotation to mathematical models of the disease. However, artificial intelligence for ARMD image analysis has become one of the main research focuses to study the progression of the disease, as accurate manual annotation of its evolution has proved difficult using traditional methods even for experienced practicians. In this paper, we propose a deep learning architecture that can detect changes in the eye fundus images and assess the progression of the disease. Our method is based on joint autoencoders and is fully unsupervised. Our algorithm has been applied to pairs of images from different eye fundus images time series of 24 ARMD patients. Our method has been shown to be quite effective when compared with other methods from the literature, including non-neural network based algorithms that still are the current standard to follow the disease progression and change detection methods from other fields.
Collapse
Affiliation(s)
| | | | - Jérémie Sublime
- ISEP, DaSSIP Team, 92130 Issy-Les-Moulineaux, France
- Université Paris 13, LIPN - CNRS UMR 7030, 93430 Villetaneuse, France
| | | | - Michel Pâques
- Clinical Imaging Center 1423, Quinze-Vingts Hospital, INSERM-DGOS Clinical Investigation Center, 75012 Paris, France
| |
Collapse
|
15
|
Caetano Dos Santos FL, Laforest T, Künzi M, Kowalczuk L, Behar-Cohen F, Moser C. Fully automated detection, segmentation, and analysis of in vivo RPE single cells. Eye (Lond) 2020; 35:1473-1481. [PMID: 32555522 DOI: 10.1038/s41433-020-1036-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2020] [Accepted: 06/09/2020] [Indexed: 11/09/2022] Open
Abstract
OBJECTIVE To develop a fully automated method of retinal pigmented epithelium (RPE) cells detection, segmentation and analysis based on in vivo cellular resolution images obtained with the transscleral optical phase imaging method (TOPI). METHODS Fourteen TOPI-RPE images from 11 healthy individuals were analysed. The developed image processing method encompassed image filtering and normalisation, detection and removal of blood vessels, cell detection and cell membrane segmentation. The produced measures were cellular density of RPE layer, cell area, number of neighbouring cells, eccentricity, circularity and solidity. In addition, we proposed coefficient of variation (CV) of RPE cellular membrane (CMDCV) and the solidity of the RPE cell membrane-shape as new metrics for the assessment of RPE single cells. RESULTS The observed median cellular density of the RPE layer was 3743 cells/µm2 (interquartile rate (IQR) 1687), with a median observed RPE cell area of 193 µm2 (IQR 141). The mean number of neighbouring cells was 5.22 (standard deviation (SD) 0.05) per RPE cell. The mean RPE cell eccentricity was 0.67 (SD 0.02), median circularity 0.83 (IQR 0.01), and median solidity 0.92 (IQR 0.00). The median CMDCV was 0.19 (IQR 0.02). The method is characterised by a median image processing and analysis time of 48 sec (IQR 12) per image. CONCLUSIONS The present study provides the first fully automated quantitative assessment of human RPE single cells in vivo. The method provides a baseline for future research in the field of clinical ophthalmology, enabling characterisation and diagnostics of retinal diseases at the single-cell level.
Collapse
Affiliation(s)
| | - Timothé Laforest
- Laboratory of Applied Photonic Devices (LAPD), School of Engineering, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | | | - Laura Kowalczuk
- Laboratory of Applied Photonic Devices (LAPD), School of Engineering, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.,Department of Ophthalmology, Jules-Gonin Eye Hospital, Fondation Asile des aveugles, University of Lausanne, Lausanne, Switzerland
| | - Francine Behar-Cohen
- INSERM UMR_S 1138, Team 17, Centre de Recherche des Cordeliers, University of Pierre et Marie Curie, Paris Descartes University, Sorbonne Paris Cité, Paris, France.,Department of Ophthalmology, Ophthalmopole, Cochin Hospital, Assistance Publique, Hôpitaux de Paris, Paris, France
| | - Christophe Moser
- Laboratory of Applied Photonic Devices (LAPD), School of Engineering, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| |
Collapse
|
16
|
Hu Z, Wang Z, Sadda SR. Automated choroidal segmentation in spectral optical coherence tomography images with geographic atrophy using multimodal complementary information. J Med Imaging (Bellingham) 2019. [DOI: 10.1117/1.jmi.6.2.024009] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Affiliation(s)
- Zhihong Hu
- University of California, Doheny Eye Institute, Los Angeles, California
| | - Ziyuan Wang
- University of California, Doheny Eye Institute, Los Angeles, California
| | - Srinivas R. Sadda
- University of California, Doheny Eye Institute, Los Angeles, California
| |
Collapse
|
17
|
Ji Z, Chen Q, Niu S, Leng T, Rubin DL. Beyond Retinal Layers: A Deep Voting Model for Automated Geographic Atrophy Segmentation in SD-OCT Images. Transl Vis Sci Technol 2018; 7:1. [PMID: 29302382 PMCID: PMC5749649 DOI: 10.1167/tvst.7.1.1] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2017] [Accepted: 11/01/2017] [Indexed: 01/12/2023] Open
Abstract
PURPOSE To automatically and accurately segment geographic atrophy (GA) in spectral-domain optical coherence tomography (SD-OCT) images by constructing a voting system with deep neural networks without the use of retinal layer segmentation. METHODS An automatic GA segmentation method for SD-OCT images based on the deep network was constructed. The structure of the deep network was composed of five layers, including one input layer, three hidden layers, and one output layer. During the training phase, the labeled A-scans with 1024 features were directly fed into the network as the input layer to obtain the deep representations. Then a soft-max classifier was trained to determine the label of each individual pixel. Finally, a voting decision strategy was used to refine the segmentation results among 10 trained models. RESULTS Two image data sets with GA were used to evaluate the model. For the first dataset, our algorithm obtained a mean overlap ratio (OR) 86.94% ± 8.75%, absolute area difference (AAD) 11.49% ± 11.50%, and correlation coefficients (CC) 0.9857; for the second dataset, the mean OR, AAD, and CC of the proposed method were 81.66% ± 10.93%, 8.30% ± 9.09%, and 0.9952, respectively. The proposed algorithm was capable of improving over 5% and 10% segmentation accuracy, respectively, when compared with several state-of-the-art algorithms on two data sets. CONCLUSIONS Without retinal layer segmentation, the proposed algorithm could produce higher segmentation accuracy and was more stable when compared with state-of-the-art methods that relied on retinal layer segmentation results. Our model may provide reliable GA segmentations from SD-OCT images and be useful in the clinical diagnosis of advanced nonexudative AMD. TRANSLATIONAL RELEVANCE Based on the deep neural networks, this study presents an accurate GA segmentation method for SD-OCT images without using any retinal layer segmentation results, and may contribute to improved understanding of advanced nonexudative AMD.
Collapse
Affiliation(s)
- Zexuan Ji
- School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
| | - Qiang Chen
- School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
| | - Sijie Niu
- School of Information Science and Engineering, University of Jinan, Jinan, China
| | - Theodore Leng
- Byers Eye Institute at Stanford, Stanford University School of Medicine, Palo Alto, CA, USA
| | - Daniel L Rubin
- Department of Radiology, Stanford University, Stanford, CA, USA
- Medicine (Biomedical Informatics Research), Stanford University, Stanford, CA, USA
| |
Collapse
|
18
|
|
19
|
Feeny AK, Tadarati M, Freund DE, Bressler NM, Burlina P. Automated segmentation of geographic atrophy of the retinal epithelium via random forests in AREDS color fundus images. Comput Biol Med 2015; 65:124-36. [PMID: 26318113 DOI: 10.1016/j.compbiomed.2015.06.018] [Citation(s) in RCA: 35] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2015] [Revised: 06/19/2015] [Accepted: 06/20/2015] [Indexed: 10/23/2022]
Abstract
BACKGROUND Age-related macular degeneration (AMD), left untreated, is the leading cause of vision loss in people older than 55. Severe central vision loss occurs in the advanced stage of the disease, characterized by either the in growth of choroidal neovascularization (CNV), termed the "wet" form, or by geographic atrophy (GA) of the retinal pigment epithelium (RPE) involving the center of the macula, termed the "dry" form. Tracking the change in GA area over time is important since it allows for the characterization of the effectiveness of GA treatments. Tracking GA evolution can be achieved by physicians performing manual delineation of GA area on retinal fundus images. However, manual GA delineation is time-consuming and subject to inter-and intra-observer variability. METHODS We have developed a fully automated GA segmentation algorithm in color fundus images that uses a supervised machine learning approach employing a random forest classifier. This algorithm is developed and tested using a dataset of images from the NIH-sponsored Age Related Eye Disease Study (AREDS). GA segmentation output was compared against a manual delineation by a retina specialist. RESULTS Using 143 color fundus images from 55 different patient eyes, our algorithm achieved PPV of 0.82±0.19, and NPV of 0:95±0.07. DISCUSSION This is the first study, to our knowledge, applying machine learning methods to GA segmentation on color fundus images and using AREDS imagery for testing. These preliminary results show promising evidence that machine learning methods may have utility in automated characterization of GA from color fundus images.
Collapse
Affiliation(s)
- Albert K Feeny
- Applied Physics Laboratory, The Johns Hopkins University, MD, USA; Department of Biomedical Engineering, The Johns Hopkins University, MD, USA
| | - Mongkol Tadarati
- Retina Division, Wilmer Eye Institute, The Johns Hopkins University, MD, USA; Rajavithi Hospital, College of Medicine, Rangsit University, Bangkok, Thailand
| | - David E Freund
- Applied Physics Laboratory, The Johns Hopkins University, MD, USA
| | - Neil M Bressler
- Retina Division, Wilmer Eye Institute, The Johns Hopkins University, MD, USA
| | - Philippe Burlina
- Applied Physics Laboratory, The Johns Hopkins University, MD, USA; Retina Division, Wilmer Eye Institute, The Johns Hopkins University, MD, USA; Department of Computer Science, The Johns Hopkins University, MD, USA
| |
Collapse
|