Multi-granularity learning of explicit geometric constraint and contrast for label-efficient medical image segmentation and differentiable clinical function assessment

Automated segmentation is a challenging task in medical image analysis that usually requires a large amount of manually labeled data. However, most current supervised learning based algorithms suffer from insufficient manual annotations, posing a significant difficulty for accurate and robust segmentation. In addition, most current semi-supervised methods lack explicit representations of geometric structure and semantic information, restricting segmentation accuracy. In this work, we propose a hybrid framework to learn polygon vertices, region masks, and their boundaries in a weakly/semi-supervised manner that significantly advances geometric and semantic representations. Firstly, we propose multi-granularity learning of explicit geometric structure constraints via polygon vertices (PolyV) and pixel-wise region (PixelR) segmentation masks in a semi-supervised manner. Secondly, we propose eliminating boundary ambiguity by using an explicit contrastive objective to learn a discriminative feature space of boundary contours at the pixel level with limited annotations. Thirdly, we exploit the task-specific clinical domain knowledge to differentiate the clinical function assessment end-to-end. The ground truth of clinical function assessment, on the other hand, can serve as auxiliary weak supervision for PolyV and PixelR learning. We evaluate the proposed framework on two tasks, including optic disc (OD) and cup (OC) segmentation along with vertical cup-to-disc ratio (vCDR) estimation in fundus images; left ventricle (LV) segmentation at end-diastolic and end-systolic frames along with ejection fraction (LVEF) estimation in two-dimensional echocardiography images. Experiments on nine large-scale datasets of the two tasks under different label settings demonstrate our model's superior performance on segmentation and clinical function assessment.

Development and validation of a semi-supervised deep learning model for automatic retinopathy of prematurity staging

Retinopathy of prematurity (ROP) is currently one of the leading causes of infant blindness worldwide. Recently significant progress has been made in deep learning-based computer-aided diagnostic methods. However, deep learning often requires a large amount of annotated data for model optimization, but this requires long hours of effort by experienced doctors in clinical scenarios. In contrast, a large number of unlabeled images are relatively easy to obtain. In this paper, we propose a new semi-supervised learning framework to reduce annotation costs for automatic ROP staging. We design two consistency regularization strategies, prediction consistency loss and semantic structure consistency loss, which can help the model mine useful discriminative information from unlabeled data, thus improving the generalization performance of the classification model. Extensive experiments on a real clinical dataset show that the proposed method promises to greatly reduce the labeling requirements in clinical scenarios while achieving good classification performance.

Artificial intelligence in glaucoma: opportunities, challenges, and future directions

Artificial intelligence (AI) has shown excellent diagnostic performance in detecting various complex problems related to many areas of healthcare including ophthalmology. AI diagnostic systems developed from fundus images have become state-of-the-art tools in diagnosing retinal conditions and glaucoma as well as other ocular diseases. However, designing and implementing AI models using large imaging data is challenging. In this study, we review different machine learning (ML) and deep learning (DL) techniques applied to multiple modalities of retinal data, such as fundus images and visual fields for glaucoma detection, progression assessment, staging and so on. We summarize findings and provide several taxonomies to help the reader understand the evolution of conventional and emerging AI models in glaucoma. We discuss opportunities and challenges facing AI application in glaucoma and highlight some key themes from the existing literature that may help to explore future studies. Our goal in this systematic review is to help readers and researchers to understand critical aspects of AI related to glaucoma as well as determine the necessary steps and requirements for the successful development of AI models in glaucoma.

Deep Learning Application to Detect Glaucoma with a Mixed Training Approach: Public Database and Expert-Labeled Glaucoma Population

INTRODUCTION

Artificial intelligence has real potential for early identification of ocular diseases such as glaucoma. An important challenge is the requirement for large databases properly selected, which are not easily obtained. We used a relatively original strategy: a glaucoma recognition algorithm trained with fundus images from public databases and then tested and retrained with a carefully selected patient database.

METHODS

The study's supervised deep learning method was an adapted version of the ResNet-50 architecture previously trained from 10,658 optic head images (glaucomatous or non-glaucomatous) from seven public databases. A total of 1,158 new images labeled by experts from 616 patients were added. The images were categorized after clinical examination including visual fields in 304 (26%) control images or those with ocular hypertension and 347 (30%) images with early, 290 (25%) with moderate, and 217 (19%) with advanced glaucoma. The initial algorithm was tested using 30% of the selected glaucoma database and then re-trained with 70% of this database and tested again.

RESULTS

The results in the initial sample showed an area under the curve (AUC) of 76% for all images, and 66% for early, 82% for moderate, and 84% for advanced glaucoma. After retraining the algorithm, the respective AUC results were 82%, 72%, 89%, and 91%.

CONCLUSION

Using combined data from public databases and data selected and labeled by experts facilitated improvement of the system's precision and identified interesting possibilities for obtaining tools for automatic screening of glaucomatous eyes more affordably.

A transfer learning-based multimodal neural network combining metadata and multiple medical images for glaucoma type diagnosis

Glaucoma is an acquired optic neuropathy, which can lead to irreversible vision loss. Deep learning(DL), especially convolutional neural networks(CNN), has achieved considerable success in the field of medical image recognition due to the availability of large-scale annotated datasets and CNNs. However, obtaining fully annotated datasets like ImageNet in the medical field is still a challenge. Meanwhile, single-modal approaches remain both unreliable and inaccurate due to the diversity of glaucoma disease types and the complexity of symptoms. In this paper, a new multimodal dataset for glaucoma is constructed and a new multimodal neural network for glaucoma diagnosis and classification (GMNNnet) is proposed aiming to address both of these issues. Specifically, the dataset includes the five most important types of glaucoma labels, electronic medical records and four kinds of high-resolution medical images. The structure of GMNNnet consists of three branches. Branch 1 consisting of convolutional, cyclic and transposition layers processes patient metadata, branch 2 uses Unet to extract features from glaucoma segmentation based on domain knowledge, and branch 3 uses ResFormer to directly process glaucoma medical images.Branch one and branch two are mixed together and then processed by the Catboost classifier. We introduce a gradient-weighted class activation mapping (Grad-GAM) method to increase the interpretability of the model and a transfer learning method for the case of insufficient training data,i.e.,fine-tuning CNN models pre-trained from natural image dataset to medical image tasks. The results show that GMNNnet can better present the high-dimensional information of glaucoma and achieves excellent performance under multimodal data.

The Definition of Glaucomatous Optic Neuropathy in Artificial Intelligence Research and Clinical Applications

OBJECTIVE

Although artificial intelligence (AI) models may offer innovative and powerful ways to use the wealth of data generated by diagnostic tools, there are important challenges related to their development and validation. Most notable is the lack of a perfect reference standard for glaucomatous optic neuropathy (GON). Because AI models are trained to predict presence of glaucoma or its progression, they generally rely on a reference standard that is used to train the model and assess its validity. If an improper reference standard is used, the model may be trained to detect or predict something that has little or no clinical value. This article summarizes the issues and discussions related to the definition of GON in AI applications as presented by the Glaucoma Workgroup from the Collaborative Community for Ophthalmic Imaging (CCOI) US Food and Drug Administration Virtual Workshop, on September 3 and 4, 2020, and on January 28, 2022.

DESIGN

Review and conference proceedings.

SUBJECTS

No human or animal subjects or data therefrom were used in the production of this article.

METHODS

A summary of the Workshop was produced with input and approval from all participants.

MAIN OUTCOME MEASURES

Consensus position of the CCOI Workgroup on the challenges in defining GON and possible solutions.

RESULTS

The Workshop reviewed existing challenges that arise from the use of subjective definitions of GON and highlighted the need for a more objective approach to characterize GON that could facilitate replication and comparability of AI studies and allow for better clinical validation of proposed AI tools. Different tests and combination of parameters for defining a reference standard for GON have been proposed. Different reference standards may need to be considered depending on the scenario in which the AI models are going to be applied, such as community-based or opportunistic screening versus detection or monitoring of glaucoma in tertiary care.

CONCLUSIONS

The development and validation of new AI-based diagnostic tests should be based on rigorous methodology with clear determination of how the reference standards for glaucomatous damage are constructed and the settings where the tests are going to be applied.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found after the references.

Automated Glaucoma Screening and Diagnosis Based on Retinal Fundus Images Using Deep Learning Approaches: A Comprehensive Review

Glaucoma is a chronic eye disease that may lead to permanent vision loss if it is not diagnosed and treated at an early stage. The disease originates from an irregular behavior in the drainage flow of the eye that eventually leads to an increase in intraocular pressure, which in the severe stage of the disease deteriorates the optic nerve head and leads to vision loss. Medical follow-ups to observe the retinal area are needed periodically by ophthalmologists, who require an extensive degree of skill and experience to interpret the results appropriately. To improve on this issue, algorithms based on deep learning techniques have been designed to screen and diagnose glaucoma based on retinal fundus image input and to analyze images of the optic nerve and retinal structures. Therefore, the objective of this paper is to provide a systematic analysis of 52 state-of-the-art relevant studies on the screening and diagnosis of glaucoma, which include a particular dataset used in the development of the algorithms, performance metrics, and modalities employed in each article. Furthermore, this review analyzes and evaluates the used methods and compares their strengths and weaknesses in an organized manner. It also explored a wide range of diagnostic procedures, such as image pre-processing, localization, classification, and segmentation. In conclusion, automated glaucoma diagnosis has shown considerable promise when deep learning algorithms are applied. Such algorithms could increase the accuracy and efficiency of glaucoma diagnosis in a better and faster manner.

Artificial intelligence in glaucoma: posterior segment optical coherence tomography

PURPOSE OF REVIEW

To summarize the recent literature on deep learning (DL) model applications in glaucoma detection and surveillance using posterior segment optical coherence tomography (OCT) imaging.

RECENT FINDINGS

DL models use OCT derived parameters including retinal nerve fiber layer (RNFL) scans, macular scans, and optic nerve head (ONH) scans, as well as a combination of these parameters, to achieve high diagnostic accuracy in detecting glaucomatous optic neuropathy (GON). Although RNFL segmentation is the most widely used OCT parameter for glaucoma detection by ophthalmologists, newer DL models most commonly use a combination of parameters, which provide a more comprehensive approach. Compared to DL models for diagnosing glaucoma, DL models predicting glaucoma progression are less commonly studied but have also been developed.

SUMMARY

DL models offer time-efficient, objective, and potential options in the management of glaucoma. Although artificial intelligence models have already been commercially accepted as diagnostic tools for other ophthalmic diseases, there is no commercially approved DL tool for the diagnosis of glaucoma, most likely in part due to the lack of a universal definition of glaucoma defined by OCT derived parameters alone (see Supplemental Digital Content 1 for video abstract, http://links.lww.com/COOP/A54 ).

Joint optic disk and cup segmentation for glaucoma screening using a region-based deep learning network

OBJECTIVES

To develop and validate an end-to-end region-based deep convolutional neural network (R-DCNN) to jointly segment the optic disc (OD) and optic cup (OC) in retinal fundus images for precise cup-to-disc ratio (CDR) measurement and glaucoma screening.

METHODS

In total, 2440 retinal fundus images were retrospectively obtained from 2033 participants. An R-DCNN was presented for joint OD and OC segmentation, where the OD and OC segmentation problems were formulated into object detection problems. We compared R-DCNN's segmentation performance on our in-house dataset with that of four ophthalmologists while performing quantitative, qualitative and generalization analyses on the publicly available both DRISHIT-GS and RIM-ONE v3 datasets. The Dice similarity coefficient (DC), Jaccard coefficient (JC), overlapping error (E), sensitivity (SE), specificity (SP) and area under the curve (AUC) were measured.

RESULTS

On our in-house dataset, the proposed model achieved a 98.51% DC and a 97.07% JC for OD segmentation, and a 97.63% DC and a 95.39% JC for OC segmentation, achieving a performance level comparable to that of the ophthalmologists. On the DRISHTI-GS dataset, our approach achieved 97.23% and 94.17% results in DC and JC results for OD segmentation, respectively, while it achieved a 94.56% DC and an 89.92% JC for OC segmentation. Additionally, on the RIM-ONE v3 dataset, our model generated DC and JC values of 96.89% and 91.32% on the OD segmentation task, respectively, whereas the DC and JC values acquired for OC segmentation were 88.94% and 78.21%, respectively.

CONCLUSION

The proposed approach achieved very encouraging performance on the OD and OC segmentation tasks, as well as in glaucoma screening. It has the potential to serve as a useful tool for computer-assisted glaucoma screening.

Precision Medicine in Glaucoma: Artificial Intelligence, Biomarkers, Genetics and Redox State

Glaucoma is a multifactorial neurodegenerative illness requiring early diagnosis and strict monitoring of the disease progression. Current exams for diagnosis and prognosis are based on clinical examination, intraocular pressure (IOP) measurements, visual field tests, and optical coherence tomography (OCT). In this scenario, there is a critical unmet demand for glaucoma-related biomarkers to enhance clinical testing for early diagnosis and tracking of the disease's development. The introduction of validated biomarkers would allow for prompt intervention in the clinic to help with prognosis prediction and treatment response monitoring. This review aims to report the latest acquisitions on biomarkers in glaucoma, from imaging analysis to genetics and metabolic markers.

Dual Consistency Enabled Weakly and Semi-Supervised Optic Disc and Cup Segmentation With Dual Adaptive Graph Convolutional Networks

Glaucoma is a progressive eye disease that results in permanent vision loss, and the vertical cup to disc ratio (vCDR) in colour fundus images is essential in glaucoma screening and assessment. Previous fully supervised convolution neural networks segment the optic disc (OD) and optic cup (OC) from color fundus images and then calculate the vCDR offline. However, they rely on a large set of labeled masks for training, which is expensive and time-consuming to acquire. To address this, we propose a weakly and semi-supervised graph-based network that investigates geometric associations and domain knowledge between segmentation probability maps (PM), modified signed distance function representations (mSDF), and boundary region of interest characteristics (B-ROI) in three aspects. Firstly, we propose a novel Dual Adaptive Graph Convolutional Network (DAGCN) to reason the long-range features of the PM and the mSDF w.r.t. the regional uniformity. Secondly, we propose a dual consistency regularization-based semi-supervised learning paradigm. The regional consistency between the PM and the mSDF, and the marginal consistency between the derived B-ROI from each of them boost the proposed model's performance due to the inherent geometric associations. Thirdly, we exploit the task-specific domain knowledge via the oval shapes of OD & OC, where a differentiable vCDR estimating layer is proposed. Furthermore, without additional annotations, the supervision on vCDR serves as weakly-supervisions for segmentation tasks. Experiments on six large-scale datasets demonstrate our model's superior performance on OD & OC segmentation and vCDR estimation. The implementation code has been made available.https://github.com/smallmax00/Dual_Adaptive_Graph_Reasoning.

An empirical study of preprocessing techniques with convolutional neural networks for accurate detection of chronic ocular diseases using fundus images

Chronic Ocular Diseases (COD) such as myopia, diabetic retinopathy, age-related macular degeneration, glaucoma, and cataract can affect the eye and may even lead to severe vision impairment or blindness. According to a recent World Health Organization (WHO) report on vision, at least 2.2 billion individuals worldwide suffer from vision impairment. Often, overt signs indicative of COD do not manifest until the disease has progressed to an advanced stage. However, if COD is detected early, vision impairment can be avoided by early intervention and cost-effective treatment. Ophthalmologists are trained to detect COD by examining certain minute changes in the retina, such as microaneurysms, macular edema, hemorrhages, and alterations in the blood vessels. The range of eye conditions is diverse, and each of these conditions requires a unique patient-specific treatment. Convolutional neural networks (CNNs) have demonstrated significant potential in multi-disciplinary fields, including the detection of a variety of eye diseases. In this study, we combined several preprocessing approaches with convolutional neural networks to accurately detect COD in eye fundus images. To the best of our knowledge, this is the first work that provides a qualitative analysis of preprocessing approaches for COD classification using CNN models. Experimental results demonstrate that CNNs trained on the region of interest segmented images outperform the models trained on the original input images by a substantial margin. Additionally, an ensemble of three preprocessing techniques outperformed other state-of-the-art approaches by 30% and 3%, in terms of Kappa and F ₁ scores, respectively. The developed prototype has been extensively tested and can be evaluated on more comprehensive COD datasets for deployment in the clinical setup.

Artificial intelligence and deep learning in ophthalmology: Current status and future perspectives

Background

The ophthalmology field was among the first to adopt artificial intelligence (AI) in medicine. The availability of digitized ocular images and substantial data have made deep learning (DL) a popular topic.

Main text

At the moment, AI in ophthalmology is mostly used to improve disease diagnosis and assist decision-making aiming at ophthalmic diseases like diabetic retinopathy (DR), glaucoma, age-related macular degeneration (AMD), cataract and other anterior segment diseases. However, most of the AI systems developed to date are still in the experimental stages, with only a few having achieved clinical applications. There are a number of reasons for this phenomenon, including security, privacy, poor pervasiveness, trust and explainability concerns.

Conclusions

This review summarizes AI applications in ophthalmology, highlighting significant clinical considerations for adopting AI techniques and discussing the potential challenges and future directions.

MTCLF: A multitask curriculum learning framework for unbiased glaucoma screenings

BACKGROUND AND OBJECTIVE

Glaucoma is a disease that causes irreversible damage to the optic nerve. Research on accurate automatic screening algorithms is essential for the prevention and treatment of glaucoma. However, due to the imbalance of existing datasets and the existence of some hard samples that accompany other diverse and complex fundus diseases, the performance of current glaucoma screening algorithms is limited. In addition, the lack of interpretability also makes it difficult for the current algorithms to meet the requirements of clinical applications.

METHOD

In this paper, we propose a new multitask curriculum learning framework (MTCLF) for unbiased glaucoma screenings and visualizations of model decision-making areas. MTCLF is a teacher-student framework. The teacher network is used to generate the label evidence map. The student network can diagnose glaucoma and predict the evidence map at the same time with the well-designed dual-branch CNN structure and collaborative learning module. We design two curriculum coefficients θ and σ to guide the training process of the student network in the sample space so that the student network can adaptively balance the sample contribution, reduce the prediction bias and mine hard samples.

RESULTS

The experimental results show that the accuracy, sensitivity, specificity, AUC and F₂-score of MTCLF based on the LAG dataset for glaucoma diagnoses are 0.967, 0.961, 0.970, 0.996, and 0.958, respectively. These results are superior to those of the state-of-the-art methods.

CONCLUSION

MTCLF not only achieves the best performance for unbiased glaucoma diagnoses but also generates a reliable evidence map to help clinicians explore fine lesion areas.

The use of deep learning technology for the detection of optic neuropathy

The emergence of computer graphics processing units (GPUs), improvements in mathematical models, and the availability of big data, has allowed artificial intelligence (AI) to use machine learning and deep learning (DL) technology to achieve robust performance in various fields of medicine. The DL system provides improved capabilities, especially in image recognition and image processing. Recent progress in the sorting of AI data sets has stimulated great interest in the development of DL algorithms. Compared with subjective evaluation and other traditional methods, DL algorithms can identify diseases faster and more accurately in diagnostic tests. Medical imaging is of great significance in the clinical diagnosis and individualized treatment of ophthalmic diseases. Based on the morphological data sets of millions of data points, various image-related diagnostic techniques can now impart high-resolution information on anatomical and functional changes, thereby providing unprecedented insights in ophthalmic clinical practice. As ophthalmology relies heavily on imaging examinations, it is one of the first medical fields to apply DL algorithms in clinical practice. Such algorithms can assist in the analysis of large amounts of data acquired from the examination of auxiliary images. In recent years, rapid advancements in imaging technology have facilitated the application of DL in the automatic identification and classification of pathologies that are characteristic of ophthalmic diseases, thereby providing high quality diagnostic information. This paper reviews the origins, development, and application of DL technology. The technical and clinical problems associated with building DL systems to meet clinical needs and the potential challenges of clinical application are discussed, especially in relation to the field of optic nerve diseases.

Literature Review on Artificial Intelligence Methods for Glaucoma Screening, Segmentation, and Classification

Artificial intelligence techniques are now being applied in different medical solutions ranging from disease screening to activity recognition and computer-aided diagnosis. The combination of computer science methods and medical knowledge facilitates and improves the accuracy of the different processes and tools. Inspired by these advances, this paper performs a literature review focused on state-of-the-art glaucoma screening, segmentation, and classification based on images of the papilla and excavation using deep learning techniques. These techniques have been shown to have high sensitivity and specificity in glaucoma screening based on papilla and excavation images. The automatic segmentation of the contours of the optic disc and the excavation then allows the identification and assessment of the glaucomatous disease’s progression. As a result, we verified whether deep learning techniques may be helpful in performing accurate and low-cost measurements related to glaucoma, which may promote patient empowerment and help medical doctors better monitor patients.

Accuracy of Using Generative Adversarial Networks for Glaucoma Detection During the COVID-19 Pandemic: A Systematic Review and Bibliometric Analysis

Background

Glaucoma leads to irreversible blindness. Globally, it is the second most common retinal disease that leads to blindness, slightly less common than cataracts. Therefore, there is a great need to avoid the silent growth of this disease using recently developed generative adversarial networks (GANs).

Objective

This paper aims to introduce a GAN technology for the diagnosis of eye disorders, particularly glaucoma. This paper illustrates deep adversarial learning as a potential diagnostic tool and the challenges involved in its implementation. This study describes and analyzes many of the pitfalls and problems that researchers will need to overcome to implement this kind of technology.

Methods

To organize this review comprehensively, articles and reviews were collected using the following keywords: (“Glaucoma,” “optic disc,” “blood vessels”) and (“receptive field,” “loss function,” “GAN,” “Generative Adversarial Network,” “Deep learning,” “CNN,” “convolutional neural network” OR encoder). The records were identified from 5 highly reputed databases: IEEE Xplore, Web of Science, Scopus, ScienceDirect, and PubMed. These libraries broadly cover the technical and medical literature. Publications within the last 5 years, specifically 2015-2020, were included because the target GAN technique was invented only in 2014 and the publishing date of the collected papers was not earlier than 2016. Duplicate records were removed, and irrelevant titles and abstracts were excluded. In addition, we excluded papers that used optical coherence tomography and visual field images, except for those with 2D images. A large-scale systematic analysis was performed, and then a summarized taxonomy was generated. Furthermore, the results of the collected articles were summarized and a visual representation of the results was presented on a T-shaped matrix diagram. This study was conducted between March 2020 and November 2020.

Results

We found 59 articles after conducting a comprehensive survey of the literature. Among the 59 articles, 30 present actual attempts to synthesize images and provide accurate segmentation/classification using single/multiple landmarks or share certain experiences. The other 29 articles discuss the recent advances in GANs, do practical experiments, and contain analytical studies of retinal disease.

Conclusions

Recent deep learning techniques, namely GANs, have shown encouraging performance in retinal disease detection. Although this methodology involves an extensive computing budget and optimization process, it saturates the greedy nature of deep learning techniques by synthesizing images and solves major medical issues. This paper contributes to this research field by offering a thorough analysis of existing works, highlighting current limitations, and suggesting alternatives to support other researchers and participants in further improving and strengthening future work. Finally, new directions for this research have been identified.

[Diagnostics of diseases of the optic nerve head in times of artificial intelligence and big data]

Hintergrund

Der Einsatz von künstlicher Intelligenz (KI) ist unter anderem in der automatischen Bildsegmentierung, -analyse und Klassifikation interessant und bereits für verschiedene Bereiche der Augenheilkunde beschrieben.

Fragestellung

Diese Arbeit soll einen Überblick über aktuelle Ansätze und Fortschritte bei der Anwendung von Big Data und KI bei verschiedenen Erkrankungen des Sehnervenkopfes geben.

Material und Methode

Es wurde eine PubMed-Recherche durchgeführt. Gesucht wurde nach Studien, die klinische Fragestellungen mithilfe von Big-Data-Ansätzen beantworteten oder klassische Methoden des maschinellen Lernens bei der Analyse von multimodaler Bildgebung des Sehnervenkopfes verwendeten.

Ergebnisse

Big Data kann bei Volkskrankheiten wie dem Glaukom helfen, klinische Fragestellungen zu beantworten. KI findet sowohl bei der Segmentierung von multimodaler Bildgebung des Sehnervenkopfes als auch bei der Klassifikation von Erkrankungen wie dem Glaukom oder der Stauungspapille auf diesen Bilddaten Anwendung.

Schlussfolgerung

Mithilfe von Big Data und KI können Zusammenhänge besser erkannt und die Diagnostik und Verlaufsbeurteilung von Erkrankungen des Sehnervenkopfes erleichtert oder automatisiert werden. Eine Voraussetzung für die klinische Anwendung ist in Europa die CE-Kennzeichnung als ein Medizinprodukt und in den USA die Zulassung durch die Food and Drug Administration.

Predicting the clinical management of skin lesions using deep learning

Automated machine learning approaches to skin lesion diagnosis from images are approaching dermatologist-level performance. However, current machine learning approaches that suggest management decisions rely on predicting the underlying skin condition to infer a management decision without considering the variability of management decisions that may exist within a single condition. We present the first work to explore image-based prediction of clinical management decisions directly without explicitly predicting the diagnosis. In particular, we use clinical and dermoscopic images of skin lesions along with patient metadata from the Interactive Atlas of Dermoscopy dataset (1011 cases; 20 disease labels; 3 management decisions) and demonstrate that predicting management labels directly is more accurate than predicting the diagnosis and then inferring the management decision ([Formula: see text] and [Formula: see text] improvement in overall accuracy and AUROC respectively), statistically significant at [Formula: see text]. Directly predicting management decisions also considerably reduces the over-excision rate as compared to management decisions inferred from diagnosis predictions (24.56% fewer cases wrongly predicted to be excised). Furthermore, we show that training a model to also simultaneously predict the seven-point criteria and the diagnosis of skin lesions yields an even higher accuracy (improvements of [Formula: see text] and [Formula: see text] in overall accuracy and AUROC respectively) of management predictions. Finally, we demonstrate our model's generalizability by evaluating on the publicly available MClass-D dataset and show that our model agrees with the clinical management recommendations of 157 dermatologists as much as they agree amongst each other.

Artificial intelligence and complex statistical modeling in glaucoma diagnosis and management

PURPOSE OF REVIEW

The field of artificial intelligence has grown exponentially in recent years with new technology, methods, and applications emerging at a rapid rate. Many of these advancements have been used to improve the diagnosis and management of glaucoma. We aim to provide an overview of recent publications regarding the use of artificial intelligence to enhance the detection and treatment of glaucoma.

RECENT FINDINGS

Machine learning classifiers and deep learning algorithms have been developed to autonomously detect early structural and functional changes of glaucoma using different imaging and testing modalities such as fundus photography, optical coherence tomography, and standard automated perimetry. Artificial intelligence has also been used to further delineate structure-function correlation in glaucoma. Additional 'structure-structure' predictions have been successfully estimated. Other machine learning techniques utilizing complex statistical modeling have been used to detect glaucoma progression, as well as to predict future progression. Although not yet approved for clinical use, these artificial intelligence techniques have the potential to significantly improve glaucoma diagnosis and management.

SUMMARY

Rapidly emerging artificial intelligence algorithms have been used for the detection and management of glaucoma. These algorithms may aid the clinician in caring for patients with this complex disease. Further validation is required prior to employing these techniques widely in clinical practice.

Applications of deep learning in fundus images: A review

The use of fundus images for the early screening of eye diseases is of great clinical importance. Due to its powerful performance, deep learning is becoming more and more popular in related applications, such as lesion segmentation, biomarkers segmentation, disease diagnosis and image synthesis. Therefore, it is very necessary to summarize the recent developments in deep learning for fundus images with a review paper. In this review, we introduce 143 application papers with a carefully designed hierarchy. Moreover, 33 publicly available datasets are presented. Summaries and analyses are provided for each task. Finally, limitations common to all tasks are revealed and possible solutions are given. We will also release and regularly update the state-of-the-art results and newly-released datasets at https://github.com/nkicsl/Fundus_Review to adapt to the rapid development of this field.

Applications of deep learning in detection of glaucoma: A systematic review

Glaucoma is the leading cause of irreversible blindness and disability worldwide. Nevertheless, the majority of patients do not know they have the disease and detection of glaucoma progression using standard technology remains a challenge in clinical practice. Artificial intelligence (AI) is an expanding field that offers the potential to improve diagnosis and screening for glaucoma with minimal reliance on human input. Deep learning (DL) algorithms have risen to the forefront of AI by providing nearly human-level performance, at times exceeding the performance of humans for detection of glaucoma on structural and functional tests. A succinct summary of present studies and challenges to be addressed in this field is needed. Following PRISMA guidelines, we conducted a systematic review of studies that applied DL methods for detection of glaucoma using color fundus photographs, optical coherence tomography (OCT), or standard automated perimetry (SAP). In this review article we describe recent advances in DL as applied to the diagnosis of glaucoma and glaucoma progression for application in screening and clinical settings, as well as the challenges that remain when applying this novel technique in glaucoma.

Automatic glaucoma screening using optic nerve head measurements and random forest classifier on fundus images

Glaucoma is an optic neuropathy that gradually steals the patient's sight by damaging the optic nerve head (which is responsible for transferring images from the eye to the brain). Causing an estimated 12.3% of global blindness, glaucoma is considered as the first leading cause of irreversible blindness in the world. This paper presents a novel eye fundus image analysis algorithm for the automatic measurement of fundus related glaucoma indicators; Cup to Disc Ratio (CDR), verification of the ISNT rule, Disc Damage Likelihood Scale (DDLS), and the classification of the input fundus into glaucoma or non-glaucoma case using a random forest model. The proposed method is applied on the public image database 'HRF', and a local database containing both, normal and glaucoma cases, and resulted sensitivity, specificity, and accuracy of 1, 0.93 and 0.97 respectively. This technique presented the highest classification accuracy compared to previous works studied in the state of the art; hence, it can be used as a computer aided glaucoma diagnosis system by ophthalmologists to assist in their screening routine.

A Review of Deep Learning for Screening, Diagnosis, and Detection of Glaucoma Progression

Because of recent advances in computing technology and the availability of large datasets, deep learning has risen to the forefront of artificial intelligence, with performances that often equal, or sometimes even exceed, those of human subjects on a variety of tasks, especially those related to image classification and pattern recognition. As one of the medical fields that is highly dependent on ancillary imaging tests, ophthalmology has been in a prime position to witness the application of deep learning algorithms that can help analyze the vast amount of data coming from those tests. In particular, glaucoma stands as one of the conditions where application of deep learning algorithms could potentially lead to better use of the vast amount of information coming from structural and functional tests evaluating the optic nerve and macula. The purpose of this article is to critically review recent applications of deep learning models in glaucoma, discussing their advantages but also focusing on the challenges inherent to the development of such models for screening, diagnosis and detection of progression. After a brief general overview of deep learning and how it compares to traditional machine learning classifiers, we discuss issues related to the training and validation of deep learning models and how they specifically apply to glaucoma. We then discuss specific scenarios where deep learning has been proposed for use in glaucoma, such as screening with fundus photography, and diagnosis and detection of glaucoma progression with optical coherence tomography and standard automated perimetry.

Translational Relevance

Deep learning algorithms have the potential to significantly improve diagnostic capabilities in glaucoma, but their application in clinical practice requires careful validation, with consideration of the target population, the reference standards used to build the models, and potential sources of bias.

Channel and Spatial Attention Regression Network for Cup-to-Disc Ratio Estimation

Cup-to-disc ratio (CDR) is of great importance during assessing structural changes at the optic nerve head (ONH) and diagnosis of glaucoma. While most efforts have been put on acquiring the CDR number through CNN-based segmentation algorithms followed by the calculation of CDR, these methods usually only focus on the features in the convolution kernel, which is, after all, the operation of the local region, ignoring the contribution of rich global features (such as distant pixels) to the current features. In this paper, a new end-to-end channel and spatial attention regression deep learning network is proposed to deduces CDR number from the regression perspective and combine the self-attention mechanism with the regression network. Our network consists of four modules: the feature extraction module to extract deep features expressing the complicated pattern of optic disc (OD) and optic cup (OC), the attention module including the channel attention block (CAB) and the spatial attention block (SAB) to improve feature representation by aggregating long-range contextual information, the regression module to deduce CDR number directly, and the segmentation-auxiliary module to focus the model’s attention on the relevant features instead of the background region. Especially, the CAB selects relatively important feature maps in channel dimension, shifting the emphasis on the OD and OC region; meanwhile, the SAB learns the discriminative ability of feature representation at pixel level by capturing the relationship of intra-feature map. The experimental results of ORIGA dataset show that our method obtains absolute CDR error of 0.067 and the Pearson’s correlation coefficient of 0.694 in estimating CDR and our method has a great potential in predicting the CDR number.

A novel glaucomatous representation method based on Radon and wavelet transform

Background

Glaucoma is an irreversible eye disease caused by the optic nerve injury. Therefore, it usually changes the structure of the optic nerve head (ONH). Clinically, ONH assessment based on fundus image is one of the most useful way for glaucoma detection. However, the effective representation for ONH assessment is a challenging task because its structural changes result in the complex and mixed visual patterns.

Method

We proposed a novel feature representation based on Radon and Wavelet transform to capture these visual patterns. Firstly, Radon transform (RT) is used to map the fundus image into Radon domain, in which the spatial radial variations of ONH are converted to a discrete signal for the description of image structural features. Secondly, the discrete wavelet transform (DWT) is utilized to capture differences and get quantitative representation. Finally, principal component analysis (PCA) and support vector machine (SVM) are used for dimensionality reduction and glaucoma detection.

Results

The proposed method achieves the state-of-the-art detection performance on RIMONE-r2 dataset with the accuracy and area under the curve (AUC) at 0.861 and 0.906, respectively.

Conclusion

In conclusion, we showed that the proposed method has the capacity as an effective tool for large-scale glaucoma screening, and it can provide a reference for the clinical diagnosis on glaucoma.