1
|
Diao T, Kushzad F, Patel MD, Bindiganavale MP, Wasi M, Kochenderfer MJ, Moss HE. Comparison of Machine Learning Approaches to Improve Diagnosis of Optic Neuropathy Using Photopic Negative Response Measured Using a Handheld Device. Front Med (Lausanne) 2021; 8:771713. [PMID: 34926514 PMCID: PMC8677942 DOI: 10.3389/fmed.2021.771713] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2021] [Accepted: 11/05/2021] [Indexed: 11/20/2022] Open
Abstract
The photopic negative response of the full-field electroretinogram (ERG) is reduced in optic neuropathies. However, technical requirements for measurement and poor classification performance have limited widespread clinical application. Recent advances in hardware facilitate efficient clinic-based recording of the full-field ERG. Time series classification, a machine learning approach, may improve classification by using the entire ERG waveform as the input. In this study, full-field ERGs were recorded in 217 eyes (109 optic neuropathy and 108 controls) of 155 subjects. User-defined ERG features including photopic negative response were reduced in optic neuropathy eyes (p < 0.0005, generalized estimating equation models accounting for age). However, classification of optic neuropathy based on user-defined features was only fair with receiver operating characteristic area under the curve ranging between 0.62 and 0.68 and F1 score at the optimal cutoff ranging between 0.30 and 0.33. In comparison, machine learning classifiers using a variety of time series analysis approaches had F1 scores of 0.58–0.76 on a test data set. Time series classifications are promising for improving optic neuropathy diagnosis using ERG waveforms. Larger sample sizes will be important to refine the models.
Collapse
Affiliation(s)
- Tina Diao
- Department of Management Science & Engineering, Stanford University, Stanford, CA, United States
| | - Fareshta Kushzad
- Department of Ophthalmology, Stanford University, Palo Alto, CA, United States
| | - Megh D Patel
- Department of Ophthalmology, Stanford University, Palo Alto, CA, United States
| | | | - Munam Wasi
- Department of Ophthalmology, Stanford University, Palo Alto, CA, United States
| | - Mykel J Kochenderfer
- Department of Ophthalmology, Stanford University, Palo Alto, CA, United States.,Department of Aeronautics and Astronautics, Stanford University, Stanford, CA, United States
| | - Heather E Moss
- Department of Ophthalmology, Stanford University, Palo Alto, CA, United States.,Department of Neurology and Neurological Sciences, Stanford University, Palo Alto, CA, United States
| |
Collapse
|
2
|
Ranking Information Extracted from Uncertainty Quantification of the Prediction of a Deep Learning Model on Medical Time Series Data. MATHEMATICS 2020. [DOI: 10.3390/math8071078] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/23/2023]
Abstract
Uncertainty quantification in deep learning models is especially important for the medical applications of this complex and successful type of neural architectures. One popular technique is Monte Carlo dropout that gives a sample output for a record, which can be measured statistically in terms of average probability and variance for each diagnostic class of the problem. The current paper puts forward a convolutional–long short-term memory network model with a Monte Carlo dropout layer for obtaining information regarding the model uncertainty for saccadic records of all patients. These are next used in assessing the uncertainty of the learning model at the higher level of sets of multiple records (i.e., registers) that are gathered for one patient case by the examining physician towards an accurate diagnosis. Means and standard deviations are additionally calculated for the Monte Carlo uncertainty estimates of groups of predictions. These serve as a new collection where a random forest model can perform both classification and ranking of variable importance. The approach is validated on a real-world problem of classifying electrooculography time series for an early detection of spinocerebellar ataxia 2 and reaches an accuracy of 88.59% in distinguishing between the three classes of patients.
Collapse
|
3
|
Stoean C, Stoean R, Atencia M, Abdar M, Velázquez-Pérez L, Khosravi A, Nahavandi S, Acharya UR, Joya G. Automated Detection of Presymptomatic Conditions in Spinocerebellar Ataxia Type 2 Using Monte Carlo Dropout and Deep Neural Network Techniques with Electrooculogram Signals. SENSORS (BASEL, SWITZERLAND) 2020; 20:E3032. [PMID: 32471077 PMCID: PMC7309035 DOI: 10.3390/s20113032] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/21/2020] [Revised: 05/25/2020] [Accepted: 05/25/2020] [Indexed: 12/21/2022]
Abstract
Application of deep learning (DL) to the field of healthcare is aiding clinicians to make an accurate diagnosis. DL provides reliable results for image processing and sensor interpretation problems most of the time. However, model uncertainty should also be thoroughly quantified. This paper therefore addresses the employment of Monte Carlo dropout within the DL structure to automatically discriminate presymptomatic signs of spinocerebellar ataxia type 2 in saccadic samples obtained from electrooculograms. The current work goes beyond the common incorporation of this special type of dropout into deep neural networks and uses the uncertainty derived from the validation samples to construct a decision tree at the register level of the patients. The decision tree built from the uncertainty estimates obtained a classification accuracy of 81.18% in automatically discriminating control, presymptomatic and sick classes. This paper proposes a novel method to address both uncertainty quantification and explainability to develop reliable healthcare support systems.
Collapse
Affiliation(s)
- Catalin Stoean
- Romanian Institute of Science and Technology, 400022 Cluj-Napoca, Romania;
| | - Ruxandra Stoean
- Romanian Institute of Science and Technology, 400022 Cluj-Napoca, Romania;
| | - Miguel Atencia
- Department of Applied Mathematics, Universidad de Málaga, 29071 Málaga, Spain;
| | - Moloud Abdar
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong 3216, Australia; (M.A.); (A.K.); (S.N.)
| | - Luis Velázquez-Pérez
- Cuban Academy of Sciences, La Habana 10100, Cuba;
- Center for Research and Rehabilitation of Hereditary Ataxias, Holguín 80100, Cuba
| | - Abbas Khosravi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong 3216, Australia; (M.A.); (A.K.); (S.N.)
| | - Saeid Nahavandi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong 3216, Australia; (M.A.); (A.K.); (S.N.)
| | - U. Rajendra Acharya
- Department of Electronics and Computer Engineering, Ngee Ann Polytechnic, Clementi 599489, Singapore;
- Department of Bioinformatics and Medical Engineering, Asia University, Taichung 41354, Taiwan
- International Research Organization for Advanced Science and Technology (IROAST), Kumamoto University, Kumamoto 860-8555, Japan
| | - Gonzalo Joya
- Department of Electronic Technology, Universidad de Málaga, 29071 Málaga, Spain;
| |
Collapse
|