1
|
Giesser SD, Turgut F, Saad A, Zoellin JR, Sommer C, Zhou Y, Wagner SK, Keane PA, Becker M, DeBuc DC, Somfai GM. Evaluating the Impact of Retinal Vessel Segmentation Metrics on Retest Reliability in a Clinical Setting: A Comparative Analysis Using AutoMorph. Invest Ophthalmol Vis Sci 2024; 65:24. [PMID: 39540861 PMCID: PMC11572755 DOI: 10.1167/iovs.65.13.24] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2024] [Accepted: 08/29/2024] [Indexed: 11/16/2024] Open
Abstract
Purpose Current research on artificial intelligence-based fundus photography biomarkers has demonstrated inconsistent results. Consequently, we aimed to evaluate and predict the test-retest reliability of retinal parameters extracted from fundus photography. Methods Two groups of patients were recruited for the study: an intervisit group (n = 28) to assess retest reliability over a period of 1 to 14 days and an intravisit group (n = 44) to evaluate retest reliability within a single session. Using AutoMorph, we generated test and retest vessel segmentation maps; measured segmentation map agreement via accuracy, sensitivity, F1 score and Jaccard index; and calculated 76 metrics from each fundus image. The retest reliability of each metric was analyzed in terms of the Spearman correlation coefficient, intraclass correlation coefficient (ICC), and relative percentage change. A linear model with the input variables contrast-to-noise-ratio and fractal dimension, chosen by a P-value-based backward selection process, was developed to predict the median percentage difference on retest per image based on image-quality metrics. This model was trained on the intravisit dataset and validated using the intervisit dataset. Results In the intervisit group, retest reliability varied between Spearman correlation coefficients of 0.34 and 0.99, ICC values of 0.31 to 0.99, and mean absolute percentage differences of 0.96% to 223.67%. Similarly, in the intravisit group, the retest reliability ranged from Spearman correlation coefficients of 0.55 and 0.96, ICC values of 0.40 to 0.97, and mean percentage differences of 0.49% to 371.23%. Segmentation map accuracy between test and retest never dropped below 97%; the mean F1 scores were 0.85 for the intravisit dataset and 0.82 for the intervisit dataset. The best retest was achieved with disc-width regarding the Spearman correlation coefficient in both datasets. In terms of the Spearman correlation coefficient, the worst retests of the intervisit and intravisit groups were tortuosity density and artery tortuosity density, respectively. The intravisit group exhibited better retest reliability than the intervisit group (P < 0.001). Our linear model, with the two independent variables contrast-to-noise ratio and fractal dimension predicted the median retest reliability per image on its validation dataset, the intervisit group, with an R2 of 0.53 (P < 0.001). Conclusions Our findings highlight a considerable volatility in the reliability of some retinal biomarkers. Improving retest could allow disease progression modeling in smaller datasets or an individualized treatment approach. Image quality is moderately predictive of retest reliability, and further work is warranted to understand the reasons behind our observations better and thus ensure consistent retest results.
Collapse
Affiliation(s)
- Samuel D Giesser
- Department of Ophthalmology, Stadtspital Zürich, Zurich, Switzerland
- Spross Research Institute, Zurich, Switzerland
| | - Ferhat Turgut
- Department of Ophthalmology, Stadtspital Zürich, Zurich, Switzerland
- Spross Research Institute, Zurich, Switzerland
- Gutblick Research, Pfäffikon, Switzerland
- Department of Ophthalmology, Semmelweis University, Budapest, Hungary
| | - Amr Saad
- Department of Ophthalmology, Stadtspital Zürich, Zurich, Switzerland
- Spross Research Institute, Zurich, Switzerland
| | - Jay R Zoellin
- Department of Ophthalmology, Stadtspital Zürich, Zurich, Switzerland
- Spross Research Institute, Zurich, Switzerland
| | - Chiara Sommer
- Department of Ophthalmology, Stadtspital Zürich, Zurich, Switzerland
- Spross Research Institute, Zurich, Switzerland
| | - Yukun Zhou
- NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
- Institute of Ophthalmology, University College London, London, United Kingdom
- Department of Medical Physics and Biomedical Engineering, University College London, London, United Kingdom
| | - Siegfried K Wagner
- NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
- Institute of Ophthalmology, University College London, London, United Kingdom
| | - Pearse A Keane
- NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
- Institute of Ophthalmology, University College London, London, United Kingdom
| | - Matthias Becker
- Department of Ophthalmology, Stadtspital Zürich, Zurich, Switzerland
- Spross Research Institute, Zurich, Switzerland
- Department of Ophthalmology, University of Heidelberg, Heidelberg, Germany
| | - Delia Cabrera DeBuc
- Bascom Palmer Eye Institute, Miller School of Medicine, University of Miami, Miami, Florida, United States
- iScreen 2 Prevent LLC, Miami, FL, United States
| | - Gábor Márk Somfai
- Department of Ophthalmology, Stadtspital Zürich, Zurich, Switzerland
- Spross Research Institute, Zurich, Switzerland
- Department of Ophthalmology, Semmelweis University, Budapest, Hungary
| |
Collapse
|
2
|
Brummer AB, Xella A, Woodall R, Adhikarla V, Cho H, Gutova M, Brown CE, Rockne RC. Data driven model discovery and interpretation for CAR T-cell killing using sparse identification and latent variables. Front Immunol 2023; 14:1115536. [PMID: 37256133 PMCID: PMC10226275 DOI: 10.3389/fimmu.2023.1115536] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2022] [Accepted: 03/27/2023] [Indexed: 06/01/2023] Open
Abstract
In the development of cell-based cancer therapies, quantitative mathematical models of cellular interactions are instrumental in understanding treatment efficacy. Efforts to validate and interpret mathematical models of cancer cell growth and death hinge first on proposing a precise mathematical model, then analyzing experimental data in the context of the chosen model. In this work, we present the first application of the sparse identification of non-linear dynamics (SINDy) algorithm to a real biological system in order discover cell-cell interaction dynamics in in vitro experimental data, using chimeric antigen receptor (CAR) T-cells and patient-derived glioblastoma cells. By combining the techniques of latent variable analysis and SINDy, we infer key aspects of the interaction dynamics of CAR T-cell populations and cancer. Importantly, we show how the model terms can be interpreted biologically in relation to different CAR T-cell functional responses, single or double CAR T-cell-cancer cell binding models, and density-dependent growth dynamics in either of the CAR T-cell or cancer cell populations. We show how this data-driven model-discovery based approach provides unique insight into CAR T-cell dynamics when compared to an established model-first approach. These results demonstrate the potential for SINDy to improve the implementation and efficacy of CAR T-cell therapy in the clinic through an improved understanding of CAR T-cell dynamics.
Collapse
Affiliation(s)
- Alexander B. Brummer
- Division of Mathematical Oncology, Department of Computational and Quantitative Medicine, Beckman Research Institute, City of Hope National Medical Center, Duarte, CA, United States
- Department of Physics and Astronomy, College of Charleston, Charleston, SC, United States
| | - Agata Xella
- Department of Hemtaology and Hematopoietic Cell Translation and Immuno-Oncology, Beckman Research Institute, City of Hope National Medical Center, Duarte, CA, United States
| | - Ryan Woodall
- Division of Mathematical Oncology, Department of Computational and Quantitative Medicine, Beckman Research Institute, City of Hope National Medical Center, Duarte, CA, United States
| | - Vikram Adhikarla
- Division of Mathematical Oncology, Department of Computational and Quantitative Medicine, Beckman Research Institute, City of Hope National Medical Center, Duarte, CA, United States
| | - Heyrim Cho
- Department of Mathematics, University of California, Riverside, Riverside, CA, United States
| | - Margarita Gutova
- Department of Stem Cell Biology and Regenerative Medicine, Beckman Research Institute, City of Hope National Medical Center, Duarte, CA, United States
| | - Christine E. Brown
- Department of Hemtaology and Hematopoietic Cell Translation and Immuno-Oncology, Beckman Research Institute, City of Hope National Medical Center, Duarte, CA, United States
| | - Russell C. Rockne
- Division of Mathematical Oncology, Department of Computational and Quantitative Medicine, Beckman Research Institute, City of Hope National Medical Center, Duarte, CA, United States
| |
Collapse
|
4
|
Abstract
Biological allometries, such as the scaling of metabolism to mass, are hypothesized to result from natural selection to maximize how vascular networks fill space yet minimize internal transport distances and resistance to blood flow. Metabolic scaling theory argues two guiding principles—conservation of fluid flow and space-filling fractal distributions—describe a diversity of biological networks and predict how the geometry of these networks influences organismal metabolism. Yet, mostly absent from past efforts are studies that directly, and independently, measure metabolic rate from respiration and vascular architecture for the same organ, organism, or tissue. Lack of these measures may lead to inconsistent results and conclusions about metabolism, growth, and allometric scaling. We present simultaneous and consistent measurements of metabolic scaling exponents from clinical images of lung cancer, serving as a first-of-its-kind test of metabolic scaling theory, and identifying potential quantitative imaging biomarkers indicative of tumor growth. We analyze data for 535 clinical PET-CT scans of patients with non-small cell lung carcinoma to establish the presence of metabolic scaling between tumor metabolism and tumor volume. Furthermore, we use computer vision and mathematical modeling to examine predictions of metabolic scaling based on the branching geometry of the tumor-supplying blood vessel networks in a subset of 56 patients diagnosed with stage II-IV lung cancer. Examination of the scaling of maximum standard uptake value with metabolic tumor volume, and metabolic tumor volume with gross tumor volume, yield metabolic scaling exponents of 0.64 (0.20) and 0.70 (0.17), respectively. We compare these to the value of 0.85 (0.06) derived from the geometric scaling of the tumor-supplying vasculature. These results: (1) inform energetic models of growth and development for tumor forecasting; (2) identify imaging biomarkers in vascular geometry related to blood volume and flow; and (3) highlight unique opportunities to develop and test the metabolic scaling theory of ecology in tumors transitioning from avascular to vascular geometries.
Collapse
|