Optimal surface segmentation in volumetric images--a graph-theoretic approach

Assessment of OCT-Based Macular Curvature and Its Relationship with Macular Microvasculature in Children with Anisomyopia

INTRODUCTION

To evaluate the intraocular differences in optical coherence tomography (OCT)-based macular curvature index (MCI) among children with anisomyopia and to investigate the relationship between MCI and the macular microvasculature.

METHODS

Fifty-two schoolchildren with anisometropia > 2.00 D were enrolled and underwent comprehensive examinations including cycloplegic refraction, axial length (AL), and swept source OCT/OCT angiography. OCT-based MCIs were determined from horizontal and vertical B-scans by a customized curve fitting model in MATLAB R2022 at 1-mm-, 3-mm-, and 6-mm-diameter circles at fovea. Characteristics and topographic variation of MCI was analyzed, and the relationships with microvascularity and its associated factors were investigated.

RESULTS

MCI achieved high reliability and repeatability. There were overall larger MCIs in the more myopic eyes than the less myopic eyes in 1-mm-, 3-mm-, and 6-mm-diameter circles at fovea (all p < 0.001). For the topographic variation, horizontal MCI was significantly greater than vertical MCI (all p < 0.001), and was the largest in 6-mm circle, followed by 3-mm and 1-mm circles. Stronger correlation of horizontal MCI with myopic severity than vertical MCI was found. Partial Pearson's correlation found MCI was negatively associated with deep capillary plexus (DCP) vessel density (p = 0.016). Eyes with a higher MCI in a 6-mm circle were more likely to have longer AL (p < 0.001), lower DCP vessel density (p = 0.037), and thinner choroidal thickness (ChT) (p = 0.045).

CONCLUSION

Larger MCI was found in the more myopic eyes of children with anisomyopia and was significantly associated with smaller DCP density, suggesting that MCI was an important indicator of myopia-related retinal microvascularity change, and it could be a valuable metric for myopia assessment in children.

Approved AI-based fluid monitoring to identify morphological and functional treatment outcomes in neovascular age-related macular degeneration in real-world routine

AIM

To predict antivascular endothelial growth factor (VEGF) treatment requirements, visual acuity and morphological outcomes in neovascular age-related macular degeneration (nAMD) using fluid quantification by artificial intelligence (AI) in a real-world cohort.

METHODS

Spectral-domain optical coherence tomography data of 158 treatment-naïve patients with nAMD from the Fight Retinal Blindness! registry in Zurich were processed at baseline, and after initial treatment using intravitreal anti-VEGF to predict subsequent 1-year and 4-year outcomes. Intraretinal and subretinal fluid and pigment epithelial detachment volumes were segmented using a deep learning algorithm (Vienna Fluid Monitor, RetInSight, Vienna, Austria). A predictive machine learning model for future treatment requirements and morphological outcomes was built using the computed set of quantitative features.

RESULTS

Two hundred and two eyes from 158 patients were evaluated. 107 eyes had a lower median (≤7) and 95 eyes had an upper median (≥8) number of injections in the first year, with a mean accuracy of prediction of 0.77 (95% CI 0.71 to 0.83) area under the curve (AUC). Best-corrected visual acuity at baseline was the most relevant predictive factor determining final visual outcomes after 1 year. Over 4 years, half of the eyes had progressed to macular atrophy (MA) with the model being able to distinguish MA from non-MA eyes with a mean AUC of 0.70 (95% CI 0.61 to 0.79). Prediction for subretinal fibrosis reached an AUC of 0.74 (95% CI 0.63 to 0.81).

CONCLUSIONS

The regulatory approved AI-based fluid monitoring allows clinicians to use automated algorithms in prospectively guided patient treatment in AMD. Furthermore, retinal fluid localisation and quantification can predict long-term morphological outcomes.

Hybrid deep learning and optimal graph search method for optical coherence tomography layer segmentation in diseases affecting the optic nerve

Accurate segmentation of retinal layers in optical coherence tomography (OCT) images is critical for assessing diseases that affect the optic nerve, but existing automated algorithms often fail when pathology causes irregular layer topology, such as extreme thinning of the ganglion cell-inner plexiform layer (GCIPL). Deep LOGISMOS, a hybrid approach that combines the strengths of deep learning and 3D graph search to overcome their limitations, was developed to improve the accuracy, robustness and generalizability of retinal layer segmentation. The method was trained on 124 OCT volumes from both eyes of 31 non-arteritic anterior ischemic optic neuropathy (NAION) patients and tested on three cross-sectional datasets with available reference tracings: Test-NAION (40 volumes from both eyes of 20 NAION subjects), Test-G (29 volumes from 29 glaucoma subjects/eyes), and Test-JHU (35 volumes from 21 multiple sclerosis and 14 control subjects/eyes) and one longitudinal dataset without reference tracings: Test-G-L (155 volumes from 15 glaucoma patients/eyes). In the three test datasets with reference tracings (Test-NAION, Test-G, and Test-JHU), Deep LOGISMOS achieved very high Dice similarity coefficients (%) on GCIPL: 89.97±3.59, 90.63±2.56, and 94.06±1.76, respectively. In the same context, Deep LOGISMOS outperformed the Iowa reference algorithms by improving the Dice score by 17.5, 5.4, and 7.5, and also surpassed the deep learning framework nnU-Net with improvements of 4.4, 3.7, and 1.0. For the 15 severe glaucoma eyes with marked GCIPL thinning (Test-G-L), it demonstrated reliable regional GCIPL thickness measurement over five years. The proposed Deep LOGISMOS approach has potential to enhance precise quantification of retinal structures, aiding diagnosis and treatment management of optic nerve diseases.

Automatic segmentation of layers in chorio-retinal complex using Graph-based method for ultra-speed 1.7 MHz wide field swept source FDML optical coherence tomography

The posterior segment of the human eye complex contains two discrete microstructure and vasculature network systems, namely, the retina and choroid. We present a single segmentation framework technique for segmenting the entire layers present in the chorio-retinal complex of the human eye using optical coherence tomography (OCT) images. This automatic program is based on the graph theory method. This single program is capable of segmenting seven layers of the retina and choroid scleral interface. The graph theory was utilized to find the probability matrix and subsequent boundaries of different layers. The program was also implemented to segment angiographic maps of different chorio-retinal layers using "segmentation matrices." The method was tested and successfully validated on OCT images from six normal human eyes as well as eyes with non-exudative age-related macular degeneration (AMD). The thickness of microstructure and microvasculature for different layers located in the chorio-retinal segment of the eye was also generated and compared. A decent efficiency in terms of processing time, sensitivity, and accuracy was observed compared to the manual segmentation and other existing methods. The proposed method automatically segments whole OCT images of chorio-retinal complex with augmented probability maps generation in OCT volume dataset. We have also evaluated the segmentation results using quantitative metrics such as Dice coefficient and Hausdorff distance This method realizes a mean descent Dice similarity coefficient (DSC) value of 0.82 (range, 0.816-0.864) for RPE and CSI layer.

Signature of Altered Retinal Microstructures and Electrophysiology in Schizophrenia Spectrum Disorders Is Associated With Disease Severity and Polygenic Risk

BACKGROUND

Optical coherence tomography and electroretinography studies have revealed structural and functional retinal alterations in individuals with schizophrenia spectrum disorders (SSDs). However, it remains unclear which specific retinal layers are affected; how the retina, brain, and clinical symptomatology are connected; and how alterations of the visual system are related to genetic disease risk.

METHODS

Optical coherence tomography, electroretinography, and brain magnetic resonance imaging were applied to comprehensively investigate the visual system in a cohort of 103 patients with SSDs and 130 healthy control individuals. The sparse partial least squares algorithm was used to identify multivariate associations between clinical disease phenotype and biological alterations of the visual system. The association of the revealed patterns with individual polygenic disease risk for schizophrenia was explored in a post hoc analysis. In addition, covariate-adjusted case-control comparisons were performed for each individual optical coherence tomography and electroretinography parameter.

RESULTS

The sparse partial least squares analysis yielded a phenotype-eye-brain signature of SSDs in which greater disease severity, longer duration of illness, and impaired cognition were associated with electrophysiological alterations and microstructural thinning of most retinal layers. Higher individual loading onto this disease-relevant signature of the visual system was significantly associated with elevated polygenic risk for schizophrenia. In case-control comparisons, patients with SSDs had lower macular thickness, thinner retinal nerve fiber and inner plexiform layers, less negative a-wave amplitude, and lower b-wave amplitude.

CONCLUSIONS

This study demonstrates multimodal microstructural and electrophysiological retinal alterations in individuals with SSDs that are associated with disease severity and individual polygenic burden.

Long-term impact of low-energy femtosecond laser and manual cataract surgery on macular layer thickness: A prospective randomized study

PURPOSE

To evaluate change in retinal layers 18 months after femtosecond laser-assisted cataract surgery (LCS) and manual cataract surgery (MCS) in a representative age-related cataract population using artificial intelligence (AI)-based automated retinal layer segmentation.

METHODS

This was a prospective, randomized and intraindividual-controlled study including 60 patients at the Medical University of Vienna, Austria. Bilateral same-day LCS and MCS were performed in a randomized sequence. To provide insight into the development of cystoid macular oedema (CME), retinal layer thickness was measured pre-operatively and up to 18 months post-operatively in the central 1 mm, 3 mm and 6 mm.

RESULTS

Fifty-six patients completed all follow-up visits. LCS compared to MCS did not impact any of the investigated retinal layers at any follow-up visit (p > 0.05). For the central 1 mm, a significant increase in total retinal thickness (TRT) was seen after 1 week followed by an elevated plateau thereafter. For the 3 mm and 6 mm, TRT increased only after 3 weeks and 6 weeks and decreased again until 18 months. TRT remained significantly increased compared to pre-operative thickness (p < 0.001). Visual acuity remained unaffected by the macular thickening and no case of CME was observed. Inner nuclear layer (INL) and outer nuclear layer (ONL) were the main causative layers for the total TRT increase. Photoreceptors (PR) declined 1 week after surgery but regained pre-operative values 18 months after surgery.

CONCLUSION

Low-energy femtosecond laser pre-treatment did not influence thickness of the retinal layers in any topographic zone compared to manual high fluidic phacoemulsification. TRT did not return to pre-operative values 18 months after surgery. The causative layers for subclinical development of CME were successfully identified.

Deep Learning-Based Retinal Layer Segmentation in Optical Coherence Tomography Scans of Patients with Inherited Retinal Diseases

BACKGROUND

In optical coherence tomography (OCT) scans of patients with inherited retinal diseases (IRDs), the measurement of the thickness of the outer nuclear layer (ONL) has been well established as a surrogate marker for photoreceptor preservation. Current automatic segmentation tools fail in OCT segmentation in IRDs, and manual segmentation is time-consuming.

METHODS AND MATERIAL

Patients with IRD and an available OCT scan were screened for the present study. Additionally, OCT scans of patients without retinal disease were included to provide training data for artificial intelligence (AI). We trained a U-net-based model on healthy patients and applied a domain adaption technique to the IRD patients' scans.

RESULTS

We established an AI-based image segmentation algorithm that reliably segments the ONL in OCT scans of IRD patients. In a test dataset, the dice score of the algorithm was 98.7%. Furthermore, we generated thickness maps of the full retinal thickness and the ONL layer for each patient.

CONCLUSION

Accurate segmentation of anatomical layers on OCT scans plays a crucial role for predictive models linking retinal structure to visual function. Our algorithm for segmentation of OCT images could provide the basis for further studies on IRDs.

Visualization of Optic Nerve Structural Patterns in Papilledema Using Deep Learning Variational Autoencoders

Purpose

To visualize and quantify structural patterns of optic nerve edema encountered in papilledema during treatment.

Methods

A novel bi-channel deep-learning variational autoencoder (biVAE) model was trained using 1498 optical coherence tomography (OCT) scans of 125 subjects over time from the Idiopathic Intracranial Hypertension Treatment Trial (IIHTT) and 791 OCT scans of 96 control subjects from the University of Iowa. An independent test dataset of 70 eyes from 70 papilledema subjects was used to evaluate the ability of the biVAE model to quantify and reconstruct the papilledema spatial patterns from input OCT scans using only two variables.

Results

The montage color maps of the retinal nerve fiber layer (RNFL) and total retinal thickness (TRT) produced by the biVAE model provided an organized visualization of the variety of morphological patterns of optic disc edema (including differing patterns at similar thickness levels). Treatment effects of acetazolamide versus placebo in the IIHTT were also demonstrated in the latent space. In image reconstruction, the mean signed peripapillary retinal nerve fiber layer thickness (pRNFLT) difference ± SD was -0.12 ± 17.34 µm, the absolute pRNFLT difference was 13.68 ± 10.65 µm, and the RNFL structural similarity index reached 0.91 ± 0.05.

Conclusions

A wide array of structural patterns of papilledema, integrating the magnitude of disc edema with underlying disc and retinal morphology, can be quantified by just two latent variables.

Translational Relevance

A biVAE model encodes structural patterns, as well as the correlation between channels, and may be applied to visualize individuals or populations with papilledema throughout treatment.

Reduced Thickness of the Retina in de novo Parkinson's Disease Shows A Distinct Pattern, Different from Glaucoma

Background

Parkinson's disease (PD) patients experience visual symptoms and retinal degeneration. Studies using optical coherence tomography (OCT) have shown reduced thickness of the retina in PD, also a key characteristic of glaucoma.

Objective

To identify the presence and pattern of retinal changes in de novo, treatment-naive PD patients compared to healthy controls (HC) and early primary open angle glaucoma (POAG) patients.

Methods

Macular OCT data (10×10 mm) were collected from HC, PD, and early POAG patients, at the University Medical Center Groningen. Bayesian informative hypotheses statistical analyses were carried out comparing HC, PD-, and POAG patients, within each retinal cell layer.

Results

In total 100 HC, 121 PD, and 78 POAG patients were included. We showed significant reduced thickness of the inner plexiform layer and retinal pigment epithelium in PD compared to HC. POAG patients presented with a significantly thinner retinal nerve fiber layer, ganglion cell layer, inner plexiform layer, outer plexiform layer, and outer photoreceptor and subretinal virtual space compared to PD. Only the outer segment layer and retinal pigment epithelium were significantly thinner in PD compared to POAG.

Conclusions

De novo PD patients show reduced thickness of the retina compared to HC, especially of the inner plexiform layer, which differs significantly from POAG, showing a more extensive and widespread pattern of reduced thickness across layers. OCT is a useful tool to detect retinal changes in de novo PD, but its specificity versus other neurodegenerative disorders has to be established.

A Technique to Enable Efficient Adaptive Radiation Therapy: Automated Contouring of Prostate and Adjacent Organs

Purpose

The purpose of this work was to investigate the use of a segmentation approach that could potentially improve the speed and reproducibility of contouring during magnetic resonance-guided adaptive radiation therapy.

Methods and Materials

The segmentation algorithm was based on a hybrid deep neural network and graph optimization approach that also allows rapid user intervention (Deep layered optimal graph image segmentation of multiple objects and surfaces [LOGISMOS] + just enough interaction [JEI]). A total of 115 magnetic resonance-data sets were used for training and quantitative assessment. Expert segmentations were used as the independent standard for the prostate, seminal vesicles, bladder, rectum, and femoral heads for all 115 data sets. In addition, 3 independent radiation oncologists contoured the prostate, seminal vesicles, and rectum for a subset of patients such that the interobserver variability could be quantified. Consensus contours were then generated from these independent contours using a simultaneous truth and performance level estimation approach, and the deviation of Deep LOGISMOS + JEI contours to the consensus contours was evaluated and compared with the interobserver variability.

Results

The absolute accuracy of Deep LOGISMOS + JEI generated contours was evaluated using median absolute surface-to-surface distance which ranged from a minimum of 0.20 mm for the bladder to a maximum of 0.93 mm for the prostate compared with the independent standard across all data sets. The median relative surface-to-surface distance was less than 0.17 mm for all organs, indicating that the Deep LOGISMOS + JEI algorithm did not exhibit a systematic under- or oversegmentation. Interobserver variability testing yielded a mean absolute surface-to-surface distance of 0.93, 1.04, and 0.81 mm for the prostate, seminal vesicles, and rectum, respectively. In comparison, the deviation of Deep LOGISMOS + JEI from consensus simultaneous truth and performance level estimation contours was 0.57, 0.64, and 0.55 mm for the same organs. On average, the Deep LOGISMOS algorithm took less than 26 seconds for contour segmentation.

Conclusions

Deep LOGISMOS + JEI segmentation efficiently generated clinically acceptable prostate and normal tissue contours, potentially limiting the need for time intensive manual contouring with each fraction.

Relationship Between Retinal Layer Thickness and Genetic Susceptibility to Age-Related Macular Degeneration in Asian Populations

Purpose

For OCT retinal thickness measurements to be used as a prodromal age-related macular degeneration (AMD) risk marker, the 3-dimensional (3D) topographic variation of the relationship between genetic susceptibility to AMD and retinal thickness needs to be assessed. We aimed to evaluate individual retinal layer thickness changes and topography at the macula that are associated with AMD genetic susceptibility.

Design

Genetic association study.

Participants

A total of 1579 healthy participants (782 Chinese, 353 Malays, and 444 Indians) from the multiethnic Singapore Epidemiology of Eye Diseases study were included.

Methods

Spectral-domain OCT and automatic segmentation of individual retinal layers were performed to produce 10 retinal layer thickness measurements at each ETDRS subfield, producing 3D topographic information. Age-related macular degeneration genetic susceptibility was represented via single nucleotide polymorphisms (SNPs) and aggregated via whole genome (overall) and pathway-specific age-related macular degeneration polygenic risk score (PRSAMD).

Main Outcome Measures

Associations of individual SNPs, overall PRSAMD, and pathway-specific PRSAMD with retinal thickness were analyzed by individual retinal layer and ETDRS subfield.

Results

CFH rs10922109, ARMS2-HTRA1 rs3750846, and LIPC rs2043085 were the top AMD susceptibility SNPs associated with retinal thickness of individual layers (P < 1.67 × 10-3), all at the central subfield. The overall PRSAMD was most associated with thinner L9 (outer segment photoreceptor/retinal pigment epithelium complex) thickness at the central subfield (β = -0.63 μm; P = 5.45 × 10-9). Pathway-specific PRSAMD for the complement cascade (β = -0.53 μm; P = 9.42 × 10-7) and lipoprotein metabolism (β = -0.05 μm; P = 0.0061) were associated with thinner photoreceptor layers (L9 and L7 [photoreceptor inner/outer segments], respectively) at the central subfield. The mean PRSAMD score was larger among Indians compared with that of the Chinese and had the thinnest thickness at the L9 central subfield (β = -1.00 μm; P = 2.91 × 10-7; R2 = 5.5%). Associations at other retinal layers and ETDRS regions were more heterogeneous.

Conclusions

Overall genetic susceptibility to AMD and the aggregate effects of the complement cascade and lipoprotein metabolism pathway are associated most significantly with L7 and L9 photoreceptor thinning at the central macula in healthy individuals. Photoreceptor thinning has potential to be a prodromal AMD risk marker, and topographic variation should be considered.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Point-to-point associations of drusen and hyperreflective foci volumes with retinal sensitivity in non-exudative age-related macular degeneration

OBJECTIVES

To evaluate the quantitative impact of drusen and hyperreflective foci (HRF) volumes on mesopic retinal sensitivity in non-exudative age-related macular degeneration (AMD).

METHODS

In a standardized follow-up scheme of every three months, retinal sensitivity of patients with early or intermediate AMD was assessed by microperimetry using a custom pattern of 45 stimuli (Nidek MP-3, Gamagori, Japan). Eyes were consecutively scanned using Spectralis SD-OCT (20° × 20°, 1024 × 97 × 496). Fundus photographs obtained by the MP-3 allowed to map the stimuli locations onto the corresponding OCT scans. The volume and mean thickness of drusen and HRF within a circle of 240 µm centred at each stimulus point was determined using automated AI-based image segmentation algorithms.

RESULTS

8055 individual stimuli from 179 visits from 51 eyes of 35 consecutive patients were matched with the respective OCT images in a point-to-point manner. The patients mean age was 76.85 ± 6.6 years. Mean retinal sensitivity at baseline was 25.7 dB. 73.47% of all MP-spots covered drusen area and 2.02% of MP-spots covered HRF. A negative association between retinal sensitivity and the volume of underlying drusen (p < 0.001, Estimate -0.991 db/µm3) and HRF volume (p = 0.002, Estimate -5.230 db/µm3) was found. During observation time, no eye showed conversion to advanced AMD.

CONCLUSION

A direct correlation between drusen and lower sensitivity of the overlying photoreceptors can be observed. For HRF, a small but significant correlation was shown, which is compromised by their small size. Biomarker quantification using AI-methods allows to determine the impact of sub-clinical features in the progression of AMD.

Cost-efficient and glaucoma-specifical model by exploiting normal OCT images with knowledge transfer learning

Monitoring the progression of glaucoma is crucial for preventing further vision loss. However, deep learning-based models emphasize early glaucoma detection, resulting in a significant performance gap to glaucoma-confirmed subjects. Moreover, developing a fully-supervised model is suffering from insufficient annotated glaucoma datasets. Currently, sufficient and low-cost normal OCT images with pixel-level annotations can serve as valuable resources, but effectively transferring shared knowledge from normal datasets is a challenge. To alleviate the issue, we propose a knowledge transfer learning model for exploiting shared knowledge from low-cost and sufficient annotated normal OCT images by explicitly establishing the relationship between the normal domain and the glaucoma domain. Specifically, we directly introduce glaucoma domain information to the training stage through a three-step adversarial-based strategy. Additionally, our proposed model exploits different level shared features in both output space and encoding space with a suitable output size by a multi-level strategy. We have collected and collated a dataset called the TongRen OCT glaucoma dataset, including pixel-level annotated glaucoma OCT images and diagnostic information. The results on the dataset demonstrate our proposed model outperforms the un-supervised model and the mixed training strategy, achieving an increase of 5.28% and 5.77% on mIoU, respectively. Moreover, our proposed model narrows performance gap to the fully-supervised model decreased by only 1.01% on mIoU. Therefore, our proposed model can serve as a valuable tool for extracting glaucoma-related features, facilitating the tracking progression of glaucoma.

The Occurrence of Intraretinal and Subretinal Fluid in Anterior Ischemic Optic Neuropathy: Pathogenesis, Prognosis, and Treatment

PURPOSE

To describe the frequency and characteristics of intraretinal and subretinal fluid in nonarteritic anterior ischemic optic neuropathy (NAAION) and to assess the influence on the visual deficit and optic nerve fiber/ganglion cell loss.

DESIGN

A retrospective, single-center study.

PARTICIPANTS

Thirty-two patients with NAAION referred to our Neuro-ophthalmology Department between 2014 and 2021.

METHODS

The study was carried out at the University Hospital of Liège, Belgium. For participants in whom subretinal fluid was identified on standard OCT (Carl Zeiss Meditec) an additional macular OCT (Spectralis Heidelberg) had been performed. The pattern and the maximal height of the retinal fluid were determined manually, and thicknesses of retinal layers were obtained using the OCT protocol analysis.

RESULTS

The mean age of the cohort was 60 years (standard deviation, ±12.5; range, 22-88 years), and 65.6% were male. In the 21 eyes (46.7%) in which retinal fluid was observed, macular OCT findings were categorized according to fluid localization: 19 cases had parafoveal fluid (of whom 9 also had subfoveal fluid). One patient had subfoveal fluid alone, and 1 patient had peripapillary subretinal fluid alone. Specific patterns of optic disc (OD) swelling were associated with the occurrence and distribution of retinal edema. Visual acuity, visual field loss, and foveal thresholds were stable over the period of observation (P = 0.74, P = 0.42, and P = 0.36, respectively). No difference was found in visual function at 6 months between patients with retinal fluid treated (n = 10) or not treated (n = 11) with corticosteroids (visual acuity, P = 0.13; foveal threshold, P = 0.59; mean deviation, P = 0.66).

CONCLUSIONS

Subretinal fluid is found in a high proportion of cases of NAAION. Visual function remained largely stable from presentation in this cohort. Corticosteroid intake at presentation did not influence visual recovery or timing of the resorption of tissue edema. Our findings do not support treatment of NAAION with corticosteroids with or without evidence of subretinal fluid acutely. With regard to pathogenesis, we propose that the volume of transudate generated at the OD is the critical factor rather than dysfunction of retinal mechanisms subserving reabsorption.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Contractile ring composition dictates kinetics of in silico contractility

Constriction kinetics of the cytokinetic ring are expected to depend on dynamic adjustment of contractile ring composition, but the impact of ring component abundance dynamics on ring constriction is understudied. Computational models generally assume that contractile networks maintain constant total amounts of components, which is not always true. To test how compositional dynamics affect constriction kinetics, we first measured F-actin, non-muscle myosin II, septin, and anillin during Caenorhabditis elegans zygotic mitosis. A custom microfluidic device that positioned the cell with the division plane parallel to a light sheet allowed even illumination of the cytokinetic ring. Measured component abundances were implemented in a three-dimensional agent-based model of a membrane-associated contractile ring. With constant network component amounts, constriction completed with biologically unrealistic kinetics. However, imposing the measured changes in component quantities allowed this model to elicit realistic constriction kinetics. Simulated networks were more sensitive to changes in motor and filament amounts than those of crosslinkers and tethers. Our findings highlight the importance of network composition for actomyosin contraction kinetics.

Deep learning network with differentiable dynamic programming for retina OCT surface segmentation

Multiple-surface segmentation in optical coherence tomography (OCT) images is a challenging problem, further complicated by the frequent presence of weak image boundaries. Recently, many deep learning-based methods have been developed for this task and yield remarkable performance. Unfortunately, due to the scarcity of training data in medical imaging, it is challenging for deep learning networks to learn the global structure of the target surfaces, including surface smoothness. To bridge this gap, this study proposes to seamlessly unify a U-Net for feature learning with a constrained differentiable dynamic programming module to achieve end-to-end learning for retina OCT surface segmentation to explicitly enforce surface smoothness. It effectively utilizes the feedback from the downstream model optimization module to guide feature learning, yielding better enforcement of global structures of the target surfaces. Experiments on Duke AMD (age-related macular degeneration) and JHU MS (multiple sclerosis) OCT data sets for retinal layer segmentation demonstrated that the proposed method was able to achieve subvoxel accuracy on both datasets, with the mean absolute surface distance (MASD) errors of 1.88 ± 1.96μm and 2.75 ± 0.94μm, respectively, over all the segmented surfaces.

Automated inter-device 3D OCT image registration using deep learning and retinal layer segmentation

Optical coherence tomography (OCT) is the most widely used imaging modality in ophthalmology. There are multiple variations of OCT imaging capable of producing complementary information. Thus, registering these complementary volumes is desirable in order to combine their information. In this work, we propose a novel automated pipeline to register OCT images produced by different devices. This pipeline is based on two steps: a multi-modal 2D en-face registration based on deep learning, and a Z-axis (axial axis) registration based on the retinal layer segmentation. We evaluate our method using data from a Heidelberg Spectralis and an experimental PS-OCT device. The empirical results demonstrated high-quality registrations, with mean errors of approximately 46 µm for the 2D registration and 9.59 µm for the Z-axis registration. These registrations may help in multiple clinical applications such as the validation of layer segmentations among others.

Characteristics of Henle's fiber layer in healthy and glaucoma eyes assessed by polarization-sensitive optical coherence tomography

Using conventional optical coherence tomography (OCT), it is difficult to image Henle fibers (HF) due to their low backscattering potential. However, fibrous structures exhibit form birefringence, which can be exploited to visualize the presence of HF by polarization-sensitive (PS) OCT. We found a slight asymmetry in the retardation pattern of HF in the fovea region that can be associated with the asymmetric decrease of cone density with eccentricity from the fovea. We introduce a new measure based on a PS-OCT assessment of optic axis orientation to estimate the presence of HF at various eccentricities from the fovea in a large cohort of 150 healthy subjects. By comparing a healthy age-matched sub-group (N = 87) to a cohort of 64 early-stage glaucoma patients, we found no significant difference in HF extension but a slightly decreased retardation at about 2° to 7.5° eccentricity from the fovea in the glaucoma patients. This potentially indicates that glaucoma affects this neuronal tissue at an early state.

Retinal photoreceptor layer thickness has disease specificity and distinguishes predicted FTLD-Tau from biomarker-determined Alzheimer's disease

While Alzheimer's disease (AD) is associated with inner retina thinning (retinal nerve fiber layer and ganglion cell layer), we have observed photoreceptor outer nuclear layer (ONL) thinning in patients with frontotemporal lobar degeneration tauopathy (FTLD-Tau) compared to normal controls. We hypothesized that ONL thinning may distinguish FTLD-Tau from patients with biomarker evidence of AD neuropathologic change (ADNC) and will correlate with FTLD-Tau disease severity. Predicted FTLD-Tau (pFTLD-Tau; n = 21; 33 eyes) and predicted ADNC (pADNC; n = 24; 46 eyes) patients were consecutively enrolled, underwent optical coherence tomography macula imaging, and disease was categorized (pFTLD-Tau vs. pADNC) with cerebrospinal fluid biomarkers, genetic testing, and autopsy data when available. Adjusting for age, sex, and race, pFTLD-Tau patients had a thinner ONL compared to pADNC, while retinal nerve fiber layer and ganglion cell layer were not significantly different. Reduced ONL thickness correlated with worse performance on Folstein Mini-Mental State Examination and clinical dementia rating plus frontotemporal dementia sum of boxes for pFTLD-Tau but not pADNC. Photoreceptor ONL thickness may serve as an important noninvasive diagnostic marker that distinguishes FTLD-Tau from AD neuropathologic change.

Using computed tomography to recover hidden medieval fragments beneath early modern leather bindings, first results

Medieval bindings fragments have become increasingly interesting to Humanities researchers as sources for the textual and material history of medieval Europeans. Later book binders used these discarded and repurposed pieces of earlier medieval manuscripts to reinforce the structures of other manuscripts and printed books. That many of these fragments are contained within and obscured by decorative bindings that cannot be dismantled ethically has limited their discovery and description. Although previous attempts to recover these texts using IRT and MA-XRF scanning have been successful, the extensive time required to scan a single book, and the need to modify or create specialized IRT or MA-XRF equipment for this method are drawbacks. Our research proposes and tests the capabilities of medical CT scanning technologies (commonly available at research university medical schools) for making visible and legible these fragments hidden under leather bindings. Our research team identified three sixteenth-century printed codices in our university libraries that were evidently bound in tawed leather by one workshop. The damaged cover of one of these three had revealed medieval manuscript fragments on the book spine; this codex served as a control for testing the other two volumes to see if they, too, contain fragments. The use of a medical CT scanner proved successful in visualizing interior book-spine structures and some letterforms, but not all of the text was made visible. The partial success of CT-scanning points to the value of further experimentation, given the relatively wide availability of medical imaging technologies, with their potential for short, non-destructive, 3D imaging times.

Therapeutic response in the HAWK and HARRIER trials using deep learning in retinal fluid volume and compartment analysis

OBJECTIVES

To assess the therapeutic response to brolucizumab and aflibercept by deep learning/OCT-based analysis of macular fluid volumes in neovascular age-related macular degeneration.

METHODS

In this post-hoc analysis of two phase III, randomised, multi-centre studies (HAWK/HARRIER), 1078 and 739 treatment-naive eyes receiving brolucizumab or aflibercept according to protocol-specified criteria in HAWK and HARRIER, respectively, were included. Macular fluid on 41,840 OCT scans was localised and quantified using a validated deep learning-based algorithm. Volumes of intraretinal fluid (IRF), subretinal fluid (SRF), pigment epithelial detachment (PED) for all central macular areas (1, 3 and 6 mm) in nanolitres (nL) and best corrected visual acuity (BCVA) change in ETDRS letters were associated using mixed models for repeated measures.

RESULTS

Baseline IRF volumes decreased by >92% following the first intravitreal injection and consistently remained low during follow-up. Baseline SRF volumes decreased by >74% following the first injection, while PED volume resolved by 68-79% of its baseline volume. Resolution of SRF and PED was dependent on the substance and regimen used. Larger residual post-loading IRF, SRF and PED volumes were all independently associated with progressive vision loss during maintenance, where the differences in mean BCVA change between high and low fluid volume subgroups for IRF, SRF and PED were 3.4 letters (p < 0.0001), 1.7 letters (p < 0.001) and 2.5 letters (p < 0.0001), respectively.

CONCLUSIONS

Deep-learning methods allow an accurate assessment of substance and regimen efficacy. Irrespectively, all fluid compartments were found to be important markers of disease activity and were relevant for visual outcomes.

Protocol for Multi-modality MEdical imaging sTudy bAsed on KaiLuan Study (META-KLS): rationale, design and database building

INTRODUCTION

Multi-modality medical imaging study, especially brain MRI, greatly facilitates the research on subclinical brain disease. However, there is still a lack of such studies with a wider age span of participants. The Multi-modality MEdical imaging sTudy bAsed on KaiLuan Study (META-KLS) was designed to address this issue with a large sample size population.

METHODS AND ANALYSIS

We aim to enrol at least 1000 subjects in META-KLS. All the participants without contraindications will perform multi-modality medical imaging, including brain MRI, retinal fundus photograph, fundus optical coherence tomography (OCT) and ultrasonography of the internal carotid artery (ICA) every 2-4 years. The acquired medical imaging will be further processed with a standardised and validated workflow. The clinical data at baseline and follow-up will be collected from the KaiLuan Study. The associations between multiple risk factors and subclinical brain disease are able to be fully investigated. Researches based on META-KLS will provide a series of state-of-the-art evidence for the prevention of neurological diseases and common chronic diseases.

ETHICS AND DISSEMINATION

The Kailuan Study and META-KLS have been approved by the Medical Ethics Committee of Kailuan General Hospital (IRB number: 2008 No. 1 and 2021002, respectively). Written informed consent will be acquired from each participant. Results are expected to be published in professional peer-reviewed journals beginning in 2023.

TRIAL REGISTRATION NUMBER

NCT05453877.

Review of Serial and Parallel Min-Cut/Max-Flow Algorithms for Computer Vision

Minimum cut/maximum flow (min-cut/max-flow) algorithms solve a variety of problems in computer vision and thus significant effort has been put into developing fast min-cut/max-flow algorithms. As a result, it is difficult to choose an ideal algorithm for a given problem. Furthermore, parallel algorithms have not been thoroughly compared. In this paper, we evaluate the state-of-the-art serial and parallel min-cut/max-flow algorithms on the largest set of computer vision problems yet. We focus on generic algorithms, i.e., for unstructured graphs, but also compare with the specialized GridCut implementation. When applicable, GridCut performs best. Otherwise, the two pseudoflow algorithms, Hochbaum pseudoflow and excesses incremental breadth first search, achieves the overall best performance. The most memory efficient implementation tested is the Boykov-Kolmogorov algorithm. Amongst generic parallel algorithms, we find the bottom-up merging approach by Liu and Sun to be best, but no method is dominant. Of the generic parallel methods, only the parallel preflow push-relabel algorithm is able to efficiently scale with many processors across problem sizes, and no generic parallel method consistently outperforms serial algorithms. Finally, we provide and evaluate strategies for algorithm selection to obtain good expected performance. We make our dataset and implementations publicly available for further research.

Quantitative assessment of depolarization by the retinal pigment epithelium in healthy and glaucoma subjects measured over a large field of view

We present measurements of depolarization introduced by the retinal pigment epithelium (RPE) over a 45° field of view using polarization sensitive optical coherence tomography. A detailed spatial distribution analysis of depolarization caused by the RPE is presented in a total of 153 subjects including both healthy and diseased eyes. Age and sex related differences in the depolarizing character of the RPE are investigated.

A multi-regression framework to improve diagnostic ability of optical coherence tomography retinal biomarkers to discriminate mild cognitive impairment and Alzheimer’s disease

Background

Diagnostic performance of optical coherence tomography (OCT) to detect Alzheimer’s disease (AD) and mild cognitive impairment (MCI) remains limited. We assessed whether compensating the circumpapillary retinal nerve fiber layer (cpRNFL) thickness for multiple demographic and anatomical factors as well as the combination of macular layers improves the detection of MCI and AD.

Methods

This cross-sectional study of 62 AD (n = 92 eyes), 108 MCI (n = 158 eyes), and 55 cognitively normal control (n = 86 eyes) participants. Macular ganglion cell complex (mGCC) thickness was extracted. Circumpapillary retinal nerve fiber layer (cpRNFL) measurement was compensated for several ocular factors. Thickness measurements and their corresponding areas under the receiver operating characteristic curves (AUCs) were compared between the groups. The main outcome measure was OCT thickness measurements.

Results

Participants with MCI/AD showed significantly thinner measured and compensated cpRNFL, mGCC, and altered retinal vessel density (p < 0.05). Compensated RNFL outperformed measured RNFL for discrimination of MCI/AD (AUC = 0.74 vs 0.69; p = 0.026). Combining macular and compensated cpRNFL parameters provided the best detection of MCI/AD (AUC = 0.80 vs 0.69; p < 0.001).

Conclusions and relevance

Accounting for interindividual variations of ocular anatomical features in cpRNFL measurements and incorporating macular information may improve the identification of high-risk individuals with early cognitive impairment.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13195-022-00982-0.

Collective cell migration during optic cup formation features changing cell-matrix interactions linked to matrix topology

Cell migration is crucial for organismal development and shapes organisms in health and disease. Although a lot of research has revealed the role of intracellular components and extracellular signaling in driving single and collective cell migration, the influence of physical properties of the tissue and the environment on migration phenomena in vivo remains less explored. In particular, the role of the extracellular matrix (ECM), which many cells move upon, is currently unclear. To overcome this gap, we use zebrafish optic cup formation, and by combining novel transgenic lines and image analysis pipelines, we study how ECM properties influence cell migration in vivo. We show that collectively migrating rim cells actively move over an immobile extracellular matrix. These cell movements require cryptic lamellipodia that are extended in the direction of migration. Quantitative analysis of matrix properties revealed that the topology of the matrix changes along the migration path. These changes in matrix topologies are accompanied by changes in the dynamics of cell-matrix interactions. Experiments and theoretical modeling suggest that matrix porosity could be linked to efficient migration. Indeed, interfering with matrix topology by increasing its porosity results in a loss of cryptic lamellipodia, less-directed cell-matrix interactions, and overall inefficient migration. Thus, matrix topology is linked to the dynamics of cell-matrix interactions and the efficiency of directed collective rim cell migration during vertebrate optic cup morphogenesis.

Influence of signal‐to‐noise ratio, glaucoma stage and segmentation algorithm on OCT usability for quantifying layer thicknesses in the peripapillary retina

PURPOSE

OCT can be used for glaucoma assessment, but its usefulness may depend on image quality, disease stage and segmentation algorithm. We aimed to determine how layer thicknesses as assessed with different algorithms depend on image quality and disease stage, how reproducible the algorithms are, and if the algorithms (dis)agree.

METHODS

Optic disc OCT data (Canon OCT-HS100) from 20 healthy subjects and 28 early, 29 moderate, and 23 severe glaucoma patients were assessed with four different algorithms (CANON, IOWA, FWHM, DOCTRAP). We measured retinal nerve fibre layer thickness (RNFLT) and total retinal thickness (TRT) along the 1.7-mm-radius OCT measurement circle centred at the optic disc. In healthy subjects, image quality was degraded with neutral density filters (0.3-0.9 optical density [OD]); three scans were made to assess repeatability. Results were analysed with ANOVA with Bonferroni corrected t-tests for post hoc analysis and with intraclass correlation coefficient (ICC) analysis.

RESULTS

For all algorithms, RNFLT was more sensitive to image quality than TRT. Both RNFLT and TRT showed differences between healthy and glaucoma (all algorithms p < 0.001 for both RNFLT and TRT) and between early and moderate glaucoma (RNFLT: p = 0.001 to p = 0.09; TRT: p < 0.001 to p = 0.03); neither was able to discriminate between moderate and severe glaucoma (p = 0.08 to p = 1.0). Generally, repeatability was excellent (ICC >0.75); agreement between algorithms varied from moderate to excellent.

CONCLUSIONS

OCT becomes less informative with glaucoma progression, irrespective of the algorithm. For good-quality scans, RNFLT and TRT perform similarly; TRT may be advantageous with poor image quality.

LOCTseg: A lightweight fully convolutional network for end-to-end optical coherence tomography segmentation

This article presents a novel end-to-end automatic solution for semantic segmentation of optical coherence tomography (OCT) images. OCT is a non-invasive imaging technology widely used in clinical practice due to its ability to acquire high-resolution cross-sectional images of the ocular fundus. Due to the large variability of the retinal structures, OCT segmentation is usually carried out manually and requires expert knowledge. This study introduces a novel fully convolutional network (FCN) architecture designated by LOCTSeg, for end-to-end automatic segmentation of diagnostic markers in OCT b-scans. LOCTSeg is a lightweight deep FCN optimized for balancing performance and efficiency. Unlike state-of-the-art FCNs used in image segmentation, LOCTSeg achieves competitive inference speed without sacrificing segmentation accuracy. The proposed LOCTSeg is evaluated on two publicly available benchmarking datasets: (1) annotated retinal OCT image database (AROI) comprising 1136 images, and (2) healthy controls and multiple sclerosis lesions (HCMS) consisting of 1715 images. Moreover, we evaluated the proposed LOCTSeg with a private dataset of 250 OCT b-scans acquired from epiretinal membrane (ERM) and healthy patients. Results of the evaluation demonstrate empirically the effectiveness of the proposed algorithm, which improves the state-of-the-art Dice score from 69% to 73% and from 91% to 92% on AROI and HCMS datasets, respectively. Furthermore, LOCTSeg outperforms comparable lightweight FCNs' Dice score by margins between 4% and 15% on ERM segmentation.

Extending field-of-view of retinal imaging by optical coherence tomography using convolutional Lissajous and slow scan patterns

Optical coherence tomography (OCT) is a high-speed non-invasive cross-sectional imaging technique. Although its imaging speed is high, three-dimensional high-spatial-sampling-density imaging of in vivo tissues with a wide field-of-view (FOV) is challenging. We employed convolved Lissajous and slow circular scanning patterns to extend the FOV of retinal OCT imaging with a 1-µm, 100-kHz-sweep-rate swept-source OCT prototype system. Displacements of sampling points due to eye movements are corrected by post-processing based on a Lissajous scan. Wide FOV three-dimensional retinal imaging with high sampling density and motion correction is achieved. Three-dimensional structures obtained using repeated imaging sessions of a healthy volunteer and two patients showed good agreement. The demonstrated technique will extend the FOV of simple point-scanning OCT, such as commercial ophthalmic OCT devices, without sacrificing sampling density.

Muscular hydraulics drive larva-polyp morphogenesis

Development is a highly dynamic process in which organisms often experience changes in both form and behavior, which are typically coupled to each other. However, little is known about how organismal-scale behaviors such as body contractility and motility impact morphogenesis. Here, we use the cnidarian Nematostella vectensis as a developmental model to uncover a mechanistic link between organismal size, shape, and behavior. Using quantitative live imaging in a large population of developing animals, combined with molecular and biophysical experiments, we demonstrate that the muscular-hydraulic machinery that controls body movement also drives larva-polyp morphogenesis. We show that organismal size largely depends on cavity inflation through fluid uptake, whereas body shape is constrained by the organization of the muscular system. The generation of ethograms identifies different trajectories of size and shape development in sessile and motile animals, which display distinct patterns of body contractions. With a simple theoretical model, we conceptualize how pressures generated by muscular hydraulics can act as a global mechanical regulator that coordinates tissue remodeling. Altogether, our findings illustrate how organismal contractility and motility behaviors can influence morphogenesis.

A comparison of deep learning U-Net architectures for posterior segment OCT retinal layer segmentation

Deep learning methods have enabled a fast, accurate and automated approach for retinal layer segmentation in posterior segment OCT images. Due to the success of semantic segmentation methods adopting the U-Net, a wide range of variants and improvements have been developed and applied to OCT segmentation. Unfortunately, the relative performance of these methods is difficult to ascertain for OCT retinal layer segmentation due to a lack of comprehensive comparative studies, and a lack of proper matching between networks in previous comparisons, as well as the use of different OCT datasets between studies. In this paper, a detailed and unbiased comparison is performed between eight U-Net architecture variants across four different OCT datasets from a range of different populations, ocular pathologies, acquisition parameters, instruments and segmentation tasks. The U-Net architecture variants evaluated include some which have not been previously explored for OCT segmentation. Using the Dice coefficient to evaluate segmentation performance, minimal differences were noted between most of the tested architectures across the four datasets. Using an extra convolutional layer per pooling block gave a small improvement in segmentation performance for all architectures across all four datasets. This finding highlights the importance of careful architecture comparison (e.g. ensuring networks are matched using an equivalent number of layers) to obtain a true and unbiased performance assessment of fully semantic models. Overall, this study demonstrates that the vanilla U-Net is sufficient for OCT retinal layer segmentation and that state-of-the-art methods and other architectural changes are potentially unnecessary for this particular task, especially given the associated increased complexity and slower speed for the marginal performance gains observed. Given the U-Net model and its variants represent one of the most commonly applied image segmentation methods, the consistent findings across several datasets here are likely to translate to many other OCT datasets and studies. This will provide significant value by saving time and cost in experimentation and model development as well as reduced inference time in practice by selecting simpler models.

Extracting multiple surfaces from 3D microscopy images in complex biological tissues with the Zellige software tool

Background

Efficient tools allowing the extraction of 2D surfaces from 3D-microscopy data are essential for studies aiming to decipher the complex cellular choreography through which epithelium morphogenesis takes place during development. Most existing methods allow for the extraction of a single and smooth manifold of sufficiently high signal intensity and contrast, and usually fail when the surface of interest has a rough topography or when its localization is hampered by other surrounding structures of higher contrast. Multiple surface segmentation entails laborious manual annotations of the various surfaces separately.

Results

As automating this task is critical in studies involving tissue-tissue or tissue-matrix interaction, we developed the Zellige software, which allows the extraction of a non-prescribed number of surfaces of varying inclination, contrast, and texture from a 3D image. The tool requires the adjustment of a small set of control parameters, for which we provide an intuitive interface implemented as a Fiji plugin.

Conclusions

As a proof of principle of the versatility of Zellige, we demonstrate its performance and robustness on synthetic images and on four different types of biological samples, covering a wide range of biological contexts.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12915-022-01378-0.

Retinal layer segmentation in optical coherence tomography (OCT) using a 3D deep-convolutional regression network for patients with age-related macular degeneration

Introduction - Retinal layer segmentation in optical coherence tomography (OCT) images is an important approach for detecting and prognosing disease. Automating segmentation using robust machine learning techniques lead to computationally efficient solutions and significantly reduces the cost of labor-intensive labeling, which is traditionally performed by trained graders at a reading center, sometimes aided by semi-automated algorithms. Although several algorithms have been proposed since the revival of deep learning, eyes with severe pathological conditions continue to challenge fully automated segmentation approaches. There remains an opportunity to leverage the underlying spatial correlations between the retinal surfaces in the segmentation approach. Methods - Some of these proposed traditional methods can be expanded to utilize the three-dimensional spatial context governing the retinal image volumes by replacing the use of 2D filters with 3D filters. Towards this purpose, we propose a spatial-context, continuity and anatomical relationship preserving semantic segmentation algorithm, which utilizes the 3D spatial context from the image volumes with the use of 3D filters. We propose a 3D deep neural network capable of learning the surface positions of the layers in the retinal volumes. Results - We utilize a dataset of OCT images from patients with Age-related Macular Degeneration (AMD) to assess performance of our model and provide both qualitative (including segmentation maps and thickness maps) and quantitative (including error metric comparisons and volumetric comparisons) results, which demonstrate that our proposed method performs favorably even for eyes with pathological changes caused by severe retinal diseases. The Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) for patients with a wide range of AMD severity scores (0-11) were within 0.84±0.41 and 1.33±0.73 pixels, respectively, which are significantly better than some of the other state-of-the-art algorithms. Conclusion - The results demonstrate the utility of extracting features from the entire OCT volume by treating the volume as a correlated entity and show the benefit of utilizing 3D autoencoder based regression networks for smoothing the approximated retinal layers by inducing shape based regularization constraints.

Intraretinal Layer Segmentation Using Cascaded Compressed U-Nets

Reliable biomarkers quantifying neurodegeneration and neuroinflammation in central nervous system disorders such as Multiple Sclerosis, Alzheimer's dementia or Parkinson's disease are an unmet clinical need. Intraretinal layer thicknesses on macular optical coherence tomography (OCT) images are promising noninvasive biomarkers querying neuroretinal structures with near cellular resolution. However, changes are typically subtle, while tissue gradients can be weak, making intraretinal segmentation a challenging task. A robust and efficient method that requires no or minimal manual correction is an unmet need to foster reliable and reproducible research as well as clinical application. Here, we propose and validate a cascaded two-stage network for intraretinal layer segmentation, with both networks being compressed versions of U-Net (CCU-INSEG). The first network is responsible for retinal tissue segmentation from OCT B-scans. The second network segments eight intraretinal layers with high fidelity. At the post-processing stage, we introduce Laplacian-based outlier detection with layer surface hole filling by adaptive non-linear interpolation. Additionally, we propose a weighted version of focal loss to minimize the foreground-background pixel imbalance in the training data. We train our method using 17,458 B-scans from patients with autoimmune optic neuropathies, i.e., multiple sclerosis, and healthy controls. Voxel-wise comparison against manual segmentation produces a mean absolute error of 2.3 μm, outperforming current state-of-the-art methods on the same data set. Voxel-wise comparison against external glaucoma data leads to a mean absolute error of 2.6 μm when using the same gold standard segmentation approach, and 3.7 μm mean absolute error in an externally segmented data set. In scans from patients with severe optic atrophy, 3.5% of B-scan segmentation results were rejected by an experienced grader, whereas this was the case in 41.4% of B-scans segmented with a graph-based reference method. The validation results suggest that the proposed method can robustly segment macular scans from eyes with even severe neuroretinal changes.

Synergistically segmenting choroidal layer and vessel using deep learning for choroid structure analysis

Objective. The choroid is the most vascularized structure in the human eye, whose layer structure and vessel distribution are both critical for the physiology of the retina, and disease pathogenesis of the eye. Although some works have used graph-based methods or convolutional neural networks to separate the choroid layer from the outer-choroid structure, few works focused on further distinguishing the inner-choroid structure, such as the choroid vessel and choroid stroma. Approach. Inspired by the multi-task learning strategy, in this paper, we propose a segmentation pipeline for choroid analysis which can separate the choroid layer from other structures and segment the choroid vessel synergistically. The key component of this pipeline is the proposed choroidal U-shape network (CUNet), which catches both correlation features and specific features between the choroid layer and the choroid vessel. Then pixel-wise classification is generated based on these two types of features to obtain choroid layer segmentation and vessel segmentation. Besides, the training process of CUNet is supervised by a proposed adaptive multi-task segmentation loss which adds a regularization term that is used to balance the performance of the two tasks. Main results. Experiments show the high performance (4% higher dice score) and less computational complexity (18.85 M lower size) of our proposed strategy. Significance. The high performance and generalization on both choroid layer and vessel segmentation indicate the clinical potential of our proposed pipeline.

How AI Can Help in the Diagnostic Dilemma of Pulmonary Nodules

Simple Summary

Pulmonary nodules are considered a sign of bronchogenic carcinoma, detecting them early will reduce their progression and can save lives. Lung cancer is the second most common type of cancer in both men and women. This manuscript discusses the current applications of artificial intelligence (AI) in lung segmentation as well as pulmonary nodule segmentation and classification using computed tomography (CT) scans, published in the last two decades, in addition to the limitations and future prospects in the field of AI.

Abstract

Pulmonary nodules are the precursors of bronchogenic carcinoma, its early detection facilitates early treatment which save a lot of lives. Unfortunately, pulmonary nodule detection and classification are liable to subjective variations with high rate of missing small cancerous lesions which opens the way for implementation of artificial intelligence (AI) and computer aided diagnosis (CAD) systems. The field of deep learning and neural networks is expanding every day with new models designed to overcome diagnostic problems and provide more applicable and simply used models. We aim in this review to briefly discuss the current applications of AI in lung segmentation, pulmonary nodule detection and classification.

Performance of a Machine-Learning Computational Image Analysis Algorithm in Retinal Fluid Quantification for Patients With Diabetic Macular Edema and Retinal Vein Occlusions

BACKGROUND AND OBJECTIVE

The objective is to validate an automated artificial intelligence model in detecting and quantifying fluid in diabetic macular edema (DME) and retinal vein occlusion (RVO) optical coherence tomography images.

PATIENTS AND METHODS

DME (n = 100) and RVO (n = 100) images of adult patients were reviewed. The performance of machine-learning (ML) computational image analysis algorithm was evaluated against consensus manual grading. Main outcomes were accuracy and sensitivity for detection and Pearson's correlation coefficients for quantification.

RESULTS

The ML algorithm had a high accuracy and sensitivity in both DME (intraretinal fluid [IRF]: 0.92, 0.97; subretinal fluid [SRF]: 0.93, 1.00) and RVO (IRF: 0.94, 0.99; SRF: 0.93, 1.00). It had moderate-high correlation in quantifying fluid in DME (total retinal fluid: 0.88; IRF: 0.88; SRF: 0.97) and RVO (total retinal fluid: 0.83; IRF: 0.76; SRF: 0.64).

CONCLUSION

The ML algorithm is highly accurate and sensitive in detecting fluid in DME and RVO optical coherence tomography images and effectively quantifies IRF and SRF in both disease states, particularly in images with low to moderate fluid burden. [Ophthalmic Surg Lasers Imaging Retina. 2022;53:123-131.].

Globally optimal OCT surface segmentation using a constrained IPM optimization

Segmentation of multiple surfaces in optical coherence tomography (OCT) images is a challenging problem, further complicated by the frequent presence of weak boundaries, varying layer thicknesses, and mutual influence between adjacent surfaces. The traditional graph-based optimal surface segmentation method has proven its effectiveness with its ability to capture various surface priors in a uniform graph model. However, its efficacy heavily relies on handcrafted features that are used to define the surface cost for the "goodness" of a surface. Recently, deep learning (DL) is emerging as a powerful tool for medical image segmentation thanks to its superior feature learning capability. Unfortunately, due to the scarcity of training data in medical imaging, it is nontrivial for DL networks to implicitly learn the global structure of the target surfaces, including surface interactions. This study proposes to parameterize the surface cost functions in the graph model and leverage DL to learn those parameters. The multiple optimal surfaces are then simultaneously detected by minimizing the total surface cost while explicitly enforcing the mutual surface interaction constraints. The optimization problem is solved by the primal-dual interior-point method (IPM), which can be implemented by a layer of neural networks, enabling efficient end-to-end training of the whole network. Experiments on spectral-domain optical coherence tomography (SD-OCT) retinal layer segmentation demonstrated promising segmentation results with sub-pixel accuracy.

Segmentation and Quantitative Analysis of Epithelial Tissues

Epithelial tissues regulate exchanges with the environment. They are highly dynamic and can acquire virtually any shape. At the cellular level, they are composed of cells tightly connected by junctions. Most often epithelia are amenable to live imaging; however, the vast number of cells composing an epithelium makes large-scale studies tedious. Here, we present Tissue Analyzer (TA), an open-source tool that can be used to segment epithelia and monitor cell and tissue dynamics.

Automatic 3-D segmentation and volumetric light fluence correction for photoacoustic tomography based on optimal 3-D graph search

Preclinical imaging with photoacoustic tomography (PAT) has attracted wide attention in recent years since it is capable of providing molecular contrast with deep imaging depth. The automatic extraction and segmentation of the animal in PAT images is crucial for improving image analysis efficiency and enabling advanced image post-processing, such as light fluence (LF) correction for quantitative PAT imaging. Previous automatic segmentation methods are mostly two-dimensional approaches, which failed to conserve the 3-D surface continuity because the image slices were processed separately. This discontinuity problem further hampers LF correction, which, ideally, should be carried out in 3-D due to spatially diffused illumination. Here, to solve these problems, we propose a volumetric auto-segmentation method for small animal PAT imaging based on the 3-D optimal graph search (3-D GS) algorithm. The 3-D GS algorithm takes into account the relation among image slices by constructing a 3-D node-weighted directed graph, and thus ensures surface continuity. In view of the characteristics of PAT images, we improve the original 3-D GS algorithm on graph construction, solution guidance and cost assignment, such that the accuracy and smoothness of the segmented animal surface were guaranteed. We tested the performance of the proposed method by conducting in vivo nude mice imaging experiments with a commercial preclinical cross-sectional PAT system. The results showed that our method successfully retained the continuous global surface structure of the whole 3-D animal body, as well as smooth local subcutaneous tumor boundaries at different development stages. Moreover, based on the 3-D segmentation result, we were able to simulate volumetric LF distribution of the entire animal body and obtained LF corrected PAT images with enhanced structural visibility and uniform image intensity.

Retina tissue validation of optical coherence tomography determined outer nuclear layer loss in FTLD-tau

Alzheimer's disease (AD) is associated with inner retina (nerve fiber and ganglion cell layers) thinning. In contrast, we have seen outer retina thinning driven by photoreceptor outer nuclear layer (ONL) thinning with antemortem optical coherence tomography (OCT) among patients considered to have a frontotemporal degeneration tauopathy (FTLD-Tau). Our objective was to determine if postmortem retinal tissue from FTLD-Tau patients demonstrates ONL loss observed antemortem on OCT. Two probable FTLD-Tau patients that were deeply phenotyped by clinical and genetic testing were imaged with OCT and followed to autopsy. Postmortem brain and retinal tissue were evaluated by a neuropathologist and ocular pathologist, respectively, masked to diagnosis. OCT findings were correlated with retinal histology. The two patients had autopsy-confirmed FTLD-Tau neuropathology and had antemortem OCT measurements showing ONL thinning (66.9 μm, patient #1; 74.9 μm, patient #2) below the 95% confidence interval of normal limits (75.1-120.7 μm) in our healthy control cohort. Postmortem, retinal tissue from both patients demonstrated loss of nuclei in the ONL, matching ONL loss visualized on antemortem OCT. Nuclei counts from each area of ONL loss (2 - 3 nuclei per column) seen in patient eyes were below the 95% confidence interval (4 - 8 nuclei per column for ONL) of 3 normal control retinas analyzed at the same location. Our evaluation of retinal tissue from FTLD-Tau patients confirms ONL loss seen antemortem by OCT. Continued investigation of ONL thinning as a biomarker that may distinguish FTLD-Tau from other dementias is warranted.

3D Interactive Segmentation With Semi-Implicit Representation and Active Learning

Segmenting complex 3D geometry is a challenging task due to rich structural details and complex appearance variations of target object. Shape representation and foreground-background delineation are two of the core components of segmentation. Explicit shape models, such as mesh based representations, suffer from poor handling of topological changes. On the other hand, implicit shape models, such as level-set based representations, have limited capacity for interactive manipulation. Fully automatic segmentation for separating foreground objects from background generally utilizes non-interoperable machine learning methods, which heavily rely on the off-line training dataset and are limited to the discrimination power of the chosen model. To address these issues, we propose a novel semi-implicit representation method, namely Non-Uniform Implicit B-spline Surface (NU-IBS), which adaptively distributes parametrically blended patches according to geometrical complexity. Then, a two-stage cascade classifier is introduced to carry out efficient foreground and background delineation, where a simplistic Naïve-Bayesian model is trained for fast background elimination, followed by a stronger pseudo-3D Convolutional Neural Network (CNN) multi-scale classifier to precisely identify the foreground objects. A localized interactive and adaptive segmentation scheme is incorporated to boost the delineation accuracy by utilizing the information iteratively gained from user intervention. The segmentation result is obtained via deforming an NU-IBS according to the probabilistic interpretation of delineated regions, which also imposes a homogeneity constrain for individual segments. The proposed method is evaluated on a 3D cardiovascular Computed Tomography Angiography (CTA) image dataset and Brain Tumor Image Segmentation Benchmark 2015 (BraTS2015) 3D Magnetic Resonance Imaging (MRI) dataset.

Assignment Flow for Order-Constrained OCT Segmentation

At the present time optical coherence tomography (OCT) is among the most commonly used non-invasive imaging methods for the acquisition of large volumetric scans of human retinal tissues and vasculature. The substantial increase of accessible highly resolved 3D samples at the optic nerve head and the macula is directly linked to medical advancements in early detection of eye diseases. To resolve decisive information from extracted OCT volumes and to make it applicable for further diagnostic analysis, the exact measurement of retinal layer thicknesses serves as an essential task be done for each patient separately. However, manual examination of OCT scans is a demanding and time consuming task, which is typically made difficult by the presence of tissue-dependent speckle noise. Therefore, the elaboration of automated segmentation models has become an important task in the field of medical image processing. We propose a novel, purely data driven geometric approach to order-constrained 3D OCT retinal cell layer segmentation which takes as input data in any metric space and can be implemented using only simple, highly parallelizable operations. As opposed to many established retinal layer segmentation methods, we use only locally extracted features as input and do not employ any global shape prior. The physiological order of retinal cell layers and membranes is achieved through the introduction of a smoothed energy term. This is combined with additional regularization of local smoothness to yield highly accurate 3D segmentations. The approach thereby systematically avoid bias pertaining to global shape and is hence suited for the detection of anatomical changes of retinal tissue structure. To demonstrate its robustness, we compare two different choices of features on a data set of manually annotated 3D OCT volumes of healthy human retina. The quality of computed segmentations is compared to the state of the art in automatic retinal layer segmention as well as to manually annotated ground truth data in terms of mean absolute error and Dice similarity coefficient. Visualizations of segmented volumes are also provided.

QUANTITATIVE ANALYSIS OF LONGITUDINAL CHANGES IN MULTIMODAL IMAGING OF LATE-ONSET RETINAL DEGENERATION

PURPOSE

To quantitatively analyze clinically relevant features on longitudinal multimodal imaging of late-onset retinal degeneration to characterize disease progression.

METHODS

Fundus autofluorescence (FAF), infrared reflectance, and optical coherence tomography imaging of 4 patients with late-onset retinal degeneration were acquired over 3 to 15 years (20 visits total). Corresponding regions of interest were analyzed on FAF (reticular pseudodrusen [RPD], "speckled FAF," and chorioretinal atrophy) and infrared reflectance (hyporeflective RPD and target RPD) using quantitative measurements, including contour area, distance to fovea, contour overlap, retinal thickness, and texture features.

RESULTS

Cross-sectional analysis revealed a moderate correlation (RPD FAF ∩ RPD infrared reflectance = 63%) between contour area across modalities. Quantification of retinal thickness and texture analysis of areas contoured on FAF objectively differentiated the contour types. A longitudinal analysis of aligned images demonstrates that the contoured region of atrophy both encroaches toward the fovea and grows monotonically with a rate of 0.531 mm/year to 1.969 mm/year (square root of area, n = 5 eyes). A retrospective analysis of precursor lesions of atrophy reveals quantifiable progression from RPD to speckled FAF to atrophy.

CONCLUSION

Image analysis of time points before the development of atrophy reveals consistent patterns over time and space in late-onset retinal degeneration that may provide useful outcomes for this and other degenerative retinal diseases.

LocalZProjector and DeProj: a toolbox for local 2D projection and accurate morphometrics of large 3D microscopy images

Background

Quantitative imaging of epithelial tissues requires bioimage analysis tools that are widely applicable and accurate. In the case of imaging 3D tissues, a common preprocessing step consists of projecting the acquired 3D volume on a 2D plane mapping the tissue surface. While segmenting the tissue cells is amenable on 2D projections, it is still very difficult and cumbersome in 3D. However, for many specimen and models used in developmental and cell biology, the complex content of the image volume surrounding the epithelium in a tissue often reduces the visibility of the biological object in the projection, compromising its subsequent analysis. In addition, the projection may distort the geometry of the tissue and can lead to strong artifacts in the morphology measurement.

Results

Here we introduce a user-friendly toolbox built to robustly project epithelia on their 2D surface from 3D volumes and to produce accurate morphology measurement corrected for the projection distortion, even for very curved tissues. Our toolbox is built upon two components. LocalZProjector is a configurable Fiji plugin that generates 2D projections and height-maps from potentially large 3D stacks (larger than 40 GB per time-point) by only incorporating signal of the planes with local highest variance/mean intensity, despite a possibly complex image content. DeProj is a MATLAB tool that generates correct morphology measurements by combining the height-map output (such as the one offered by LocalZProjector) and the results of a cell segmentation on the 2D projection, hence effectively deprojecting the 2D segmentation in 3D. In this paper, we demonstrate their effectiveness over a wide range of different biological samples. We then compare its performance and accuracy against similar existing tools.

Conclusions

We find that LocalZProjector performs well even in situations where the volume to project also contains unwanted signal in other layers. We show that it can process large images without a pre-processing step. We study the impact of geometrical distortions on morphological measurements induced by the projection. We measured very large distortions which are then corrected by DeProj, providing accurate outputs.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12915-021-01037-w).

Identification and quantification of fibrotic areas in the human retina using polarization-sensitive OCT

Subretinal fibrosis is one of the most prevalent causes of blindness in the elderly population, but a true gold standard to objectively diagnose fibrosis is still lacking. Since fibrotic tissue is birefringent, it can be detected by polarization-sensitive optical coherence tomography (PS-OCT). We present a new algorithm to automatically detect, segment, and quantify fibrotic lesions within 3D data sets recorded by PS-OCT. The algorithm first compensates for the birefringence of anterior ocular tissues and then uses the uniformity of the birefringent optic axis as an indicator to identify fibrotic tissue, which is then segmented and quantified. The algorithm was applied to 3D volumes recorded in 57 eyes of 57 patients with neovascular age-related macular degeneration using a spectral domain PS-OCT system. The results of fibrosis detection were compared to the clinical diagnosis based on color fundus photography (CFP), and the precision of fibrotic area measurement was assessed by three repeated measurements in a sub-set of 15 eyes. The average standard deviation of the fibrotic area obtained in eyes with a lesion area > 0.7 mm2 was 15%. Fibrosis detection by CFP and PS-OCT agreed in 48 cases, discrepancies were only observed in cases of lesion area < 0.7 mm2. These remaining discrepancies are discussed, and a new method to treat ambiguous cases is presented.

Automatic Segmentation in Multiple OCT Layers For Stargardt Disease Characterization Via Deep Learning

Purpose

This study sought to perform automated segmentation of 11 retinal layers and Stargardt-associated features on spectral-domain optical coherence tomography (SD-OCT) images and to analyze differences between normal eyes and eyes diagnosed with Stargardt disease.

Methods

Automated segmentation was accomplished through application of the deep learning–shortest path (DL-SP) framework, a shortest path segmentation approach that is enhanced by a deep learning fully convolutional neural network. To compare normal eyes and eyes diagnosed with Stargardt disease, various retinal layer thickness and intensity feature maps associated with the outer retinal layers were generated.

Results

The automated DL-SP approach achieved a mean difference within a subpixel accuracy range for all layers when compared to manually traced layers by expert graders. The algorithm achieved mean and absolute mean differences in border positions for Stargardt features of −0.11 ± 4.17 pixels and 1.92 ± 3.71 pixels, respectively. In several of the feature maps generated, the characteristic Stargardt features of flecks and atrophic-appearing lesions were readily visualized.

Conclusions

To the best of our knowledge, this is the first automated algorithm for 11 retinal layer segmentation on OCT in eyes with Stargardt disease, and, furthermore, the feature differences found between eyes diagnosed with Stargardt disease and normal eyes may inform new insights and the better understanding of retinal characteristic morphologic changes caused by Stargardt disease.

Translational Relevance

The automated algorithm's performance and the feature differences found using the algorithm's segmentation support the future applications of SD-OCT for the quantitative monitoring of Stargardt disease.

Automated segmentation of choroidal layers from 3-dimensional macular optical coherence tomography scans

BACKGROUND

Changes in choroidal thickness are associated with various ocular diseases, and the choroid can be imaged using spectral-domain optical coherence tomography (SD-OCT) and enhanced depth imaging OCT (EDI-OCT).

NEW METHOD

Eighty macular SD-OCT volumes from 80 patients were obtained using the Zeiss Cirrus machine. Eleven additional control subjects had two Cirrus scans done in one visit along with enhanced depth imaging (EDI-OCT) using the Heidelberg Spectralis machine. To automatically segment choroidal layers from the OCT volumes, our graph-theoretic approach was utilized. The segmentation results were compared with reference standards from two independent graders, and the accuracy of automated segmentation was calculated using unsigned/signed border positioning/thickness errors and Dice similarity coefficient (DSC). The repeatability and reproducibility of our choroidal thicknesses were determined by intraclass correlation coefficient (ICC), coefficient of variation (CV), and repeatability coefficient (RC).

RESULTS

The mean unsigned/signed border positioning errors for the choroidal inner and outer surfaces are 3.39 ± 1.26 µm (mean ± standard deviation)/- 1.52 ± 1.63 µm and 16.09 ± 6.21 µm/4.73 ± 9.53 µm, respectively. The mean unsigned/signed choroidal thickness errors are 16.54 ± 6.47 µm/6.25 ± 9.91 µm, and the mean DSC is 0.949 ± 0.025. The ICC (95% confidence interval), CV, RC values are 0.991 (0.977-0.997), 2.48%, 14.25 µm for the repeatability and 0.991 (0.977-0.997), 2.49%, 14.30 µm for the reproducibility studies, respectively.

COMPARISON WITH EXISTING METHOD(S)

The proposed method outperformed our previous method using choroidal vessel segmentation and inter-grader variability.

CONCLUSIONS

This automated segmentation method can reliably measure choroidal thickness using different OCT platforms.