Contactless optical coherence tomography of the eyes of freestanding individuals with a robotic scanner

Large area kidney imaging for pre-transplant evaluation using real-time robotic optical coherence tomography

Optical coherence tomography (OCT) can be used to image microstructures of human kidneys. However, current OCT probes exhibit inadequate field-of-view, leading to potentially biased kidney assessment. Here we present a robotic OCT system where the probe is integrated to a robot manipulator, enabling wider area (covers an area of 106.39 mm by 37.70 mm) spatially-resolved imaging. Our system comprehensively scans the kidney surface at the optimal altitude with preoperative path planning and OCT image-based feedback control scheme. It further parameterizes and visualizes microstructures of large area. We verified the system positioning accuracy on a phantom as 0.0762 ± 0.0727 mm and showed the clinical feasibility by scanning ex vivo kidneys. The parameterization reveals vasculatures beneath the kidney surface. Quantification on the proximal convoluted tubule of a human kidney yields clinical-relevant information. The system promises to assess kidney viability for transplantation after collecting a vast amount of whole-organ parameterization and patient outcomes data.

Cost-effectiveness of incorporating self-imaging optical coherence tomography into fundus photography-based diabetic retinopathy screening

Diabetic macular edema (DME) has emerged as the foremost cause of vision loss in the population with diabetes. Early detection of DME is paramount, yet the prevailing screening, relying on two-dimensional and labor-intensive fundus photography (FP), results in frequent unwarranted referrals and overlooked diagnoses. Self-imaging optical coherence tomography (SI-OCT), offering fully automated, three-dimensional macular imaging, holds the potential to enhance DR screening. We conducted an observational study within a cohort of 1822 participants with diabetes, who received comprehensive assessments, including visual acuity testing, FP, and SI-OCT examinations. We compared the performance of three screening strategies: the conventional FP-based strategy, a combination strategy of FP and SI-OCT, and a simulated combination strategy of FP and manual SD-OCT. Additionally, we undertook a cost-effectiveness analysis utilizing Markov models to evaluate the costs and benefits of the three strategies for referable DR. We found that the FP + SI-OCT strategy demonstrated superior sensitivity (87.69% vs 61.53%) and specificity (98.29% vs 92.47%) in detecting DME when compared to the FP-based strategy. Importantly, the FP + SI-OCT strategy outperformed the FP-based strategy, with an incremental cost-effectiveness ratio (ICER) of $8016 per quality-adjusted life year (QALY), while the FP + SD-OCT strategy was less cost-effective, with an ICER of $45,754/QALY. Our results were robust to extensive sensitivity analyses, with the FP + SI-OCT strategy standing as the dominant choice in 69.36% of simulations conducted at the current willingness-to-pay threshold. In summary, incorporating SI-OCT into FP-based screening offers substantial enhancements in sensitivity, specificity for detecting DME, and most notably, cost-effectiveness for DR screening.

Large area robotically assisted optical coherence tomography (LARA-OCT)

We demonstrate large-area robotically assisted optical coherence tomography (LARA-OCT), utilizing a seven-degree-of-freedom robotic arm in conjunction with a 3.3 MHz swept-source OCT to raster scan samples of arbitrary shape. By combining multiple fields of view (FOV), LARA-OCT can probe a much larger area than conventional OCT. Also, nonplanar and curved surfaces like skin on arms and legs can be probed. The lenses in the LARA-OCT scanner with their normal FOV can have fewer aberrations and less complex optics compared to a single wide field design. This may be especially critical for high resolution scans. We directly use our fast MHz-OCT for tracking and stitching, making additional machine vision systems like cameras, positioning, tracking or navigation devices obsolete. This also eliminates the need for complex coordinate system registration between OCT and the machine vision system. We implemented a real time probe-to-surface control that maintains the probe alignment orthogonal to the sample by only using surface information from the OCT images. We present OCT data sets with volume sizes of 140 × 170 × 20 mm3, captured in 2.5 minutes.

Freeform robotic optical coherence tomography beyond the optical field-of-view limit

Imaging complex, non-planar anatomies with optical coherence tomography (OCT) is limited by the optical field of view (FOV) in a single volumetric acquisition. Combining linear mechanical translation with OCT extends the FOV but suffers from inflexibility in imaging non-planar anatomies. We report the freeform robotic OCT to fill this gap. To address challenges in volumetric reconstruction associated with the robotic movement accuracy being two orders of magnitudes worse than OCT imaging resolution, we developed a volumetric registration algorithm based on simultaneous localization and mapping (SLAM) to overcome this limitation. We imaged the entire aqueous humor outflow pathway, whose imaging has the potential to customize glaucoma surgeries but is typically constrained by the FOV, circumferentially in mice as a test. We acquired volumetric OCT data at different robotic poses and reconstructed the entire anterior segment of the eye. The reconstructed volumes showed heterogeneous Schlemm's canal (SC) morphology in the reconstructed anterior segment and revealed a segmental nature in the circumferential distribution of collector channels (CC) with spatial features as small as a few micrometers.

Megahertz multi-parametric ophthalmic OCT system for whole eye imaging

An ultrahigh-speed, wide-field OCT system for the imaging of anterior, posterior, and ocular biometers is crucial for obtaining comprehensive ocular parameters and quantifying ocular pathology size. Here, we demonstrate a multi-parametric ophthalmic OCT system with a speed of up to 1 MHz for wide-field imaging of the retina and 50 kHz for anterior chamber and ocular biometric measurement. A spectrum correction algorithm is proposed to ensure the accurate pairing of adjacent A-lines and elevate the A-scan speed from 500 kHz to 1 MHz for retinal imaging. A registration method employing position feedback signals was introduced, reducing pixel offsets between forward and reverse galvanometer scanning by 2.3 times. Experimental validation on glass sheets and the human eye confirms feasibility and efficacy. Meanwhile, we propose a revised formula to determine the "true" fundus size using all-axial length parameters from different fields of view. The efficient algorithms and compact design enhance system compatibility with clinical requirements, showing promise for widespread commercialization.

Automatic and real-time tissue sensing for autonomous intestinal anastomosis using hybrid MLP-DC-CNN classifier-based optical coherence tomography

Anastomosis is a common and critical part of reconstructive procedures within gastrointestinal, urologic, and gynecologic surgery. The use of autonomous surgical robots such as the smart tissue autonomous robot (STAR) system demonstrates an improved efficiency and consistency of the laparoscopic small bowel anastomosis over the current da Vinci surgical system. However, the STAR workflow requires auxiliary manual monitoring during the suturing procedure to avoid missed or wrong stitches. To eliminate this monitoring task from the operators, we integrated an optical coherence tomography (OCT) fiber sensor with the suture tool and developed an automatic tissue classification algorithm for detecting missed or wrong stitches in real time. The classification results were updated and sent to the control loop of STAR robot in real time. The suture tool was guided to approach the object by a dual-camera system. If the tissue inside the tool jaw was inconsistent with the desired suture pattern, a warning message would be generated. The proposed hybrid multilayer perceptron dual-channel convolutional neural network (MLP-DC-CNN) classification platform can automatically classify eight different abdominal tissue types that require different suture strategies for anastomosis. In MLP, numerous handcrafted features (∼1955) were utilized including optical properties and morphological features of one-dimensional (1D) OCT A-line signals. In DC-CNN, intensity-based features and depth-resolved tissues' attenuation coefficients were fully exploited. A decision fusion technique was applied to leverage the information collected from both classifiers to further increase the accuracy. The algorithm was evaluated on 69,773 testing A-line data. The results showed that our model can classify the 1D OCT signals of small bowels in real time with an accuracy of 90.06%, a precision of 88.34%, and a sensitivity of 87.29%, respectively. The refresh rate of the displayed A-line signals was set as 300 Hz, the maximum sensing depth of the fiber was 3.6 mm, and the running time of the image processing algorithm was ∼1.56 s for 1,024 A-lines. The proposed fully automated tissue sensing model outperformed the single classifier of CNN, MLP, or SVM with optimized architectures, showing the complementarity of different feature sets and network architectures in classifying intestinal OCT A-line signals. It can potentially reduce the manual involvement of robotic laparoscopic surgery, which is a crucial step towards a fully autonomous STAR system.

RobOCTNet: Robotics and Deep Learning for Referable Posterior Segment Pathology Detection in an Emergency Department Population

Purpose

To evaluate the diagnostic performance of a robotically aligned optical coherence tomography (RAOCT) system coupled with a deep learning model in detecting referable posterior segment pathology in OCT images of emergency department patients.

Methods

A deep learning model, RobOCTNet, was trained and internally tested to classify OCT images as referable versus non-referable for ophthalmology consultation. For external testing, emergency department patients with signs or symptoms warranting evaluation of the posterior segment were imaged with RAOCT. RobOCTNet was used to classify the images. Model performance was evaluated against a reference standard based on clinical diagnosis and retina specialist OCT review.

Results

We included 90,250 OCT images for training and 1489 images for internal testing. RobOCTNet achieved an area under the curve (AUC) of 1.00 (95% confidence interval [CI], 0.99-1.00) for detection of referable posterior segment pathology in the internal test set. For external testing, RAOCT was used to image 72 eyes of 38 emergency department patients. In this set, RobOCTNet had an AUC of 0.91 (95% CI, 0.82-0.97), a sensitivity of 95% (95% CI, 87%-100%), and a specificity of 76% (95% CI, 62%-91%). The model's performance was comparable to two human experts' performance.

Conclusions

A robotically aligned OCT coupled with a deep learning model demonstrated high diagnostic performance in detecting referable posterior segment pathology in a cohort of emergency department patients.

Translational Relevance

Robotically aligned OCT coupled with a deep learning model may have the potential to improve emergency department patient triage for ophthalmology referral.

Evaluation of peripapillary retinal nerve fiber layer thickness in intracranial atherosclerotic stenosis

PURPOSE

To evaluate the peripapillary retinal nerve fiber layer thickness (pRNFL) in patients with intracranial atherosclerotic stenosis (ICAS).

METHODS

A cross-sectional study was performed in a general hospital. The intracranial atherosclerotic stenosis was evaluated by digital subtraction angiography (DSA), computed tomography angiography (CTA) or magnetic resonance angiography (MRA). High-definition optical coherence tomography (HD-OCT) was used to evaluate the peripapillary retinal nerve fiber layer thickness.

RESULTS

A total of 102 patients, including 59(57.8%) patients with ICAS and 43(42.2%) patients without ICAS, were finally analysed in the study. The peripapillary retinal nerve fiber layer thickness (pRNFL) was reduced significantly in the average, the superior and the inferior quadrants of the ipsilateral eyes and in the superior quadrant of the contralateral eyes in patients with ICAS compared with patients without ICAS. After multivariate analysis, only the superior pRNFL thickness in the ipsilateral eyes was significantly associated with ICAS (OR,0.968; 95% CI,0.946-0.991; p = 0.006). The area under receiver operator curve was 0.679 (95% CI,0.576-0.782) for it to identify the presence of ICAS. The cut-off value of the superior pRNFL was 109.5 μm, and the sensitivity and specificity were 50.8% and 83.7%, respectively.

CONCLUSION

The superior pRNFL in the ipsilateral eye was significantly associated with ICAS in this study. Larger studies are needed to explore the relation between pRNFL and ICAS further.

Large Area Kidney Imaging for Pre-transplant Evaluation using Real-Time Robotic Optical Coherence Tomography

Optical coherence tomography (OCT) is a high-resolution imaging modality that can be used to image microstructures of human kidneys. These images can be analyzed to evaluate the viability of the organ for transplantation. However, current OCT devices suffer from insufficient field-of-view, leading to biased examination outcomes when only small portions of the kidney can be assessed. Here we present a robotic OCT system where an OCT probe is integrated with a robotic manipulator, enabling wider area spatially-resolved imaging. With the proposed system, it becomes possible to comprehensively scan the kidney surface and provide large area parameterization of the microstructures. We verified the probe tracking accuracy with a phantom as 0.0762±0.0727 mm and demonstrated its clinical feasibility by scanning ex vivo kidneys. The parametric map exhibits fine vasculatures beneath the kidney surface. Quantitative analysis on the proximal convoluted tubule from the ex vivo human kidney yields highly clinical-relevant information.

Painting unknown worlds

This paper sets out to discover more about the name 'Tarrant' whose ophthalmic paintings have regularly featured in ophthalmic textbooks over the past 50 years. Through a series of telephone calls, I have spoken to Tarrant about his life and work while I research the origins of ophthalmic illustrations charting the story behind this art movement. The paper also explores the eventual decline of retinal painting and the emergence of photography, concluding that due to the continuing advance of technology the ophthalmic photographer may eventually succumb to the same fate as the artist.

Robotic-OCT guided inspection and microsurgery of monolithic storage devices

Data recovery from monolithic storage devices (MSDs) is in high demand for legal or business purposes. However, the conventional data recovery methods are destructive, complicated, and time-consuming. We develop a robotic-arm-assisted optical coherence tomography (robotic-OCT) for non-destructive inspection of MSDs, offering ~7 μm lateral resolution, ~4 μm axial resolution and an adjustable field-of-view to accommodate various MSD sizes. Using a continuous scanning strategy, robotic-OCT achieves automated volumetric imaging of a micro-SD card in ~37 seconds, significantly faster than the traditional stop-and-stare scanning that typically takes tens of minutes. We also demonstrate the robotic-OCT-guided laser ablation as a microsurgical tool for targeted area removal with precision of ±10 μm and accuracy of ~50 μm, eliminating the need to remove the entire insulating layer and operator intervention, thus greatly improving the data recovery efficiency. This work has diverse potential applications in digital forensics, failure analysis, materials testing, and quality control.

Annotation-efficient learning for OCT segmentation

Deep learning has been successfully applied to OCT segmentation. However, for data from different manufacturers and imaging protocols, and for different regions of interest (ROIs), it requires laborious and time-consuming data annotation and training, which is undesirable in many scenarios, such as surgical navigation and multi-center clinical trials. Here we propose an annotation-efficient learning method for OCT segmentation that could significantly reduce annotation costs. Leveraging self-supervised generative learning, we train a Transformer-based model to learn the OCT imagery. Then we connect the trained Transformer-based encoder to a CNN-based decoder, to learn the dense pixel-wise prediction in OCT segmentation. These training phases use open-access data and thus incur no annotation costs, and the pre-trained model can be adapted to different data and ROIs without re-training. Based on the greedy approximation for the k-center problem, we also introduce an algorithm for the selective annotation of the target data. We verified our method on publicly-available and private OCT datasets. Compared to the widely-used U-Net model with 100% training data, our method only requires ∼10% of the data for achieving the same segmentation accuracy, and it speeds the training up to ∼3.5 times. Furthermore, our proposed method outperforms other potential strategies that could improve annotation efficiency. We think this emphasis on learning efficiency may help improve the intelligence and application penetration of OCT-based technologies.

Robotic Optical Coherence Tomography Retinal Imaging for Emergency Department Patients: A Pilot Study for Emergency Physicians' Diagnostic Performance

STUDY OBJECTIVE

To evaluate the diagnostic performance of emergency physicians' interpretation of robotically acquired retinal optical coherence tomography images for detecting posterior eye abnormalities in patients seen in the emergency department (ED).

METHODS

Adult patients presenting to Duke University Hospital emergency department from November 2020 through October 2021 with acute visual changes, headache, or focal neurologic deficit(s) who received an ophthalmology consultation were enrolled in this pilot study. Emergency physicians provided standard clinical care, including direct ophthalmoscopy, at their discretion. Retinal optical coherence tomography images of these patients were obtained with a robotic, semi-autonomous optical coherence tomography system. We compared the detection of abnormalities in optical coherence tomography images by emergency physicians with a reference standard, a combination of ophthalmology consultation diagnosis and retina specialist optical coherence tomography review.

RESULTS

Nine emergency physicians reviewed the optical coherence tomography images of 72 eyes from 38 patients. Based on the reference standard, 33 (46%) eyes were normal, 16 (22%) had at least 1 urgent/emergency abnormality, and the remaining 23 (32%) had at least 1 nonurgent abnormality. Emergency physicians' optical coherence tomography interpretation had 69% (95% confidence interval [CI], 49% to 89%) sensitivity for any abnormality, 100% (95% CI, 79% to 100%) sensitivity for urgent/emergency abnormalities, 48% (95% CI, 28% to 68%) sensitivity for nonurgent abnormalities, and 64% (95% CI, 44% to 84%) overall specificity. In contrast, emergency physicians providing standard clinical care did not detect any abnormality with direct ophthalmoscopy.

CONCLUSION

Robotic, semi-autonomous optical coherence tomography enabled ocular imaging of emergency department patients with a broad range of posterior eye abnormalities. In addition, emergency provider optical coherence tomography interpretation was more sensitive than direct ophthalmoscopy for any abnormalities, urgent/emergency abnormalities, and nonurgent abnormalities in this pilot study with a small sample of patients and emergency physicians.

Robotic Optical Coherence Tomography of Human Subjects with Posture-Invariant Head and Eye Alignment in Six Degrees of Freedom

Ophthalmic optical coherence tomography (OCT) has achieved remarkable clinical success but remains sequestered in ophthalmology specialty offices. Recently introduced robotic OCT systems seek to expand patient access but fall short of their full potential due to significant imaging workspace and motion planning restrictions. Here, we present a next-generation robotic OCT system capable of imaging in any head orientation or posture that is mechanically reachable. This system overcomes prior restrictions by eliminating fixed-base tracking components, extending robot reach, and planning alignment in six degrees of freedom. With this robotic system, we show repeatable subject imaging independent of posture (standing, seated, reclined, and supine) under widely varying head orientations for multiple human subjects. For each subject, we obtained a consistent view of the retina, including the fovea, retinal vasculature, and edge of the optic nerve head. We believe this robotic approach can extend OCT as an eye disease screening, diagnosis, and monitoring tool to previously unreached patient populations.

Contactless, autonomous robotic alignment of optical coherence tomography for in vivo evaluation of diseased retinas

During the COVID-19 pandemic, an emphasis was placed on contactless, physical distancing and improved telehealth; contrariwise, standard-of-care ophthalmic imaging of patients required present, trained personnel. Here, we introduce contactless, autonomous robotic alignment of optical coherence tomography (RAOCT) for in vivo imaging of retinal disease and compare measured retinal thickness and diagnostic readability to technician operated clinical OCT. In a powered study, we found no statistically significant difference in retinal thickness in both healthy and diseased retinas (p > 0.7) or across a variety of demographics (gender, race, and age) between RAOCT and clinical OCT. In a secondary study, a retina specialist labeled a given volume as normal/abnormal. Compared to the clinical diagnostic label, sensitivity/specificity for RAOCT were equal or improved over clinical OCT. Contactless, autonomous RAOCT, that improves upon current clinical OCT, could play a role in both ophthalmic care and non-ophthalmic settings that would benefit from improved eye care.

Volumetric directional optical coherence tomography

Photoreceptor loss and resultant thinning of the outer nuclear layer (ONL) is an important pathological feature of retinal degenerations and may serve as a useful imaging biomarker for age-related macular degeneration. However, the demarcation between the ONL and the adjacent Henle's fiber layer (HFL) is difficult to visualize with standard optical coherence tomography (OCT). A dedicated OCT system that can precisely control and continuously and synchronously update the imaging beam entry points during scanning has not been realized yet. In this paper, we introduce a novel imaging technology, Volumetric Directional OCT (VD-OCT), which can dynamically adjust the incident beam on the pupil without manual adjustment during a volumetric OCT scan. We also implement a customized spoke-circular scanning pattern to observe the appearance of HFL with sufficient optical contrast in continuous cross-sectional scans through the entire volume. The application of VD-OCT for retinal imaging to exploit directional reflectivity properties of tissue layers has the potential to allow for early identification of retinal diseases.

Robotically aligned optical coherence tomography with 5 degree of freedom eye tracking for subject motion and gaze compensation

Optical coherence tomography (OCT) has revolutionized diagnostics in ophthalmology. However, OCT requires a trained operator and patient cooperation to carefully align a scanner with the subject's eye and orient it in such a way that it images a desired region of interest at the retina. With the goal of automating this process of orienting and aligning the scanner, we developed a robot-mounted OCT scanner that automatically aligned with the pupil while matching its optical axis with the target region of interest at the retina. The system used two 3D cameras for face tracking and three high-resolution 2D cameras for pupil and gaze tracking. The tracking software identified 5 degrees of freedom for robot alignment and ray aiming through the ocular pupil: 3 degrees of translation (x, y, z) and 2 degrees of orientation (yaw, pitch). We evaluated the accuracy, precision, and range of our tracking system and demonstrated imaging performance on free-standing human subjects. Our results demonstrate that the system stabilized images and that the addition of gaze tracking and aiming allowed for region-of-interest specific alignment at any gaze orientation within a 28° range.