1
|
Jiang Z, Wen Y, Song L, Li D, Zhao X. Optical fiber bundle differential compressive imaging. OPTICS LETTERS 2024; 49:2297-2300. [PMID: 38691703 DOI: 10.1364/ol.519161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Accepted: 03/31/2024] [Indexed: 05/03/2024]
Abstract
We present a differential compressive imaging method for an optical fiber bundle (OFB), which provides a solution for an ultrathin bend-resistant endoscope with high resolution. This method uses an OFB and a diffuser to generate speckle illumination patterns. Differential operation is additionally applied to the speckle patterns to produce sensing matrices, by which the correlation between the matrices is greatly reduced from 0.875 to 0.0275, which ensures the high quality of image reconstruction. Pixilation artifacts from the fiber core arrangement are also effectively eliminated with this configuration. We demonstrate high-resolution reconstruction of images of 132 × 132 pixels with a compression rate of 12% using 77 fiber cores, the total diameter of which is only about 91 µm. An experimental verification proves that this method is tolerant to a limited degree of fiber bending, which provides a potential approach for robust high-resolution fiber endoscopy.
Collapse
|
2
|
Hughes MR. Real-timing processing of fiber bundle endomicroscopy images in Python using PyFibreBundle. APPLIED OPTICS 2023; 62:9041-9050. [PMID: 38108740 DOI: 10.1364/ao.503700] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 10/30/2023] [Indexed: 12/19/2023]
Abstract
Fiber imaging bundles allow the transfer of optical images from place-to-place along narrow and flexible conduits. Traditionally used extensively in medical endoscopy, bundles are now finding new applications in endoscopic microscopy and other emerging techniques. PyFibreBundle is an open-source Python package for fast processing of images acquired through imaging bundles. This includes detection and removal of the fiber core pattern by filtering or interpolation, and application of background and flat-field corrections. It also allows images to be stitched together to create mosaics and resolution to be improved by combining multiple shifted images. This paper describes the technical implementation of PyFibreBundle and provides example results from three endomicroscopy imaging systems: color transmission, monochrome transmission, and confocal fluorescence. This allows various processing options to be compared quantitatively and qualitatively, and benchmarking demonstrates that PyFibreBundle can achieve state-of-the-art performance in an open-source package. The paper demonstrates core removal by interpolation and mosaicing at over 100 fps, real-time multi-frame resolution enhancement and the first demonstration of real-time endomicroscopy image processing, including core removal, on a Raspberry Pi single board computer. This demonstrates that PyFibreBundle is potentially a valuable tool for the development of low-cost, high-performance fiber bundle imaging systems.
Collapse
|
3
|
Jiang Z, Zhao X, Wen Y, Peng Q, Li D, Song L. Block-based compressed sensing for fast optic fiber bundle imaging with high spatial resolution. OPTICS EXPRESS 2023; 31:17235-17249. [PMID: 37381463 DOI: 10.1364/oe.488171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Accepted: 04/23/2023] [Indexed: 06/30/2023]
Abstract
The resolution of traditional fiber bundle imaging is usually limited by the density and the diameter of the fiber cores. To improve the resolution, compression sensing was introduced to resolve multiple pixels from a single fiber core, but current methods have the drawbacks of excessive sampling and long reconstruction time. In this paper, we present, what we believe to be, a novel block-based compressed sensing scheme for fast realization of high-resolution optic fiber bundle imaging. In this method, the target image is segmented into multiple small blocks, each of which covers the projection area of one fiber core. All block images are independently and simultaneously sampled and the intensities are recorded by a two-dimensional detector after they are collected and transmitted through corresponding fiber cores. Because the size of sampling patterns and the sampling numbers are greatly reduced, the reconstruction complexity and reconstruction time are also decreased. According to the simulation analysis, our method is 23 times faster than the current compressed sensing optical fiber imaging for reconstructing a fiber image of 128 × 128 pixels, while the sampling number is only 0.39%. Experiment results demonstrate that the method is also effective for reconstructing large target images and the number of sampling does not increase with the size of the image. Our finding may provide a new idea for high-resolution real-time imaging of fiber bundle endoscope.
Collapse
|
4
|
Xu X, Tang R, Zuo S. A novel OCT servoing control method for endomicroscopy surface scanning. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104719] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2023]
|
5
|
Yao B, Huang B, Li X, Qi J, Li Y, Shao Y, Qu J, Gu Y, Li J. Depixelation and image restoration with meta-learning in fiber-bundle-based endomicroscopy. OPTICS EXPRESS 2022; 30:5038-5050. [PMID: 35209475 DOI: 10.1364/oe.447495] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Accepted: 01/18/2022] [Indexed: 05/25/2023]
Abstract
In order to efficiently remove honeycomb artifacts and restore images in fiber-bundle-based endomicroscopy, we develop a meta-learning algorithm in this work. Two sub-networks are used to extract different levels of features. Meta-training is employed to train the network with small amount of simulated training data, enabling the optimal model to generalize to new tasks not seen in the training set. Numerical results on both USAF target and endomicroscopy images of living mice tissues demonstrate that the algorithm can restore high contrast image without pixilated noise using shorter time. Additionally, no prior information on the shape of the underlying tissues and the distribution of fiber bundles is required, making the method applicable to a variety of fiber-bundle-based endomicroscopy imaging conditions.
Collapse
|
6
|
Li Q, Rohringer W, Preißer S, Erkkilä MT, Haindl R, Sattmann H, Liu M, Fischer B, Leitgeb R, Drexler W. Depixelation of coherent fiber bundle imaging by fiber-core-targeted scanning. APPLIED OPTICS 2021; 60:7955-7962. [PMID: 34613055 DOI: 10.1364/ao.430537] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/05/2021] [Accepted: 08/09/2021] [Indexed: 06/13/2023]
Abstract
A novel fast proximal scanning method, to the best of our knowledge, termed fiber-core-targeted scanning (FCTS), is proposed for illuminating individual fiber cores sequentially to remove the pixelation effect in fiber bundle (FB) imaging. FCTS is based on a galvanometer scanning system. Through a dynamic control of the scan trajectory and speed using the prior knowledge of fiber core positions, FCTS experimentally verifies a precise sequential delivery of laser pulses into fiber cores at a maximal speed of 45,000 cores per second. By applying FCTS on a FB-based photoacoustic forward-imaging probe, the results demonstrate that FCTS eliminates the pixelation effect and improves the imaging quality.
Collapse
|
7
|
Xu X, Tang R, Gong L, Chen B, Zuo S. Two Dimensional Position-Based Visual Servoing for Soft Tissue Endomicroscopy. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3084885] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
|
8
|
Hunt B, Coole J, Brenes D, Kortum A, Mitbander R, Vohra I, Carns J, Schwarz R, Richards-Kortum R. High frame rate video mosaicking microendoscope to image large regions of intact tissue with subcellular resolution. BIOMEDICAL OPTICS EXPRESS 2021; 12:2800-2812. [PMID: 34123505 PMCID: PMC8176790 DOI: 10.1364/boe.425527] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Revised: 03/26/2021] [Accepted: 03/29/2021] [Indexed: 06/12/2023]
Abstract
High-resolution microendoscopy (HRME) is a low-cost strategy to acquire images of intact tissue with subcellular resolution at frame rates ranging from 11 to 18 fps. Current HRME imaging strategies are limited by the small microendoscope field of view (∼0.5 mm2); multiple images must be acquired and reliably registered to assess large regions of clinical interest. Image mosaics have been assembled from co-registered frames of video acquired as a microendoscope is slowly moved across the tissue surface, but the slow frame rate of previous HRME systems made this approach impractical for acquiring quality mosaicked images from large regions of interest. Here, we present a novel video mosaicking microendoscope incorporating a high frame rate CMOS sensor and optical probe holder to enable high-speed, high quality interrogation of large tissue regions of interest. Microendoscopy videos acquired at >90 fps are assembled into an image mosaic. We assessed registration accuracy and image sharpness across the mosaic for images acquired with a handheld probe over a range of translational speeds. This high frame rate video mosaicking microendoscope enables in vivo probe translation at >15 millimeters per second while preserving high image quality and accurate mosaicking, increasing the size of the region of interest that can be interrogated at high resolution from 0.5 mm2 to >30 mm2. Real-time deployment of this high-frame rate system is demonstrated in vivo and source code made publicly available.
Collapse
Affiliation(s)
- Brady Hunt
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77025, USA
| | - Jackson Coole
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77025, USA
| | - David Brenes
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77025, USA
| | - Alex Kortum
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77025, USA
| | - Ruchika Mitbander
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77025, USA
| | - Imran Vohra
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77025, USA
| | - Jennifer Carns
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77025, USA
| | - Richard Schwarz
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77025, USA
| | | |
Collapse
|
9
|
Laparoscopic surgical device with modular end tools for real-time endomicroscopy and therapy. Med Biol Eng Comput 2021; 59:787-797. [PMID: 33677776 DOI: 10.1007/s11517-021-02341-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2020] [Accepted: 02/23/2021] [Indexed: 10/22/2022]
Abstract
In recent years, laparoscopic surgery attracts considerable attention for its distinct advantages owing to minimized invasiveness. However, traditional surgical instruments have a single function and lack the ability to provide high-resolution images for real-time tissue assessment. Thus, it remains a challenge to perform complex procedures with the traditional surgical instruments. In this paper, a novel laparoscopic surgical device for real-time endomicroscopy and therapy is proposed. The device is integrated with a visualization unit to provide endoscopic images in a constrained cavity for real-time observation. Moreover, modular end tools can be assembled and switched with the multi-DOF manipulator. The experimental results demonstrate that the surface-scanning tool can perform smooth scanning and obtain a large area mosaic of 6.99 mm2 in 40.2 s for tissue assessment. The spring-loaded forceps can generate an output force of 1.47 N to manipulate tissue. These experimental results demonstrate the potential value of the device for intraoperative endomicroscopy-guided surgery and robot-assisted surgery.
Collapse
|
10
|
Wieslander H, Wählby C, Sintorn IM. TEM image restoration from fast image streams. PLoS One 2021; 16:e0246336. [PMID: 33524053 PMCID: PMC7850474 DOI: 10.1371/journal.pone.0246336] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2020] [Accepted: 01/16/2021] [Indexed: 11/19/2022] Open
Abstract
Microscopy imaging experiments generate vast amounts of data, and there is a high demand for smart acquisition and analysis methods. This is especially true for transmission electron microscopy (TEM) where terabytes of data are produced if imaging a full sample at high resolution, and analysis can take several hours. One way to tackle this issue is to collect a continuous stream of low resolution images whilst moving the sample under the microscope, and thereafter use this data to find the parts of the sample deemed most valuable for high-resolution imaging. However, such image streams are degraded by both motion blur and noise. Building on deep learning based approaches developed for deblurring videos of natural scenes we explore the opportunities and limitations of deblurring and denoising images captured from a fast image stream collected by a TEM microscope. We start from existing neural network architectures and make adjustments of convolution blocks and loss functions to better fit TEM data. We present deblurring results on two real datasets of images of kidney tissue and a calibration grid. Both datasets consist of low quality images from a fast image stream captured by moving the sample under the microscope, and the corresponding high quality images of the same region, captured after stopping the movement at each position to let all motion settle. We also explore the generalizability and overfitting on real and synthetically generated data. The quality of the restored images, evaluated both quantitatively and visually, show that using deep learning for image restoration of TEM live image streams has great potential but also comes with some limitations.
Collapse
Affiliation(s)
- Håkan Wieslander
- Department of Information Technology, Uppsala University, Uppsala, Sweden
| | - Carolina Wählby
- Department of Information Technology, Uppsala University, Uppsala, Sweden
- BioImage Informatics Facility of SciLifeLab, Uppsala, Sweden
| | - Ida-Maria Sintorn
- Department of Information Technology, Uppsala University, Uppsala, Sweden
- Vironova AB, Stockholm, Sweden
| |
Collapse
|
11
|
Gong L, Zheng J, Ping Z, Wang Y, Wang S, Zuo S. Robust Mosaicing of Endomicroscopic Videos via Context-Weighted Correlation Ratio. IEEE Trans Biomed Eng 2021; 68:579-591. [PMID: 32746056 DOI: 10.1109/tbme.2020.3007768] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2022]
Abstract
Probe-based confocal laser endomicroscopy (pCLE) is a promising imaging tool that provides in situ and in vivo optical imaging to perform real-time pathological assessments. However, due to limited field of view, it is difficult for clinicians to get a full understanding of the scanned tissues. In this paper, we develop a novel mosaicing framework to assemble all frame sequences into a full view image. First, a hybrid rigid registration that combines feature matching and template matching is presented to achieve a global alignment of all frames. Then, the parametric free-form deformation (FFD) model with a multiresolution architecture is implemented to accommodate non-rigid tissue distortions. More importantly, we devise a robust similarity metric called context-weighted correlation ratio (CWCR) to promote registration accuracy, where spatial and geometric contexts are incorporated into the estimation of functional intensity dependence. Experiments on both robotic setup and manual manipulation have demonstrated that the proposed scheme significantly precedes some state-of-the-art mosaicing schemes in the presence of intensity fluctuations, insufficient overlap and tissue distortions. Moreover, the comparisons of the proposed CWCR metric and two other metrics have validated the effectiveness of the context-weighted strategy in quantifying the differences between two frames. Benefiting from more rational and delicate mosaics, the proposed scheme is more suitable to instruct diagnosis and treatment during optical biopsies.
Collapse
|
12
|
Aktar R, Huxley VH, Guidoboni G, AliAkbarpour H, Bunyak F, Palaniappan K. MOSAICING OF DYNAMIC MESENTERY VIDEO WITH GRADIENT BLENDING. PROCEEDINGS. INTERNATIONAL CONFERENCE ON IMAGE PROCESSING 2020; 2020:563-567. [PMID: 35656332 PMCID: PMC9159528 DOI: 10.1109/icip40778.2020.9191045] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/24/2023]
Abstract
In biomedical imaging using video microscopy, understanding large tissue structures at cellular and finer resolution poses many image acquisition challenges including limited field-of-view and tissue dynamics during imaging. Automated mosaicing or stitching of live tissue video microscopy enables the visualization and analysis of subtle morphological structures and large scale vessel network architecture in tissues like the mesentery. But mosacing can be challenging if there are deformable, motion-blurred, textureless, feature-poor frames. Feature-based methods perform poorly in such cases for the lack of distinctive keypoints. Standard single block correlation matching strategies might not provide robust registration due to deformable content. In addition, the panorama suffers if there is motion blur present in a sequence. To handle these challenges, we propose a novel algorithm, Deformable Normalized Cross Correlation (DNCC) image matching with RANSAC to establish robust registration. Besides, to produce seamless panorama from motion-blurred frames we present gradient blending method based on image edge information. The DNCC algorithm is applied on Frog Mesentery sequences. Our result is compared with PSS/AutoStitch [1, 2] to establish the efficiency and robustness of the proposed DNCC method.
Collapse
Affiliation(s)
- Rumana Aktar
- Department of Electrical Engineering and Computer Science
| | - V H Huxley
- Department of Medical Pharmacology and Physiology, University of Missouri-Columbia, Columbia, MO-65211, USA
| | - G Guidoboni
- Department of Electrical Engineering and Computer Science
| | - H AliAkbarpour
- Department of Electrical Engineering and Computer Science
| | - F Bunyak
- Department of Electrical Engineering and Computer Science
| | - K Palaniappan
- Department of Electrical Engineering and Computer Science
| |
Collapse
|
13
|
Yin C, Wei L, Kose K, Glaser AK, Peterson G, Rajadhyaksha M, Liu JT. Real-time video mosaicking to guide handheld in vivo microscopy. JOURNAL OF BIOPHOTONICS 2020; 13:e202000048. [PMID: 32246558 PMCID: PMC7969124 DOI: 10.1002/jbio.202000048] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/13/2020] [Revised: 03/16/2020] [Accepted: 03/24/2020] [Indexed: 05/05/2023]
Abstract
Handheld and endoscopic optical-sectioning microscopes are being developed for noninvasive screening and intraoperative consultation. Imaging a large extent of tissue is often desired, but miniature in vivo microscopes tend to suffer from limited fields of view. To extend the imaging field during clinical use, we have developed a real-time video mosaicking method, which allows users to efficiently survey larger areas of tissue. Here, we modified a previous post-processing mosaicking method so that real-time mosaicking is possible at >30 frames/second when using a device that outputs images that are 400 × 400 pixels in size. Unlike other real-time mosaicking methods, our strategy can accommodate image rotations and deformations that often occur during clinical use of a handheld microscope. We perform a feasibility study to demonstrate that the use of real-time mosaicking is necessary to enable efficient sampling of a desired imaging field when using a handheld dual-axis confocal microscope.
Collapse
Affiliation(s)
- Chengbo Yin
- University of Washington, Department of Mechanical Engineering, Seattle, WA, 98195, USA
| | - Linpeng Wei
- University of Washington, Department of Mechanical Engineering, Seattle, WA, 98195, USA
| | - Kivanc Kose
- Memorial Sloan-Kettering Cancer Center, Dermatology Service, New York, NY, 10021, USA
| | - Adam K. Glaser
- University of Washington, Department of Mechanical Engineering, Seattle, WA, 98195, USA
| | - Gary Peterson
- Memorial Sloan-Kettering Cancer Center, Dermatology Service, New York, NY, 10021, USA
| | - Milind Rajadhyaksha
- Memorial Sloan-Kettering Cancer Center, Dermatology Service, New York, NY, 10021, USA
| | - Jonathan T.C. Liu
- University of Washington, Department of Mechanical Engineering, Seattle, WA, 98195, USA
- University of Washington School of Medicine, Department of Pathology, Seattle, WA 98195, USA
- University of Washington, Department of Bioengineering, Seattle, WA 98195, USA
| |
Collapse
|
14
|
Perperidis A, Dhaliwal K, McLaughlin S, Vercauteren T. Image computing for fibre-bundle endomicroscopy: A review. Med Image Anal 2020; 62:101620. [PMID: 32279053 PMCID: PMC7611433 DOI: 10.1016/j.media.2019.101620] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2018] [Accepted: 11/18/2019] [Indexed: 12/12/2022]
Abstract
Endomicroscopy is an emerging imaging modality, that facilitates the acquisition of in vivo, in situ optical biopsies, assisting diagnostic and potentially therapeutic interventions. While there is a diverse and constantly expanding range of commercial and experimental optical biopsy platforms available, fibre-bundle endomicroscopy is currently the most widely used platform and is approved for clinical use in a range of clinical indications. Miniaturised, flexible fibre-bundles, guided through the working channel of endoscopes, needles and catheters, enable high-resolution imaging across a variety of organ systems. Yet, the nature of image acquisition though a fibre-bundle gives rise to several inherent characteristics and limitations necessitating novel and effective image pre- and post-processing algorithms, ranging from image formation, enhancement and mosaicing to pathology detection and quantification. This paper introduces the underlying technology and most prevalent clinical applications of fibre-bundle endomicroscopy, and provides a comprehensive, up-to-date, review of relevant image reconstruction, analysis and understanding/inference methodologies. Furthermore, current limitations as well as future challenges and opportunities in fibre-bundle endomicroscopy computing are identified and discussed.
Collapse
Affiliation(s)
- Antonios Perperidis
- Institute of Sensors, Signals and Systems (ISSS), Heriot Watt University, EH14 4AS, UK; EPSRC IRC "Hub" in Optical Molecular Sensing & Imaging, MRC Centre for Inflammation Research, Queen's Medical Research Institute (QMRI), University of Edinburgh, EH16 4TJ, UK.
| | - Kevin Dhaliwal
- EPSRC IRC "Hub" in Optical Molecular Sensing & Imaging, MRC Centre for Inflammation Research, Queen's Medical Research Institute (QMRI), University of Edinburgh, EH16 4TJ, UK.
| | - Stephen McLaughlin
- Institute of Sensors, Signals and Systems (ISSS), Heriot Watt University, EH14 4AS, UK.
| | - Tom Vercauteren
- School of Biomedical Engineering and Imaging Sciences, King's College London, WC2R 2LS, UK.
| |
Collapse
|
15
|
Li K, Yang Z, Liang W, Shang J, Liang Y, Wan S. Low-cost, ultracompact handheld optical coherence tomography probe for in vivo oral maxillofacial tissue imaging. JOURNAL OF BIOMEDICAL OPTICS 2020; 25:1-13. [PMID: 32314560 PMCID: PMC7167599 DOI: 10.1117/1.jbo.25.4.046003] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/10/2019] [Accepted: 04/01/2020] [Indexed: 05/13/2023]
Abstract
SIGNIFICANCE Optical coherence tomography (OCT) has proven useful for detecting various oral maxillofacial abnormalities. To apply it to clinical applications including biopsy guidance and routine screening, a handheld imaging probe is indispensable. OCT probes reported for oral maxillofacial imaging were either based on a bulky galvanometric mirror pair (not compact or long enough) or a distal-end microelectromechanical systems (MEMS) scanner (raised safety concerns), or adapted from fiber-optic catheters (ill-suited for oral cavity geometry). AIM To develop a handheld probe featuring great compactness and excellent maneuverability for oral maxillofacial tissue imaging. APPROACH A dual-axis MEMS scanner was deployed at the proximal end of the probe and the scanned beam was relayed to the distal end through a 4f configuration. Such design provides both a perfect dual-axis telecentric scan and excellent compactness. RESULTS A handheld probe with a rigid part 70 mm in length and 7 mm in diameter and weighing 25 g in total was demonstrated through both ex vivo and in vivo experiments, including structural visualization of various oral maxillofacial tissues and monitoring the recovery process of an oral mucosa canker sore. CONCLUSIONS The proposed probe exhibits excellent maneuverability and imaging performance showing great potential in clinical applications.
Collapse
Affiliation(s)
- Kaiyan Li
- Southeast University, School of Biological Science and Medical Engineering, Nanjing, Jiangsu, China
| | - Zihan Yang
- Nankai University, Institute of Modern Optics, Tianjin, China
| | - Wenxuan Liang
- Columbia University, Mortimer B. Zuckerman Mind Brain Behavior Institute, New York, United States
| | - Jianwei Shang
- Nankai University, Tianjin Stomatological Hospital, Hospital of Stomatology, Department of Oral Pathology, Tianjin, China
| | - Yanmei Liang
- Nankai University, Institute of Modern Optics, Tianjin, China
- Address all correspondence to Yanmei Liang, E-mail: ; Suiren Wan, E-mail:
| | - Suiren Wan
- Southeast University, School of Biological Science and Medical Engineering, Nanjing, Jiangsu, China
- Address all correspondence to Yanmei Liang, E-mail: ; Suiren Wan, E-mail:
| |
Collapse
|
16
|
Loewke NO, Qiu Z, Mandella MJ, Ertsey R, Loewke A, Gunaydin LA, Rosenthal EL, Contag CH, Solgaard O. Software-Based Phase Control, Video-Rate Imaging, and Real-Time Mosaicing With a Lissajous-Scanned Confocal Microscope. IEEE TRANSACTIONS ON MEDICAL IMAGING 2020; 39:1127-1137. [PMID: 31567074 PMCID: PMC8837204 DOI: 10.1109/tmi.2019.2942552] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2023]
Abstract
We present software-based methods for automatic phase control and for mosaicing high-speed, Lissajous-scanned images. To achieve imaging speeds fast enough for mosaicing, we first increase the image update rate tenfold from 3 to 30 Hz, then vertically interpolate each sparse image in real-time to eliminate fixed pattern noise. We validate our methods by imaging fluorescent beads and automatically maintaining phase control over the course of one hour. We then image fixed mouse brain tissues at varying update rates and compare the resulting mosaics. Using reconstructed image data as feedback for phase control eliminates the need for phase sensors and feedback controllers, enabling long-term imaging experiments without additional hardware. Mosaicing subsampled images results in video-rate imaging speeds, nearly fully recovered spatial resolution, and millimeter-scale fields of view.
Collapse
|
17
|
Thrapp AD, Hughes MR. Automatic motion compensation for structured illumination endomicroscopy using a flexible fiber bundle. JOURNAL OF BIOMEDICAL OPTICS 2020; 25:1-13. [PMID: 32100492 PMCID: PMC7040435 DOI: 10.1117/1.jbo.25.2.026501] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Accepted: 01/21/2020] [Indexed: 05/13/2023]
Abstract
SIGNIFICANCE Confocal laser scanning enables optical sectioning in clinical fiber bundle endomicroscopes, but lower-cost, simplified endomicroscopes use widefield incoherent illumination instead. Optical sectioning can be introduced in these simple systems using structured illumination microscopy (SIM), a multiframe digital subtraction process. However, SIM results in artifacts when the probe is in motion, making the technique difficult to use in vivo and preventing the use of mosaicking to synthesize a larger effective field of view (FOV). AIM We report and validate an automatic motion compensation technique to overcome motion artifacts and allow generation of mosaics in SIM endomicroscopy. APPROACH Motion compensation is achieved using image registration and real-time pattern orientation correction via a digital micromirror device. We quantify the similarity of moving probe reconstructions to those acquired with a stationary probe using the relative mean of the absolute differences (MAD). We further demonstrate mosaicking with a moving probe in mechanical and freehand operation. RESULTS Reconstructed SIM images show an improvement in the MAD from 0.85 to 0.13 for lens paper and from 0.27 to 0.12 for bovine tissue. Mosaics also show vastly reduced artifacts. CONCLUSION The reduction in motion artifacts in individual SIM reconstructions leads to mosaics that more faithfully represent the morphology of tissue, giving clinicians a larger effective FOV than the probe itself can provide.
Collapse
Affiliation(s)
- Andrew D. Thrapp
- University of Kent, School of Physical Sciences, Applied Optics Group, Canterbury, United Kingdom
- Address all correspondence to Andrew D. Thrapp, E-mail:
| | - Michael R. Hughes
- University of Kent, School of Physical Sciences, Applied Optics Group, Canterbury, United Kingdom
| |
Collapse
|
18
|
RENTERIA CARLOS, SUáREZ JAVIER, LICUDINE ALYSSA, BOPPART STEPHENA. Depixelation and enhancement of fiber bundle images by bundle rotation. APPLIED OPTICS 2020; 59:536-544. [PMID: 32225338 PMCID: PMC7286003 DOI: 10.1364/ao.59.000536] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Fiber bundles have become widely adopted for use in endoscopy, live-organism imaging, and other imaging applications. An inherent consequence of imaging with these bundles is the introduction of a honeycomb-like artifact that arises from the inter-fiber spacing, which obscures features of objects in the image. This artifact subsequently limits applicability and can make interpretation of the image-based data difficult. This work presents a method to reduce this artifact by on-axis rotation of the fiber bundle. Fiber bundle images were first low-pass and median filtered to improve image quality. Consecutive filtered images with rotated samples were then co-registered and averaged to generate a final, reconstructed image. The results demonstrate removal of the artifacts, in addition to increased signal contrast and signal-to-noise ratio. This approach combines digital filtering and spatial resampling to reconstruct higher-quality images, enhancing the utility of images acquired using fiber bundles.
Collapse
Affiliation(s)
- CARLOS RENTERIA
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
| | - JAVIER SUáREZ
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
| | - ALYSSA LICUDINE
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
| | - STEPHEN A. BOPPART
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
- Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
- Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
- Carle Illinois College of Medicine, University of Illinois at Urbana-Champaign, Champaign, Illinois 61820, USA
- Corresponding author:
| |
Collapse
|
19
|
Lang RT, Tatz J, Kercher EM, Palanisami A, Brooks DH, Spring BQ. Multichannel correlation improves the noise tolerance of real-time hyperspectral microimage mosaicking. JOURNAL OF BIOMEDICAL OPTICS 2019; 24:1-9. [PMID: 31828983 PMCID: PMC6905180 DOI: 10.1117/1.jbo.24.12.126002] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/21/2019] [Accepted: 11/14/2019] [Indexed: 05/08/2023]
Abstract
Live-subject microscopies, including microendoscopy and other related technologies, offer promise for basic biology research as well as the optical biopsy of disease in the clinic. However, cellular resolution generally comes with the trade-off of a microscopic field-of-view. Microimage mosaicking enables stitching many small scenes together to aid visualization, quantitative interpretation, and mapping of microscale features, for example, to guide surgical intervention. The development of hyperspectral and multispectral systems for biomedical applications provides motivation for adapting mosaicking algorithms to process a number of simultaneous spectral channels. We present an algorithm that mosaics multichannel video by correlating channels of consecutive frames as a basis for efficiently calculating image alignments. We characterize the noise tolerance of the algorithm by using simulated video with known ground-truth alignments to quantify mosaicking accuracy and speed, showing that multiplexed molecular imaging enhances mosaic accuracy by leveraging observations of distinct molecular constituents to inform frame alignment. A simple mathematical model is introduced to characterize the noise suppression provided by a given group of spectral channels, thus predicting the performance of selected subsets of data channels in order to balance mosaic computation accuracy and speed. The characteristic noise tolerance of a given number of channels is shown to improve through selection of an optimal subset of channels that maximizes this model. We also demonstrate that the multichannel algorithm produces higher quality mosaics than the analogous single-channel methods in an empirical test case. To compensate for the increased data rate of hyperspectral video compared to single-channel systems, we employ parallel processing via GPUs to alleviate computational bottlenecks and to achieve real-time mosaicking even for video-rate multichannel systems anticipated in the future. This implementation paves the way for real-time multichannel mosaicking to accompany next-generation hyperspectral and multispectral video microscopy.
Collapse
Affiliation(s)
- Ryan T. Lang
- Northeastern University, Translational Biophotonics Cluster, Boston, United States
- Northeastern University, Department of Physics, Boston, United States
| | - Julia Tatz
- Northeastern University, Translational Biophotonics Cluster, Boston, United States
- Northeastern University, Department of Physics, Boston, United States
| | - Eric M. Kercher
- Northeastern University, Translational Biophotonics Cluster, Boston, United States
- Northeastern University, Department of Physics, Boston, United States
| | - Akilan Palanisami
- Massachusetts General Hospital and Harvard Medical School, Wellman Center for Photomedicine, Boston, United States
| | - Dana H. Brooks
- Northeastern University, Department of Electrical and Computer Engineering, Boston, United States
| | - Bryan Q. Spring
- Northeastern University, Translational Biophotonics Cluster, Boston, United States
- Northeastern University, Department of Physics, Boston, United States
- Northeastern University, Department of Bioengineering, Boston, United States
- Address all correspondence to Bryan Q. Spring, E-mail:
| |
Collapse
|
20
|
Gawedzinski J, Schmeler KM, Milbourne A, Ramalingam P, Moghaddam PA, Richards-Kortum R, Tkaczyk TS. Toward development of a large field-of-view cancer screening patch (CASP) to detect cervical intraepithelial neoplasia. BIOMEDICAL OPTICS EXPRESS 2019; 10:6145-6159. [PMID: 31853391 PMCID: PMC6913391 DOI: 10.1364/boe.10.006145] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/28/2019] [Revised: 10/24/2019] [Accepted: 10/27/2019] [Indexed: 06/10/2023]
Abstract
Cervical cancers are primarily diagnosed via colposcopy, in which the tissue is visually assessed by a clinician for abnormalities, followed by directed biopsies and histologic analysis of excised tissue. Optical biopsy technologies offer a less invasive method of imaging such that subcellular features can be resolved without removing tissue. These techniques, however, are limited in field-of-view by the distal end of the probe. We present a prototype that incorporates a rigid, machinable waveguide that is in direct contact with a fluorescently-labeled sample paired with a scanning fluorescent microscope. The system is capable of imaging large areas of tissue without the need to re-position the tissue-probe interface. A mosaicing algorithm was developed to quantify scanning shifts and stitch neighboring frames together to increase the field-of-view. Our prototype can yield a maximum axial resolution of <5 µm for individual frames and can produce mosaiced images with a field-of-view greater than 15 mm x 15 mm without sacrificing resolution. We validated the system with a 1951 USAF resolution target, fluorescent in vitro standards, and a patient study where ex vivo conization samples of squamous cervical epithelium were imaged. The results of the patient study indicate that architectural features of subcellular components could be detected and differentiated between normal tissue and precancerous lesions.
Collapse
Affiliation(s)
- John Gawedzinski
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77005, USA
| | - Kathleen M. Schmeler
- Departments of Gynecologic Oncology and Pathology, The University of Texas M.D. Anderson Cancer Center, 1515 Holcombe Boulevard, Houston, TX 77030, USA
| | - Andrea Milbourne
- Departments of Gynecologic Oncology and Pathology, The University of Texas M.D. Anderson Cancer Center, 1515 Holcombe Boulevard, Houston, TX 77030, USA
| | - Preetha Ramalingam
- Department of Pathology, The University of Texas M.D.
Anderson Cancer Center, 1515 Holcombe Boulevard, Houston, TX 77030,
USA
| | - Parnian A. Moghaddam
- Department of Pathology and Laboratory Medicine, University of Texas Health Science Center, Lyndon B Johnson Hospital, 5656 Kelley St, Houston, TX 77026, USA
| | - Rebecca Richards-Kortum
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77005, USA
- Department of Electrical and Computer Engineering, Rice University, 6100 Main Street, Houston, TX 77005, USA
| | - Tomasz S. Tkaczyk
- Department of Bioengineering, Rice University, 6100 Main Street, Houston, TX 77005, USA
- Department of Electrical and Computer Engineering, Rice University, 6100 Main Street, Houston, TX 77005, USA
| |
Collapse
|
21
|
Peterson G, Zanoni DK, Ardigo M, Migliacci JC, Patel SG, Rajadhyaksha M. Feasibility of a Video-Mosaicking Approach to Extend the Field-of-View For Reflectance Confocal Microscopy in the Oral Cavity In Vivo. Lasers Surg Med 2019; 51:439-451. [PMID: 31067360 PMCID: PMC6842028 DOI: 10.1002/lsm.23090] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/22/2019] [Indexed: 12/24/2022]
Abstract
BACKGROUND Reflectance confocal microscopy (RCM) is a developing approach for noninvasive detection of oral lesions with label-free contrast and cellular-level resolution. For access into the oral cavity, confocal microscopes are being configured with small-diameter telescopic probes and small objective lenses. However, a small probe and objective lens allows for a rather small field-of-view relative to the large areas of tissue that must be examined for diagnosis. To extend the field-of-view for intraoral RCM imaging, we are investigating a video-mosaicking approach. METHODS A relay telescope and objective lens were adapted to an existing confocal microscope for access into the oral cavity. Imaging was performed using metal three-dimensional-printed objective lens front-end caps with coverslip windows to contact and stabilize the tissue and set depth. Four healthy volunteers (normal oral mucosa), one patient (with an amalgam tattoo) in a clinical setting, and 20 anesthetized patients (with oral squamous cell carcinoma [OSCC]) in a surgical setting were imaged. Instead of the usual still RCM images, videos were recorded and then processed into video-mosaics. Thirty video-mosaics were read and qualitatively assessed by an expert reader of RCM images of the oral mucosa. RESULTS Whereas the objective lens' native field-of-view is 0.75 mm × 0.75 mm, the video-mosaics display larger areas, ranging from 2 mm × 2 mm to 4 mm × 2 mm, with resolution, morphologic detail, and image quality that is preserved relative to that observed in the original videos (individual images). Video-mosaics in healthy volunteers' and the patients' images showed cellular morphologic patterns in the lower epithelium and at the epithelial junction, and connective tissue along with capillary loops and blood flow in the deeper lamina propria. In OSCC, tumor nests could be observed along with normal looking mucosa in margin areas. CONCLUSIONS Video-mosaicking is a reasonably quick and efficient approach for extending the field-of-view of RCM imaging, which can, to some extent, overcome the inherent limitation of an intraoral probe's small field-of-view. Reading video-mosaics can mimic the procedure for examining pathology: initial visualization of the spatial cellular and morphologic patterns of the tumor and the spread of tumor margins over larger areas of the lesion, followed by digitally zooming (magnifying) for closer inspection of suspicious areas. However, faster processing of videos into video-mosaics will be necessary, to allow examination of video-mosaics in real-time at the bedside. Lasers Surg. Med. 51:439-451, 2019. © 2019 Wiley Periodicals, Inc.
Collapse
Affiliation(s)
- Gary Peterson
- Dermatology Service, Memorial Sloan Kettering Cancer Center, New York, New York, 10022, USA
| | - Daniella Karassawa Zanoni
- Head and Neck Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, New York, 10065, USA
| | - Marco Ardigo
- Department of Clinical Dermatology, San Gallicano Dermatological Institute, 00144, Rome, Italy
| | - Jocelyn C Migliacci
- Head and Neck Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, New York, 10065, USA
| | - Snehal G Patel
- Head and Neck Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, New York, 10065, USA
| | - Milind Rajadhyaksha
- Dermatology Service, Memorial Sloan Kettering Cancer Center, New York, New York, 10022, USA
| |
Collapse
|
22
|
Marques MJ, Hughes MR, Vyas K, Thrapp A, Zhang H, Bradu A, Gelikonov G, Giataganas P, Payne CJ, Yang GZ, Podoleanu A. En-face optical coherence tomography/fluorescence endomicroscopy for minimally invasive imaging using a robotic scanner. JOURNAL OF BIOMEDICAL OPTICS 2019; 24:1-15. [PMID: 31222989 PMCID: PMC6977172 DOI: 10.1117/1.jbo.24.6.066006] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/11/2019] [Accepted: 05/30/2019] [Indexed: 05/27/2023]
Abstract
We report a compact rigid instrument capable of delivering en-face optical coherence tomography (OCT) images alongside (epi)-fluorescence endomicroscopy (FEM) images by means of a robotic scanning device. Two working imaging channels are included: one for a one-dimensional scanning, forward-viewing OCT probe and another for a fiber bundle used for the FEM system. The robotic scanning system provides the second axis of scanning for the OCT channel while allowing the field of view (FoV) of the FEM channel to be increased by mosaicking. The OCT channel has resolutions of 25 / 60 μm (axial/lateral) and can provide en-face images with an FoV of 1.6 × 2.7 mm2. The FEM channel has a lateral resolution of better than 8 μm and can generate an FoV of 0.53 × 3.25 mm2 through mosaicking. The reproducibility of the scanning was determined using phantoms to be better than the lateral resolution of the OCT channel. Combined OCT and FEM imaging were validated with ex-vivo ovine and porcine tissues, with the instrument mounted on an arm to ensure constant contact of the probe with the tissue. The OCT imaging system alone was validated for in-vivo human dermal imaging with the handheld instrument. In both cases, the instrument was capable of resolving fine features such as the sweat glands in human dermal tissue and the alveoli in porcine lung tissue.
Collapse
Affiliation(s)
- Manuel J. Marques
- University of Kent, School of Physical Sciences, Applied Optics Group, Canterbury, United Kingdom
| | - Michael R. Hughes
- University of Kent, School of Physical Sciences, Applied Optics Group, Canterbury, United Kingdom
| | - Khushi Vyas
- Imperial College London, Hamlyn Centre for Robotic Surgery, London, United Kingdom
| | - Andrew Thrapp
- University of Kent, School of Physical Sciences, Applied Optics Group, Canterbury, United Kingdom
| | - Haojie Zhang
- Imperial College London, Hamlyn Centre for Robotic Surgery, London, United Kingdom
| | - Adrian Bradu
- University of Kent, School of Physical Sciences, Applied Optics Group, Canterbury, United Kingdom
| | | | - Petros Giataganas
- Imperial College London, Hamlyn Centre for Robotic Surgery, London, United Kingdom
| | - Christopher J. Payne
- Imperial College London, Hamlyn Centre for Robotic Surgery, London, United Kingdom
- Boston Children’s Hospital, Department of Cardiac Surgery, Boston, Massachusetts, United States
| | - Guang-Zhong Yang
- Imperial College London, Hamlyn Centre for Robotic Surgery, London, United Kingdom
| | - Adrian Podoleanu
- University of Kent, School of Physical Sciences, Applied Optics Group, Canterbury, United Kingdom
| |
Collapse
|
23
|
Osman H, Elsahy D, Slivova V, Thompson C, Georges J, Yocom S, Cohen-Gadol AA. Neurosurgical Flexible Probe Microscopy with Enhanced Architectural and Cytological Detail. World Neurosurg 2019; 128:e929-e937. [PMID: 31100529 DOI: 10.1016/j.wneu.2019.05.039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2019] [Revised: 05/03/2019] [Accepted: 05/04/2019] [Indexed: 10/26/2022]
Abstract
BACKGROUND Microscopic delineation and clearance of tumor cells at neurosurgical excision margins potentially reduce tumor recurrence and increase patient survival. Probe-based in vivo fluorescence microscopy technologies are promising for neurosurgical in vivo microscopy. OBJECTIVE We sought to demonstrate a flexible fiberoptic epifluorescence microscope capable of enhanced architectural and cytological imaging for in vivo microscopy during neurosurgical procedures. METHODS Eighteen specimens were procured from neurosurgical procedures. These specimens were stained with acridine orange and imaged with a 3-dimensional (3D)-printed epifluorescent microscope that incorporates a flexible fiberoptic probe. Still images and video sequence frames were processed using frame alignment, signal projection, and pseudo-coloring, resulting in resolution enhancement and an increased field of view. RESULTS Images produced displayed good nuclear contrast and architectural detail. Grade 1 meningiomas demonstrated 3D chords and whorls. Low-grade meningothelial nuclei showed streaming and displayed regularity in size, shape, and distribution. Oligodendrogliomas showed regular round nuclei and a variably staining background. Glioblastomas showed high degrees of nuclear pleomorphism and disarray. Mitoses, vascular proliferation, and necrosis were evident. CONCLUSIONS We demonstrate the utility of a 3D-printed, flexible probe microscope for high-resolution microscopic imaging with increased architectural detail. Enhanced in vivo imaging using this device may improve our ability to detect and decrease microscopic tumor burden at excision margins during neurosurgical procedures.
Collapse
Affiliation(s)
- Hany Osman
- Wellman Center for Photomedicine, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, USA
| | - Deena Elsahy
- Department of Obstetrics and Gynecology, Indiana University School of Medicine, Indianapolis, Indiana, USA
| | - Veronika Slivova
- Enterprise Clinical Research Operations Biorepository, Indiana University Health, Indianapolis, Indiana, USA
| | - Corey Thompson
- Enterprise Clinical Research Operations Biorepository, Indiana University Health, Indianapolis, Indiana, USA
| | - Joseph Georges
- Department of Neurosurgery, Philadelphia College of Osteopathic Medicine, Philadelphia, Pennsylvania, USA
| | - Steven Yocom
- Department of Neurosurgery, Philadelphia College of Osteopathic Medicine, Philadelphia, Pennsylvania, USA
| | - Aaron A Cohen-Gadol
- Goodman Campbell Brain and Spine and Indiana University Department of Neurological Surgery, Indianapolis, Indiana, USA.
| |
Collapse
|
24
|
A mobile-phone based high-resolution microendoscope to image cervical precancer. PLoS One 2019; 14:e0211045. [PMID: 30726252 PMCID: PMC6364962 DOI: 10.1371/journal.pone.0211045] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2018] [Accepted: 01/07/2019] [Indexed: 01/12/2023] Open
Abstract
Nearly 90% of cervical cancer cases and deaths occur in low- and middle-income countries that lack comprehensive national HPV immunization and cervical cancer screening programs. In these settings, it is difficult to implement screening programs due to a lack of infrastructure and shortage of trained personnel. Screening programs based on visual inspection with acetic acid (VIA) have been successfully implemented in some low-resource settings. However, VIA has poor specificity and up to 90% of patients receiving treatment based on a positive VIA exam are over-treated. A number of studies have suggested that high-resolution cervical imaging to visualize nuclear morphology in vivo can improve specificity by better distinguishing precancerous and benign lesions. To enable high-resolution imaging in low-resource settings, we developed a portable, low-cost, high-resolution microendoscope that uses a mobile phone to detect and display images of cervical epithelium in vivo with subcellular resolution. The device was fabricated for less than $2,000 using commercially available optical components including filters, an LED and triplet lenses assembled in a 3D-printed opto-mechanical mount. We show that the mobile high-resolution microendoscope achieves similar resolution and signal-to-background ratio as previously reported high-resolution microendoscope systems using traditional cameras and computers to detect and display images. Finally, we demonstrate the ability of the mobile high-resolution microendoscope to image normal and precancerous squamous epithelium of the cervix in vivo in a gynecological referral clinic in Barretos, Brazil.
Collapse
|
25
|
Tang Y, Polydorides AD, Anandasabapathy S, Richards-Kortum RR. Quantitative analysis of in vivo high-resolution microendoscopic images for the detection of neoplastic colorectal polyps. JOURNAL OF BIOMEDICAL OPTICS 2018; 23:1-6. [PMID: 30460794 PMCID: PMC6276307 DOI: 10.1117/1.jbo.23.11.116003] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/23/2018] [Accepted: 10/26/2018] [Indexed: 05/04/2023]
Abstract
Colonoscopy is routinely performed for colorectal cancer screening but lacks the capability to accurately characterize precursor lesions and early cancers. High-resolution microendoscopy (HRME) is a low-cost imaging tool to visualize colorectal polyps with subcellular resolution. We present a computer-aided algorithm to evaluate HRME images of colorectal polyps and classify neoplastic from benign lesions. Using histopathology as the gold standard, clinically relevant features based on luminal morphology and texture are quantified to build the classification algorithm. We demonstrate that adenomatous polyps can be identified with a sensitivity and specificity of 100% and 80% using a two-feature linear discriminant model in a pilot test set. The classification algorithm presented here offers an objective framework to detect adenomatous lesions in the colon with high accuracy and can potentially improve real-time assessment of colorectal polyps.
Collapse
Affiliation(s)
- Yubo Tang
- Rice University, Department of Bioengineering, Houston, Texas, United States
- Address all correspondence to: Yubo Tang, E-mail:
| | | | - Sharmila Anandasabapathy
- Baylor College of Medicine, Section of Gastroenterology and Hepatology, Houston, Texas, United States
| | | |
Collapse
|
26
|
Dsouza R, Won J, Monroy GL, Spillman DR, Boppart SA. Economical and compact briefcase spectral-domain optical coherence tomography system for primary care and point-of-care applications. JOURNAL OF BIOMEDICAL OPTICS 2018; 23:1-11. [PMID: 30251484 PMCID: PMC6170142 DOI: 10.1117/1.jbo.23.9.096003] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/14/2018] [Accepted: 08/29/2018] [Indexed: 05/28/2023]
Abstract
Development of low-cost and portable optical coherence tomography (OCT) systems is of global interest in the OCT research community. Such systems enable utility broadly throughout a clinical facility, or in remote areas that often lack clinical infrastructure. We report the development and validation of a low-cost, portable briefcase spectral-domain optical coherence tomography (SD-OCT) system for point-of-care diagnostics in primary care centers and/or in remote settings. The self-contained briefcase OCT contains all associated optical hardware, including light source, spectrometer, hand-held probe, and a laptop. Additionally, this system utilizes unique real-time mosaicking of surface video images that are synchronized with rapid A-scan acquisition to eliminate the need for lateral scanning hardware, and enable the construction of cross-sectional B-mode images over extended lateral distances. The entire briefcase system weighs 9 kg and costs ∼USD$8000 using off-the-shelf components. System performance was validated by acquiring images of in vivo human skin on the fingertip, palm, and nail fold. The efficiency, portability, and low-cost enable accessibility and utility in primary care centers and low-resource settings.
Collapse
Affiliation(s)
- Roshan Dsouza
- University of Illinois at Urbana-Champaign, Beckman Institute for Advanced Science and Technology, Urbana, Illinois, United States
| | - Jungeun Won
- University of Illinois at Urbana-Champaign, Beckman Institute for Advanced Science and Technology, Urbana, Illinois, United States
- University of Illinois at Urbana-Champaign, Department of Bioengineering, Urbana, Illinois, United States
| | - Guillermo L. Monroy
- University of Illinois at Urbana-Champaign, Beckman Institute for Advanced Science and Technology, Urbana, Illinois, United States
- University of Illinois at Urbana-Champaign, Department of Bioengineering, Urbana, Illinois, United States
| | - Darold R. Spillman
- University of Illinois at Urbana-Champaign, Beckman Institute for Advanced Science and Technology, Urbana, Illinois, United States
| | - Stephen A. Boppart
- University of Illinois at Urbana-Champaign, Beckman Institute for Advanced Science and Technology, Urbana, Illinois, United States
- University of Illinois at Urbana-Champaign, Department of Bioengineering, Urbana, Illinois, United States
- University of Illinois at Urbana-Champaign, Department of Electrical and Computer Engineering, Urbana, Illinois, United States
- University of Illinois at Urbana-Champaign, Carle-Illinois College of Medicine, Urbana, Illinois, United States
| |
Collapse
|
27
|
Wu C, Gleysteen J, Teraphongphom NT, Li Y, Rosenthal E. In-vivo optical imaging in head and neck oncology: basic principles, clinical applications and future directions. Int J Oral Sci 2018; 10:10. [PMID: 29555901 PMCID: PMC5944254 DOI: 10.1038/s41368-018-0011-4] [Citation(s) in RCA: 46] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2017] [Revised: 12/29/2017] [Accepted: 01/10/2018] [Indexed: 02/05/2023] Open
Abstract
Head and neck cancers become a severe threat to human's health nowadays and represent the sixth most common cancer worldwide. Surgery remains the first-line choice for head and neck cancer patients. Limited resectable tissue mass and complicated anatomy structures in the head and neck region put the surgeons in a dilemma between the extensive resection and a better quality of life for the patients. Early diagnosis and treatment of the pre-malignancies, as well as real-time in vivo detection of surgical margins during en bloc resection, could be leveraged to minimize the resection of normal tissues. With the understanding of the head and neck oncology, recent advances in optical hardware and reagents have provided unique opportunities for real-time pre-malignancies and cancer imaging in the clinic or operating room. Optical imaging in the head and neck has been reported using autofluorescence imaging, targeted fluorescence imaging, high-resolution microendoscopy, narrow band imaging and the Raman spectroscopy. In this study, we reviewed the basic theories and clinical applications of optical imaging for the diagnosis and treatment in the field of head and neck oncology with the goal of identifying limitations and facilitating future advancements in the field.
Collapse
Affiliation(s)
- Chenzhou Wu
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases & Department of Head and Neck Oncology, West China Hospital of Stomatology, Sichuan University, Chengdu, China
| | - John Gleysteen
- Department of Otolaryngology, University of Tennessee Health Science Center, 38163, Memphis, TN, USA
| | | | - Yi Li
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases & Department of Head and Neck Oncology, West China Hospital of Stomatology, Sichuan University, Chengdu, China.
| | - Eben Rosenthal
- Department of Otolaryngology and Radiology, Stanford University, 94305, Stanford, CA, USA.
| |
Collapse
|
28
|
High-resolution microendoscopy: a point-of-care diagnostic for cervical dysplasia in low-resource settings. Eur J Cancer Prev 2018; 26:63-70. [PMID: 26637074 DOI: 10.1097/cej.0000000000000219] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Abstract
Cervical cancer is the third leading cause of cancer-related death among women in low-to-middle income countries. Pap testing and pathological services are difficult to implement under these settings. Alternative techniques for the diagnosis of cervical precancer in these settings are needed to reduce the burden of the disease. The objective of this study was to evaluate the diagnostic accuracy of a low-cost, high-resolution microendoscope imaging system in identifying precancerous lesions of the cervix in vivo. A retrospective study of 59 patients undergoing colposcopy for an abnormal Pap test was performed at Hospital de Câncer de Barretos in Brazil. All patients underwent colposcopy as per standard of care, and acetowhite lesions were recorded. High-resolution microendoscopy (HRME) images were obtained from one colposcopically normal region and from all lesions observed on colposcopy. Biopsies of abnormal areas were obtained and reviewed by three independent, blinded pathologists and compared with HRME findings. The mean nuclear area and the median nuclear eccentricity were calculated from HRME images acquired from each site. A diagnostic algorithm to distinguish histopathologically diagnosed cervical intraepithelial neoplasias of grade 2 or more severe lesions (high grade) from less severe lesions (low grade) was developed using these parameters. A test of trend was used to analyze the relationship between HRME positivity and severity of histopathogical diagnosis. Fisher's exact test was used to analyze differences in HRME positivity between high-grade and low-grade lesions. Evaluable images were obtained from 108 of 143 discrete sites. Of these, 71 sites were colposcopically normal or low grade according to histopathology and 37 were diagnosed as high grade on the basis of histopathology. Using the mean nuclear area and the median nuclear eccentricity, HRME images from 59 colposcopically abnormal sites were classified as high grade or low grade with 92% sensitivity and 77% specificity compared with histopathological findings. Increasing HRME positivity showed a significant trend with increasing severity of diagnosis (Ptrend<0.001). We found a strong association (P<0.001) between HRME positivity and a histopathological diagnosis of cervical intraepithelial neoplasia of grade 2 or higher. HRME demonstrated an accurate in-situ diagnosis of high-grade dysplasia. In low-resource settings in which colposcopy and histopathology services are severely limited or unavailable, HRME may provide a low-cost, accurate method for diagnosis of cervical precancer without the need for biopsy, allowing for a single 'screen-and-treat' approach.
Collapse
|
29
|
Wang H, Wang S, Li J, Zuo S. Robotic Scanning Device for Intraoperative Thyroid Gland Endomicroscopy. Ann Biomed Eng 2018; 46:543-554. [PMID: 29340932 DOI: 10.1007/s10439-018-1978-x] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2017] [Accepted: 01/04/2018] [Indexed: 01/02/2023]
Abstract
Probe-based confocal laser endomicroscopy has a high potential to be a promising tool that can provide intraoperative high-resolution in vivo morphological imaging at cellular and subcellular levels for thyroidectomy, and allow real-time assessment of tumor margins. However, the typical images acquired with this technique cover a very small area limited by the field of view of the probe, accompanied by tissue deformation and inconsistent probe-tissue contact when operated manually. In this paper, a novel compact robotic device for large area scanning has been developed. The device can scan a large surface in a spiral trajectory by rotating the tip frame along the spiral groove of the base frame. The fiber Bragg grating sensor with a passive linear structure is used to detect and maintain a stable probe-tissue contact force during scanning. An active linear actuation is also integrated for adjusting the probe-tissue contact level prior to each scan. Results demonstrate that the scanning device ensures a suitable probe-tissue contact force and compensates for simulated hand tremor. Mosaicing results of lens tissue paper and porcine belly tissue with both bench and hand-held experiments show the effectiveness and usability of the device, demonstrating the potential clinical value of the system.
Collapse
Affiliation(s)
- Haibo Wang
- The Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Shuxin Wang
- The Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Jinhua Li
- The Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China
| | - Siyang Zuo
- The Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, 300072, China.
| |
Collapse
|
30
|
Automated video-mosaicking approach for confocal microscopic imaging in vivo: an approach to address challenges in imaging living tissue and extend field of view. Sci Rep 2017; 7:10759. [PMID: 28883434 PMCID: PMC5589933 DOI: 10.1038/s41598-017-11072-9] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2017] [Accepted: 08/11/2017] [Indexed: 12/12/2022] Open
Abstract
We describe a computer vision-based mosaicking method for in vivo videos of reflectance confocal microscopy (RCM). RCM is a microscopic imaging technique, which enables the users to rapidly examine tissue in vivo. Providing resolution at cellular-level morphology, RCM imaging combined with mosaicking has shown to be highly sensitive and specific for non-invasively guiding skin cancer diagnosis. However, current RCM mosaicking techniques with existing microscopes have been limited to two-dimensional sequences of individual still images, acquired in a highly controlled manner, and along a specific predefined raster path, covering a limited area. The recent advent of smaller handheld microscopes is enabling acquisition of videos, acquired in a relatively uncontrolled manner and along an ad-hoc arbitrarily free-form, non-rastered path. Mosaicking of video-images (video-mosaicking) is necessary to display large areas of tissue. Our video-mosaicking methods addresses this need. The method can handle unique challenges encountered during video capture such as motion blur artifacts due to rapid motion of the microscope over the imaged area, warping in frames due to changes in contact angle and varying resolution with depth. We present test examples of video-mosaics of melanoma and non-melanoma skin cancers, to demonstrate potential clinical utility.
Collapse
|
31
|
Olsovsky C, Hinsdale T, Cuenca R, Cheng YSL, Wright JM, Rees TD, Jo JA, Maitland KC. Handheld tunable focus confocal microscope utilizing a double-clad fiber coupler for in vivo imaging of oral epithelium. JOURNAL OF BIOMEDICAL OPTICS 2017; 22:56008. [PMID: 28541447 PMCID: PMC5444308 DOI: 10.1117/1.jbo.22.5.056008] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/16/2016] [Accepted: 05/08/2017] [Indexed: 05/08/2023]
Abstract
A reflectance confocal endomicroscope with double-clad fiber coupler and electrically tunable focus lens is applied to imaging of the oral mucosa. The instrument is designed to be lightweight and robust for clinical use. The tunable lens allows axial scanning through > 250 ?? ? m in the epithelium when the probe tip is placed in contact with tissue. Images are acquired at 6.6 frames per second with a field of view diameter up to 850 ?? ? m . In vivo imaging of a wide range of normal sites in the oral cavity demonstrates the accessibility of the handheld probe. In vivo imaging of clinical lesions diagnosed as inflammation and dysplasia illustrates the ability of reflectance confocal endomicroscopy to image cellular changes associated with pathology.
Collapse
Affiliation(s)
- Cory Olsovsky
- Texas A&M University, Biomedical Engineering Department, College Station, Texas, United States
| | - Taylor Hinsdale
- Texas A&M University, Biomedical Engineering Department, College Station, Texas, United States
| | - Rodrigo Cuenca
- Texas A&M University, Biomedical Engineering Department, College Station, Texas, United States
| | - Yi-Shing Lisa Cheng
- Texas A&M University College of Dentistry, Department of Diagnostic Sciences, Dallas, Texas, United States
| | - John M. Wright
- Texas A&M University College of Dentistry, Department of Diagnostic Sciences, Dallas, Texas, United States
| | - Terry D. Rees
- Texas A&M University College of Dentistry, Department of Periodontics, Dallas, Texas, United States
| | - Javier A. Jo
- Texas A&M University, Biomedical Engineering Department, College Station, Texas, United States
| | - Kristen C. Maitland
- Texas A&M University, Biomedical Engineering Department, College Station, Texas, United States
- Address all correspondence to: Kristen C. Maitland, E-mail:
| |
Collapse
|
32
|
Abstract
Endomicroscopy is a new technique that allows human tissue to be characterized in vivo and in situ, circumventing the need for conventional biopsy and histology. Despite increased application and growing research interests in this area, the clinical application of endomicroscopy, however, is limited by difficulties in ergonomic control, consistent probe-tissue contact, large area surveillance, and retargeting. Recently, advances in high-speed imaging, mosaicing, and robotics have aimed to address these difficulties. The development of robot-assisted devices in particular has shown great promises in extending the clinical potential of endomicroscopy. Issues related to miniaturization, adaptation to tissue deformation, control stability, force and position compensation, cost, and sterility are being pursued by both research and commercial communities. In this review, recent clinical and technical developments in different aspects of computer and robotic assisted endomicroscopy interventions including instrumentation, multiscale integration, and high-speed imaging techniques are presented. We further address emerging trends and new research opportunities toward more widespread clinical acceptance of robotically assisted endomicroscopy technologies.
Collapse
|
33
|
Linked color imaging application for improving the endoscopic diagnosis accuracy: a pilot study. Sci Rep 2016; 6:33473. [PMID: 27641243 PMCID: PMC5027569 DOI: 10.1038/srep33473] [Citation(s) in RCA: 41] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2016] [Accepted: 08/30/2016] [Indexed: 12/13/2022] Open
Abstract
Endoscopy has been widely used in diagnosing gastrointestinal mucosal lesions. However, there are still lack of objective endoscopic criteria. Linked color imaging (LCI) is newly developed endoscopic technique which enhances color contrast. Thus, we investigated the clinical application of LCI and further analyzed pixel brightness for RGB color model. All the lesions were observed by white light endoscopy (WLE), LCI and blue laser imaging (BLI). Matlab software was used to calculate pixel brightness for red (R), green (G) and blue color (B). Of the endoscopic images for lesions, LCI had significantly higher R compared with BLI but higher G compared with WLE (all P < 0.05). R/(G + B) was significantly different among 3 techniques and qualified as a composite LCI marker. Our correlation analysis of endoscopic diagnosis with pathology revealed that LCI was quite consistent with pathological diagnosis (P = 0.000) and the color could predict certain kinds of lesions. ROC curve demonstrated at the cutoff of R/(G+B) = 0.646, the area under curve was 0.646, and the sensitivity and specificity was 0.514 and 0.773. Taken together, LCI could improve efficiency and accuracy of diagnosing gastrointestinal mucosal lesions and benefit target biopsy. R/(G + B) based on pixel brightness may be introduced as a objective criterion for evaluating endoscopic images.
Collapse
|
34
|
Differential structured illumination microendoscopy for in vivo imaging of molecular contrast agents. Proc Natl Acad Sci U S A 2016; 113:10769-73. [PMID: 27621464 DOI: 10.1073/pnas.1613497113] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022] Open
Abstract
Fiber optic microendoscopy has shown promise for visualization of molecular contrast agents used to study disease in vivo. However, fiber optic microendoscopes have limited optical sectioning capability, and image contrast is limited by out-of-focus light generated in highly scattering tissue. Optical sectioning techniques have been used in microendoscopes to remove out-of-focus light but reduce imaging speed or rely on bulky optical elements that prevent in vivo imaging. Here, we present differential structured illumination microendoscopy (DSIMe), a fiber optic system that can perform structured illumination in real time for optical sectioning without any opto-mechanical components attached to the distal tip of the fiber bundle. We demonstrate the use of DSIMe during in vivo fluorescence imaging in patients undergoing surgery for cervical adenocarcinoma in situ. Images acquired using DSIMe show greater contrast than standard microendoscopy, improving the ability to detect cellular atypia associated with neoplasia.
Collapse
|
35
|
Prieto SP, Lai KK, Laryea JA, Mizell JS, Muldoon TJ. Quantitative analysis of ex vivo colorectal epithelium using an automated feature extraction algorithm for microendoscopy image data. J Med Imaging (Bellingham) 2016; 3:024502. [PMID: 27335893 DOI: 10.1117/1.jmi.3.2.024502] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2015] [Accepted: 04/28/2016] [Indexed: 12/20/2022] Open
Abstract
Qualitative screening for colorectal polyps via fiber bundle microendoscopy imaging has shown promising results, with studies reporting high rates of sensitivity and specificity, as well as low interobserver variability with trained clinicians. A quantitative image quality control and image feature extraction algorithm (QFEA) was designed to lessen the burden of training and provide objective data for improved clinical efficacy of this method. After a quantitative image quality control step, QFEA extracts field-of-view area, crypt area, crypt circularity, and crypt number per image. To develop and validate this QFEA, a training set of microendoscopy images was collected from freshly resected porcine colon epithelium. The algorithm was then further validated on ex vivo image data collected from eight human subjects, selected from clinically normal appearing regions distant from grossly visible tumor in surgically resected colorectal tissue. QFEA has proven flexible in application to both mosaics and individual images, and its automated crypt detection sensitivity ranges from 71 to 94% despite intensity and contrast variation within the field of view. It also demonstrates the ability to detect and quantify differences in grossly normal regions among different subjects, suggesting the potential efficacy of this approach in detecting occult regions of dysplasia.
Collapse
Affiliation(s)
- Sandra P Prieto
- University of Arkansas , Department of Biomedical Engineering, 1 University Boulevard, Fayetteville, Arkansas 72701, United States
| | - Keith K Lai
- University of Arkansas for Medical Sciences , Department of Pathology, 4301 West Markham Street, Little Rock, Arkansas 72205, United States
| | - Jonathan A Laryea
- University of Arkansas for Medical Sciences , Department of Gastrointestinal Surgery, 4301 West Markham Street, Little Rock, Arkansas 72205, United States
| | - Jason S Mizell
- University of Arkansas for Medical Sciences , Department of Gastrointestinal Surgery, 4301 West Markham Street, Little Rock, Arkansas 72205, United States
| | - Timothy J Muldoon
- University of Arkansas , Department of Biomedical Engineering, 1 University Boulevard, Fayetteville, Arkansas 72701, United States
| |
Collapse
|
36
|
Shin D, Lee MH, Polydorides AD, Pierce MC, Vila PM, Parikh ND, Rosen DG, Anandasabapathy S, Richards-Kortum RR. Quantitative analysis of high-resolution microendoscopic images for diagnosis of neoplasia in patients with Barrett's esophagus. Gastrointest Endosc 2016; 83:107-14. [PMID: 26253018 PMCID: PMC4691546 DOI: 10.1016/j.gie.2015.06.045] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/17/2014] [Accepted: 06/20/2015] [Indexed: 02/08/2023]
Abstract
BACKGROUND AND AIMS Previous studies show that microendoscopic images can be interpreted visually to identify the presence of neoplasia in patients with Barrett's esophagus (BE), but this approach is subjective and requires clinical expertise. This study describes an approach for quantitative image analysis of microendoscopic images to identify neoplastic lesions in patients with BE. METHODS Images were acquired from 230 sites from 58 patients by using a fiberoptic high-resolution microendoscope during standard endoscopic procedures. Images were analyzed by a fully automated image processing algorithm, which automatically selected a region of interest and calculated quantitative image features. Image features were used to develop an algorithm to identify the presence of neoplasia; results were compared with a histopathology diagnosis. RESULTS A sequential classification algorithm that used image features related to glandular and cellular morphology resulted in a sensitivity of 84% and a specificity of 85%. Applying the algorithm to an independent validation set resulted in a sensitivity of 88% and a specificity of 85%. CONCLUSIONS This pilot study demonstrates that automated analysis of microendoscopic images can provide an objective, quantitative framework to assist clinicians in evaluating esophageal lesions from patients with BE. ( CLINICAL TRIAL REGISTRATION NUMBER NCT01384227 and NCT02018367.).
Collapse
Affiliation(s)
- Dongsuk Shin
- Department of Bioengineering, Rice University, Houston, Texas,Department of Neurosurgery, The University of Texas Medical School at Houston, Houston, Texas
| | - Michelle H. Lee
- Division of Gastroenterology, The Mount Sinai Medical Center, New York, New York
| | | | - Mark C. Pierce
- Department of Biomedical Engineering, Rutgers, The State University of New Jersey, Piscataway, New Jersey
| | - Peter M. Vila
- Division of Gastroenterology, The Mount Sinai Medical Center, New York, New York,Department of Otolaryngology-Head and Neck Surgery, Washington University School of Medicine, St. Louis, Missouri
| | - Neil D. Parikh
- Division of Gastroenterology, The Mount Sinai Medical Center, New York, New York,Division of Digestive Diseases, Yale-New Haven Hospital, New Haven, Connecticut
| | - Daniel G. Rosen
- Department of Pathology, Baylor College of Medicine, Houston, Texas
| | | | | |
Collapse
|
37
|
High-resolution microendoscopy in differentiating neoplastic from non-neoplastic colorectal polyps. Best Pract Res Clin Gastroenterol 2015; 29:663-73. [PMID: 26381310 PMCID: PMC4578798 DOI: 10.1016/j.bpg.2015.05.011] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/08/2015] [Revised: 04/16/2015] [Accepted: 05/20/2015] [Indexed: 01/31/2023]
Abstract
Colorectal cancer is one of the leading causes of death worldwide. The progression from adenoma to cancer is a well known phenomenon. Current clinical practice favors colonoscopy as the preferred modality for colorectal cancer screening. Many novel endoscopic technologies are emerging for the purposes of performing "optical biopsy" to allow real-time histologic diagnosis of polyps. High resolution microendoscopy is a low-cost endoscopic technology that has demonstrated high sensitivity and specificity in differentiating neoplastic and non-neoplastic polyps. With the ability to make real-time conclusions based on the endoscopic appearance of polyps, it is becoming increasingly possible to decrease the rate of unnecessary polypectomies and utilize a "resect and discard" strategy to decrease costs of pathology evaluation. Future directions for this technology include surveillance of premalignant conditions such as inflammatory bowel disease. Moreover, the low cost and relative ease of use of this technology lends itself to widespread applicability.
Collapse
|
38
|
Patsias A, Giraldez-Rodriguez L, Polydorides AD, Richards-Kortum R, Anandasabapathy S, Quang T, Sikora AG, Miles B. Feasibility of transoral robotic-assisted high-resolution microendoscopic imaging of oropharyngeal squamous cell carcinoma. Head Neck 2015; 37:E99-102. [PMID: 25327825 DOI: 10.1002/hed.23892] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/14/2014] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND Transoral robotic-assisted oncologic surgery of the head and neck offers promising functional results. Nonetheless, the efficacy of oncologic surgery remains critically dependent on obtaining negative margins. We aimed to integrate a miniaturized high-resolution fiber-optic microendoscope (HRME), which provides real-time histological assessment, with the da Vinci robotic system (Intuitive Surgical, Sunnyvale, CA). METHODS Three patients undergoing transoral robotic surgery (TORS) were prospectively enrolled in this study. Optical imaging of the oropharynx was performed intraoperatively with the robotic-assisted HRME. RESULTS All patients underwent the procedure successfully with no complications. The HRME was successfully integrated with the da Vinci robotic system. Several sites of the oropharynx and associated malignancy were imaged, which correlated with the standard histopathological analysis. CONCLUSION Transoral robotic-assisted HRME imaging of the oropharynx is a safe and technically feasible approach, providing a real-time histological assessment and may serve as a valuable aid in oncologic surgery.
Collapse
Affiliation(s)
- Alexis Patsias
- Department of Otolaryngology - Head and Neck Surgery, Head and Neck Cancer Translational Research Program, The Icahn School of Medicine at Mount Sinai, New York, New York
| | - Laureano Giraldez-Rodriguez
- Department of Otolaryngology - Head and Neck Surgery, Head and Neck Cancer Translational Research Program, The Icahn School of Medicine at Mount Sinai, New York, New York
| | | | | | | | - Timothy Quang
- Department of Bioengineering, Rice University, Houston, Texas
| | - Andrew G Sikora
- Department of Otolaryngology - Head and Neck Surgery, Head and Neck Cancer Translational Research Program, The Icahn School of Medicine at Mount Sinai, New York, New York
| | - Brett Miles
- Department of Otolaryngology - Head and Neck Surgery, Head and Neck Cancer Translational Research Program, The Icahn School of Medicine at Mount Sinai, New York, New York
| |
Collapse
|
39
|
Bergen T, Wittenberg T. Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods. IEEE J Biomed Health Inform 2014; 20:304-21. [PMID: 25532214 DOI: 10.1109/jbhi.2014.2384134] [Citation(s) in RCA: 40] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Endoscopic procedures form part of routine clinical practice for minimally invasive examinations and interventions. While they are beneficial for the patient, reducing surgical trauma and making convalescence times shorter, they make orientation and manipulation more challenging for the physician, due to the limited field of view through the endoscope. However, this drawback can be reduced by means of medical image processing and computer vision, using image stitching and surface reconstruction methods to expand the field of view. This paper provides a comprehensive overview of the current state of the art in endoscopic image stitching and surface reconstruction. The literature in the relevant fields of application and algorithmic approaches is surveyed. The technological maturity of the methods and current challenges and trends are analyzed.
Collapse
|
40
|
Louie JS, Richards-Kortum R, Anandasabapathy S. Applications and advancements in the use of high-resolution microendoscopy for detection of gastrointestinal neoplasia. Clin Gastroenterol Hepatol 2014; 12:1789-92. [PMID: 25108219 PMCID: PMC4250374 DOI: 10.1016/j.cgh.2014.08.004] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/04/2014] [Accepted: 08/04/2014] [Indexed: 02/07/2023]
Abstract
The high-resolution microendoscope (HRME) is a novel imaging modality that allows real-time epithelial imaging at subcellular resolution. Used in concert with any standard endoscope, this portable, low cost, "optical biopsy" technology has the ability to provide images of cellular morphology during a procedure. This technology has been the subject of a number of studies investigating its use in screening and surveillance of a range of gastrointestinal neoplasias, including esophageal adenocarcinoma, esophageal squamous cell cancer, colorectal neoplasia, and anal neoplasia. These studies have shown that HRME is a modality that consistently provides high specificity, negative predictive value, and accuracy across different diseases. In addition, they have illustrated that HRME users can be relatively easily trained in a short period of time, and that users have demonstrated solid inter-rater reliability. These features make HRME a potential complement to high-definition white-light imaging, narrow-band imaging, and other red flag technologies in facilitating real-time clinical diagnosis, endoscopic therapy, and margin determination. Further clinical validation is needed to determine whether this translates to reduced procedure times, pathology costs, and follow-up procedures. Finally, the HRME has a relatively simple design compared with other similar technologies, making it portable, simple to maintain, and low cost. This may allow the HRME device to function in both advanced care settings as well as in places with less resources and specialized support systems. As a whole, the HRME device has shown good performance along with low cost and portable construction, and its application in different conditions and settings has been promising.
Collapse
|
41
|
Hughes M, Giataganas P, Yang GZ. Color reflectance fiber bundle endomicroscopy without back-reflections. JOURNAL OF BIOMEDICAL OPTICS 2014; 19:30501. [PMID: 24589984 DOI: 10.1117/1.jbo.19.3.030501] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/09/2013] [Accepted: 01/28/2014] [Indexed: 05/11/2023]
Abstract
Coherent fiber imaging bundles can be used as passive probes for reflectance-mode endomicroscopy providing that the back-reflections from the fiber ends are efficiently rejected. We describe an approach specific to widefield endomicroscopy in which light is injected into a leached fiber bundle near the distal end, thereby avoiding reflections from the proximal face. We use this method to demonstrate the color widefield reflectance endomicroscopy of ex vivo animal tissue.
Collapse
|
42
|
Piccinini F, Bevilacqua A, Lucarelli E. Automated image mosaics by non-automated light microscopes: the MicroMos software tool. J Microsc 2013; 252:226-50. [PMID: 24111790 DOI: 10.1111/jmi.12084] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2013] [Accepted: 08/16/2013] [Indexed: 12/20/2022]
Abstract
Light widefield microscopes and digital imaging are the basis for most of the analyses performed in every biological laboratory. In particular, the microscope's user is typically interested in acquiring high-detailed images for analysing observed cells and tissues, meanwhile being representative of a wide area to have reliable statistics. The microscopist has to choose between higher magnification factor and extension of the observed area, due to the finite size of the camera's field of view. To overcome the need of arrangement, mosaicing techniques have been developed in the past decades for increasing the camera's field of view by stitching together more images. Nevertheless, these approaches typically work in batch mode and rely on motorized microscopes. Or alternatively, the methods are conceived just to provide visually pleasant mosaics not suitable for quantitative analyses. This work presents a tool for building mosaics of images acquired with nonautomated light microscopes. The method proposed is based on visual information only and the mosaics are built by incrementally stitching couples of images, making the approach available also for online applications. Seams in the stitching regions as well as tonal inhomogeneities are corrected by compensating the vignetting effect. In the experiments performed, we tested different registration approaches, confirming that the translation model is not always the best, despite the fact that the motion of the sample holder of the microscope is apparently translational and typically considered as such. The method's implementation is freely distributed as an open source tool called MicroMos. Its usability makes building mosaics of microscope images at subpixel accuracy easier. Furthermore, optional parameters for building mosaics according to different strategies make MicroMos an easy and reliable tool to compare different registration approaches, warping models and tonal corrections.
Collapse
Affiliation(s)
- F Piccinini
- Advanced Research Center on Electronic Systems (ARCES), University of Bologna, Italy
| | | | | |
Collapse
|
43
|
Carns J, Keahey P, Quang T, Anandasabapathy S, Richards-Kortum R. Optical molecular imaging in the gastrointestinal tract. Gastrointest Endosc Clin N Am 2013; 23:707-23. [PMID: 23735112 PMCID: PMC3746803 DOI: 10.1016/j.giec.2013.03.010] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Recent developments in optical molecular imaging allow for real-time identification of morphologic and biochemical changes in tissue associated with gastrointestinal neoplasia. This review summarizes widefield and high-resolution imaging modalities in preclinical and clinical evaluation for the detection of colorectal cancer and esophageal cancer. Widefield techniques discussed include high-definition white light endoscopy, narrow band imaging, autofluoresence imaging, and chromoendoscopy; high-resolution techniques discussed include probe-based confocal laser endomicroscopy, high-resolution microendoscopy, and optical coherence tomography. New approaches to enhance image contrast using vital dyes and molecular-specific targeted contrast agents are evaluated.
Collapse
Affiliation(s)
- Jennifer Carns
- Department of Bioengineering, Rice University, Houston, TX 77005, United States,corresponding author for proofs
| | - Pelham Keahey
- Department of Bioengineering, Rice University, Houston, TX 77005, United States
| | - Timothy Quang
- Department of Bioengineering, Rice University, Houston, TX 77005, United States
| | | | - Rebecca Richards-Kortum
- Department of Bioengineering, Rice University, Houston, TX 77005, United States,corresponding author after publication
| |
Collapse
|
44
|
Abeytunge S, Li Y, Larson B, Peterson G, Seltzer E, Toledo-Crow R, Rajadhyaksha M. Confocal microscopy with strip mosaicing for rapid imaging over large areas of excised tissue. JOURNAL OF BIOMEDICAL OPTICS 2013; 18:61227. [PMID: 23389736 PMCID: PMC3565124 DOI: 10.1117/1.jbo.18.6.061227] [Citation(s) in RCA: 56] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2012] [Revised: 12/19/2012] [Accepted: 12/21/2012] [Indexed: 05/20/2023]
Abstract
Confocal mosaicing microscopy is a developing technology platform for imaging tumor margins directly in freshly excised tissue, without the processing required for conventional pathology. Previously, mosaicing on 12-×-12 mm² of excised skin tissue from Mohs surgery and detection of basal cell carcinoma margins was demonstrated in 9 min. Last year, we reported the feasibility of a faster approach called "strip mosaicing," which was demonstrated on a 10-×-10 mm² of tissue in 3 min. Here we describe further advances in instrumentation, software, and speed. A mechanism was also developed to flatten tissue in order to enable consistent and repeatable acquisition of images over large areas. We demonstrate mosaicing on 10-×-10 mm² of skin tissue with 1-μm lateral resolution in 90 s. A 2.5-×-3.5 cm² piece of breast tissue was scanned with 0.8-μm lateral resolution in 13 min. Rapid mosaicing of confocal images on large areas of fresh tissue potentially offers a means to perform pathology at the bedside. Imaging of tumor margins with strip mosaicing confocal microscopy may serve as an adjunct to conventional (frozen or fixed) pathology for guiding surgery.
Collapse
Affiliation(s)
- Sanjee Abeytunge
- Memorial Sloan-Kettering Cancer Center, Research Engineering Laboratory, New York, New York 10065, USA.
| | | | | | | | | | | | | |
Collapse
|