1
|
Sun J, Wu J, Koukourakis N, Cao L, Kuschmierz R, Czarske J. Real-time complex light field generation through a multi-core fiber with deep learning. Sci Rep 2022; 12:7732. [PMID: 35546604 PMCID: PMC9095618 DOI: 10.1038/s41598-022-11803-7] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Accepted: 04/28/2022] [Indexed: 12/26/2022] Open
Abstract
The generation of tailored complex light fields with multi-core fiber (MCF) lensless microendoscopes is widely used in biomedicine. However, the computer-generated holograms (CGHs) used for such applications are typically generated by iterative algorithms, which demand high computation effort, limiting advanced applications like fiber-optic cell manipulation. The random and discrete distribution of the fiber cores in an MCF induces strong spatial aliasing to the CGHs, hence, an approach that can rapidly generate tailored CGHs for MCFs is highly demanded. We demonstrate a novel deep neural network-CoreNet, providing accurate tailored CGHs generation for MCFs at a near video rate. The CoreNet is trained by unsupervised learning and speeds up the computation time by two magnitudes with high fidelity light field generation compared to the previously reported CGH algorithms for MCFs. Real-time generated tailored CGHs are on-the-fly loaded to the phase-only spatial light modulator (SLM) for near video-rate complex light fields generation through the MCF microendoscope. This paves the avenue for real-time cell rotation and several further applications that require real-time high-fidelity light delivery in biomedicine.
Collapse
Affiliation(s)
- Jiawei Sun
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany. .,Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Dresden, Germany.
| | - Jiachen Wu
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany. .,State Key Laboratory of Precision Measurement Technology and Instruments, Department of Precision Instruments, Tsinghua University, Beijing, 100084, China.
| | - Nektarios Koukourakis
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany.,Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Dresden, Germany
| | - Liangcai Cao
- State Key Laboratory of Precision Measurement Technology and Instruments, Department of Precision Instruments, Tsinghua University, Beijing, 100084, China
| | - Robert Kuschmierz
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany.,Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Dresden, Germany
| | - Juergen Czarske
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany. .,Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Dresden, Germany. .,Cluster of Excellence Physics of Life, TU Dresden, Dresden, Germany. .,Institute of Applied Physics, TU Dresden, Dresden, Germany.
| |
Collapse
|
2
|
Deep Learning-Based Image Classification through a Multimode Fiber in the Presence of Wavelength Drift. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10113816] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Deep neural networks (DNNs) are employed to recover information after its propagation through a multimode fiber (MMF) in the presence of wavelength drift. The intensity distribution of the speckle patterns generated at the output of an MMF when an input wavefront propagates along its length is highly sensitive to wavelength changes. We use a tunable laser to implement a wavelength drift with a controlled bandwidth, aiming to estimate the DNN’s performance in different cases and identify the limitations. We find that when the DNNs are trained with a dataset which includes the noise induced by wavelength changes, successful classification of a speckle pattern can be performed even for a large wavelength bandwidth drift. A single training step is found to be sufficient for high classification accuracy, removing the need for time-consuming recalibration at each wavelength.
Collapse
|
3
|
Mohanan SMPC, Beck RJ, West NP, Shires M, Perry SL, Jayne DG, Hand DP, Shephard JD. Preclinical evaluation of porcine colon resection using hollow core negative curvature fibre delivered ultrafast laser pulses. JOURNAL OF BIOPHOTONICS 2019; 12:e201900055. [PMID: 31240824 DOI: 10.1002/jbio.201900055] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/08/2019] [Revised: 05/17/2019] [Accepted: 06/25/2019] [Indexed: 06/09/2023]
Abstract
Ultrashort pulse lasers offer great promise for tissue resection with exceptional precision and minimal thermal damage. Surgery in the bowel requires high precision and minimal necrotic tissue to avoid severe complications such as perforation. The deployment of ultrashort lasers in minimally invasive or endoscopic procedures has been hindered by the lack of suitable optical fibres for high peak powers. However, recent developments of hollow core microstructured fibres provide potential for delivery of such pulses throughout the body. In this study, analysis of laser ablation via a scanning galvanometer on a porcine colon tissue model is presented. A thermally damaged region (<85 μm) and fine depth control of ablation using the pulse energies 46 and 33 μJ are demonstrated. It is further demonstrated that such pulses suitable for precision porcine colon resection can be flexibly delivered via a hollow core negative curvature fibre (HC-NCF) and again ablation depth can be controlled with a thermally damaged region <85 μm. Ablation volumes are comparable to that of early stage lesions in the inner lining of the colon. This study concludes that the combination of ultrashort pulses and flexible fibre delivery via HC-NCF present a viable route to new minimally invasive surgical procedures.
Collapse
Affiliation(s)
- Syam M P C Mohanan
- School of Engineering and Physical Sciences, Institute of Photonics and Quantum Sciences, Heriot-Watt University, Edinburgh, UK
| | - Rainer J Beck
- School of Engineering and Physical Sciences, Institute of Photonics and Quantum Sciences, Heriot-Watt University, Edinburgh, UK
| | - Nicholas P West
- Leeds Institute of Medical Research at St. James's, University of Leeds, Leeds, UK
| | - Michael Shires
- Leeds Institute of Medical Research at St. James's, University of Leeds, Leeds, UK
| | - Sarah L Perry
- Leeds Institute of Medical Research at St. James's, University of Leeds, Leeds, UK
| | - David G Jayne
- Leeds Institute of Medical Research at St. James's, University of Leeds, Leeds, UK
| | - Duncan P Hand
- School of Engineering and Physical Sciences, Institute of Photonics and Quantum Sciences, Heriot-Watt University, Edinburgh, UK
| | - Jonathan D Shephard
- School of Engineering and Physical Sciences, Institute of Photonics and Quantum Sciences, Heriot-Watt University, Edinburgh, UK
| |
Collapse
|
4
|
Gordon GSD, Joseph J, Sawyer T, Macfaden AJ, Williams C, Wilkinson TD, Bohndiek SE. Full-field quantitative phase and polarisation-resolved imaging through an optical fibre bundle. OPTICS EXPRESS 2019; 27:23929-23947. [PMID: 31510290 PMCID: PMC6825613 DOI: 10.1364/oe.27.023929] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/06/2019] [Revised: 07/11/2019] [Accepted: 07/11/2019] [Indexed: 05/06/2023]
Abstract
Flexible optical fibres, used in conventional medical endoscopy and industrial inspection, scramble phase and polarisation information, restricting users to amplitude-only imaging. Here, we exploit the near-diagonality of the multi-core fibre (MCF) transmission matrix in a parallelised fibre characterisation architecture, enabling accurate imaging of quantitative phase (error <0.3 rad) and polarisation-resolved (errors <10%) properties. We first demonstrate accurate recovery of optical amplitude and phase in two polarisations through the MCF by measuring and inverting the transmission matrix, and then present a robust Bayesian inference approach to resolving 5 polarimetric properties of samples. Our method produces high-resolution (9.0±2.6μm amplitude, phase; 36.0±10.4μm polarimetric) full-field images at working distances up to 1mm over a field-of-view up to 750×750μm 2 using an MCF with potential for flexible operation. We demonstrate the potential of using quantitative phase for computational image focusing and polarisation-resolved properties in imaging birefringence.
Collapse
Affiliation(s)
- George S. D. Gordon
- Now at: Department of Electrical and Electronic Engineering, The University of Nottingham, University Park, Nottingham, NG7 2RD, UK
- Department of Engineering, University of Cambridge, 9 JJ Thomson Avenue, Cambridge, CB3 0FA, UK
| | - James Joseph
- Department of Physics, Cavendish Laboratory, JJ Thomson Avenue, Cambridge, CB3 0HE, UK
- Cancer Research UK Cambridge Institute, Li Ka Shing Centre, Robinson Way, Cambridge, CB2 0RE, UK
| | - Travis Sawyer
- Department of Physics, Cavendish Laboratory, JJ Thomson Avenue, Cambridge, CB3 0HE, UK
| | - Alexander J. Macfaden
- Now at: Department of Electrical and Electronic Engineering, The University of Nottingham, University Park, Nottingham, NG7 2RD, UK
| | - Calum Williams
- Department of Engineering, University of Cambridge, 9 JJ Thomson Avenue, Cambridge, CB3 0FA, UK
- Department of Physics, Cavendish Laboratory, JJ Thomson Avenue, Cambridge, CB3 0HE, UK
| | - Timothy D. Wilkinson
- Department of Engineering, University of Cambridge, 9 JJ Thomson Avenue, Cambridge, CB3 0FA, UK
| | - Sarah E. Bohndiek
- Department of Physics, Cavendish Laboratory, JJ Thomson Avenue, Cambridge, CB3 0HE, UK
- Cancer Research UK Cambridge Institute, Li Ka Shing Centre, Robinson Way, Cambridge, CB2 0RE, UK
| |
Collapse
|
5
|
Scharf E, Kuschmierz R, Czarske J. Holographic lensless fiber endoscope with needle size using self-calibration. ACTA ACUST UNITED AC 2019. [DOI: 10.1515/teme-2018-0087] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
Abstract
Endoscopes enable optical keyhole access in many applications for instance in biomedicine. In general, coherent fiber bundles (CFB) are used in conjunction with rigid lens systems which determine a fixed image plane. However, the lens system limits the minimum diameter of the endoscope typically to several millimeters. Additionally, only pixelated two-dimensional amplitude patterns can be transferred due to phase scrambling between adjacent cores. These limitations can be overcome by digital optical elements. Thus, in principle thinner, lensless, holographic endoscopes with a three-dimensional adjustable focus for imaging and illumination can be realized. So far, several techniques based on single mode CFB and multi mode fibers (MMF) have been presented. However, these techniques require access to both sides of the fiber, in order to calibrate the bending and temperature sensitive phase distortion, which is not possible in a real application. We present the feasibility of an in-situ calibration and compensation of a CFB with single sided access. A lensless endoscope with a diameter of only 500 µm, a spatial resolution around 1 µm and video rate capability is realized.
Collapse
Affiliation(s)
- Elias Scharf
- Professur für Mess- und Sensorsystemtechnik , TU Dresden , Helmholzstr. 18 , Dresden , Germany
| | - Robert Kuschmierz
- Professur für Mess- und Sensorsystemtechnik , TU Dresden , Helmholzstr. 18 , Dresden , Germany
| | - Jürgen Czarske
- Professur für Mess- und Sensorsystemtechnik , TU Dresden , Helmholzstr. 18 , Dresden , Germany
| |
Collapse
|
6
|
Kakkava E, Romito M, Conkey DB, Loterie D, Stankovic KM, Moser C, Psaltis D. Selective femtosecond laser ablation via two-photon fluorescence imaging through a multimode fiber. BIOMEDICAL OPTICS EXPRESS 2019; 10:423-433. [PMID: 30800490 PMCID: PMC6377891 DOI: 10.1364/boe.10.000423] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/31/2018] [Revised: 12/14/2018] [Accepted: 12/16/2018] [Indexed: 05/10/2023]
Abstract
We demonstrate the ability of a multimode fiber probe to provide two-photon fluorescence (TPF) imaging feedback that guides the femtosecond laser ablation (FLA) in biological samples for highly selective modifications. We implement the system through the propagation of high power femtosecond pulses through a graded-index (GRIN) multimode fiber and we investigate the limitations posed by the high laser peak intensities required for laser ablation. We demonstrate that the GRIN fiber probe can deliver laser intensities up to 1.5x1013 W/cm2, sufficient for the ablation of a wide range of materials, including biological samples. Wavefront shaping through an ultrathin probe of around 400 μm in diameter is used for diffraction limited focusing and digital scanning of the focus spot. Selective FLA of cochlear hair cells is performed based on the TPF images obtained through the same multimode fiber probe.
Collapse
Affiliation(s)
- Eirini Kakkava
- Optics Laboratory, School of Engineering, École Polytechnique Fédérale De Lausanne, Lausanne, Switzerland
| | - Marilisa Romito
- Optics Laboratory, School of Engineering, École Polytechnique Fédérale De Lausanne, Lausanne, Switzerland
| | - Donald B. Conkey
- Optics Laboratory, School of Engineering, École Polytechnique Fédérale De Lausanne, Lausanne, Switzerland
| | - Damien Loterie
- Laboratory of Applied Photonic Devices, School of Engineering, École Polytechnique Fédérale De Lausanne, Lausanne, Switzerland
| | - Konstantina M. Stankovic
- Department of Otolaryngology and Eaton Peabody Laboratories, Massachusetts Eye and Ear and Harvard Medical School, Boston, MA, USA
| | - Christophe Moser
- Laboratory of Applied Photonic Devices, School of Engineering, École Polytechnique Fédérale De Lausanne, Lausanne, Switzerland
| | - Demetri Psaltis
- Optics Laboratory, School of Engineering, École Polytechnique Fédérale De Lausanne, Lausanne, Switzerland
| |
Collapse
|
7
|
Kuschmierz R, Scharf E, Koukourakis N, Czarske JW. Self-calibration of lensless holographic endoscope using programmable guide stars. OPTICS LETTERS 2018; 43:2997-3000. [PMID: 29905743 DOI: 10.1364/ol.43.002997] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Coherent fiber bundle (CFB)-based endoscopes enable optical keyhole access in applications such as biophotonics. In conjunction with objective lenses, CFBs allow imaging of intensity patterns. In contrast, digital optical phase conjugation enables lensless holographic endoscopes for the generation of pixelation-free arbitrary light patterns. For real-world applications, however, this requires a non-invasive in situ calibration of the complex optical transfer function of the CFB with only single-sided access. We show that after an initial calibration in a forward direction, a differential phase measurement of the back-reflected light allows for tracking and compensating of bending-induced phase distortions. Furthermore, we present a novel in situ calibration procedure based on a programmable guide star, which requires access to only one side of the fiber.
Collapse
|