1
|
Lin Y, Mos P, Ardelean A, Bruschini C, Charbon E. Coupling a recurrent neural network to SPAD TCSPC systems for real-time fluorescence lifetime imaging. Sci Rep 2024; 14:3286. [PMID: 38331957 PMCID: PMC10853568 DOI: 10.1038/s41598-024-52966-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 01/25/2024] [Indexed: 02/10/2024] Open
Abstract
Fluorescence lifetime imaging (FLI) has been receiving increased attention in recent years as a powerful diagnostic technique in biological and medical research. However, existing FLI systems often suffer from a tradeoff between processing speed, accuracy, and robustness. Inspired by the concept of Edge Artificial Intelligence (Edge AI), we propose a robust approach that enables fast FLI with no degradation of accuracy. This approach couples a recurrent neural network (RNN), which is trained to estimate the fluorescence lifetime directly from raw timestamps without building histograms, to SPAD TCSPC systems, thereby drastically reducing transfer data volumes and hardware resource utilization, and enabling real-time FLI acquisition. We train two variants of the RNN on a synthetic dataset and compare the results to those obtained using center-of-mass method (CMM) and least squares fitting (LS fitting). Results demonstrate that two RNN variants, gated recurrent unit (GRU) and long short-term memory (LSTM), are comparable to CMM and LS fitting in terms of accuracy, while outperforming them in the presence of background noise by a large margin. To explore the ultimate limits of the approach, we derive the Cramer-Rao lower bound of the measurement, showing that RNN yields lifetime estimations with near-optimal precision. To demonstrate real-time operation, we build a FLI microscope based on an existing SPAD TCSPC system comprising a 32[Formula: see text]32 SPAD sensor named Piccolo. Four quantized GRU cores, capable of processing up to 4 million photons per second, are deployed on the Xilinx Kintex-7 FPGA that controls the Piccolo. Powered by the GRU, the FLI setup can retrieve real-time fluorescence lifetime images at up to 10 frames per second. The proposed FLI system is promising and ideally suited for biomedical applications, including biological imaging, biomedical diagnostics, and fluorescence-assisted surgery, etc.
Collapse
Affiliation(s)
- Yang Lin
- Advanced Quantum Architecture Laboratory, École polytechnique fédérale de Lausanne, Neuchâtel, 2002, Switzerland
| | - Paul Mos
- Advanced Quantum Architecture Laboratory, École polytechnique fédérale de Lausanne, Neuchâtel, 2002, Switzerland
| | - Andrei Ardelean
- Advanced Quantum Architecture Laboratory, École polytechnique fédérale de Lausanne, Neuchâtel, 2002, Switzerland
| | - Claudio Bruschini
- Advanced Quantum Architecture Laboratory, École polytechnique fédérale de Lausanne, Neuchâtel, 2002, Switzerland
| | - Edoardo Charbon
- Advanced Quantum Architecture Laboratory, École polytechnique fédérale de Lausanne, Neuchâtel, 2002, Switzerland.
| |
Collapse
|
2
|
Gouzou D, Taimori A, Haloubi T, Finlayson N, Wang Q, Hopgood JR, Vallejo M. Applications of machine learning in time-domain fluorescence lifetime imaging: a review. Methods Appl Fluoresc 2024; 12:022001. [PMID: 38055998 PMCID: PMC10851337 DOI: 10.1088/2050-6120/ad12f7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Revised: 09/25/2023] [Accepted: 12/06/2023] [Indexed: 12/08/2023]
Abstract
Many medical imaging modalities have benefited from recent advances in Machine Learning (ML), specifically in deep learning, such as neural networks. Computers can be trained to investigate and enhance medical imaging methods without using valuable human resources. In recent years, Fluorescence Lifetime Imaging (FLIm) has received increasing attention from the ML community. FLIm goes beyond conventional spectral imaging, providing additional lifetime information, and could lead to optical histopathology supporting real-time diagnostics. However, most current studies do not use the full potential of machine/deep learning models. As a developing image modality, FLIm data are not easily obtainable, which, coupled with an absence of standardisation, is pushing back the research to develop models which could advance automated diagnosis and help promote FLIm. In this paper, we describe recent developments that improve FLIm image quality, specifically time-domain systems, and we summarise sensing, signal-to-noise analysis and the advances in registration and low-level tracking. We review the two main applications of ML for FLIm: lifetime estimation and image analysis through classification and segmentation. We suggest a course of action to improve the quality of ML studies applied to FLIm. Our final goal is to promote FLIm and attract more ML practitioners to explore the potential of lifetime imaging.
Collapse
Affiliation(s)
- Dorian Gouzou
- Dorian Gouzou and Marta Vallejo are with Institute of Signals, Sensors and Systems, School of Engineering and Physical Sciences, Heriot Watt University, Edinburgh, EH14 4AS, United Kingdom
| | - Ali Taimori
- Tarek Haloubi, Ali Taimori, and James R. Hopgood are with Institute for Imaging, Data and Communication, School of Engineering, University of Edinburgh, Edinburgh, EH9 3FG, United Kingdom
| | - Tarek Haloubi
- Tarek Haloubi, Ali Taimori, and James R. Hopgood are with Institute for Imaging, Data and Communication, School of Engineering, University of Edinburgh, Edinburgh, EH9 3FG, United Kingdom
| | - Neil Finlayson
- Neil Finlayson is with Institute for Integrated Micro and Nano Systems, School of Engineering, University ofEdinburgh, Edinburgh EH9 3FF, United Kingdom
| | - Qiang Wang
- Qiang Wang is with Centre for Inflammation Research, University of Edinburgh, Edinburgh, EH16 4TJ, United Kingdom
| | - James R Hopgood
- Tarek Haloubi, Ali Taimori, and James R. Hopgood are with Institute for Imaging, Data and Communication, School of Engineering, University of Edinburgh, Edinburgh, EH9 3FG, United Kingdom
| | - Marta Vallejo
- Dorian Gouzou and Marta Vallejo are with Institute of Signals, Sensors and Systems, School of Engineering and Physical Sciences, Heriot Watt University, Edinburgh, EH14 4AS, United Kingdom
| |
Collapse
|
3
|
Pal R, Lwin TM, Krishnamoorthy M, Collins HR, Chan CD, Prilutskiy A, Nasrallah MP, Dijkhuis TH, Shukla S, Kendall AL, Marshall MS, Carp SA, Hung YP, Shih AR, Martinez-Lage M, Zukerberg L, Sadow PM, Faquin WC, Nahed BV, Feng AL, Emerick KS, Mieog JSD, Vahrmeijer AL, Rajasekaran K, Lee JYK, Rankin KS, Lozano-Calderon S, Varvares MA, Tanabe KK, Kumar ATN. Fluorescence lifetime of injected indocyanine green as a universal marker of solid tumours in patients. Nat Biomed Eng 2023; 7:1649-1666. [PMID: 37845517 DOI: 10.1038/s41551-023-01105-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Accepted: 09/10/2023] [Indexed: 10/18/2023]
Abstract
The surgical resection of solid tumours can be enhanced by fluorescence-guided imaging. However, variable tumour uptake and incomplete clearance of fluorescent dyes reduces the accuracy of distinguishing tumour from normal tissue via conventional fluorescence intensity-based imaging. Here we show that, after systemic injection of the near-infrared dye indocyanine green in patients with various types of solid tumour, the fluorescence lifetime (FLT) of tumour tissue is longer than the FLT of non-cancerous tissue. This tumour-specific shift in FLT can be used to distinguish tumours from normal tissue with an accuracy of over 97% across tumour types, and can be visualized at the cellular level using microscopy and in larger specimens through wide-field imaging. Unlike fluorescence intensity, which depends on imaging-system parameters, tissue depth and the amount of dye taken up by tumours, FLT is a photophysical property that is largely independent of these factors. FLT imaging with indocyanine green may improve the accuracy of cancer surgeries.
Collapse
Affiliation(s)
- Rahul Pal
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, USA
| | - Thinzar M Lwin
- Department of Surgical Oncology, City of Hope Hospital, Duarte, CA, USA
| | - Murali Krishnamoorthy
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, USA
| | - Hannah R Collins
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, USA
| | - Corey D Chan
- Newcastle University Centre for Cancer, Translational and Clinical Research Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Andrey Prilutskiy
- Department of Pathology, University of Wisconsin School of Medicine and Public Health, Madison, WI, USA
| | - MacLean P Nasrallah
- Department of Pathology, Hospital of the University of Pennsylvania, Philadelphia, PA, USA
| | - Tom H Dijkhuis
- Department of Surgery, Leiden University Medical Center, Leiden, the Netherlands
| | - Shriya Shukla
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, USA
| | - Amy L Kendall
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, USA
| | - Michael S Marshall
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Stefan A Carp
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, USA
| | - Yin P Hung
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Angela R Shih
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Maria Martinez-Lage
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Lawrence Zukerberg
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Peter M Sadow
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Department of Otolaryngology and Head and Neck Surgery, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA
| | - William C Faquin
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Department of Otolaryngology and Head and Neck Surgery, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA
| | - Brian V Nahed
- Department of Neurosurgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Allen L Feng
- Department of Otolaryngology and Head and Neck Surgery, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA
| | - Kevin S Emerick
- Department of Otolaryngology and Head and Neck Surgery, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA
| | - J Sven D Mieog
- Department of Surgery, Leiden University Medical Center, Leiden, the Netherlands
| | | | - Karthik Rajasekaran
- Department of Otorhinolaryngology-Head and Neck Surgery, University of Pennsylvania, Philadelphia, PA, USA
| | - John Y K Lee
- Department of Neurosurgery, Hospital of the University of Pennsylvania, Philadelphia, PA, USA
| | - Kenneth S Rankin
- The North of England Bone and Soft Tissue Tumour Service, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK
| | - Santiago Lozano-Calderon
- Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Mark A Varvares
- Department of Otolaryngology and Head and Neck Surgery, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA
| | - Kenneth K Tanabe
- Division of Gastrointestinal and Oncologic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Anand T N Kumar
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Charlestown, MA, USA.
| |
Collapse
|
4
|
Prasad K, Topf MC, Clookey S, Philips R, Curry J, Tassone P. Trends in Positive Surgical Margins in cT3-T4 Oral Cavity Squamous Cell Carcinoma. Otolaryngol Head Neck Surg 2023; 169:1200-1207. [PMID: 37232479 DOI: 10.1002/ohn.377] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2022] [Revised: 04/11/2023] [Accepted: 04/29/2023] [Indexed: 05/27/2023]
Abstract
OBJECTIVE Positive surgical margins in oral cavity squamous cell carcinoma are associated with cost escalation, treatment intensification, and greater risk of recurrence and mortality. The positive margin rate has been decreasing for cT1-T2 oral cavity cancer over the past 2 decades. We aim to evaluate positive margin rates in cT3-T4 oral cavity cancer over time, and determine factors associated with positive margins. STUDY DESIGN Retrospective analysis of a national database. SETTING National Cancer Database 2004 to 2018. METHODS All adult patients diagnosed between 2004 and 2018 who underwent primary curative intent surgery for previously untreated cT3-T4 oral cavity cancer with known margin status were included. Logistic univariable and multivariable regression analyses were performed to identify factors associated with positive margins. RESULTS Among 16,326 patients with cT3 or cT4 oral cavity cancer, positive margins were documented in 2932 patients (18.1%). Later year of treatment was not significantly associated with positive margins (odds ratio [OR] 0.98, 95% confidence interval [CI] 0.96-1.00). The proportion of patients treated at academic centers increased over time (OR 1.02, 95% CI 1.01-1.03). On multivariable analysis, positive margins were significantly associated with hard palate primary, cT4 tumors, advancing N stage, lymphovascular invasion, poorly differentiated histology, and treatment at nonacademic or low-volume centers. CONCLUSION Despite increased treatment at academic centers for locally advanced oral cavity cancer, there has been no significant decrease in positive margin rates which remains high at 18.1%. Novel techniques for margin planning and assessment may be required to decrease positive margin rates in locally advanced oral cavity cancer.
Collapse
Affiliation(s)
- Kavita Prasad
- Department of Otolaryngology-Head & Neck Surgery, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Michael C Topf
- Department of Otolaryngology-Head & Neck Surgery, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | | | - Ramez Philips
- Department of Otolaryngology-Head and Neck Surgery, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Joseph Curry
- Department of Otolaryngology-Head and Neck Surgery, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Patrick Tassone
- Department of Otolaryngology-Head & Neck Surgery, University of Missouri, Columbia, Missouri, USA
| |
Collapse
|
5
|
Intraoperative Imaging Techniques to Improve Surgical Resection Margins of Oropharyngeal Squamous Cell Cancer: A Comprehensive Review of Current Literature. Cancers (Basel) 2023; 15:cancers15030896. [PMID: 36765858 PMCID: PMC9913756 DOI: 10.3390/cancers15030896] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 01/24/2023] [Accepted: 01/26/2023] [Indexed: 02/04/2023] Open
Abstract
Inadequate resection margins in head and neck squamous cell carcinoma surgery necessitate adjuvant therapies such as re-resection and radiotherapy with or without chemotherapy and imply increasing morbidity and worse prognosis. On the other hand, taking larger margins by extending the resection also leads to avoidable increased morbidity. Oropharyngeal squamous cell carcinomas (OPSCCs) are often difficult to access; resections are limited by anatomy and functionality and thus carry an increased risk for close or positive margins. Therefore, there is a need to improve intraoperative assessment of resection margins. Several intraoperative techniques are available, but these often lead to prolonged operative time and are only suitable for a subgroup of patients. In recent years, new diagnostic tools have been the subject of investigation. This study reviews the available literature on intraoperative techniques to improve resection margins for OPSCCs. A literature search was performed in Embase, PubMed, and Cochrane. Narrow band imaging (NBI), high-resolution microendoscopic imaging, confocal laser endomicroscopy, frozen section analysis (FSA), ultrasound (US), computed tomography scan (CT), (auto) fluorescence imaging (FI), and augmented reality (AR) have all been used for OPSCC. NBI, FSA, and US are most commonly used and increase the rate of negative margins. Other techniques will become available in the future, of which fluorescence imaging has high potential for use with OPSCC.
Collapse
|
6
|
Weyers BW, Birkeland AC, Marsden MA, Tam A, Bec J, Frusciante RP, Gui D, Bewley AF, Abouyared M, Marcu L, Farwell DG. Intraoperative delineation of p16+ oropharyngeal carcinoma of unknown primary origin with fluorescence lifetime imaging: Preliminary report. Head Neck 2022; 44:1765-1776. [PMID: 35511208 PMCID: PMC9979707 DOI: 10.1002/hed.27078] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 02/23/2022] [Accepted: 04/22/2022] [Indexed: 11/08/2022] Open
Abstract
BACKGROUND This study evaluated whether fluorescence lifetime imaging (FLIm), coupled with standard diagnostic workups, could enhance primary lesion detection in patients with p16+ head and neck squamous cell carcinoma of the unknown primary (HNSCCUP). METHODS FLIm was integrated into transoral robotic surgery to acquire optical data on six HNSCCUP patients' oropharyngeal tissues. An additional 55-patient FLIm dataset, comprising conventional primary tumors, trained a machine learning classifier; the output predicted the presence and location of HNSCCUP for the six patients. Validation was performed using histopathology. RESULTS Among the six HNSCCUP patients, p16+ occult primary was surgically identified in three patients, whereas three patients ultimately had no identifiable primary site in the oropharynx. FLIm correctly detected HNSCCUP in all three patients (ROC-AUC: 0.90 ± 0.06), and correctly predicted benign oropharyngeal tissue for the remaining three patients. The mean sensitivity was 95% ± 3.5%, and specificity 89% ± 12.7%. CONCLUSIONS FLIm may be a useful diagnostic adjunct for detecting HNSCCUP.
Collapse
Affiliation(s)
- Brent W. Weyers
- Department of Biomedical Engineering, University of California, Davis, Davis, California, USA
| | - Andrew C. Birkeland
- Department of Otolaryngology – Head & Neck Surgery, University of California, Davis, Davis, California, USA
| | - Mark A. Marsden
- Department of Biomedical Engineering, University of California, Davis, Davis, California, USA
| | - Athena Tam
- Department of Biomedical Engineering, University of California, Davis, Davis, California, USA
| | - Julien Bec
- Department of Biomedical Engineering, University of California, Davis, Davis, California, USA
| | - Roberto P. Frusciante
- Department of Biomedical Engineering, University of California, Davis, Davis, California, USA
| | - Dorina Gui
- Department of Pathology and Laboratory Medicine, University of California, Davis, Davis, California, USA
| | - Arnaud F. Bewley
- Department of Otolaryngology – Head & Neck Surgery, University of California, Davis, Davis, California, USA
| | - Marianne Abouyared
- Department of Otolaryngology – Head & Neck Surgery, University of California, Davis, Davis, California, USA
| | - Laura Marcu
- Department of Biomedical Engineering, University of California, Davis, Davis, California, USA
| | - Donald Gregory Farwell
- Department of Otolaryngology – Head & Neck Surgery, University of California, Davis, Davis, California, USA,Department of Otorhinolaryngology – Head and Neck Surgery, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
7
|
Young K, Ma E, Kejriwal S, Nielsen T, Aulakh SS, Birkeland AC. Intraoperative In Vivo Imaging Modalities in Head and Neck Cancer Surgical Margin Delineation: A Systematic Review. Cancers (Basel) 2022; 14:cancers14143416. [PMID: 35884477 PMCID: PMC9323577 DOI: 10.3390/cancers14143416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Revised: 07/06/2022] [Accepted: 07/12/2022] [Indexed: 11/16/2022] Open
Abstract
Surgical margin status is one of the strongest prognosticators in predicting patient outcomes in head and neck cancer, yet head and neck surgeons continue to face challenges in the accurate detection of these margins with the current standard of care. Novel intraoperative imaging modalities have demonstrated great promise for potentially increasing the accuracy and efficiency in surgical margin delineation. In this current study, we collated and analyzed various intraoperative imaging modalities utilized in head and neck cancer to evaluate their use in discriminating malignant from healthy tissues. The authors conducted a systematic database search through PubMed/Medline, Web of Science, and EBSCOhost (CINAHL). Study screening and data extraction were performed and verified by the authors, and more studies were added through handsearching. Here, intraoperative imaging modalities are described, including optical coherence tomography, narrow band imaging, autofluorescence, and fluorescent-tagged probe techniques. Available sensitivities and specificities in delineating cancerous from healthy tissues ranged from 83.0% to 100.0% and 79.2% to 100.0%, respectively, across the different imaging modalities. Many of these initial studies are in small sample sizes, with methodological differences that preclude more extensive quantitative comparison. Thus, there is impetus for future larger studies examining and comparing the efficacy of these intraoperative imaging technologies.
Collapse
Affiliation(s)
- Kurtis Young
- John A. Burns School of Medicine, Honolulu, HI 96813, USA; (K.Y.); (E.M.); (S.K.); (T.N.)
| | - Enze Ma
- John A. Burns School of Medicine, Honolulu, HI 96813, USA; (K.Y.); (E.M.); (S.K.); (T.N.)
| | - Sameer Kejriwal
- John A. Burns School of Medicine, Honolulu, HI 96813, USA; (K.Y.); (E.M.); (S.K.); (T.N.)
| | - Torbjoern Nielsen
- John A. Burns School of Medicine, Honolulu, HI 96813, USA; (K.Y.); (E.M.); (S.K.); (T.N.)
| | | | - Andrew C. Birkeland
- Department of Otolaryngology—Head and Neck Surgery, University of California, Davis, CA 95817, USA
- Correspondence:
| |
Collapse
|
8
|
Freymüller C, Ströbl S, Aumiller M, Eisel M, Sroka R, Rühm A. Development of a microstructured tissue phantom with adaptable optical properties for use with microscopes and fluorescence lifetime imaging systems. Lasers Surg Med 2022; 54:1010-1026. [PMID: 35753039 DOI: 10.1002/lsm.23556] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2021] [Revised: 04/22/2022] [Accepted: 04/24/2022] [Indexed: 11/08/2022]
Abstract
OBJECTIVES For the development and validation of diagnostic procedures based on microscopic methods, knowledge about the imaging depth and achievable resolution in tissue is crucial. This poses the challenge to develop a microscopic artificial phantom focused on the microscopic instead of the macroscopic optical tissue characteristics. METHODS As existing artificial tissue phantoms designed for image forming systems are primarily targeted at wide field applications, they are unsuited for reaching the formulated objective. Therefore, a microscopy- and microendoscopy-suited artificial tissue phantom was developed and characterized. It is based on a microstructured glass surface coated with fluorescent beads at known depths covered by a scattering agent with modifiable optical properties. The phantom was examined with different kinds of microscopy systems in order to characterize its quality and stability and to demonstrate its usefulness for instrument comparison, for example, regarding structural as well as fluorescence lifetime analysis. RESULTS The analysis of the manufactured microstructured glass surfaces showed high regularity in their physical dimensions in accordance with the specifications. Measurements of the optical parameters of the scattering medium were consistent with simulations. The fluorescent beads coating proved to be stable for a respectable period of time (about a week). The developed artificial tissue phantom was successfully used to detect differences in image quality between a research microscope and an endoscopy based system. Plausible causes for the observed differences could be derived based on the well known microstructure of the phantom. CONCLUSIONS The artificial tissue phantom is well suited for the intended use with microscopic and microendoscopic systems. Due to its configurable design, it can be adapted to a wide range of applications. It is especially targeted at the characterization and calibration of clinical imaging systems that often lack extensive positioning capabilities such as an intrinsic z-stage.
Collapse
Affiliation(s)
- Christian Freymüller
- Laser-Forschungslabor, LIFE Center, Department of Urology, University Hospital, LMU Munich, Munich, Germany.,Department of Urology, University Hospital, LMU Munich, Munich, Germany
| | - Stephan Ströbl
- Laser-Forschungslabor, LIFE Center, Department of Urology, University Hospital, LMU Munich, Munich, Germany.,Department of Urology, University Hospital, LMU Munich, Munich, Germany.,Research Center for Microtechnology, FH Vorarlberg, Dornbirn, Vorarlberg, Austria
| | - Maximilian Aumiller
- Laser-Forschungslabor, LIFE Center, Department of Urology, University Hospital, LMU Munich, Munich, Germany.,Department of Urology, University Hospital, LMU Munich, Munich, Germany
| | - Maximilian Eisel
- Laser-Forschungslabor, LIFE Center, Department of Urology, University Hospital, LMU Munich, Munich, Germany.,Department of Urology, University Hospital, LMU Munich, Munich, Germany
| | - Ronald Sroka
- Laser-Forschungslabor, LIFE Center, Department of Urology, University Hospital, LMU Munich, Munich, Germany.,Department of Urology, University Hospital, LMU Munich, Munich, Germany
| | - Adrian Rühm
- Laser-Forschungslabor, LIFE Center, Department of Urology, University Hospital, LMU Munich, Munich, Germany.,Department of Urology, University Hospital, LMU Munich, Munich, Germany
| |
Collapse
|
9
|
Pal R, Hom M, van den Berg NS, Lwin TM, Lee YJ, Prilutskiy A, Faquin W, Yang E, Saladi SV, Varvares MA, Rosenthal EL, Kumar ATN. First Clinical Results of Fluorescence Lifetime-enhanced Tumor Imaging Using Receptor-targeted Fluorescent Probes. Clin Cancer Res 2022; 28:2373-2384. [PMID: 35302604 PMCID: PMC9167767 DOI: 10.1158/1078-0432.ccr-21-3429] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2021] [Revised: 11/23/2021] [Accepted: 03/15/2022] [Indexed: 11/16/2022]
Abstract
PURPOSE Fluorescence molecular imaging, using cancer-targeted near infrared (NIR) fluorescent probes, offers the promise of accurate tumor delineation during surgeries and the detection of cancer specific molecular expression in vivo. However, nonspecific probe accumulation in normal tissue results in poor tumor fluorescence contrast, precluding widespread clinical adoption of novel imaging agents. Here we present the first clinical evidence that fluorescence lifetime (FLT) imaging can provide tumor specificity at the cellular level in patients systemically injected with panitumumab-IRDye800CW, an EGFR-targeted NIR fluorescent probe. EXPERIMENTAL DESIGN We performed wide-field and microscopic FLT imaging of resection specimens from patients injected with panitumumab-IRDye800CW under an FDA directed clinical trial. RESULTS We show that the FLT within EGFR-overexpressing cancer cells is significantly longer than the FLT of normal tissue, providing high sensitivity (>98%) and specificity (>98%) for tumor versus normal tissue classification, despite the presence of significant nonspecific probe accumulation. We further show microscopic evidence that the mean tissue FLT is spatially correlated (r > 0.85) with tumor-specific EGFR expression in tissue and is consistent across multiple patients. These tumor cell-specific FLT changes can be detected through thick biological tissue, allowing highly specific tumor detection and noninvasive monitoring of tumor EFGR expression in vivo. CONCLUSIONS Our data indicate that FLT imaging is a promising approach for enhancing tumor contrast using an antibody-targeted NIR probe with a proven safety profile in humans, suggesting a strong potential for clinical applications in image guided surgery, cancer diagnostics, and staging.
Collapse
Affiliation(s)
- Rahul Pal
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, 13 Street, Building 149, Charlestown MA 02129
| | - Marisa Hom
- Department of Otolaryngology, Vanderbilt University Medical Center, 1211 Medical Center Dr, Nashville, TN 37232
| | | | - Thinzar M Lwin
- Department of Surgery, Massachusetts General Hospital, Harvard Medical School, 55 Fruit Street, Boston MA
| | - Yu-Jin Lee
- Department of Otolaryngology, Stanford University School of Medicine, 900 Blake Wilbur Drive, Stanford CA
| | - Andrey Prilutskiy
- Department of Pathology, University of Wisconsin School of Medicine and Public Health, Madison WI
| | - William Faquin
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, 55 Fruit Street, Boston MA
| | - Eric Yang
- Department of Pathology, Stanford University School of Medicine, 900 Blake Wilbur Drive, Stanford CA
| | - Srinivas V. Saladi
- Department of Otolaryngology and Head and Neck Surgery, Massachusetts Eye and Ear, Harvard Medical School, 55 Fruit Street, Boston MA
| | - Mark A. Varvares
- Department of Otolaryngology and Head and Neck Surgery, Massachusetts Eye and Ear, Harvard Medical School, 55 Fruit Street, Boston MA
| | - Eben L. Rosenthal
- Department of Otolaryngology, Vanderbilt University Medical Center, 1211 Medical Center Dr, Nashville, TN 37232
| | - Anand T. N. Kumar
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, 13 Street, Building 149, Charlestown MA 02129
| |
Collapse
|
10
|
Dutrieux N, Le Coupanec P, Gil H, Koenig A, Abraham P, Quesada JL, Cracowski JL, Righini C, Coll JL. Safety of use of the ENDOSWIR near-infrared optical imaging device on human tissues: prospective blind study. Lasers Med Sci 2022; 37:2873-2877. [PMID: 35650311 DOI: 10.1007/s10103-022-03556-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2021] [Accepted: 03/31/2022] [Indexed: 10/18/2022]
Abstract
Cancer surgery requires removing the tumor tissue in necessary and sufficient quantities. Spectral optical imaging in the short-wave infrared (900-1700 nm) could provide an intraoperative guidance to the surgeon based on the absorption of the tissues without contrast agent. Our objective was to ensure the safety of our ENDOSWIR device on human tissues. Histological analysis of fresh human tonsils exposed to the SWIR light or not was compared and showed no histological differences. This demonstrates the safety of using the SWIR device on human tissues and allows us to initiate a clinical study for the resection of tumors intraoperatively.
Collapse
Affiliation(s)
- Noemie Dutrieux
- Department of Oto-Rhino-Laryngology, Head and Neck Surgery, Grenoble Alpes University Hospital, Grenoble, France.,Medical Faculty, University of Grenoble Alpes, Grenoble, France.,Institute for Advanced Biosciences, INSERM UGA U1209, CNRS UMR 5309, La Tronche, France
| | - Patricia Le Coupanec
- Univ. Grenoble Alpes, F-38000, Grenoble, France.,CEA, LETI, MINATEC Campus, F-38054, Grenoble, France
| | - Hugo Gil
- Medical Faculty, University of Grenoble Alpes, Grenoble, France.,Department of Anatomo-Cytopathology, Grenoble Alpes University Hospital, Grenoble, France
| | - Anne Koenig
- Univ. Grenoble Alpes, F-38000, Grenoble, France.,CEA, LETI, MINATEC Campus, F-38054, Grenoble, France
| | | | - Jean-Louis Quesada
- Medical Faculty, University of Grenoble Alpes, Grenoble, France.,Centre d'Investigation Clinique, INSERM, Grenoble Alpes University Hospital, Grenoble, France
| | - Jean-Luc Cracowski
- Medical Faculty, University of Grenoble Alpes, Grenoble, France.,Centre d'Investigation Clinique, INSERM, Grenoble Alpes University Hospital, Grenoble, France
| | - Christian Righini
- Department of Oto-Rhino-Laryngology, Head and Neck Surgery, Grenoble Alpes University Hospital, Grenoble, France.,Medical Faculty, University of Grenoble Alpes, Grenoble, France.,Institute for Advanced Biosciences, INSERM UGA U1209, CNRS UMR 5309, La Tronche, France
| | - Jean-Luc Coll
- Institute for Advanced Biosciences, INSERM UGA U1209, CNRS UMR 5309, La Tronche, France.
| |
Collapse
|
11
|
Jiang L, Liu D, Long L, Chen J, Lan X, Zhang J. Dual-source dual-energy computed tomography-derived quantitative parameters combined with machine learning for the differential diagnosis of benign and malignant thyroid nodules. Quant Imaging Med Surg 2022; 12:967-978. [PMID: 35111598 DOI: 10.21037/qims-21-501] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2021] [Accepted: 08/12/2021] [Indexed: 01/05/2023]
Abstract
Background This study aimed to investigate the ability of quantitative parameter-derived dual-source dual-energy computed tomography (DS-DECT) combined with machine learning to distinguish between benign and malignant thyroid nodules. Methods Patients with thyroid nodules and pathological surgical results who underwent preoperative DS-DECT were selected. Quantitative parameter-derived DS-DECT was applied to classify benign and malignant nodules. Then, machine learning and binary logistic regression analysis models were constructed using the DS-DECT quantitative parameters to distinguish between benign and malignant nodules. The receiver operating characteristic curve was used to assess the diagnostic performance. The DeLong test was used to compare the diagnostic efficacy. Results One hundred and thirty patients with 139 confirmed thyroid nodules were involved in the study. The malignant group had a significantly higher iodine concentrationnodule (arterial phase) (P=0.001), normalized iodine concentration (arterial phase) (P=0.002), iodine concentration difference (P<0.001), spectral curve slope (nonenhancement) (P=0.007), spectral curve slope (arterial phase) (P=0.001), effective atomic number (nonenhancement) (P<0.001), and effective atomic number (arterial phase) (P=0.039) than the benign group. The binary logistic regression analysis model had an AUC (area under the curve) of 0.76, a sensitivity of 0.821, and a specificity of 0.667. The machine learning model had an AUC of 0.86, a sensitivity of 0.822, specificity of 0.791 in the training cohort, an AUC of 0.84, a sensitivity of 0.727, and specificity of 0.750 in the testing cohort. Conclusions Multiple quantitative parameters of DS-DECT combined with machine learning could differentiate between benign and malignant thyroid nodules.
Collapse
Affiliation(s)
- Liling Jiang
- Department of Radiology, Chongqing University Cancer Hospital, School of Medicine, Chongqing University, Chongqing, China.,Chongqing Cancer Multi-omics Big Data Application Engineering Research Center, Chongqing University Cancer Hospital, Chongqing, China
| | - Daihong Liu
- Department of Radiology, Chongqing University Cancer Hospital, School of Medicine, Chongqing University, Chongqing, China.,Chongqing Cancer Multi-omics Big Data Application Engineering Research Center, Chongqing University Cancer Hospital, Chongqing, China
| | - Ling Long
- Department of Radiology, Chongqing University Cancer Hospital, School of Medicine, Chongqing University, Chongqing, China.,Chongqing Cancer Multi-omics Big Data Application Engineering Research Center, Chongqing University Cancer Hospital, Chongqing, China
| | - Jiao Chen
- Department of Radiology, Chongqing University Cancer Hospital, School of Medicine, Chongqing University, Chongqing, China.,Chongqing Cancer Multi-omics Big Data Application Engineering Research Center, Chongqing University Cancer Hospital, Chongqing, China
| | - Xiaosong Lan
- Department of Radiology, Chongqing University Cancer Hospital, School of Medicine, Chongqing University, Chongqing, China.,Chongqing Cancer Multi-omics Big Data Application Engineering Research Center, Chongqing University Cancer Hospital, Chongqing, China
| | - Jiuquan Zhang
- Department of Radiology, Chongqing University Cancer Hospital, School of Medicine, Chongqing University, Chongqing, China.,Chongqing Cancer Multi-omics Big Data Application Engineering Research Center, Chongqing University Cancer Hospital, Chongqing, China
| |
Collapse
|
12
|
Zhou X, Bec J, Yankelevich D, Marcu L. Multispectral fluorescence lifetime imaging device with a silicon avalanche photodetector. OPTICS EXPRESS 2021; 29:20105-20120. [PMID: 34266107 PMCID: PMC8237936 DOI: 10.1364/oe.425632] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Revised: 05/26/2021] [Accepted: 05/28/2021] [Indexed: 05/08/2023]
Abstract
We report the design, development, and characterization of a novel multi-spectral fluorescence lifetime measurement device incorporating solid-state detectors and automated gain control. For every excitation pulse (∼1 µJ, 600 ps), this device records complete fluorescence decay from multiple spectral channels simultaneously within microseconds, using a dedicated UV enhanced avalanche photodetector and analog to digital convert (2.5 GS/s) in each channel. Fast (<2 ms) channel-wise dynamic range adjustment maximizes the signal-to-noise ratio. Fluorophores with known lifetime ranging from 0.5-6.0 ns were used to demonstrate the device accuracy. Current results show the clear benefits of this device compared to existing devices employing microchannel-plate photomultiplier tubes. This is demonstrated by 5-fold reduction of lifetime measurement variability in identical conditions, independent gain adjustment in each spectral band, and 4-times faster imaging speed. The use of solid-state detectors will also facilitate future improved performance and miniaturization of the instrument.
Collapse
Affiliation(s)
- Xiangnan Zhou
- Department of Biomedical Engineering, University of California, 451 Health Sciences Drive, Davis, California 95616, USA
| | - Julien Bec
- Department of Biomedical Engineering, University of California, 451 Health Sciences Drive, Davis, California 95616, USA
| | - Diego Yankelevich
- Department of Electrical and Computer Engineering, University of California, 3101 Kemper Hall, Davis, California 95616, USA
| | - Laura Marcu
- Department of Biomedical Engineering, University of California, 451 Health Sciences Drive, Davis, California 95616, USA
| |
Collapse
|
13
|
Alfonso-Garcia A, Bec J, Weyers B, Marsden M, Zhou X, Li C, Marcu L. Mesoscopic fluorescence lifetime imaging: Fundamental principles, clinical applications and future directions. JOURNAL OF BIOPHOTONICS 2021; 14:e202000472. [PMID: 33710785 PMCID: PMC8579869 DOI: 10.1002/jbio.202000472] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/27/2020] [Revised: 03/03/2021] [Accepted: 03/05/2021] [Indexed: 05/16/2023]
Abstract
Fluorescence lifetime imaging (FLIm) is an optical spectroscopic imaging technique capable of real-time assessments of tissue properties in clinical settings. Label-free FLIm is sensitive to changes in tissue structure and biochemistry resulting from pathological conditions, thus providing optical contrast to identify and monitor the progression of disease. Technical and methodological advances over the last two decades have enabled the development of FLIm instrumentation for real-time, in situ, mesoscopic imaging compatible with standard clinical workflows. Herein, we review the fundamental working principles of mesoscopic FLIm, discuss the technical characteristics of current clinical FLIm instrumentation, highlight the most commonly used analytical methods to interpret fluorescence lifetime data and discuss the recent applications of FLIm in surgical oncology and cardiovascular diagnostics. Finally, we conclude with an outlook on the future directions of clinical FLIm.
Collapse
Affiliation(s)
- Alba Alfonso-Garcia
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Julien Bec
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Brent Weyers
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Mark Marsden
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Xiangnan Zhou
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Cai Li
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Laura Marcu
- Department of Biomedical Engineering, University of California, Davis, Davis, California
- Department Neurological Surgery, University of California, Davis, California
| |
Collapse
|
14
|
Marsden M, Weaver SS, Marcu L, Campbell MJ. Intraoperative Mapping of Parathyroid Glands Using Fluorescence Lifetime Imaging. J Surg Res 2021; 265:42-48. [PMID: 33878575 DOI: 10.1016/j.jss.2021.03.023] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2020] [Revised: 01/29/2021] [Accepted: 03/03/2021] [Indexed: 12/24/2022]
Abstract
BACKGROUND Hypoparathyroidism is a common complication following thyroidectomy. There is a need for technology to aid surgeons in identifying the parathyroid glands. In contrast to near infrared technologies, fluorescence lifetime imaging (FLIm) is not affected by ambient light and may be valuable in identifying parathyroid tissue, but has never been evaluated in this capacity. METHODS We used FLIm to measure the UV induced (355 nm) time-resolved autofluorescence signatures (average lifetimes in 3 spectral emission channels) of thyroid, parathyroid, lymphoid and adipose tissue in 21 patients undergoing thyroid and parathyroid surgery. The Mann-Whitney U test was used to assess the ability of FLIm to discriminate normocellular parathyroid from each of the other tissues. Various machine learning classifiers (random forests, neural network, support vector machine) were then evaluated to recognize parathyroid through a leave-one-out cross-validation. RESULTS Statistically significant differences in average lifetime were observed between parathyroid and each of the other tissue types in spectral channels 2 and 3 respectively. The largest change was observed between adipose tissue and parathyroid (P < 0.001), while less pronounced but still significant changes were observed when comparing parathyroid with lymphoid tissue (P < 0.05) and thyroid (P < 0.01). A random forest classifier trained on average lifetimes was found to detect parathyroid tissue with 100% sensitivity and 93% specificity at the acquisition run level. CONCLUSION We found that FLIm derived parameters can distinguish the parathyroid glands and other adjacent tissue types and has promise in scanning the surgical field to identify parathyroid tissue in real-time.
Collapse
Affiliation(s)
- Mark Marsden
- University of California, Davis Department of Biomedical Engineering, Sacramento, California
| | | | - Laura Marcu
- University of California, Davis Department of Biomedical Engineering, Sacramento, California
| | - Michael J Campbell
- University of California, Davis Department of Surgery, Sacramento, California.
| |
Collapse
|
15
|
Abstract
PURPOSE OF REVIEW Image guided navigation has had significant impact in head and neck surgery, and has been most prolific in endonasal surgeries. Although conventional image guidance involves static computed tomography (CT) images attained in the preoperative setting, the continual evolution of surgical navigation technologies is fast expanding to incorporate both real-time data and bioinformation that allows for improved precision in surgical guidance. With the rapid advances in technologies, this article allows for a timely review of the current and developing techniques in surgical navigation for head and neck surgery. RECENT FINDINGS Current advances for cross-sectional-based image-guided surgery include fusion of CT with other imaging modalities (e.g., magnetic resonance imaging and positron emission tomography) as well as the uptake in intraoperative real-time 'on the table' imaging (e.g., cone-beam CT). These advances, together with the integration of virtual/augmented reality, enable potential enhancements in surgical navigation. In addition to the advances in radiological imaging, the development of optical modalities such as fluorescence and spectroscopy techniques further allows the assimilation of biological data to improve navigation particularly for head and neck surgery. SUMMARY The steady development of radiological and optical imaging techniques shows great promise in changing the paradigm of head and neck surgery.
Collapse
|
16
|
Marsden M, Weyers BW, Bec J, Sun T, Gandour-Edwards RF, Birkeland AC, Abouyared M, Bewley AF, Farwell DG, Marcu L. Intraoperative Margin Assessment in Oral and Oropharyngeal Cancer Using Label-Free Fluorescence Lifetime Imaging and Machine Learning. IEEE Trans Biomed Eng 2021; 68:857-868. [PMID: 32746066 PMCID: PMC8960054 DOI: 10.1109/tbme.2020.3010480] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/22/2023]
Abstract
OBJECTIVE To demonstrate the diagnostic ability of label-free, point-scanning, fiber-based Fluorescence Lifetime Imaging (FLIm) as a means of intraoperative guidance during oral and oropharyngeal cancer removal surgery. METHODS FLIm point-measurements acquired from 53 patients (n = 67893 pre-resection in vivo, n = 89695 post-resection ex vivo) undergoing oral or oropharyngeal cancer removal surgery were used for analysis. Discrimination of healthy tissue and cancer was investigated using various FLIm-derived parameter sets and classifiers (Support Vector Machine, Random Forests, CNN). Classifier output for the acquired set of point-measurements was visualized through an interpolation-based approach to generate a probabilistic heatmap of cancer within the surgical field. Classifier output for dysplasia at the resection margins was also investigated. RESULTS Statistically significant change (P 0.01) between healthy and cancer was observed in vivo for the acquired FLIm signal parameters (e.g., average lifetime) linked with metabolic activity. Superior classification was achieved at the tissue region level using the Random Forests method (ROC-AUC: 0.88). Classifier output for dysplasia (% probability of cancer) was observed to lie between that of cancer and healthy tissue, highlighting FLIm's ability to distinguish various conditions. CONCLUSION The developed approach demonstrates the potential of FLIm for fast, reliable intraoperative margin assessment without the need for contrast agents. SIGNIFICANCE Fiber-based FLIm has the potential to be used as a diagnostic tool during cancer resection surgery, including Transoral Robotic Surgery (TORS), helping ensure complete resections and improve the survival rate of oral and oropharyngeal cancer patients.
Collapse
|
17
|
Assessment of breast cancer surgical margins with multimodal optical microscopy: A feasibility clinical study. PLoS One 2021; 16:e0245334. [PMID: 33571221 PMCID: PMC7877783 DOI: 10.1371/journal.pone.0245334] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2020] [Accepted: 12/28/2020] [Indexed: 11/18/2022] Open
Abstract
Providing surgical margin information during breast cancer surgery is crucial for the success of the procedure. The margin is defined as the distance from the tumor to the cut surface of the resection specimen. The consensus among surgeons and radiation oncologists is that there should be no tumor left within 1 to maximum 2 mm from the surface of the surgical specimen. If a positive margin remains, there is substantial risk for tumor recurrence, which may also result in potentially reduced cosmesis and eventual need for mastectomy. In this paper we report a novel multimodal optical imaging instrument based on combined high-resolution confocal microscopy-optical coherence tomography imaging for assessing the presence of potential positive margins on surgical specimens. Since rapid specimen analysis is critical during surgery, this instrument also includes a fluorescence imaging channel to enable rapid identification of the areas of the specimen that have potential positive margins. This is possible by specimen incubation with a cancer specific agent prior to imaging. In this study we used a quenched contrast agent, which is activated by cancer specific enzymes, such as urokinase plasminogen activators (uPA). Using this agent or a similar one, one may limit the use of high-resolution optical imaging to only fluorescence-highlighted areas for visualizing tissue morphology at the sub-cellular scale and confirming or ruling out cancer presence. Preliminary evaluation of this technology was performed on 20 surgical specimens and testing of the optical imaging findings was performed against histopathology. The combination of the three imaging modes allowed for high correlation between optical image analysis and histological ground-truth. The initial results are encouraging, showing instrument capability to assess margins on clinical specimens with a positive predictive value of 1.0 and a negative predictive value of 0.83.
Collapse
|
18
|
Marsden M, Fukazawa T, Deng YC, Weyers BW, Bec J, Gregory Farwell D, Marcu L. FLImBrush: dynamic visualization of intraoperative free-hand fiber-based fluorescence lifetime imaging. BIOMEDICAL OPTICS EXPRESS 2020; 11:5166-5180. [PMID: 33014606 PMCID: PMC7510860 DOI: 10.1364/boe.398357] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/27/2020] [Revised: 07/22/2020] [Accepted: 08/11/2020] [Indexed: 05/18/2023]
Abstract
A free-hand scanning approach to medical imaging allows for flexible, lightweight probes to image intricate anatomies for modalities such as fluorescence lifetime imaging (FLIm), optical coherence tomography (OCT) and ultrasound. While very promising, this approach faces several key challenges including tissue motion during imaging, varying lighting conditions in the surgical field, and sparse sampling of the tissue surface. These challenges limit the coregistration accuracy and interpretability of the acquired imaging data. Here we report FLImBrush as a robust method for the localization and visualization of intraoperative free-hand fiber optic fluorescence lifetime imaging (FLIm). FLImBrush builds upon an existing method while employing deep learning-based image segmentation, block-matching based motion correction, and interpolation-based visualization to address the aforementioned challenges. Current results demonstrate that FLImBrush can provide accurate localization of FLIm point-measurements while producing interpretable and complete visualizations of FLIm data acquired from a tissue surface. Each of the main processing steps was shown to be capable of real-time processing (> 30 frames per second), highlighting the feasibility of FLImBrush for intraoperative imaging and surgical guidance. Current findings show the feasibility of integrating FLImBrush into a range of surgical applications including cancer margins assessment during head and neck surgery.
Collapse
Affiliation(s)
- Mark Marsden
- Department of Biomedical Engineering, University of California, Davis, CA 95616, USA
- Equal Contribution
| | - Takanori Fukazawa
- Department of Biomedical Engineering, University of California, Davis, CA 95616, USA
- Sony Imaging Products and Solutions Inc., Japan
- Equal Contribution
| | - Yu-Cheng Deng
- Department of Biomedical Engineering, University of California, Davis, CA 95616, USA
| | - Brent W Weyers
- Department of Biomedical Engineering, University of California, Davis, CA 95616, USA
| | - Julien Bec
- Department of Biomedical Engineering, University of California, Davis, CA 95616, USA
| | - D Gregory Farwell
- Department of Otolaryngology, University of California, Davis, CA 95817, USA
- Corresponding authors
| | - Laura Marcu
- Department of Biomedical Engineering, University of California, Davis, CA 95616, USA
- Corresponding authors
| |
Collapse
|
19
|
Unger J, Hebisch C, Phipps JE, Lagarto JL, Kim H, Darrow MA, Bold RJ, Marcu L. Real-time diagnosis and visualization of tumor margins in excised breast specimens using fluorescence lifetime imaging and machine learning. BIOMEDICAL OPTICS EXPRESS 2020; 11:1216-1230. [PMID: 32206404 PMCID: PMC7075618 DOI: 10.1364/boe.381358] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/25/2019] [Revised: 01/14/2020] [Accepted: 01/14/2020] [Indexed: 05/03/2023]
Abstract
Tumor-free surgical margins are critical in breast-conserving surgery. In up to 38% of the cases, however, patients undergo a second surgery since malignant cells are found at the margins of the excised resection specimen. Thus, advanced imaging tools are needed to ensure clear margins at the time of surgery. The objective of this study was to evaluate a random forest classifier that makes use of parameters derived from point-scanning label-free fluorescence lifetime imaging (FLIm) measurements of breast specimens as a means to diagnose tumor at the resection margins and to enable an intuitive visualization of a probabilistic classifier on tissue specimen. FLIm data from fresh lumpectomy and mastectomy specimens from 18 patients were used in this study. The supervised training was based on a previously developed registration technique between autofluorescence imaging data and cross-sectional histology slides. A pathologist's histology annotations provide the ground truth to distinguish between adipose, fibrous, and tumor tissue. Current results demonstrate the ability of this approach to classify the tumor with 89% sensitivity and 93% specificity and to rapidly (∼ 20 frames per second) overlay the probabilistic classifier overlaid on excised breast specimens using an intuitive color scheme. Furthermore, we show an iterative imaging refinement that allows surgeons to switch between rapid scans with a customized, low spatial resolution to quickly cover the specimen and slower scans with enhanced resolution (400 μm per point measurement) in suspicious regions where more details are required. In summary, this technique provides high diagnostic prediction accuracy, rapid acquisition, adaptive resolution, nondestructive probing, and facile interpretation of images, thus holding potential for clinical breast imaging based on label-free FLIm.
Collapse
Affiliation(s)
- Jakob Unger
- Department of Biomedical Engineering, University of California Davis, California, CA 95616, USA
- Corresponding authors
| | - Christoph Hebisch
- Department of Biomedical Engineering, University of California Davis, California, CA 95616, USA
| | - Jennifer E. Phipps
- Department of Biomedical Engineering, University of California Davis, California, CA 95616, USA
| | - João L. Lagarto
- Department of Biomedical Engineering, University of California Davis, California, CA 95616, USA
| | - Hanna Kim
- Department of Otolaryngology, University of California Davis, California, CA 95817, USA
| | - Morgan A. Darrow
- Department of Pathology and Laboratory Medicine, University of California Davis, California, CA 95817, USA
| | - Richard J. Bold
- Department of Surgery, University of California Davis, California, CA 95817, USA
| | - Laura Marcu
- Department of Biomedical Engineering, University of California Davis, California, CA 95616, USA
- Corresponding authors
| |
Collapse
|