1
|
Wang J, Li B, Zhou T, Liu C, Lu M, Gu W, Liu X, Ta D. Reconstructing Cancellous Bone From Down-Sampled Optical-Resolution Photoacoustic Microscopy Images With Deep Learning. ULTRASOUND IN MEDICINE & BIOLOGY 2024; 50:1459-1471. [PMID: 38972792 DOI: 10.1016/j.ultrasmedbio.2024.05.027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/29/2023] [Revised: 03/21/2024] [Accepted: 05/30/2024] [Indexed: 07/09/2024]
Abstract
OBJECTIVE Bone diseases deteriorate the microstructure of bone tissue. Optical-resolution photoacoustic microscopy (OR-PAM) enables high spatial resolution of imaging bone tissues. However, the spatiotemporal trade-off limits the application of OR-PAM. The purpose of this study was to improve the quality of OR-PAM images without sacrificing temporal resolution. METHODS In this study, we proposed the Photoacoustic Dense Attention U-Net (PADA U-Net) model, which was used for reconstructing full-scanning images from under-sampled images. Thereby, this approach breaks the trade-off between imaging speed and spatial resolution. RESULTS The proposed method was validated on resolution test targets and bovine cancellous bone samples to demonstrate the capability of PADA U-Net in recovering full-scanning images from under-sampled OR-PAM images. With a down-sampling ratio of [4, 1], compared to bilinear interpolation, the Peak Signal-to-Noise Ratio and Structural Similarity Index Measure values (averaged over the test set of bovine cancellous bone) of the PADA U-Net were improved by 2.325 dB and 0.117, respectively. CONCLUSION The results demonstrate that the PADA U-Net model reconstructed the OR-PAM images well with different levels of sparsity. Our proposed method can further facilitate early diagnosis and treatment of bone diseases using OR-PAM.
Collapse
Affiliation(s)
- Jingxian Wang
- Human Phenome Institute, Fudan University, Shanghai, China
| | - Boyi Li
- Academy for Engineering and Technology, Fudan University, Shanghai, China.
| | - Tianhua Zhou
- Center for Biomedical Engineering, School of Information Science and Technology, Fudan University, Shanghai, China
| | - Chengcheng Liu
- Academy for Engineering and Technology, Fudan University, Shanghai, China
| | - Mengyang Lu
- Academy for Engineering and Technology, Fudan University, Shanghai, China
| | - Wenting Gu
- Academy for Engineering and Technology, Fudan University, Shanghai, China
| | - Xin Liu
- Academy for Engineering and Technology, Fudan University, Shanghai, China
| | - Dean Ta
- Academy for Engineering and Technology, Fudan University, Shanghai, China; Center for Biomedical Engineering, School of Information Science and Technology, Fudan University, Shanghai, China
| |
Collapse
|
2
|
Hu D, Jiang Z, Shi J, Xie F, Wu K, Tang K, Cao M, Huai J, Zheng Y. Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval. Med Image Anal 2024; 95:103163. [PMID: 38626665 DOI: 10.1016/j.media.2024.103163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Revised: 03/09/2024] [Accepted: 04/02/2024] [Indexed: 04/18/2024]
Abstract
Large-scale digital whole slide image (WSI) datasets analysis have gained significant attention in computer-aided cancer diagnosis. Content-based histopathological image retrieval (CBHIR) is a technique that searches a large database for data samples matching input objects in both details and semantics, offering relevant diagnostic information to pathologists. However, the current methods are limited by the difficulty of gigapixels, the variable size of WSIs, and the dependence on manual annotations. In this work, we propose a novel histopathology language-image representation learning framework for fine-grained digital pathology cross-modal retrieval, which utilizes paired diagnosis reports to learn fine-grained semantics from the WSI. An anchor-based WSI encoder is built to extract hierarchical region features and a prompt-based text encoder is introduced to learn fine-grained semantics from the diagnosis reports. The proposed framework is trained with a multivariate cross-modal loss function to learn semantic information from the diagnosis report at both the instance level and region level. After training, it can perform four types of retrieval tasks based on the multi-modal database to support diagnostic requirements. We conducted experiments on an in-house dataset and a public dataset to evaluate the proposed method. Extensive experiments have demonstrated the effectiveness of the proposed method and its advantages to the present histopathology retrieval methods. The code is available at https://github.com/hudingyi/FGCR.
Collapse
Affiliation(s)
- Dingyi Hu
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China; Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China
| | - Zhiguo Jiang
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China; Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China
| | - Jun Shi
- School of Software, Hefei University of Technology, Hefei 230601, China
| | - Fengying Xie
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China; Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China
| | - Kun Wu
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China; Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China
| | - Kunming Tang
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China; Image Processing Center, School of Astronautics, Beihang University, Beijing, 100191, China
| | - Ming Cao
- Department of Pathology, the First People's Hospital of Wuhu, China
| | - Jianguo Huai
- Department of Pathology, the First People's Hospital of Wuhu, China
| | - Yushan Zheng
- Beijing Advanced Innovation Center for Biomedical Engineering, School of Engineering Medicine, Beihang University, Beijing, 100191, China.
| |
Collapse
|
3
|
Wang Q, Akram AR, Dorward DA, Talas S, Monks B, Thum C, Hopgood JR, Javidi M, Vallejo M. Deep learning-based virtual H& E staining from label-free autofluorescence lifetime images. NPJ IMAGING 2024; 2:17. [PMID: 38948152 PMCID: PMC11213708 DOI: 10.1038/s44303-024-00021-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Accepted: 06/11/2024] [Indexed: 07/02/2024]
Abstract
Label-free autofluorescence lifetime is a unique feature of the inherent fluorescence signals emitted by natural fluorophores in biological samples. Fluorescence lifetime imaging microscopy (FLIM) can capture these signals enabling comprehensive analyses of biological samples. Despite the fundamental importance and wide application of FLIM in biomedical and clinical sciences, existing methods for analysing FLIM images often struggle to provide rapid and precise interpretations without reliable references, such as histology images, which are usually unavailable alongside FLIM images. To address this issue, we propose a deep learning (DL)-based approach for generating virtual Hematoxylin and Eosin (H&E) staining. By combining an advanced DL model with a contemporary image quality metric, we can generate clinical-grade virtual H&E-stained images from label-free FLIM images acquired on unstained tissue samples. Our experiments also show that the inclusion of lifetime information, an extra dimension beyond intensity, results in more accurate reconstructions of virtual staining when compared to using intensity-only images. This advancement allows for the instant and accurate interpretation of FLIM images at the cellular level without the complexities associated with co-registering FLIM and histology images. Consequently, we are able to identify distinct lifetime signatures of seven different cell types commonly found in the tumour microenvironment, opening up new opportunities towards biomarker-free tissue histology using FLIM across multiple cancer types.
Collapse
Affiliation(s)
- Qiang Wang
- Centre for Inflammation Research, Institute of Regeneration and Repair, The University of Edinburgh, Edinburgh, UK
- Translational Healthcare Technologies Group, Centre for Inflammation Research, Institute of Regeneration and Repair, The University of Edinburgh, Edinburgh, UK
| | - Ahsan R. Akram
- Centre for Inflammation Research, Institute of Regeneration and Repair, The University of Edinburgh, Edinburgh, UK
- Translational Healthcare Technologies Group, Centre for Inflammation Research, Institute of Regeneration and Repair, The University of Edinburgh, Edinburgh, UK
| | - David A. Dorward
- Centre for Inflammation Research, Institute of Regeneration and Repair, The University of Edinburgh, Edinburgh, UK
- Department of Pathology, Royal Infirmary of Edinburgh, Edinburgh, UK
| | - Sophie Talas
- Centre for Inflammation Research, Institute of Regeneration and Repair, The University of Edinburgh, Edinburgh, UK
- Department of Pathology, Royal Infirmary of Edinburgh, Edinburgh, UK
| | - Basil Monks
- Department of Pathology, Royal Infirmary of Edinburgh, Edinburgh, UK
| | - Chee Thum
- Department of Pathology, Royal Infirmary of Edinburgh, Edinburgh, UK
| | - James R. Hopgood
- School of Engineering, The University of Edinburgh, Edinburgh, UK
| | - Malihe Javidi
- School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, UK
- Department of Computer Engineering, Quchan University of Technology, Quchan, Iran
| | - Marta Vallejo
- School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, UK
| |
Collapse
|
4
|
Wan M, Zhang Y, Li J, Qian Z, Gao F, Yang Y, Li W. Optical-resolution photoacoustic microelastography system for elasticity mapping: Phantom study and practical application. JOURNAL OF BIOPHOTONICS 2024:e202400032. [PMID: 38894573 DOI: 10.1002/jbio.202400032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Revised: 04/26/2024] [Accepted: 05/16/2024] [Indexed: 06/21/2024]
Abstract
Elastography is a noninvasive technique for characterizing the mechanical properties of biological tissues. Conventional methods have limitations in resolution and sensitivity, hindering disease detection in clinical diagnostics. To address these issues, this study developed an optical-resolution photoacoustic microelastography (OR-PAME) system. Using an agar tissue phantom with varying agar concentrations and contrast agents, PAME evaluated elasticity distribution under compression in both lateral and axial dimensions. It indirectly measured elastic properties by correlating photoacoustic responses, temporal lags, and induced displacement. We also applied the system to the study of the distribution of elastic characteristics of the liver tissue after ablation, which confirmed the potential of OR-PAME in the study of elastic characteristics. Quantitative analysis showed greater lateral displacement in regions with reduced agar concentrations, indicating decreased stiffness. PAME also detected vertical displacement along the axial plane, validating its efficacy in elastographic imaging. By improving resolution and penetration, PAME provides superior visualization of elasticity distribution. Its methodology correlates microstructural alterations with tissue biomechanics, holding potential implications in medical diagnostics.
Collapse
Affiliation(s)
- Min Wan
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
| | - Yameng Zhang
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
- Department of Computer Engineering, Nanjing Institute of Technology, Nanjing, Jiangsu, China
| | - Jiani Li
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
| | - Zhiyu Qian
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
| | - Fan Gao
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
| | - Yamin Yang
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
| | - Weitao Li
- Department of Biomedical Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China
| |
Collapse
|
5
|
He H, Wang H, You H, Dong X, Shi J, Dong J. 30-100 kHz, 2 ns passively Q-switched laser for fast and efficient photoacoustic microscopy. JOURNAL OF BIOPHOTONICS 2024; 17:e202300437. [PMID: 38450961 DOI: 10.1002/jbio.202300437] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Revised: 12/10/2023] [Accepted: 02/12/2024] [Indexed: 03/08/2024]
Abstract
Actively Q-switched (AQS) fiber laser and solid-state laser (SSL) are widely used for photoacoustic microscopy (PAM). In contrast, passively Q-switched (PQS) SSL not only maintains most of the merits of AQS lasers, but also exhibits unique advantages, including the pulse width (PW), pulse repetition rate (PRR) tunability, wavelength, compactness, and cost. These advantages all benefit the PAM. However, there are few reports demonstrating the performance of PQS-SSL on PA imaging. Here, we demonstrate a compact PQS-SSL for fast and efficient PA imaging. The laser uniquely maintains a constant PW (~2 ns) and pulse energy (~3 μJ) during the PRR variation (30-100 kHz), which is valuable for preserving a stabilized imaging performance at different scanning rates. The PA imaging performance is compared by a resolution target and showcased by whole-body scanning of an embryonic zebrafish in vivo. The performance indicates that PQS-SSL is a promising candidate for PAM.
Collapse
Affiliation(s)
- Hongsen He
- Laboratory of Laser and Applied Photonics (LLAP), Department of Electronic Engineering, School of Electronic Science and Engineering, Xiamen University, Xiamen, China
- Fujian Key Laboratory of Ultrafast Laser Technology and Applications, Xiamen University, Xiamen, China
| | - Hanjie Wang
- Laboratory of Laser and Applied Photonics (LLAP), Department of Electronic Engineering, School of Electronic Science and Engineering, Xiamen University, Xiamen, China
| | - Huiyue You
- Laboratory of Laser and Applied Photonics (LLAP), Department of Electronic Engineering, School of Electronic Science and Engineering, Xiamen University, Xiamen, China
| | - Xin Dong
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, China
| | - Jiawei Shi
- Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, China
| | - Jun Dong
- Laboratory of Laser and Applied Photonics (LLAP), Department of Electronic Engineering, School of Electronic Science and Engineering, Xiamen University, Xiamen, China
- Fujian Key Laboratory of Ultrafast Laser Technology and Applications, Xiamen University, Xiamen, China
| |
Collapse
|
6
|
Zhang Y, Hu P, Li L, Cao R, Khadria A, Maslov K, Tong X, Zeng Y, Jiang L, Zhou Q, Wang LV. Ultrafast longitudinal imaging of haemodynamics via single-shot volumetric photoacoustic tomography with a single-element detector. Nat Biomed Eng 2024; 8:712-725. [PMID: 38036618 PMCID: PMC11136871 DOI: 10.1038/s41551-023-01149-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Accepted: 10/25/2023] [Indexed: 12/02/2023]
Abstract
Techniques for imaging haemodynamics use ionizing radiation or contrast agents or are limited by imaging depth (within approximately 1 mm), complex and expensive data-acquisition systems, or low imaging speeds, system complexity or cost. Here we show that ultrafast volumetric photoacoustic imaging of haemodynamics in the human body at up to 1 kHz can be achieved using a single laser pulse and a single element functioning as 6,400 virtual detectors. The technique, which does not require recalibration for different objects or during long-term operation, enables the longitudinal volumetric imaging of haemodynamics in vasculature a few millimetres below the skin's surface. We demonstrate this technique in vessels in the feet of healthy human volunteers by capturing haemodynamic changes in response to vascular occlusion. Single-shot volumetric photoacoustic imaging using a single-element detector may facilitate the early detection and monitoring of peripheral vascular diseases and may be advantageous for use in biometrics and point-of-care testing.
Collapse
Affiliation(s)
- Yide Zhang
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Peng Hu
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Lei Li
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Rui Cao
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Anjul Khadria
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Konstantin Maslov
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Xin Tong
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Yushun Zeng
- USC Roski Eye Institute, University of Southern California, Los Angeles, CA, USA
- Alfred E. Mann Department of Biomedical Engineering, University of Southern California, Los Angeles, CA, USA
| | - Laiming Jiang
- USC Roski Eye Institute, University of Southern California, Los Angeles, CA, USA
- Alfred E. Mann Department of Biomedical Engineering, University of Southern California, Los Angeles, CA, USA
| | - Qifa Zhou
- USC Roski Eye Institute, University of Southern California, Los Angeles, CA, USA
- Alfred E. Mann Department of Biomedical Engineering, University of Southern California, Los Angeles, CA, USA
| | - Lihong V Wang
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA.
| |
Collapse
|
7
|
Wang Z, Han K, Liu W, Wang Z, Shi C, Liu X, Huang M, Sun G, Liu S, Guo Q. Fast Real-Time Brain Tumor Detection Based on Stimulated Raman Histology and Self-Supervised Deep Learning Model. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1160-1176. [PMID: 38326533 PMCID: PMC11169153 DOI: 10.1007/s10278-024-01001-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Revised: 12/18/2023] [Accepted: 12/21/2023] [Indexed: 02/09/2024]
Abstract
In intraoperative brain cancer procedures, real-time diagnosis is essential for ensuring safe and effective care. The prevailing workflow, which relies on histological staining with hematoxylin and eosin (H&E) for tissue processing, is resource-intensive, time-consuming, and requires considerable labor. Recently, an innovative approach combining stimulated Raman histology (SRH) and deep convolutional neural networks (CNN) has emerged, creating a new avenue for real-time cancer diagnosis during surgery. While this approach exhibits potential, there exists an opportunity for refinement in the domain of feature extraction. In this study, we employ coherent Raman scattering imaging method and a self-supervised deep learning model (VQVAE2) to enhance the speed of SRH image acquisition and feature representation, thereby enhancing the capability of automated real-time bedside diagnosis. Specifically, we propose the VQSRS network, which integrates vector quantization with a proxy task based on patch annotation for analysis of brain tumor subtypes. Training on images collected from the SRS microscopy system, our VQSRS demonstrates a significant speed enhancement over traditional techniques (e.g., 20-30 min). Comparative studies in dimensionality reduction clustering confirm the diagnostic capacity of VQSRS rivals that of CNN. By learning a hierarchical structure of recognizable histological features, VQSRS classifies major tissue pathological categories in brain tumors. Additionally, an external semantic segmentation method is applied for identifying tumor-infiltrated regions in SRH images. Collectively, these findings indicate that this automated real-time prediction technique holds the potential to streamline intraoperative cancer diagnosis, providing assistance to pathologists in simplifying the process.
Collapse
Affiliation(s)
- Zijun Wang
- Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China
| | - Kaitai Han
- Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China
| | - Wu Liu
- Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China
| | - Zhenghui Wang
- Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China
| | - Chaojing Shi
- Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China
| | - Xi Liu
- Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China
| | - Mengyuan Huang
- Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China
| | - Guocheng Sun
- Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China
| | - Shitou Liu
- Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China
| | - Qianjin Guo
- Academy of Artificial Intelligence, Beijing Institute of Petrochemical Technology, Beijing, 102617, China.
| |
Collapse
|
8
|
Tweel JED, Ecclestone BR, Boktor M, Dinakaran D, Mackey JR, Reza PH. Automated Whole Slide Imaging for Label-Free Histology Using Photon Absorption Remote Sensing Microscopy. IEEE Trans Biomed Eng 2024; 71:1901-1912. [PMID: 38231822 DOI: 10.1109/tbme.2024.3355296] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/19/2024]
Abstract
OBJECTIVE Pathologists rely on histochemical stains to impart contrast in thin translucent tissue samples, revealing tissue features necessary for identifying pathological conditions. However, the chemical labeling process is destructive and often irreversible or challenging to undo, imposing practical limits on the number of stains that can be applied to the same tissue section. Here we present an automated label-free whole slide scanner using a PARS microscope designed for imaging thin, transmissible samples. METHODS Peak SNR and in-focus acquisitions are achieved across entire tissue sections using the scattering signal from the PARS detection beam to measure the optimal focal plane. Whole slide images (WSI) are seamlessly stitched together using a custom contrast leveling algorithm. Identical tissue sections are subsequently H&E stained and brightfield imaged. The one-to-one WSIs from both modalities are visually and quantitatively compared. RESULTS PARS WSIs are presented at standard 40x magnification in malignant human breast and skin samples. We show correspondence of subcellular diagnostic details in both PARS and H&E WSIs and demonstrate virtual H&E staining of an entire PARS WSI. The one-to-one WSI from both modalities show quantitative similarity in nuclear features and structural information. CONCLUSION PARS WSIs are compatible with existing digital pathology tools, and samples remain suitable for histochemical, immunohistochemical, and other staining techniques. SIGNIFICANCE This work is a critical advance for integrating label-free optical methods into standard histopathology workflows.
Collapse
|
9
|
Ma L, Luo K, Liu Z, Ji M. Stain-Free Histopathology with Stimulated Raman Scattering Microscopy. Anal Chem 2024; 96:7907-7925. [PMID: 38713830 DOI: 10.1021/acs.analchem.4c02061] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/09/2024]
Affiliation(s)
- Liyang Ma
- State Key Laboratory of Surface Physics and Department of Physics, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Shanghai Key Laboratory of Metasurfaces for Light Manipulation, Fudan University, Shanghai 200433, China
| | - Kuan Luo
- State Key Laboratory of Surface Physics and Department of Physics, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Shanghai Key Laboratory of Metasurfaces for Light Manipulation, Fudan University, Shanghai 200433, China
| | - Zhijie Liu
- State Key Laboratory of Surface Physics and Department of Physics, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Shanghai Key Laboratory of Metasurfaces for Light Manipulation, Fudan University, Shanghai 200433, China
| | - Minbiao Ji
- State Key Laboratory of Surface Physics and Department of Physics, Academy for Engineering and Technology, Key Laboratory of Micro and Nano Photonic Structures (Ministry of Education), Shanghai Key Laboratory of Metasurfaces for Light Manipulation, Fudan University, Shanghai 200433, China
| |
Collapse
|
10
|
Song AH, Williams M, Williamson DFK, Chow SSL, Jaume G, Gao G, Zhang A, Chen B, Baras AS, Serafin R, Colling R, Downes MR, Farré X, Humphrey P, Verrill C, True LD, Parwani AV, Liu JTC, Mahmood F. Analysis of 3D pathology samples using weakly supervised AI. Cell 2024; 187:2502-2520.e17. [PMID: 38729110 PMCID: PMC11168832 DOI: 10.1016/j.cell.2024.03.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 01/15/2024] [Accepted: 03/25/2024] [Indexed: 05/12/2024]
Abstract
Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.
Collapse
Affiliation(s)
- Andrew H Song
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Mane Williams
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Drew F K Williamson
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Sarah S L Chow
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Guillaume Jaume
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Gan Gao
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Andrew Zhang
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Bowen Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Alexander S Baras
- Department of Pathology, Johns Hopkins University School of Medicine, Baltimore, MD, USA; Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Robert Serafin
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA
| | - Richard Colling
- Nuffield Department of Surgical Sciences, University of Oxford, UK; Department of Cellular Pathology, Oxford University Hospitals NHS Foundations Trust, John Radcliffe Hospital, Oxford, UK
| | - Michelle R Downes
- Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | - Xavier Farré
- Public Health Agency of Catalonia, Lleida, Spain
| | - Peter Humphrey
- Department of Pathology, Yale School of Medicine, New Haven, CT, USA
| | - Clare Verrill
- Nuffield Department of Surgical Sciences, University of Oxford, UK; Department of Cellular Pathology, Oxford University Hospitals NHS Foundations Trust, John Radcliffe Hospital, Oxford, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Lawrence D True
- Department of Laboratory Medicine & Pathology, University of Washington School of Medicine, Seattle, WA, USA
| | - Anil V Parwani
- Department of Pathology, The Ohio State University, Columbus, OH, USA
| | - Jonathan T C Liu
- Department of Mechanical Engineering, Bioengineering, and Laboratory Medicine & Pathology, University of Washington, Seattle, WA, USA.
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
| |
Collapse
|
11
|
Winetraub Y, Van Vleck A, Yuan E, Terem I, Zhao J, Yu C, Chan W, Do H, Shevidi S, Mao M, Yu J, Hong M, Blankenberg E, Rieger KE, Chu S, Aasi S, Sarin KY, de la Zerda A. Noninvasive virtual biopsy using micro-registered optical coherence tomography (OCT) in human subjects. SCIENCE ADVANCES 2024; 10:eadi5794. [PMID: 38598626 PMCID: PMC11006228 DOI: 10.1126/sciadv.adi5794] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 03/07/2024] [Indexed: 04/12/2024]
Abstract
Histological hematoxylin and eosin-stained (H&E) tissue sections are used as the gold standard for pathologic detection of cancer, tumor margin detection, and disease diagnosis. Producing H&E sections, however, is invasive and time-consuming. While deep learning has shown promise in virtual staining of unstained tissue slides, true virtual biopsy requires staining of images taken from intact tissue. In this work, we developed a micron-accuracy coregistration method [micro-registered optical coherence tomography (OCT)] that can take a two-dimensional (2D) H&E slide and find the exact corresponding section in a 3D OCT image taken from the original fresh tissue. We trained a conditional generative adversarial network using the paired dataset and showed high-fidelity conversion of noninvasive OCT images to virtually stained H&E slices in both 2D and 3D. Applying these trained neural networks to in vivo OCT images should enable physicians to readily incorporate OCT imaging into their clinical practice, reducing the number of unnecessary biopsy procedures.
Collapse
Affiliation(s)
- Yonatan Winetraub
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
- Molecular Imaging Program at Stanford, Stanford, CA 94305, USA
- The Bio-X Program, Stanford, CA 94305, USA
- Biophysics Program at Stanford, Stanford, CA 94305, USA
| | - Aidan Van Vleck
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
| | - Edwin Yuan
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
- Molecular Imaging Program at Stanford, Stanford, CA 94305, USA
- Department of Applied Physics, Stanford University, Stanford, CA 94305, USA
| | - Itamar Terem
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
- Molecular Imaging Program at Stanford, Stanford, CA 94305, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA
| | - Jinjing Zhao
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
| | - Caroline Yu
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
- Molecular Imaging Program at Stanford, Stanford, CA 94305, USA
| | - Warren Chan
- Department of Dermatology, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Hanh Do
- Department of Dermatology, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Saba Shevidi
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
- Molecular Imaging Program at Stanford, Stanford, CA 94305, USA
| | - Maiya Mao
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
- Molecular Imaging Program at Stanford, Stanford, CA 94305, USA
| | - Jacqueline Yu
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
- Molecular Imaging Program at Stanford, Stanford, CA 94305, USA
| | - Megan Hong
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
- Molecular Imaging Program at Stanford, Stanford, CA 94305, USA
| | - Erick Blankenberg
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
- Molecular Imaging Program at Stanford, Stanford, CA 94305, USA
| | - Kerri E. Rieger
- Department of Pathology, Stanford University School of Medicine and Stanford Cancer Institute, Stanford, CA 94305, USA
| | - Steven Chu
- The Bio-X Program, Stanford, CA 94305, USA
- Biophysics Program at Stanford, Stanford, CA 94305, USA
- Departments of Physics and Molecular and Cellular Physiology, Energy, Science and Engineering Stanford University, Stanford, CA 94305, USA
| | - Sumaira Aasi
- Department of Dermatology, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Kavita Y. Sarin
- Department of Dermatology, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Adam de la Zerda
- Department of Structural Biology, Stanford University, Stanford, CA 94305, USA
- Molecular Imaging Program at Stanford, Stanford, CA 94305, USA
- The Bio-X Program, Stanford, CA 94305, USA
- Biophysics Program at Stanford, Stanford, CA 94305, USA
- Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA
- The Chan Zuckerberg Biohub, San Francisco, CA 94158, USA
| |
Collapse
|
12
|
Jin L, Tang Y, Coole JB, Tan MT, Zhao X, Badaoui H, Robinson JT, Williams MD, Vigneswaran N, Gillenwater AM, Richards-Kortum RR, Veeraraghavan A. DeepDOF-SE: affordable deep-learning microscopy platform for slide-free histology. Nat Commun 2024; 15:2935. [PMID: 38580633 PMCID: PMC10997797 DOI: 10.1038/s41467-024-47065-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 03/19/2024] [Indexed: 04/07/2024] Open
Abstract
Histopathology plays a critical role in the diagnosis and surgical management of cancer. However, access to histopathology services, especially frozen section pathology during surgery, is limited in resource-constrained settings because preparing slides from resected tissue is time-consuming, labor-intensive, and requires expensive infrastructure. Here, we report a deep-learning-enabled microscope, named DeepDOF-SE, to rapidly scan intact tissue at cellular resolution without the need for physical sectioning. Three key features jointly make DeepDOF-SE practical. First, tissue specimens are stained directly with inexpensive vital fluorescent dyes and optically sectioned with ultra-violet excitation that localizes fluorescent emission to a thin surface layer. Second, a deep-learning algorithm extends the depth-of-field, allowing rapid acquisition of in-focus images from large areas of tissue even when the tissue surface is highly irregular. Finally, a semi-supervised generative adversarial network virtually stains DeepDOF-SE fluorescence images with hematoxylin-and-eosin appearance, facilitating image interpretation by pathologists without significant additional training. We developed the DeepDOF-SE platform using a data-driven approach and validated its performance by imaging surgical resections of suspected oral tumors. Our results show that DeepDOF-SE provides histological information of diagnostic importance, offering a rapid and affordable slide-free histology platform for intraoperative tumor margin assessment and in low-resource settings.
Collapse
Affiliation(s)
- Lingbo Jin
- Department of Electrical and Computer Engineering, Rice University, 6100 Main St, Houston, TX, USA
| | - Yubo Tang
- Department of Bioengineering, Rice University, 6100 Main St, Houston, TX, USA
| | - Jackson B Coole
- Department of Bioengineering, Rice University, 6100 Main St, Houston, TX, USA
| | - Melody T Tan
- Department of Bioengineering, Rice University, 6100 Main St, Houston, TX, USA
| | - Xuan Zhao
- Department of Electrical and Computer Engineering, Rice University, 6100 Main St, Houston, TX, USA
| | - Hawraa Badaoui
- Department of Head and Neck Surgery, University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, USA
| | - Jacob T Robinson
- Department of Electrical and Computer Engineering, Rice University, 6100 Main St, Houston, TX, USA
| | - Michelle D Williams
- Department of Pathology, University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, USA
| | - Nadarajah Vigneswaran
- Department of Diagnostic and Biomedical Sciences, University of Texas Health Science Center at Houston School of Dentistry, 7500 Cambridge St, Houston, TX, USA
| | - Ann M Gillenwater
- Department of Head and Neck Surgery, University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, USA
| | | | - Ashok Veeraraghavan
- Department of Electrical and Computer Engineering, Rice University, 6100 Main St, Houston, TX, USA.
| |
Collapse
|
13
|
Wong IHM, Chen Z, Shi L, Lo CTK, Kang L, Dai W, Wong TTW. Deep learning-assisted low-cost autofluorescence microscopy for rapid slide-free imaging with virtual histological staining. BIOMEDICAL OPTICS EXPRESS 2024; 15:2187-2201. [PMID: 38633074 PMCID: PMC11019672 DOI: 10.1364/boe.515018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Revised: 01/27/2024] [Accepted: 02/20/2024] [Indexed: 04/19/2024]
Abstract
Slide-free imaging techniques have shown great promise in improving the histological workflow. For example, computational high-throughput autofluorescence microscopy by pattern illumination (CHAMP) has achieved high resolution with a long depth of field, which, however, requires a costly ultraviolet laser. Here, simply using a low-cost light-emitting diode (LED), we propose a deep learning-assisted framework of enhanced widefield microscopy, termed EW-LED, to generate results similar to CHAMP (the learning target). Comparing EW-LED and CHAMP, EW-LED reduces the cost by 85×, shortening the image acquisition time and computation time by 36× and 17×, respectively. This framework can be applied to other imaging modalities, enhancing widefield images for better virtual histology.
Collapse
Affiliation(s)
| | | | - Lulin Shi
- Translational and Advanced Bioimaging Laboratory, Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology, Hong Kong, China
| | - Claudia T. K. Lo
- Translational and Advanced Bioimaging Laboratory, Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology, Hong Kong, China
| | - Lei Kang
- Translational and Advanced Bioimaging Laboratory, Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology, Hong Kong, China
| | - Weixing Dai
- Translational and Advanced Bioimaging Laboratory, Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology, Hong Kong, China
| | - Terence T. W. Wong
- Translational and Advanced Bioimaging Laboratory, Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology, Hong Kong, China
| |
Collapse
|
14
|
Abraham TM, Levenson R. Current Landscape of Advanced Imaging Tools for Pathology Diagnostics. Mod Pathol 2024; 37:100443. [PMID: 38311312 DOI: 10.1016/j.modpat.2024.100443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 12/13/2023] [Accepted: 01/26/2024] [Indexed: 02/10/2024]
Abstract
Histopathology relies on century-old workflows of formalin fixation, paraffin embedding, sectioning, and staining tissue specimens on glass slides. Despite being robust, this conventional process is slow, labor-intensive, and limited to providing two-dimensional views. Emerging technologies promise to enhance and accelerate histopathology. Slide-free microscopy allows rapid imaging of fresh, unsectioned specimens, overcoming slide preparation delays. Methods such as fluorescence confocal microscopy, multiphoton microscopy, along with more recent innovations including microscopy with UV surface excitation and fluorescence-imitating brightfield imaging can generate images resembling conventional histology directly from the surface of tissue specimens. Slide-free microscopy enable applications such as rapid intraoperative margin assessment and, with appropriate technology, three-dimensional histopathology. Multiomics profiling techniques, including imaging mass spectrometry and Raman spectroscopy, provide highly multiplexed molecular maps of tissues, although clinical translation remains challenging. Artificial intelligence is aiding the adoption of new imaging modalities via virtual staining, which converts methods such as slide-free microscopy into synthetic brightfield-like or even molecularly informed images. Although not yet commonplace, these emerging technologies collectively demonstrate the potential to modernize histopathology. Artificial intelligence-assisted workflows will ease the transition to new imaging modalities. With further validation, these advances may transform the century-old conventional histopathology pipeline to better serve 21st-century medicine. This review provides an overview of these enabling technology platforms and discusses their potential impact.
Collapse
Affiliation(s)
- Tanishq Mathew Abraham
- Department of Biomedical Engineering, University of California, Davis, Davis, California
| | - Richard Levenson
- Department of Pathology and Laboratory Medicine, UC Davis Health, Sacramento, California.
| |
Collapse
|
15
|
Cheng S, Chang S, Li Y, Novoseltseva A, Lin S, Wu Y, Zhu J, McKee AC, Rosene DL, Wang H, Bigio IJ, Boas DA, Tian L. Enhanced Multiscale Human Brain Imaging by Semi-supervised Digital Staining and Serial Sectioning Optical Coherence Tomography. RESEARCH SQUARE 2024:rs.3.rs-4014687. [PMID: 38562721 PMCID: PMC10984089 DOI: 10.21203/rs.3.rs-4014687/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
A major challenge in neuroscience is to visualize the structure of the human brain at different scales. Traditional histology reveals micro- and meso-scale brain features, but suffers from staining variability, tissue damage and distortion that impedes accurate 3D reconstructions. Here, we present a new 3D imaging framework that combines serial sectioning optical coherence tomography (S-OCT) with a deep-learning digital staining (DS) model. We develop a novel semi-supervised learning technique to facilitate DS model training on weakly paired images. The DS model performs translation from S-OCT to Gallyas silver staining. We demonstrate DS on various human cerebral cortex samples with consistent staining quality. Additionally, we show that DS enhances contrast across cortical layer boundaries. Furthermore, we showcase geometry-preserving 3D DS on cubic-centimeter tissue blocks and visualization of meso-scale vessel networks in the white matter. We believe that our technique offers the potential for high-throughput, multiscale imaging of brain tissues and may facilitate studies of brain structures.
Collapse
Affiliation(s)
- Shiyi Cheng
- Department of Electrical and Computer Engineering, Boston University, 8 St Mary’s St, Boston, MA, 02215, USA
| | - Shuaibin Chang
- Department of Electrical and Computer Engineering, Boston University, 8 St Mary’s St, Boston, MA, 02215, USA
| | - Yunzhe Li
- Department of Electrical Engineering and Computer Sciences, University of California, Cory Hall, Berkeley, California, 94720, USA
| | - Anna Novoseltseva
- Department of Biomedical Engineering, Boston University, 44 Cummington Mall, Boston MA, 02215, USA
| | - Sunni Lin
- Department of Electrical and Computer Engineering, Boston University, 8 St Mary’s St, Boston, MA, 02215, USA
- Department of Biomedical Engineering, Boston University, 44 Cummington Mall, Boston MA, 02215, USA
| | - Yicun Wu
- Department of Computer Science, Boston University, 665 Commonwealth Ave, Boston, MA, 02215, USA
| | - Jiahui Zhu
- Department of Electrical and Computer Engineering, Boston University, 8 St Mary’s St, Boston, MA, 02215, USA
| | - Ann C. McKee
- Boston University Alzheimer’s Disease Research Center and CTE Center, Boston University, Chobanian and Avedisian School of Medicine, Boston, MA, 02118, USA
- Department of Neurology, Boston University, Chobanian and Avedisian School of Medicine, Boston, MA, 02118, USA
- VA Boston Healthcare System, U.S. Department of Veteran Affairs, Jamaica Plain, MA, 02130, USA
- Department of Psychiatry and Ophthalmology, Boston University School of Medicine, Boston, MA, 02118, USA
- Department of Pathology and Laboratory Medicine, Boston University School of Medicine, Boston, MA, 02118, USA
| | - Douglas L. Rosene
- Department of Anatomy & Neurobiology, Boston University Chobanian & Avedisian School of Medicine, Boston, Massachusetts, USA
| | - Hui Wang
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital/Harvard Medical School, Charlestown, MA, 02129, USA
| | - Irving J. Bigio
- Department of Electrical and Computer Engineering, Boston University, 8 St Mary’s St, Boston, MA, 02215, USA
- Department of Biomedical Engineering, Boston University, 44 Cummington Mall, Boston MA, 02215, USA
- Neurophotonics Center, Boston University, Boston, MA, 02215, USA
| | - David A. Boas
- Department of Electrical and Computer Engineering, Boston University, 8 St Mary’s St, Boston, MA, 02215, USA
- Department of Biomedical Engineering, Boston University, 44 Cummington Mall, Boston MA, 02215, USA
- Neurophotonics Center, Boston University, Boston, MA, 02215, USA
| | - Lei Tian
- Department of Electrical and Computer Engineering, Boston University, 8 St Mary’s St, Boston, MA, 02215, USA
- Department of Biomedical Engineering, Boston University, 44 Cummington Mall, Boston MA, 02215, USA
- Neurophotonics Center, Boston University, Boston, MA, 02215, USA
| |
Collapse
|
16
|
Park WY, Yun J, Shin J, Oh BH, Yoon G, Hong SM, Kim KH. Open-top Bessel beam two-photon light sheet microscopy for three-dimensional pathology. eLife 2024; 12:RP92614. [PMID: 38488831 PMCID: PMC10942781 DOI: 10.7554/elife.92614] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/17/2024] Open
Abstract
Nondestructive pathology based on three-dimensional (3D) optical microscopy holds promise as a complement to traditional destructive hematoxylin and eosin (H&E) stained slide-based pathology by providing cellular information in high throughput manner. However, conventional techniques provided superficial information only due to shallow imaging depths. Herein, we developed open-top two-photon light sheet microscopy (OT-TP-LSM) for intraoperative 3D pathology. An extended depth of field two-photon excitation light sheet was generated by scanning a nondiffractive Bessel beam, and selective planar imaging was conducted with cameras at 400 frames/s max during the lateral translation of tissue specimens. Intrinsic second harmonic generation was collected for additional extracellular matrix (ECM) visualization. OT-TP-LSM was tested in various human cancer specimens including skin, pancreas, and prostate. High imaging depths were achieved owing to long excitation wavelengths and long wavelength fluorophores. 3D visualization of both cells and ECM enhanced the ability of cancer detection. Furthermore, an unsupervised deep learning network was employed for the style transfer of OT-TP-LSM images to virtual H&E images. The virtual H&E images exhibited comparable histological characteristics to real ones. OT-TP-LSM may have the potential for histopathological examination in surgical and biopsy applications by rapidly providing 3D information.
Collapse
Affiliation(s)
- Won Yeong Park
- Department of Mechanical Engineering, Pohang University of Science and TechnologyPohangRepublic of Korea
| | - Jieun Yun
- Department of Mechanical Engineering, Pohang University of Science and TechnologyPohangRepublic of Korea
| | - Jinho Shin
- Department of Medicine, University of Ulsan College of Medicine, SeoulSeoulRepublic of Korea
| | - Byung Ho Oh
- Department of Dermatology, College of Medicine, Yonsei UniversitySeoulRepublic of Korea
| | - Gilsuk Yoon
- Department of Pathology, School of Medicine, Kyungpook National UniversityDaeguRepublic of Korea
| | - Seung-Mo Hong
- Department of Pathology, Asan Medical Center, University of Ulsan College of MedicineSeoulRepublic of Korea
| | - Ki Hean Kim
- Department of Mechanical Engineering, Pohang University of Science and TechnologyPohangRepublic of Korea
- Medical Science and Engineering Program, School of Convergence Science and Technology, Pohang University of Science and TechnologyPohangRepublic of Korea
- Institute for Convergence Research and Education in Advanced Technology, Yonsei UniversitySeoulRepublic of Korea
| |
Collapse
|
17
|
Li X, Liu H, Song X, Marboe CC, Brott BC, Litovsky SH, Gan Y. Structurally constrained and pathology-aware convolutional transformer generative adversarial network for virtual histology staining of human coronary optical coherence tomography images. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:036004. [PMID: 38532927 PMCID: PMC10964178 DOI: 10.1117/1.jbo.29.3.036004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Revised: 01/26/2024] [Accepted: 03/11/2024] [Indexed: 03/28/2024]
Abstract
Significance There is a significant need for the generation of virtual histological information from coronary optical coherence tomography (OCT) images to better guide the treatment of coronary artery disease (CAD). However, existing methods either require a large pixel-wise paired training dataset or have limited capability to map pathological regions. Aim The aim of this work is to generate virtual histological information from coronary OCT images, without a pixel-wise paired training dataset while capable of providing pathological patterns. Approach We design a structurally constrained, pathology-aware, transformer generative adversarial network, namely structurally constrained pathology-aware convolutional transformer generative adversarial network (SCPAT-GAN), to generate virtual stained H&E histology from OCT images. We quantitatively evaluate the quality of virtual stained histology images by measuring the Fréchet inception distance (FID) and perceptual hash value (PHV). Moreover, we invite experienced pathologists to evaluate the virtual stained images. Furthermore, we visually inspect the virtual stained image generated by SCPAT-GAN. Also, we perform an ablation study to validate the design of the proposed SCPAT-GAN. Finally, we demonstrate 3D virtual stained histology images. Results Compared to previous research, the proposed SCPAT-GAN achieves better FID and PHV scores. The visual inspection suggests that the virtual histology images generated by SCPAT-GAN resemble both normal and pathological features without artifacts. As confirmed by the pathologists, the virtual stained images have good quality compared to real histology images. The ablation study confirms the effectiveness of the combination of proposed pathological awareness and structural constraining modules. Conclusions The proposed SCPAT-GAN is the first to demonstrate the feasibility of generating both normal and pathological patterns without pixel-wisely supervised training. We expect the SCPAT-GAN to assist in the clinical evaluation of treating the CAD by providing 2D and 3D histopathological visualizations.
Collapse
Affiliation(s)
- Xueshen Li
- Stevens Institute of Technology, Department of Biomedical Engineering, Hoboken, New Jersey, United States
- Stevens Institute of Technology, Semcer Center for Healthcare Innovation, Hoboken, New Jersey, United States
| | - Hongshan Liu
- Stevens Institute of Technology, Department of Biomedical Engineering, Hoboken, New Jersey, United States
- Stevens Institute of Technology, Semcer Center for Healthcare Innovation, Hoboken, New Jersey, United States
| | - Xiaoyu Song
- Icahn School of Medicine at Mount Sinai, New York, New York, United States
| | - Charles C. Marboe
- Columbia University Medical Center, New York, New York, United States
| | - Brigitta C. Brott
- The University of Alabama at Birmingham, School of Medicine, Birmingham, Alabama, United States
| | - Silvio H. Litovsky
- The University of Alabama at Birmingham, School of Medicine, Birmingham, Alabama, United States
| | - Yu Gan
- Stevens Institute of Technology, Department of Biomedical Engineering, Hoboken, New Jersey, United States
- Stevens Institute of Technology, Semcer Center for Healthcare Innovation, Hoboken, New Jersey, United States
| |
Collapse
|
18
|
Shen B, Li Z, Pan Y, Guo Y, Yin Z, Hu R, Qu J, Liu L. Noninvasive Nonlinear Optical Computational Histology. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2024; 11:e2308630. [PMID: 38095543 PMCID: PMC10916666 DOI: 10.1002/advs.202308630] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/11/2023] [Revised: 11/28/2023] [Indexed: 03/07/2024]
Abstract
Cancer remains a global health challenge, demanding early detection and accurate diagnosis for improved patient outcomes. An intelligent paradigm is introduced that elevates label-free nonlinear optical imaging with contrastive patch-wise learning, yielding stain-free nonlinear optical computational histology (NOCH). NOCH enables swift, precise diagnostic analysis of fresh tissues, reducing patient anxiety and healthcare costs. Nonlinear modalities are evaluated, including stimulated Raman scattering and multiphoton imaging, for their ability to enhance tumor microenvironment sensitivity, pathological analysis, and cancer examination. Quantitative analysis confirmed that NOCH images accurately reproduce nuclear morphometric features across different cancer stages. Key diagnostic features, such as nuclear morphology, size, and nuclear-cytoplasmic contrast, are well preserved. NOCH models also demonstrate promising generalization when applied to other pathological tissues. The study unites label-free nonlinear optical imaging with histopathology using contrastive learning to establish stain-free computational histology. NOCH provides a rapid, non-invasive, and precise approach to surgical pathology, holding immense potential for revolutionizing cancer diagnosis and surgical interventions.
Collapse
Affiliation(s)
- Binglin Shen
- Key Laboratory of Optoelectronic Devices and Systems of Guangdong Province and Ministry of EducationCollege of Physics and Optoelectronic EngineeringShenzhen UniversityShenzhen518060China
| | - Zhenglin Li
- Key Laboratory of Optoelectronic Devices and Systems of Guangdong Province and Ministry of EducationCollege of Physics and Optoelectronic EngineeringShenzhen UniversityShenzhen518060China
| | - Ying Pan
- China–Japan Union Hospital of Jilin UniversityChangchun130033China
| | - Yuan Guo
- Shaanxi Provincial Cancer HospitalXi'an710065China
| | - Zongyi Yin
- Shenzhen University General HospitalShenzhen518055China
| | - Rui Hu
- Key Laboratory of Optoelectronic Devices and Systems of Guangdong Province and Ministry of EducationCollege of Physics and Optoelectronic EngineeringShenzhen UniversityShenzhen518060China
| | - Junle Qu
- Key Laboratory of Optoelectronic Devices and Systems of Guangdong Province and Ministry of EducationCollege of Physics and Optoelectronic EngineeringShenzhen UniversityShenzhen518060China
| | - Liwei Liu
- Key Laboratory of Optoelectronic Devices and Systems of Guangdong Province and Ministry of EducationCollege of Physics and Optoelectronic EngineeringShenzhen UniversityShenzhen518060China
| |
Collapse
|
19
|
Li Y, Pillar N, Li J, Liu T, Wu D, Sun S, Ma G, de Haan K, Huang L, Zhang Y, Hamidi S, Urisman A, Keidar Haran T, Wallace WD, Zuckerman JE, Ozcan A. Virtual histological staining of unlabeled autopsy tissue. Nat Commun 2024; 15:1684. [PMID: 38396004 PMCID: PMC10891155 DOI: 10.1038/s41467-024-46077-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2023] [Accepted: 02/09/2024] [Indexed: 02/25/2024] Open
Abstract
Traditional histochemical staining of post-mortem samples often confronts inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, and such chemical staining procedures covering large tissue areas demand substantial labor, cost and time. Here, we demonstrate virtual staining of autopsy tissue using a trained neural network to rapidly transform autofluorescence images of label-free autopsy tissue sections into brightfield equivalent images, matching hematoxylin and eosin (H&E) stained versions of the same samples. The trained model can effectively accentuate nuclear, cytoplasmic and extracellular features in new autopsy tissue samples that experienced severe autolysis, such as COVID-19 samples never seen before, where the traditional histochemical staining fails to provide consistent staining quality. This virtual autopsy staining technique provides a rapid and resource-efficient solution to generate artifact-free H&E stains despite severe autolysis and cell death, also reducing labor, cost and infrastructure requirements associated with the standard histochemical staining.
Collapse
Affiliation(s)
- Yuzhu Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Nir Pillar
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Jingxi Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Tairan Liu
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Di Wu
- Computer Science Department, University of California, Los Angeles, CA, 90095, USA
| | - Songyu Sun
- Computer Science Department, University of California, Los Angeles, CA, 90095, USA
| | - Guangdong Ma
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- School of Physics, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
| | - Kevin de Haan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Luzhe Huang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Yijie Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
| | - Sepehr Hamidi
- Department of Pathology and Laboratory Medicine, David Geffen School of Medicine, University of California, Los Angeles, CA, 90095, USA
| | - Anatoly Urisman
- Department of Pathology, University of California, San Francisco, CA, 94143, USA
| | - Tal Keidar Haran
- Department of Pathology, Hadassah Hebrew University Medical Center, Jerusalem, 91120, Israel
| | - William Dean Wallace
- Department of Pathology, Keck School of Medicine, University of Southern California, Los Angeles, CA, 90033, USA
| | - Jonathan E Zuckerman
- Department of Pathology and Laboratory Medicine, David Geffen School of Medicine, University of California, Los Angeles, CA, 90095, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, CA, 90095, USA.
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA.
- Department of Surgery, University of California, Los Angeles, CA, 90095, USA.
| |
Collapse
|
20
|
Zou Z, Mao Q, Cheng R, Tao C, Liu X. Correction of high-rate motion for photoacoustic microscopy by orthogonal cross-correlation. Sci Rep 2024; 14:4264. [PMID: 38383553 PMCID: PMC10881994 DOI: 10.1038/s41598-024-53505-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 02/01/2024] [Indexed: 02/23/2024] Open
Abstract
Photoacoustic imaging is a promising technology for in vivo imaging. However, its imaging performance can be hampered by motion artifacts, especially when dealing with high-rate motion. In this paper, we propose an orthogonal motion correction method that utilizes cross-correlation along orthogonal scan directions to extract accurate motion displacements from the photoacoustic data. The extracted displacements are then applied to remove artifacts and compensate for motion-induced distortions. Phantom experiments demonstrate that the proposed method can extract the motion information and the structural similarity index measurement after correction is increased by 26.5% and 11.2% compared to no correction and the previous correction method. Then the effectiveness of our method is evaluated in vivo imaging of a mouse brain. Our method shows a stable and effective performance under high-rate motion. The high accuracy of the motion correction method makes it valuable in improving the accuracy of photoacoustic imaging.
Collapse
Affiliation(s)
- Zilong Zou
- MOE Key Laboratory of Modern Acoustics, Department of Physics, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Qiuqin Mao
- MOE Key Laboratory of Modern Acoustics, Department of Physics, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Renxiang Cheng
- School of Electronic and Information Engineering, Jinling Institute of Technology, Nanjing, 211169, China
| | - Chao Tao
- MOE Key Laboratory of Modern Acoustics, Department of Physics, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China.
| | - Xiaojun Liu
- MOE Key Laboratory of Modern Acoustics, Department of Physics, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| |
Collapse
|
21
|
Jiang D, Zhu L, Tong S, Shen Y, Gao F, Gao F. Photoacoustic imaging plus X: a review. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:S11513. [PMID: 38156064 PMCID: PMC10753847 DOI: 10.1117/1.jbo.29.s1.s11513] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 11/14/2023] [Accepted: 12/11/2023] [Indexed: 12/30/2023]
Abstract
Significance Photoacoustic (PA) imaging (PAI) represents an emerging modality within the realm of biomedical imaging technology. It seamlessly blends the wealth of optical contrast with the remarkable depth of penetration offered by ultrasound. These distinctive features of PAI hold tremendous potential for various applications, including early cancer detection, functional imaging, hybrid imaging, monitoring ablation therapy, and providing guidance during surgical procedures. The synergy between PAI and other cutting-edge technologies not only enhances its capabilities but also propels it toward broader clinical applicability. Aim The integration of PAI with advanced technology for PA signal detection, signal processing, image reconstruction, hybrid imaging, and clinical applications has significantly bolstered the capabilities of PAI. This review endeavor contributes to a deeper comprehension of how the synergy between PAI and other advanced technologies can lead to improved applications. Approach An examination of the evolving research frontiers in PAI, integrated with other advanced technologies, reveals six key categories named "PAI plus X." These categories encompass a range of topics, including but not limited to PAI plus treatment, PAI plus circuits design, PAI plus accurate positioning system, PAI plus fast scanning systems, PAI plus ultrasound sensors, PAI plus advanced laser sources, PAI plus deep learning, and PAI plus other imaging modalities. Results After conducting a comprehensive review of the existing literature and research on PAI integrated with other technologies, various proposals have emerged to advance the development of PAI plus X. These proposals aim to enhance system hardware, improve imaging quality, and address clinical challenges effectively. Conclusions The progression of innovative and sophisticated approaches within each category of PAI plus X is positioned to drive significant advancements in both the development of PAI technology and its clinical applications. Furthermore, PAI not only has the potential to integrate with the above-mentioned technologies but also to broaden its applications even further.
Collapse
Affiliation(s)
- Daohuai Jiang
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
- Fujian Normal University, College of Photonic and Electronic Engineering, Fuzhou, China
| | - Luyao Zhu
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
| | - Shangqing Tong
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
| | - Yuting Shen
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
| | - Feng Gao
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
| | - Fei Gao
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
- Shanghai Engineering Research Center of Energy Efficient and Custom AI IC, Shanghai, China
- Shanghai Clinical Research and Trial Center, Shanghai, China
| |
Collapse
|
22
|
Wang X, Anastasio M, Zhang H, Sakadzic S, Hu S, Gao L. Introducing the Special Issue Honoring Lihong V. Wang, Pioneer in Biomedical Optics. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:S11500. [PMID: 38846410 PMCID: PMC11153774 DOI: 10.1117/1.jbo.29.s1.s11500] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/09/2024]
Abstract
The editorial concludes the JBO Special Issue Honoring Lihong V. Wang, outlining Prof. Wang's salient contributions to advancing the field of biomedical optics.
Collapse
Affiliation(s)
- Xueding Wang
- University of Michigan, School of Medicine, Ann Arbor, Michigan, United States
| | - Mark Anastasio
- University of Illinois Urbana - Champaign, The Grainger College of Engineering, Department of Bioengineering, Urbana, Illinois, United States
| | - Hao Zhang
- Northwestern University, Department of Biomedical Engineering, Evanston, Illinois, United States
| | - Sava Sakadzic
- Harvard Medical School, Department of Radiology, Boston, Massachusetts, United States
- Athinoula A. Martinos Center for Biomedical Imaging, Mass General Brigham, Charlestown, Massachusetts, United States
| | - Song Hu
- Washington University in St. Louis, Department of Biomedical Engineering, St. Louis, Missouri, United States
| | - Liang Gao
- University of California Los Angeles, Department of Bioengineering, Los Angeles, California, United States
| |
Collapse
|
23
|
Fan Y, Liu S, Gao E, Guo R, Dong G, Li Y, Gao T, Tang X, Liao H. The LMIT: Light-mediated minimally-invasive theranostics in oncology. Theranostics 2024; 14:341-362. [PMID: 38164160 PMCID: PMC10750201 DOI: 10.7150/thno.87783] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 10/18/2023] [Indexed: 01/03/2024] Open
Abstract
Minimally-invasive diagnosis and therapy have gradually become the trend and research hotspot of current medical applications. The integration of intraoperative diagnosis and treatment is a development important direction for real-time detection, minimally-invasive diagnosis and therapy to reduce mortality and improve the quality of life of patients, so called minimally-invasive theranostics (MIT). Light is an important theranostic tool for the treatment of cancerous tissues. Light-mediated minimally-invasive theranostics (LMIT) is a novel evolutionary technology that integrates diagnosis and therapeutics for the less invasive treatment of diseased tissues. Intelligent theranostics would promote precision surgery based on the optical characterization of cancerous tissues. Furthermore, MIT also requires the assistance of smart medical devices or robots. And, optical multimodality lay a solid foundation for intelligent MIT. In this review, we summarize the important state-of-the-arts of optical MIT or LMIT in oncology. Multimodal optical image-guided intelligent treatment is another focus. Intraoperative imaging and real-time analysis-guided optical treatment are also systemically discussed. Finally, the potential challenges and future perspectives of intelligent optical MIT are discussed.
Collapse
Affiliation(s)
- Yingwei Fan
- School of Medical Technology, Beijing Institute of Technology, Beijing, China, 100081
| | - Shuai Liu
- School of Medical Technology, Beijing Institute of Technology, Beijing, China, 100081
| | - Enze Gao
- School of Medical Technology, Beijing Institute of Technology, Beijing, China, 100081
| | - Rui Guo
- School of Medical Technology, Beijing Institute of Technology, Beijing, China, 100081
| | - Guozhao Dong
- School of Medical Technology, Beijing Institute of Technology, Beijing, China, 100081
| | - Yangxi Li
- Dept. of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China, 100084
| | - Tianxin Gao
- School of Medical Technology, Beijing Institute of Technology, Beijing, China, 100081
| | - Xiaoying Tang
- School of Medical Technology, Beijing Institute of Technology, Beijing, China, 100081
| | - Hongen Liao
- Dept. of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China, 100084
| |
Collapse
|
24
|
Harary T, Nagli M, Suleymanov N, Goykhman I, Rosenthal A. Large-field-of-view optical-resolution optoacoustic microscopy using a stationary silicon-photonics acoustic detector. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:S11511. [PMID: 38187934 PMCID: PMC10768684 DOI: 10.1117/1.jbo.29.s1.s11511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Revised: 11/22/2023] [Accepted: 11/29/2023] [Indexed: 01/09/2024]
Abstract
Significance Optical-resolution optoacoustic microscopy (OR-OAM) enables label-free imaging of the microvasculature by using optical pulse excitation and acoustic detection, commonly performed by a focused optical beam and an ultrasound transducer. One of the main challenges of OR-OAM is the need to combine the excitation and detection in a coaxial configuration, often leading to a bulky setup that requires physically scanning the ultrasound transducer to achieve a large field of view. Aim The aim of this work is to develop an OR-OAM configuration that does not require physically scanning the ultrasound transducer or the acoustic beam path. Approach Our OR-OAM system is based on a non-coaxial configuration in which the detection is performed by a silicon-photonics acoustic detector (SPADE) with a semi-isotropic sensitivity. The system is demonstrated in both epi- and trans-illumination configurations, where in both configurations SPADE remains stationary during the imaging procedure and only the optical excitation beam is scanned. Results The system is showcased for imaging resolution targets and for the in vivo visualization of the microvasculature in a mouse ear. Optoacoustic imaging with focal spots down to 1.3 μ m , lateral resolution of 4 μ m , and a field of view higher than 4 mm in both lateral dimensions were demonstrated. Conclusions We showcase a new OR-OAM design, compatible with epi-illumination configuration. This setup enables relatively large fields of view without scanning the acoustic detector or acoustic beam path. Furthermore, it offers the potential for high-speed imaging within compact, miniature probe and could potentially facilitate the clinical translation of OR-OAM technology.
Collapse
Affiliation(s)
- Tamar Harary
- Technion - Israel Institute of Technology, The Andrew and Erna Viterbi Faculty of Electrical and Computer Engineering, Haifa, Israel
| | - Michael Nagli
- Technion - Israel Institute of Technology, The Andrew and Erna Viterbi Faculty of Electrical and Computer Engineering, Haifa, Israel
| | - Nathan Suleymanov
- Technion - Israel Institute of Technology, The Andrew and Erna Viterbi Faculty of Electrical and Computer Engineering, Haifa, Israel
| | - Ilya Goykhman
- Technion - Israel Institute of Technology, The Andrew and Erna Viterbi Faculty of Electrical and Computer Engineering, Haifa, Israel
- The Hebrew University of Jerusalem, Institute of Applied Physics and Institute of Chemistry, Faculty of Science, Jerusalem, Israel
| | - Amir Rosenthal
- Technion - Israel Institute of Technology, The Andrew and Erna Viterbi Faculty of Electrical and Computer Engineering, Haifa, Israel
| |
Collapse
|
25
|
Yan B, Song B, Mu G, Fan Y, Zhao Y. Compressed single-shot 3D photoacoustic imaging with a single-element transducer. PHOTOACOUSTICS 2023; 34:100570. [PMID: 38027529 PMCID: PMC10661598 DOI: 10.1016/j.pacs.2023.100570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 10/14/2023] [Accepted: 11/06/2023] [Indexed: 12/01/2023]
Abstract
Three-dimensional (3D) photoacoustic imaging (PAI) can provide rich information content and has gained increasingly more attention in various biomedical applications. However, current 3D PAI methods either involves pointwise scanning of the 3D volume using a single-element transducer, which can be time-consuming, or requires an array of transducers, which is known to be complex and expensive. By utilizing a 3D encoder and compressed sensing techniques, we develop a new imaging modality that is capable of single-shot 3D PAI using a single-element transducer. The proposed method is validated with phantom study, which demonstrates single-shot 3D imaging of different objects and 3D tracking of a moving object. After one-time calibration, while the system could perform single-shot 3D imaging for different objects, the calibration could remain effective over 7 days, which is highly beneficial for practical translation. Overall, the experimental results showcase the potential of this technique for both scientific research and clinical applications.
Collapse
Affiliation(s)
- Bingbao Yan
- Beijing Advanced Innovation Center for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, School of Engineering Medicine, Beihang University, Beijing 100191, China
| | - Bowen Song
- Beijing Advanced Innovation Center for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, School of Engineering Medicine, Beihang University, Beijing 100191, China
| | - Gen Mu
- Beijing Advanced Innovation Center for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, School of Engineering Medicine, Beihang University, Beijing 100191, China
| | - Yubo Fan
- Beijing Advanced Innovation Center for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, School of Engineering Medicine, Beihang University, Beijing 100191, China
| | - Yanyu Zhao
- Beijing Advanced Innovation Center for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, School of Engineering Medicine, Beihang University, Beijing 100191, China
| |
Collapse
|
26
|
Huang J, Tang X, Chen Z, Li X, Zhang Y, Huang X, Zhang D, An G, Lee HJ. Rapid azoospermia classification by stimulated Raman scattering and second harmonic generation microscopy. BIOMEDICAL OPTICS EXPRESS 2023; 14:5569-5582. [PMID: 38021145 PMCID: PMC10659792 DOI: 10.1364/boe.501623] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 09/26/2023] [Accepted: 09/26/2023] [Indexed: 12/01/2023]
Abstract
Disease diagnosis and classification pose significant challenges due to the limited capabilities of traditional methods to obtain molecular information with spatial distribution. Optical imaging techniques, utilizing (auto)fluorescence and nonlinear optical signals, introduce new dimensions for biomarkers exploration that can improve diagnosis and classification. Nevertheless, these signals often cover only a limited number of species, impeding a comprehensive assessment of the tissue microenvironment, which is crucial for effective disease diagnosis and therapy. To address this challenge, we developed a multimodal platform, termed stimulated Raman scattering and second harmonic generation microscopy (SRASH), capable of simultaneously providing both chemical bonds and structural information of tissues. Applying SRASH imaging to azoospermia patient samples, we successfully identified lipids, protein, and collagen contrasts, unveiling molecular and structural signatures for non-obstructive azoospermia. This achievement is facilitated by LiteBlendNet-Dx (LBNet-Dx), our diagnostic algorithm, which achieved an outstanding 100% sample-level accuracy in classifying azoospermia, surpassing conventional imaging modalities. As a label-free technique, SRASH imaging eliminates the requirement for sample pre-treatment, demonstrating great potential for clinical translation and enabling molecular imaging-based diagnosis and therapy.
Collapse
Affiliation(s)
- Jie Huang
- Zhejiang Polytechnic Institute, Polytechnic Institute, Zhejiang University, Hangzhou 310058, China
- College of Biomedical Engineering & Instrument Science; Key Laboratory for Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310058, China
| | - Xiaobin Tang
- Interdisciplinary Centre for Quantum Information, Zhejiang Province Key Laboratory of Quantum Technology and Device, and Department of Physics, Zhejiang University; Hangzhou 310027, China
| | - Zhicong Chen
- Department of Obstetrics and Gynecology, Center for Reproductive Medicine; Guangdong Provincial Key Laboratory of Major Obstetric Diseases; Guangdong Provincial Clinical Research Center for Obstetrics and Gynecology; Guangdong-Hong Kong-Macao Greater Bay Area Higher Education Joint Laboratory of Maternal-Fetal Medicine; The Third Affiliated Hospital of Guangzhou Medical University; Guangzhou 510150, China
| | - Xiaomin Li
- Department of Obstetrics and Gynecology, Center for Reproductive Medicine; Guangdong Provincial Key Laboratory of Major Obstetric Diseases; Guangdong Provincial Clinical Research Center for Obstetrics and Gynecology; Guangdong-Hong Kong-Macao Greater Bay Area Higher Education Joint Laboratory of Maternal-Fetal Medicine; The Third Affiliated Hospital of Guangzhou Medical University; Guangzhou 510150, China
| | - Yongqing Zhang
- Interdisciplinary Centre for Quantum Information, Zhejiang Province Key Laboratory of Quantum Technology and Device, and Department of Physics, Zhejiang University; Hangzhou 310027, China
| | - Xiangjie Huang
- College of Biomedical Engineering & Instrument Science; Key Laboratory for Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310058, China
| | - Delong Zhang
- Interdisciplinary Centre for Quantum Information, Zhejiang Province Key Laboratory of Quantum Technology and Device, and Department of Physics, Zhejiang University; Hangzhou 310027, China
| | - Geng An
- Department of Obstetrics and Gynecology, Center for Reproductive Medicine; Guangdong Provincial Key Laboratory of Major Obstetric Diseases; Guangdong Provincial Clinical Research Center for Obstetrics and Gynecology; Guangdong-Hong Kong-Macao Greater Bay Area Higher Education Joint Laboratory of Maternal-Fetal Medicine; The Third Affiliated Hospital of Guangzhou Medical University; Guangzhou 510150, China
| | - Hyeon Jeong Lee
- College of Biomedical Engineering & Instrument Science; Key Laboratory for Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310058, China
| |
Collapse
|
27
|
Wang N, Zhang C, Wei X, Yan T, Zhou W, Zhang J, Kang H, Yuan Z, Chen X. Harnessing the power of optical microscopy for visualization and analysis of histopathological images. BIOMEDICAL OPTICS EXPRESS 2023; 14:5451-5465. [PMID: 37854561 PMCID: PMC10581782 DOI: 10.1364/boe.501893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Revised: 08/29/2023] [Accepted: 09/01/2023] [Indexed: 10/20/2023]
Abstract
Histopathology is the foundation and gold standard for identifying diseases, and precise quantification of histopathological images can provide the pathologist with objective clues to make a more convincing diagnosis. Optical microscopy (OM), an important branch of optical imaging technology that provides high-resolution images of tissue cytology and structural morphology, has been used in the diagnosis of histopathology and evolved into a new disciplinary direction of optical microscopic histopathology (OMH). There are a number of ex-vivo studies providing applicability of different OMH approaches, and a transfer of these techniques toward in vivo diagnosis is currently in progress. Furthermore, combined with advanced artificial intelligence algorithms, OMH allows for improved diagnostic reliability and convenience due to the complementarity of retrieval information. In this review, we cover recent advances in OMH, including the exploration of new techniques in OMH as well as their applications, and look ahead to new challenges in OMH. These typical application examples well demonstrate the application potential and clinical value of OMH techniques in histopathological diagnosis.
Collapse
Affiliation(s)
- Nan Wang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Chang Zhang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
| | - Xinyu Wei
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
| | - Tianyu Yan
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Wangting Zhou
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Jiaojiao Zhang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Huan Kang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Zhen Yuan
- Faculty of Health Sciences, University of Macau, Macau, 999078, China
| | - Xueli Chen
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
- Inovation Center for Advanced Medical Imaging and Intelligent Medicine, Guangzhou Institute of Technology, Xidian University, Guangzhou, Guangdong 510555, China
| |
Collapse
|
28
|
Martell MT, Haven NJM, Cikaluk BD, Restall BS, McAlister EA, Mittal R, Adam BA, Giannakopoulos N, Peiris L, Silverman S, Deschenes J, Li X, Zemp RJ. Deep learning-enabled realistic virtual histology with ultraviolet photoacoustic remote sensing microscopy. Nat Commun 2023; 14:5967. [PMID: 37749108 PMCID: PMC10519961 DOI: 10.1038/s41467-023-41574-2] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2022] [Accepted: 09/11/2023] [Indexed: 09/27/2023] Open
Abstract
The goal of oncologic surgeries is complete tumor resection, yet positive margins are frequently found postoperatively using gold standard H&E-stained histology methods. Frozen section analysis is sometimes performed for rapid intraoperative margin evaluation, albeit with known inaccuracies. Here, we introduce a label-free histological imaging method based on an ultraviolet photoacoustic remote sensing and scattering microscope, combined with unsupervised deep learning using a cycle-consistent generative adversarial network for realistic virtual staining. Unstained tissues are scanned at rates of up to 7 mins/cm2, at resolution equivalent to 400x digital histopathology. Quantitative validation suggests strong concordance with conventional histology in benign and malignant prostate and breast tissues. In diagnostic utility studies we demonstrate a mean sensitivity and specificity of 0.96 and 0.91 in breast specimens, and respectively 0.87 and 0.94 in prostate specimens. We also find virtual stain quality is preferred (P = 0.03) compared to frozen section analysis in a blinded survey of pathologists.
Collapse
Affiliation(s)
- Matthew T Martell
- Department of Electrical and Computer Engineering, University of Alberta, 116 Street & 85 Avenue, Edmonton, AB, T6G 2R3, Canada
| | - Nathaniel J M Haven
- Department of Electrical and Computer Engineering, University of Alberta, 116 Street & 85 Avenue, Edmonton, AB, T6G 2R3, Canada
| | - Brendyn D Cikaluk
- Department of Electrical and Computer Engineering, University of Alberta, 116 Street & 85 Avenue, Edmonton, AB, T6G 2R3, Canada
| | - Brendon S Restall
- Department of Electrical and Computer Engineering, University of Alberta, 116 Street & 85 Avenue, Edmonton, AB, T6G 2R3, Canada
| | - Ewan A McAlister
- Department of Electrical and Computer Engineering, University of Alberta, 116 Street & 85 Avenue, Edmonton, AB, T6G 2R3, Canada
| | - Rohan Mittal
- Department of Laboratory Medicine and Pathology, University of Alberta, 11405 87 Avenue NW, Edmonton, AB, T6G 1C9, Canada
| | - Benjamin A Adam
- Department of Laboratory Medicine and Pathology, University of Alberta, 11405 87 Avenue NW, Edmonton, AB, T6G 1C9, Canada
| | - Nadia Giannakopoulos
- Department of Laboratory Medicine and Pathology, University of Alberta, 11405 87 Avenue NW, Edmonton, AB, T6G 1C9, Canada
| | - Lashan Peiris
- Department of Surgery, University of Alberta, 8440 - 112 Street, Edmonton, AB, T6G 2B7, Canada
| | - Sveta Silverman
- Department of Laboratory Medicine and Pathology, University of Alberta, 11405 87 Avenue NW, Edmonton, AB, T6G 1C9, Canada
| | - Jean Deschenes
- Department of Laboratory Medicine and Pathology, University of Alberta, 11405 87 Avenue NW, Edmonton, AB, T6G 1C9, Canada
| | - Xingyu Li
- Department of Electrical and Computer Engineering, University of Alberta, 116 Street & 85 Avenue, Edmonton, AB, T6G 2R3, Canada
| | - Roger J Zemp
- Department of Electrical and Computer Engineering, University of Alberta, 116 Street & 85 Avenue, Edmonton, AB, T6G 2R3, Canada.
| |
Collapse
|
29
|
Barulin A, Park H, Park B, Kim I. Dual-wavelength UV-visible metalens for multispectral photoacoustic microscopy: A simulation study. PHOTOACOUSTICS 2023; 32:100545. [PMID: 37645253 PMCID: PMC10461252 DOI: 10.1016/j.pacs.2023.100545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 08/01/2023] [Accepted: 08/13/2023] [Indexed: 08/31/2023]
Abstract
Photoacoustic microscopy is advancing with research on utilizing ultraviolet and visible light. Dual-wavelength approaches are sought for observing DNA/RNA- and vascular-related disorders. However, the availability of high numerical aperture lenses covering both ultraviolet and visible wavelengths is severely limited due to challenges such as chromatic aberration in the optics. Herein, we present a groundbreaking proposal as a pioneering simulation study for incorporating multilayer metalenses into ultraviolet-visible photoacoustic microscopy. The proposed metalens has a thickness of 1.4 µm and high numerical aperture of 0.8. By arranging cylindrical hafnium oxide nanopillars, we design an achromatic transmissive lens for 266 and 532 nm wavelengths. The metalens achieves a diffraction-limited focal spot, surpassing commercially available objective lenses. Through three-dimensional photoacoustic simulation, we demonstrate high-resolution imaging with superior endogenous contrast of targets with ultraviolet and visible optical absorption bands. This metalens will open new possibilities for downsized multispectral photoacoustic microscopy in clinical and preclinical applications.
Collapse
Affiliation(s)
- Aleksandr Barulin
- Department of Biophysics, Institute of Quantum Biophysics, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Hyemi Park
- Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Byullee Park
- Department of Biophysics, Institute of Quantum Biophysics, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Inki Kim
- Department of Biophysics, Institute of Quantum Biophysics, Sungkyunkwan University, Suwon 16419, Republic of Korea
- Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University, Suwon 16419, Republic of Korea
| |
Collapse
|
30
|
Song W, Guo C, Zhao Y, Wang YC, Zhu S, Min C, Yuan X. Ultraviolet metasurface-assisted photoacoustic microscopy with great enhancement in DOF for fast histology imaging. PHOTOACOUSTICS 2023; 32:100525. [PMID: 37645256 PMCID: PMC10461204 DOI: 10.1016/j.pacs.2023.100525] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Revised: 06/14/2023] [Accepted: 06/22/2023] [Indexed: 08/31/2023]
Abstract
Pathology interpretations of tissue rely on the gold standard of histology imaging, potentially hampering timely access to critical information for diagnosis and management of neoplasms because of tedious sample preparations. Slide-free capture of cell nuclei in unprocessed specimens without staining is preferable; however, inevitable irregular surfaces in fresh tissues results in limitations. An ultraviolet metasurface with the ability to generate an ultraviolet optical focus maintaining < 1.1-µm in lateral resolution and ∼290 µm in depth of field (DOF) is proposed for fast, high resolution, label-free photoacoustic histological imaging of unprocessed tissues with uneven surfaces. Microanatomical characteristics of the cell nuclei can be observed, as demonstrated by the mouse brain samples that were cut by hand and a ∼3 × 3-mm2 field of view was imaged in ∼27 min. Therefore, ultraviolet metasurface-assisted photoacoustic microscopy is anticipated to benefit intraoperative pathological assessments and basic scientific research by alleviating laborious tissue preparations.
Collapse
Affiliation(s)
- Wei Song
- Nanophotonics Research Center, Shenzhen Key Laboratory of Micro-Scale Optical Information Technology, Institute of Microscale Optoelectronics & State Key Laboratory of Radio Frequency Heterogeneous, Shenzhen University, Shenzhen 518060, China
| | - Changkui Guo
- Nanophotonics Research Center, Shenzhen Key Laboratory of Micro-Scale Optical Information Technology, Institute of Microscale Optoelectronics & State Key Laboratory of Radio Frequency Heterogeneous, Shenzhen University, Shenzhen 518060, China
| | - Yuting Zhao
- Nanophotonics Research Center, Shenzhen Key Laboratory of Micro-Scale Optical Information Technology, Institute of Microscale Optoelectronics & State Key Laboratory of Radio Frequency Heterogeneous, Shenzhen University, Shenzhen 518060, China
| | - Ya-chao Wang
- Depart of Neurosurgery, The First Affiliated Hospital of Shenzhen University, Shenzhen Second People's Hospital, Shenzhen 518060, China
| | - Siwei Zhu
- The Institute of Translational Medicine, Tianjin Union Medical Center of Nankai University, Tianjin 300121, China
| | - Changjun Min
- Nanophotonics Research Center, Shenzhen Key Laboratory of Micro-Scale Optical Information Technology, Institute of Microscale Optoelectronics & State Key Laboratory of Radio Frequency Heterogeneous, Shenzhen University, Shenzhen 518060, China
| | - Xiaocong Yuan
- Nanophotonics Research Center, Shenzhen Key Laboratory of Micro-Scale Optical Information Technology, Institute of Microscale Optoelectronics & State Key Laboratory of Radio Frequency Heterogeneous, Shenzhen University, Shenzhen 518060, China
- Research Center for Humanoid Sensing, Zhejiang Laboratory, Hangzhou 311100, China
| |
Collapse
|
31
|
He P, Chen G, Huang M, Jing L, Wu W, Kuo HC, Tu CC, Chen SL. Biodegradable germanium nanoparticles as contrast agents for near-infrared-II photoacoustic imaging. NANOSCALE 2023. [PMID: 37366254 DOI: 10.1039/d3nr01594g] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/28/2023]
Abstract
Photoacoustic (PA) imaging using contrast agents with strong near-infrared-II (NIR-II, 1000-1700 nm) absorption enables deep penetration into biological tissue. Besides, biocompatibility and biodegradability are essential for clinical translation. Herein, we developed biocompatible and biodegradable germanium nanoparticles (GeNPs) with high photothermal stability as well as strong and broad absorption for NIR-II PA imaging. We first demonstrate the excellent biocompatibility of the GeNPs through experiments, including the zebrafish embryo survival rates, nude mouse body weight curves, and histological images of the major organs. Then, comprehensive PA imaging demonstrations are presented to showcase the versatile imaging capabilities and excellent biodegradability, including in vitro PA imaging which can bypass blood absorption, in vivo dual-wavelength PA imaging which can clearly distinguish the injected GeNPs from the background blood vessels, in vivo and ex vivo PA imaging with deep penetration, in vivo time-lapse PA imaging of a mouse ear for observing biodegradation, ex vivo time-lapse PA imaging of the major organs of a mouse model for observing the biodistribution after intravenous injection, and notably in vivo dual-modality fluorescence and PA imaging of osteosarcoma tumors. The in vivo biodegradation of GeNPs is observed not only in the normal tissue but also in the tumor, making the GeNPs a promising candidate for clinical NIR-II PA imaging applications.
Collapse
Affiliation(s)
- Pengbo He
- University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University, Shanghai 200240, China.
| | - Guo Chen
- University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University, Shanghai 200240, China.
| | - Mengling Huang
- School of Pharmacy, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Lili Jing
- School of Pharmacy, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Wen Wu
- Shanghai Key Laboratory of Orthopaedic Implants, Department of Orthopaedics, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200025, China.
- Engineering Research Center of Digital Medicine and Clinical Translation, Ministry of Education, Shanghai 200030, China
| | - Hao-Chung Kuo
- Hon Hai Research Institute, Foxconn Technology Group, Shenzhen 518109, China.
| | - Chang-Ching Tu
- University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University, Shanghai 200240, China.
- Hon Hai Research Institute, Foxconn Technology Group, Shenzhen 518109, China.
- Engineering Research Center of Digital Medicine and Clinical Translation, Ministry of Education, Shanghai 200030, China
| | - Sung-Liang Chen
- University of Michigan-Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University, Shanghai 200240, China.
- Engineering Research Center of Digital Medicine and Clinical Translation, Ministry of Education, Shanghai 200030, China
- State Key Laboratory of Advanced Optical Communication Systems and Networks, Shanghai Jiao Tong University, Shanghai 200240, China
| |
Collapse
|
32
|
Xu M, Ouyang Y, Yuan Z. Deep Learning Aided Neuroimaging and Brain Regulation. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23114993. [PMID: 37299724 DOI: 10.3390/s23114993] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Revised: 05/15/2023] [Accepted: 05/22/2023] [Indexed: 06/12/2023]
Abstract
Currently, deep learning aided medical imaging is becoming the hot spot of AI frontier application and the future development trend of precision neuroscience. This review aimed to render comprehensive and informative insights into the recent progress of deep learning and its applications in medical imaging for brain monitoring and regulation. The article starts by providing an overview of the current methods for brain imaging, highlighting their limitations and introducing the potential benefits of using deep learning techniques to overcome these limitations. Then, we further delve into the details of deep learning, explaining the basic concepts and providing examples of how it can be used in medical imaging. One of the key strengths is its thorough discussion of the different types of deep learning models that can be used in medical imaging including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial network (GAN) assisted magnetic resonance imaging (MRI), positron emission tomography (PET)/computed tomography (CT), electroencephalography (EEG)/magnetoencephalography (MEG), optical imaging, and other imaging modalities. Overall, our review on deep learning aided medical imaging for brain monitoring and regulation provides a referrable glance for the intersection of deep learning aided neuroimaging and brain regulation.
Collapse
Affiliation(s)
- Mengze Xu
- Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Zhuhai 519087, China
- Centre for Cognitive and Brain Sciences, Institute of Collaborative Innovation, University of Macau, Macau SAR 999078, China
| | - Yuanyuan Ouyang
- Nanomicro Sino-Europe Technology Company Limited, Zhuhai 519031, China
- Jiangfeng China-Portugal Technology Co., Ltd., Macau SAR 999078, China
| | - Zhen Yuan
- Centre for Cognitive and Brain Sciences, Institute of Collaborative Innovation, University of Macau, Macau SAR 999078, China
| |
Collapse
|
33
|
Cheng Z, Wang D, Zhang Z, Wang Z, Yang F, Zeng L, Ji X. Photoacoustic maximum amplitude projection microscopy by ultra-low data sampling. OPTICS LETTERS 2023; 48:1718-1721. [PMID: 37221749 DOI: 10.1364/ol.485628] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 02/14/2023] [Indexed: 05/25/2023]
Abstract
Photoacoustic microscopy (PAM) has attracted increasing research interest in the biomedical field due to its unique merit of combining light and sound. In general, the bandwidth of a photoacoustic signal reaches up to tens or even hundreds of MHz, which requires a high-performance acquisition card to meet the high requirement of precision of sampling and control. For most depth-insensitive scenes, it is complex and costly to capture the photoacoustic maximum amplitude projection (MAP) images. Herein, we propose a simple and low-cost MAP-PAM system based on a custom-made peak holding circuit to obtain the extremum values by Hz data sampling. The dynamic range of the input signal is 0.01-2.5 V, and the -6-dB bandwidth of the input signal can be up to 45 MHz. Through in vitro and in vivo experiments, we have verified that the system has the same imaging ability as conventional PAM. Owing to its compact size and ultra-low price (approximately $18), it provides a new performance paradigm for PAM and opens up a new way for an optimal photoacoustic sensing and imaging device.
Collapse
|
34
|
Zhang Y, Hu P, Li L, Cao R, Khadria A, Maslov K, Tong X, Zeng Y, Jiang L, Zhou Q, Wang LV. Single-shot 3D photoacoustic tomography using a single-element detector for ultrafast imaging of hemodynamics. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.03.14.532661. [PMID: 36993341 PMCID: PMC10055152 DOI: 10.1101/2023.03.14.532661] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 04/29/2023]
Abstract
Imaging hemodynamics is crucial for the diagnosis, treatment, and prevention of vascular diseases. However, current imaging techniques are limited due to the use of ionizing radiation or contrast agents, short penetration depth, or complex and expensive data acquisition systems. Photoacoustic tomography shows promise as a solution to these issues. However, existing photoacoustic tomography methods collect signals either sequentially or through numerous detector elements, leading to either low imaging speed or high system complexity and cost. To address these issues, here we introduce a method to capture a 3D photoacoustic image of vasculature using a single laser pulse and a single-element detector that functions as 6,400 virtual ones. Our method enables ultrafast volumetric imaging of hemodynamics in the human body at up to 1 kHz and requires only a single calibration for different objects and for long-term operations. We demonstrate 3D imaging of hemodynamics at depth in humans and small animals, capturing the variability in blood flow speeds. This concept can inspire other imaging technologies and find applications such as home-care monitoring, biometrics, point-of-care testing, and wearable monitoring.
Collapse
Affiliation(s)
- Yide Zhang
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125, USA
| | - Peng Hu
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125, USA
| | - Lei Li
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125, USA
| | - Rui Cao
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125, USA
| | - Anjul Khadria
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125, USA
| | - Konstantin Maslov
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125, USA
| | - Xin Tong
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125, USA
| | - Yushun Zeng
- Department of Biomedical Engineering and Ophthalmology, University of Southern California, Los Angeles, CA 90089, USA
| | - Laiming Jiang
- Department of Biomedical Engineering and Ophthalmology, University of Southern California, Los Angeles, CA 90089, USA
| | - Qifa Zhou
- Department of Biomedical Engineering and Ophthalmology, University of Southern California, Los Angeles, CA 90089, USA
| | - Lihong V. Wang
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125, USA
| |
Collapse
|
35
|
Bai B, Yang X, Li Y, Zhang Y, Pillar N, Ozcan A. Deep learning-enabled virtual histological staining of biological samples. LIGHT, SCIENCE & APPLICATIONS 2023; 12:57. [PMID: 36864032 PMCID: PMC9981740 DOI: 10.1038/s41377-023-01104-7] [Citation(s) in RCA: 33] [Impact Index Per Article: 33.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 02/10/2023] [Accepted: 02/14/2023] [Indexed: 06/18/2023]
Abstract
Histological staining is the gold standard for tissue examination in clinical pathology and life-science research, which visualizes the tissue and cellular structures using chromatic dyes or fluorescence labels to aid the microscopic assessment of tissue. However, the current histological staining workflow requires tedious sample preparation steps, specialized laboratory infrastructure, and trained histotechnologists, making it expensive, time-consuming, and not accessible in resource-limited settings. Deep learning techniques created new opportunities to revolutionize staining methods by digitally generating histological stains using trained neural networks, providing rapid, cost-effective, and accurate alternatives to standard chemical staining methods. These techniques, broadly referred to as virtual staining, were extensively explored by multiple research groups and demonstrated to be successful in generating various types of histological stains from label-free microscopic images of unstained samples; similar approaches were also used for transforming images of an already stained tissue sample into another type of stain, performing virtual stain-to-stain transformations. In this Review, we provide a comprehensive overview of the recent research advances in deep learning-enabled virtual histological staining techniques. The basic concepts and the typical workflow of virtual staining are introduced, followed by a discussion of representative works and their technical innovations. We also share our perspectives on the future of this emerging field, aiming to inspire readers from diverse scientific fields to further expand the scope of deep learning-enabled virtual histological staining techniques and their applications.
Collapse
Affiliation(s)
- Bijie Bai
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Xilin Yang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Yuzhu Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Yijie Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Nir Pillar
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
- Bioengineering Department, University of California, Los Angeles, 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA.
- Bioengineering Department, University of California, Los Angeles, 90095, USA.
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA.
| |
Collapse
|