1
|
Zhang X, Song J, Fan J, Zeng N, He H, Tuchin VV, Ma H. Stereoscopic spatial graphical method of Mueller matrix: Global-Polarization Stokes Ellipsoid. FRONTIERS OF OPTOELECTRONICS 2024; 17:29. [PMID: 39150587 PMCID: PMC11329479 DOI: 10.1007/s12200-024-00132-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2024] [Accepted: 07/14/2024] [Indexed: 08/17/2024]
Abstract
A Mueller matrix covers all the polarization information of the measured sample, however the combination of its 16 elements is sometimes not intuitive enough to describe and identify the key characteristics of polarization changes. Within the Poincaré sphere system, this study achieves a spatial representation of the Mueller matrix: the Global-Polarization Stokes Ellipsoid (GPSE). With the help of Monte Carlo simulations combined with anisotropic tissue models, three basic characteristic parameters of GPSE are proposed and explained, where the V parameter represents polarization maintenance ability, and the E and D† parameters represent the degree of anisotropy. Furthermore, based on GPSE system, a dynamic analysis of skeletal muscle dehydration process demonstrates the monitoring effect of GPSE from an application perspective, while confirming its robustness and accuracy.
Collapse
Affiliation(s)
- Xinxian Zhang
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
| | - Jiawei Song
- School of Teacher Education, Nanjing Normal University, Nanjing, 210097, China
| | - Jiahao Fan
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
| | - Nan Zeng
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China.
| | - Honghui He
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
| | - Valery V Tuchin
- Institute of Physics, Saratov State University, Saratov, 410012, Russia
| | - Hui Ma
- Department of Physics, Tsinghua University, Beijing, 100084, China
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen, 518055, China
| |
Collapse
|
2
|
Chen Y, Chu J, Xin B, Qi J. Mechanical stability of polarization signatures in biological tissue characterization. BIOMEDICAL OPTICS EXPRESS 2024; 15:2652-2665. [PMID: 38633097 PMCID: PMC11019670 DOI: 10.1364/boe.518756] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 03/19/2024] [Accepted: 03/19/2024] [Indexed: 04/19/2024]
Abstract
Mueller matrix imaging polarimetry (MMIP) is a promising technique for investigating structural abnormalities in pathological diagnosis. The characterization stability of polarization signatures, described by Mueller matrix parameters (MMPs), correlates with the mechanical state of the biological medium. In this study, we developed an MMIP system capable of applying quantitative forces to samples and measuring the resulting polarization signatures. Mechanical stretching experiments were conducted on a mimicking phantom and a tissue sample at different force scales. We analyzed the textural features and data distribution of MMP images and evaluated the force effect on the characterization of MMPs using the structural similarity index. The results demonstrate that changes in the mechanical microenvironment (CMM) can cause textural fluctuations in MMP images, interfering with the stability of polarization signatures. Specifically, parameters of anisotropic orientation, retardance, and optical rotation are the most sensitive to CMM, inducing a dramatic change in the overall image texture, while other parameters (e.g., polarization, diattenuation, and depolarization) exhibit locality in their response to CMM. For some MMPs, CMM can enhance regional textural contrasts. This study elucidates the mechanical stability of polarization signatures in biological tissue characterization and provides a valuable reference for further research toward minimizing CMM influence.
Collapse
Affiliation(s)
- Yongtai Chen
- Research Center for Frontier Fundamental Studies, Zhejiang Lab, Hangzhou 311100, China
- School of Mechanical Engineering, Dalian University of Technology, Dalian 116024, China
| | - Jinkui Chu
- School of Mechanical Engineering, Dalian University of Technology, Dalian 116024, China
| | - Benda Xin
- School of Mechanical Engineering, Dalian University of Technology, Dalian 116024, China
| | - Ji Qi
- Research Center for Frontier Fundamental Studies, Zhejiang Lab, Hangzhou 311100, China
| |
Collapse
|
3
|
Hao R, Zeng N, Zhang Z, He H, He C, Ma H. Discrepancy of coordinate system selection in backscattering Mueller matrix polarimetry: exploring photon coordinate system transformation invariants. OPTICS EXPRESS 2024; 32:3804-3816. [PMID: 38297593 DOI: 10.1364/oe.513999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Accepted: 01/09/2024] [Indexed: 02/02/2024]
Abstract
In biomedical studies, Mueller matrix polarimetry is gaining increasing attention because it can comprehensively characterize polarization-related vectorial properties of the sample, which are crucial for microstructural identification and evaluation. For backscattering Mueller matrix polarimetry, there are two photon coordinate selection conventions, which can affect the following Mueller matrix parameters calculation and information acquisition quantitatively. In this study, we systematically analyze the influence of photon coordinate system selection on the backscattering Mueller matrix polarimetry. We compare the Mueller matrix elements in the right-handed-nonunitary and non-right-handed-unitary coordinate systems, and specifically deduce the changes of Mueller matrix polar decomposition, Mueller matrix Cloude decomposition and Mueller matrix transformation parameters widely used in backscattering Mueller matrix imaging as the photon coordinate system varied. Based on the theoretical analysis and phantom experiments, we provide a group of photon coordinate system transformation invariants for backscattering Mueller matrix polarimetry. The findings presented in this study give a crucial criterion of parameters selection for backscattering Mueller matrix imaging under different photon coordinate systems.
Collapse
|
4
|
Wei S, Si L, Huang T, Du S, Yao Y, Dong Y, Ma H. Deep-learning-based cross-modality translation from Stokes image to bright-field contrast. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:102911. [PMID: 37867633 PMCID: PMC10587695 DOI: 10.1117/1.jbo.28.10.102911] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 08/25/2023] [Accepted: 09/25/2023] [Indexed: 10/24/2023]
Abstract
Significance Mueller matrix (MM) microscopy has proven to be a powerful tool for probing microstructural characteristics of biological samples down to subwavelength scale. However, in clinical practice, doctors usually rely on bright-field microscopy images of stained tissue slides to identify characteristic features of specific diseases and make accurate diagnosis. Cross-modality translation based on polarization imaging helps to improve the efficiency and stability in analyzing sample properties from different modalities for pathologists. Aim In this work, we propose a computational image translation technique based on deep learning to enable bright-field microscopy contrast using snapshot Stokes images of stained pathological tissue slides. Taking Stokes images as input instead of MM images allows the translated bright-field images to be unaffected by variations of light source and samples. Approach We adopted CycleGAN as the translation model to avoid requirements on co-registered image pairs in the training. This method can generate images that are equivalent to the bright-field images with different staining styles on the same region. Results Pathological slices of liver and breast tissues with hematoxylin and eosin staining and lung tissues with two types of immunohistochemistry staining, i.e., thyroid transcription factor-1 and Ki-67, were used to demonstrate the effectiveness of our method. The output results were evaluated by four image quality assessment methods. Conclusions By comparing the cross-modality translation performance with MM images, we found that the Stokes images, with the advantages of faster acquisition and independence from light intensity and image registration, can be well translated to bright-field images.
Collapse
Affiliation(s)
- Shilong Wei
- Tsinghua University, Shenzhen International Graduate School, Shenzhen, China
| | - Lu Si
- Tsinghua University, Shenzhen International Graduate School, Shenzhen, China
| | - Tongyu Huang
- Tsinghua University, Shenzhen International Graduate School, Shenzhen, China
- Tsinghua University, Department of Biomedical Engineering, Beijing, China
| | - Shan Du
- University of Chinese Academy of Sciences, Shenzhen Hospital, Department of Pathology, Shenzhen, China
| | - Yue Yao
- Tsinghua University, Shenzhen International Graduate School, Shenzhen, China
| | - Yang Dong
- Tsinghua University, Shenzhen International Graduate School, Shenzhen, China
| | - Hui Ma
- Tsinghua University, Shenzhen International Graduate School, Shenzhen, China
- Tsinghua University, Department of Biomedical Engineering, Beijing, China
- Tsinghua University, Department of Physics, Beijing, China
| |
Collapse
|
5
|
Sampaio P, Lopez-Antuña M, Storni F, Wicht J, Sökeland G, Wartenberg M, Márquez-Neila P, Candinas D, Demory BO, Perren A, Sznitman R. Müller matrix polarimetry for pancreatic tissue characterization. Sci Rep 2023; 13:16417. [PMID: 37775538 PMCID: PMC10541901 DOI: 10.1038/s41598-023-43195-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 09/20/2023] [Indexed: 10/01/2023] Open
Abstract
Polarimetry is an optical characterization technique capable of analyzing the polarization state of light reflected by materials and biological samples. In this study, we investigate the potential of Müller matrix polarimetry (MMP) to analyze fresh pancreatic tissue samples. Due to its highly heterogeneous appearance, pancreatic tissue type differentiation is a complex task. Furthermore, its challenging location in the body makes creating direct imaging difficult. However, accurate and reliable methods for diagnosing pancreatic diseases are critical for improving patient outcomes. To this end, we measured the Müller matrices of ex-vivo unfixed human pancreatic tissue and leverage the feature-learning capabilities of a machine-learning model to derive an optimized data representation that minimizes normal-abnormal classification error. We show experimentally that our approach accurately differentiates between normal and abnormal pancreatic tissue. This is, to our knowledge, the first study to use ex-vivo unfixed human pancreatic tissue combined with feature-learning from raw Müller matrix readings for this purpose.
Collapse
Affiliation(s)
- Paulo Sampaio
- ARTORG Center, University of Bern, Bern, Switzerland.
| | | | - Federico Storni
- Department of Visceral surgery and medicine, Bern University Hospital, Bern, Switzerland
| | - Jonatan Wicht
- ARTORG Center, University of Bern, Bern, Switzerland
- Center for Space and Habitability, University of Bern, Bern, Switzerland
| | - Greta Sökeland
- Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| | - Martin Wartenberg
- Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| | | | - Daniel Candinas
- Department of Visceral surgery and medicine, Bern University Hospital, Bern, Switzerland
| | | | - Aurel Perren
- Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| | | |
Collapse
|
6
|
Zou X, Zhai J, Qian S, Li A, Tian F, Cao X, Wang R. Improved breast ultrasound tumor classification using dual-input CNN with GAP-guided attention loss. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:15244-15264. [PMID: 37679179 DOI: 10.3934/mbe.2023682] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/09/2023]
Abstract
Ultrasonography is a widely used medical imaging technique for detecting breast cancer. While manual diagnostic methods are subject to variability and time-consuming, computer-aided diagnostic (CAD) methods have proven to be more efficient. However, current CAD approaches neglect the impact of noise and artifacts on the accuracy of image analysis. To enhance the precision of breast ultrasound image analysis for identifying tissues, organs and lesions, we propose a novel approach for improved tumor classification through a dual-input model and global average pooling (GAP)-guided attention loss function. Our approach leverages a convolutional neural network with transformer architecture and modifies the single-input model for dual-input. This technique employs a fusion module and GAP operation-guided attention loss function simultaneously to supervise the extraction of effective features from the target region and mitigate the effect of information loss or redundancy on misclassification. Our proposed method has three key features: (i) ResNet and MobileViT are combined to enhance local and global information extraction. In addition, a dual-input channel is designed to include both attention images and original breast ultrasound images, mitigating the impact of noise and artifacts in ultrasound images. (ii) A fusion module and GAP operation-guided attention loss function are proposed to improve the fusion of dual-channel feature information, as well as supervise and constrain the weight of the attention mechanism on the fused focus region. (iii) Using the collected uterine fibroid ultrasound dataset to train ResNet18 and load the pre-trained weights, our experiments on the BUSI and BUSC public datasets demonstrate that the proposed method outperforms some state-of-the-art methods. The code will be publicly released at https://github.com/425877/Improved-Breast-Ultrasound-Tumor-Classification.
Collapse
Affiliation(s)
- Xiao Zou
- School of Physics and Electronics, Hunan Normal University, Changsha 410081, China
| | - Jintao Zhai
- School of Physics and Electronics, Hunan Normal University, Changsha 410081, China
| | - Shengyou Qian
- School of Physics and Electronics, Hunan Normal University, Changsha 410081, China
| | - Ang Li
- School of Physics and Electronics, Hunan Normal University, Changsha 410081, China
| | - Feng Tian
- School of Physics and Electronics, Hunan Normal University, Changsha 410081, China
| | - Xiaofei Cao
- College of Information Science and Engineering, Hunan Normal University, Changsha 410081, China
| | - Runmin Wang
- College of Information Science and Engineering, Hunan Normal University, Changsha 410081, China
| |
Collapse
|
7
|
Chen Y, Chu J, Lin F, Jiang B, Liu Y, Huang B, Zhang R, Xin B, Ding X. Polarization clustering of biological structures with Mueller matrix parameters. JOURNAL OF BIOPHOTONICS 2023; 16:e202200255. [PMID: 36259128 DOI: 10.1002/jbio.202200255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/11/2022] [Revised: 10/07/2022] [Accepted: 10/17/2022] [Indexed: 06/16/2023]
Abstract
Mueller matrix imaging polarimetry (MMIP) is a promising technique for the characterization of biological tissues, including the classification of microstructures in pathological diagnosis. To expand the parameter space of Mueller matrix parameters, we propose new vector parameters (VPs) according to the Mueller matrix polar decomposition method. We measure invasive bladder cancer (IBC) with extensive necrosis and high-grade ductal carcinoma in situ (DCIS) with MMIP, and the regions of cancer cells and fibrotic stroma are classified with the VPs. Then the proposed and existing VPs are mapped on the Poincaré sphere with 3D visualization, and an indicator of spatial feature is defined based on the minimum enclosing sphere to evaluate the classification capability of the VPs. For both IBC and DCIS, the results show that the proposed VPs exhibit evident contrast between the regions of cancer cells and fibrotic stroma. This study broadens the fundamental Mueller matrix parameters and helps to improve the characterization ability of the MMIP technique.
Collapse
Affiliation(s)
- Yongtai Chen
- School of Mechanical Engineering, Dalian University of Technology, Dalian, China
| | - Jinkui Chu
- School of Mechanical Engineering, Dalian University of Technology, Dalian, China
| | - Fanlu Lin
- Department of Urology, Linyi Central Hospital, Linyi, China
| | - Bing Jiang
- Department of Pathology, Cancer Hospital of Dalian University of Technology, Liaoning Cancer Hospital & Institute, Shenyang, Liaoning Province, China
| | - Yadong Liu
- Institute of Ultrasound Imaging, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Bo Huang
- Department of Pathology, Cancer Hospital of Dalian University of Technology, Liaoning Cancer Hospital & Institute, Shenyang, Liaoning Province, China
| | - Ran Zhang
- School of Mechanical Engineering, Dalian University of Technology, Dalian, China
| | - Benda Xin
- School of Mechanical Engineering, Dalian University of Technology, Dalian, China
| | - Xiaohan Ding
- School of Mechanical Engineering, Dalian University of Technology, Dalian, China
| |
Collapse
|
8
|
Leng Y, Huang T, Pei H, Hu Z, Guo B, Liao R, Ma H. Mueller matrix imaging with a spatially modulated polarization light source. OPTICS EXPRESS 2022; 30:40441-40454. [PMID: 36298977 DOI: 10.1364/oe.474360] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Accepted: 10/04/2022] [Indexed: 06/16/2023]
Abstract
In this paper, we present a Mueller matrix imaging system consisting of a spatially modulated polarization light source (SMPL) and a dual division-of-focal-plane (DoFP) polarimeters as the PSA and 2D detector. The system does not contain moving parts such as a rotating stage, which leads to more robust and reliable operations for applications in hostile settings. By taking Muller matrix images at variable distances between the SMPL and the target, we examine in details errors due to different spatial distributions in angle and intensity of different polarized lights. A calibration method is proposed to reduce such errors introduced by SMPL. The performances of the new imaging technique and the calibration method are tested in Mueller matrix imaging of different samples.
Collapse
|
9
|
Polarimetric biomarkers of peri-tumoral stroma can correlate with 5-year survival in patients with left-sided colorectal cancer. Sci Rep 2022; 12:12652. [PMID: 35879367 PMCID: PMC9314438 DOI: 10.1038/s41598-022-16178-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Accepted: 07/06/2022] [Indexed: 12/24/2022] Open
Abstract
Using a novel variant of polarized light microscopy for high-contrast imaging and quantification of unstained histology slides, the current study assesses the prognostic potential of peri-tumoral collagenous stroma architecture in 32 human stage III colorectal cancer (CRC) patient samples. We analyze three distinct polarimetrically-derived images and their associated texture features, explore different unsupervised clustering algorithm models to group the data, and compare the resultant groupings with patient survival. The results demonstrate an appreciable total accuracy of ~ 78% with significant separation (p < 0.05) across all approaches for the binary classification of 5-year patient survival outcomes. Surviving patients preferentially belonged to Cluster 1 irrespective of model approach, suggesting similar stromal microstructural characteristics in this sub-population. The results suggest that polarimetrically-derived stromal biomarkers may possess prognostic value that could improve clinical management/treatment stratification in CRC patients.
Collapse
|
10
|
Shao C, Chen B, He H, He C, Shen Y, Zhai H, Ma H. Analyzing the Influence of Imaging Resolution on Polarization Properties of Scattering Media Obtained From Mueller Matrix. Front Chem 2022; 10:936255. [PMID: 35903191 PMCID: PMC9315153 DOI: 10.3389/fchem.2022.936255] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2022] [Accepted: 06/09/2022] [Indexed: 11/23/2022] Open
Abstract
The Mueller matrix contains abundant micro- and even nanostructural information of media. Especially, it can be used as a powerful tool to characterize anisotropic structures quantitatively, such as the particle size, density, and orientation information of fibers in the sample. Compared with unpolarized microscopic imaging techniques, Mueller matrix microscopy can also obtain some essential structural information about the sample from the derived parameters images at low resolution. Here, to analyze the comprehensive effects of imaging resolution on polarization properties obtained from the Mueller matrix, we, first, measure the microscopic Mueller matrices of unstained rat dorsal skin tissue slices rich in collagen fibers using a series of magnifications or numerical aperture (NA) values of objectives. Then, the first-order moments and image texture parameters are quantified and analyzed in conjunction with the polarization parameter images. The results show that the Mueller matrix polar decomposition parameters diattenuation D, linear retardance δ, and depolarization Δ images obtained using low NA objective retain most of the structural information of the sample and can provide fast imaging speed. In addition, the scattering phase function analysis and Monte Carlo simulation based on the cylindrical scatterers reveal that the diattenuation parameter D images with different imaging resolutions are expected to be used to distinguish among the fibrous scatterers in the medium with different particle sizes. This study provides a criterion to decide which structural information can be accurately and rapidly obtained using a transmission Mueller matrix microscope with low NA objectives to assist pathological diagnosis and other applications.
Collapse
Affiliation(s)
- Conghui Shao
- Department of Physics, Tsinghua University, Beijing, China
- Guangdong Research Center of Polarization Imaging and Measurement Engineering Technology, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
| | - Binguo Chen
- Guangdong Research Center of Polarization Imaging and Measurement Engineering Technology, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Department of Biomedical Engineering, Tsinghua University, Beijing, China
| | - Honghui He
- Guangdong Research Center of Polarization Imaging and Measurement Engineering Technology, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- *Correspondence: Honghui He, ; Chao He,
| | - Chao He
- Department of Engineering Science, University of Oxford, Oxford, United Kingdom
- *Correspondence: Honghui He, ; Chao He,
| | - Yuanxing Shen
- Guangdong Research Center of Polarization Imaging and Measurement Engineering Technology, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Department of Biomedical Engineering, Tsinghua University, Beijing, China
| | - Haoyu Zhai
- Guangdong Research Center of Polarization Imaging and Measurement Engineering Technology, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Department of Biomedical Engineering, Tsinghua University, Beijing, China
| | - Hui Ma
- Department of Physics, Tsinghua University, Beijing, China
- Guangdong Research Center of Polarization Imaging and Measurement Engineering Technology, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
| |
Collapse
|
11
|
Yang X, Zhao Q, Huang T, Hu Z, Bu T, He H, Hou A, Li M, Xiao Y, Ma H. Deep learning for denoising in a Mueller matrix microscope. BIOMEDICAL OPTICS EXPRESS 2022; 13:3535-3551. [PMID: 35781954 PMCID: PMC9208591 DOI: 10.1364/boe.457219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 05/05/2022] [Accepted: 05/17/2022] [Indexed: 06/15/2023]
Abstract
The Mueller matrix microscope is a powerful tool for characterizing the microstructural features of a complex biological sample. Performance of a Mueller matrix microscope usually relies on two major specifications: measurement accuracy and acquisition time, which may conflict with each other but both contribute to the complexity and expenses of the apparatus. In this paper, we report a learning-based method to improve both specifications of a Mueller matrix microscope using a rotating polarizer and a rotating waveplate polarization state generator. Low noise data from long acquisition time are used as the ground truth. A modified U-Net structured network incorporating channel attention effectively reduces the noise in lower quality Mueller matrix images obtained with much shorter acquisition time. The experimental results show that using high quality Mueller matrix data as ground truth, such a learning-based method can achieve both high measurement accuracy and short acquisition time in polarization imaging.
Collapse
Affiliation(s)
- Xiongjie Yang
- Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
- Contributed equally
| | - Qianhao Zhao
- Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
- Contributed equally
| | - Tongyu Huang
- Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
- Department of Biomedical Engineering, Tsinghua University, Beijing 100084, China
| | - Zheng Hu
- Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
| | - Tongjun Bu
- Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
| | - Honghui He
- Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
| | - Anli Hou
- Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
- Department of Gynaecology, University of Chinese Academy of Sciences Shenzhen Hospital, Shenzhen 518106, China
| | - Migao Li
- Guangdong Liss Optical Instrument Co., Ltd., Guangzhou 510095, China
| | - Yucheng Xiao
- Beijing Normal University - Hong Kong Baptist University United International College, Zhuhai 519085, China
| | - Hui Ma
- Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
- Department of Biomedical Engineering, Tsinghua University, Beijing 100084, China
- Department of Physics, Tsinghua University, Beijing 100084, China
| |
Collapse
|
12
|
Wan J, Dong Y, Xue JH, Lin L, Du S, Dong J, Yao Y, Li C, Ma H. Polarization-based probabilistic discriminative model for quantitative characterization of cancer cells. BIOMEDICAL OPTICS EXPRESS 2022; 13:3339-3354. [PMID: 35781945 PMCID: PMC9208602 DOI: 10.1364/boe.456649] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Revised: 04/05/2022] [Accepted: 04/26/2022] [Indexed: 05/25/2023]
Abstract
We propose a polarization-based probabilistic discriminative model for deriving a set of new sigmoid-transformed polarimetry feature parameters, which not only enables accurate and quantitative characterization of cancer cells at pixel level, but also accomplish the task with a simple and stable model. By taking advantages of polarization imaging techniques, these parameters enable a low-magnification and wide-field imaging system to separate the types of cells into more specific categories that previously were distinctive under high magnification. Instead of blindly choosing the model, the L0 regularization method is used to obtain the simplified and stable polarimetry feature parameter. We demonstrate the model viability by using the pathological tissues of breast cancer and liver cancer, in each of which there are two derived parameters that can characterize the cells and cancer cells respectively with satisfactory accuracy and sensitivity. The stability of the final model opens the possibility for physical interpretation and analysis. This technique may bypass the typically labor-intensive and subjective tumor evaluating system, and could be used as a blueprint for an objective and automated procedure for cancer cell screening.
Collapse
Affiliation(s)
- Jiachen Wan
- Guangdong Engineering Center of
Polarization Imaging and Sensing Technology, Tsinghua Shenzhen
International Graduate School, Tsinghua
University, Shenzhen 518055, China
- Equal contributors
| | - Yang Dong
- Guangdong Engineering Center of
Polarization Imaging and Sensing Technology, Tsinghua Shenzhen
International Graduate School, Tsinghua
University, Shenzhen 518055, China
- Center for Precision Medicine and
Healthcare, Tsinghua-Berkeley Shenzhen Institute,
Tsinghua University, Shenzhen 518071,
China
- Equal contributors
| | - Jing-Hao Xue
- Department of Statistical Science,
University College London, London WC1E 6BT,
UK
| | - Liyan Lin
- Department of Pathology,
Fujian Medical University Cancer Hospital,
Fujian Cancer Hospital, Fuzhou 350014, China
| | - Shan Du
- Department of Pathology,
University of Chinese Academy of Sciences Shenzhen
Hospital, Shenzhen 518106, China
| | - Jia Dong
- Guangdong Engineering Center of
Polarization Imaging and Sensing Technology, Tsinghua Shenzhen
International Graduate School, Tsinghua
University, Shenzhen 518055, China
| | - Yue Yao
- Guangdong Engineering Center of
Polarization Imaging and Sensing Technology, Tsinghua Shenzhen
International Graduate School, Tsinghua
University, Shenzhen 518055, China
- Center for Precision Medicine and
Healthcare, Tsinghua-Berkeley Shenzhen Institute,
Tsinghua University, Shenzhen 518071,
China
| | - Chao Li
- Department of Pathology,
Fujian Medical University Cancer Hospital,
Fujian Cancer Hospital, Fuzhou 350014, China
| | - Hui Ma
- Guangdong Engineering Center of
Polarization Imaging and Sensing Technology, Tsinghua Shenzhen
International Graduate School, Tsinghua
University, Shenzhen 518055, China
- Center for Precision Medicine and
Healthcare, Tsinghua-Berkeley Shenzhen Institute,
Tsinghua University, Shenzhen 518071,
China
- Department of Physics,
Tsinghua University, Beijing 100084,
China
| |
Collapse
|
13
|
Polarization Aberrations in High-Numerical-Aperture Lens Systems and Their Effects on Vectorial-Information Sensing. REMOTE SENSING 2022. [DOI: 10.3390/rs14081932] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
The importance of polarization aberrations has been recognized and studied in numerous optical systems and related applications. It is known that polarization aberrations are particularly crucial in certain photogrammetry and microscopy techniques that are related to vectorial information—such as polarization imaging, stimulated emission depletion microscopy, and structured illumination microscopy. Hence, a reduction in polarization aberrations would be beneficial to different types of optical imaging/sensing techniques with enhanced vectorial information. In this work, we first analyzed the intrinsic polarization aberrations induced by a high-NA lens theoretically and experimentally. The aberrations of depolarization, diattenuation, and linear retardance were studied in detail using the Mueller matrix polar-decomposition method. Based on an analysis of the results, we proposed strategies to compensate the polarization aberrations induced by high-NA lenses for hardware-based solutions. The preliminary imaging results obtained using a Mueller matrix polarimeter equipped with multiple coated aspheric lenses for polarization-aberration reduction confirmed that the conclusions and strategies proposed in this study had the potential to provide more precise polarization information of the targets for applications spanning across classical optics, remote sensing, biomedical imaging, photogrammetry, and vectorial optical-information extraction.
Collapse
|
14
|
Texture Image Compression Algorithm Based on Self-Organizing Neural Network. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:4865808. [PMID: 35440945 PMCID: PMC9013571 DOI: 10.1155/2022/4865808] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/18/2022] [Revised: 02/21/2022] [Accepted: 02/28/2022] [Indexed: 11/23/2022]
Abstract
With the rapid development of science and technology, human beings have gradually stepped into a brand-new digital era. Virtual reality technology has brought people an immersive experience. In order to enable users to get a better virtual reality experience, the pictures produced by virtual skillfully must be realistic enough and support users' real-time interaction. So interactive real-time photorealistic rendering becomes the focus of research. Texture mapping is a technology proposed to solve the contradiction between real time and reality. It has been widely studied and used since it was proposed. However, due to limited bandwidth and memory storage, it brings challenges to the stain dyeing of many large texture images, so texture compression is introduced. Texture compression can improve the utilization rate of cache but also greatly reduce the pressure on data transmission caused by the system, which largely solves the problem of real-time rendering of realistic graphics. Due to the particularity of texture image compression, it is necessary to consider not only the quality of texture image after compression ratio and decompression but also whether the algorithm is compatible with mainstream graphics cards. On this basis, we put forward the texture image compression method based on self-organizing mapping, the experiment results show that our method has achieved good results, and it is superior to other methods in most performance indexes.
Collapse
|
15
|
Si L, Huang T, Wang X, Yao Y, Dong Y, Liao R, Ma H. Deep learning Mueller matrix feature retrieval from a snapshot Stokes image. OPTICS EXPRESS 2022; 30:8676-8689. [PMID: 35299314 DOI: 10.1364/oe.451612] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Accepted: 02/18/2022] [Indexed: 06/14/2023]
Abstract
A Mueller matrix (MM) provides a comprehensive representation of the polarization properties of a complex medium and encodes very rich information on the macro- and microstructural features. Histopathological features can be characterized by polarization parameters derived from MM. However, a MM must be derived from at least four Stokes vectors corresponding to four different incident polarization states, which makes the qualities of MM very sensitive to small changes in the imaging system or the sample during the exposures, such as fluctuations in illumination light and co-registration of polarization component images. In this work, we use a deep learning approach to retrieve MM-based specific polarimetry basis parameters (PBPs) from a snapshot Stokes vector. This data post-processing method is capable of eliminating errors introduced by multi-exposure, as well as reducing the imaging time and hardware complexity. It shows the potential for accurate MM imaging on dynamic samples or in unstable environments. The translation model is designed based on generative adversarial network with customized loss functions. The effectiveness of the approach was demonstrated on liver and breast tissue slices and blood smears. Finally, we evaluated the performance by quantitative similarity assessment methods in both pixel and image levels.
Collapse
|
16
|
Probing Dynamic Variation of Layered Microstructure Using Backscattering Polarization Imaging. PHOTONICS 2022. [DOI: 10.3390/photonics9030153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Polarization imaging can quantitatively probe the microscopic structure of biological tissues which can be complex and consist of layered structures. In this paper, we established a fast-backscattering Mueller matrix imaging system to characterize the dynamic variation in the microstructure of single-layer and double-layer tissues as glycerin solution penetrated into the samples. The characteristic response of Mueller matrix elements, as well as polarization parameters with clearer physics meanings, show that polarization imaging can capture the dynamic variation in the layered microstructure. The experimental results are confirmed by Monte Carlo simulations. Further examination on the accuracy of Mueller matrix measurements also shows that much faster speed has to be considered when backscattering Mueller matrix imaging is applied to living samples.
Collapse
|
17
|
Yao Y, Zhang F, Wang B, Wan J, Si L, Dong Y, Zhu Y, Liu X, Chen L, Ma H. Polarization imaging-based radiomics approach for the staging of liver fibrosis. BIOMEDICAL OPTICS EXPRESS 2022; 13:1564-1580. [PMID: 35414973 PMCID: PMC8973194 DOI: 10.1364/boe.450294] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/02/2021] [Revised: 01/21/2022] [Accepted: 02/04/2022] [Indexed: 05/25/2023]
Abstract
Mueller matrix imaging contains abundant biological microstructure information and has shown promising potential in clinical applications. Compared with the ordinary unpolarized light microscopy that relies on the spatial resolution to reveal detailed histological features, Mueller matrix imaging encodes rich information on the microstructures even at low-resolution and wide-field conditions. Accurate staging of liver fibrosis is essential for the therapeutic diagnosis and prognosis of chronic liver diseases. In the clinic, pathologists commonly use semiquantitative numerical scoring systems to determine the stages of liver fibrosis based on the visualization of stained characteristic morphological changes, which require skilled staining technicians and well-trained pathologists. A polarization imaging based quantitative diagnostic method can help to reduce the time-consuming multiple staining processes and provide quantitative information to facilitate the accurate staging of liver fibrosis. In this study, we report a polarization imaging based radiomics approach to provide quantitative diagnostic features for the staging of liver fibrosis. Comparisons between polarization image features under a 4× objective lens with H&E image features under 4×, 10×, 20×, and 40× objective lenses were performed to highlight the superiority of the high dimensional polarization image features in the characterization of the histological microstructures of liver fibrosis tissues at low-resolution and wide-field conditions.
Collapse
Affiliation(s)
- Yue Yao
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Center for Precision Medicine and Healthcare, Shenzhen 518071, China
- Tsinghua Shenzhen International Graduate School, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen 518055, China
| | - Fengdi Zhang
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Center for Precision Medicine and Healthcare, Shenzhen 518071, China
- Tsinghua Shenzhen International Graduate School, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen 518055, China
| | - Bin Wang
- Fujian Medical University, Department of Pathology and Institute of Oncology, School of Basic Medical Sciences, Fuzhou 350014, China
- Fujian Medical University, Diagnostic Pathology Center, Fuzhou 350014, China
- Fujian Medical University, Mengchao Hepatobiliary Hospital, Fuzhou 350014, China
| | - Jiachen Wan
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Center for Precision Medicine and Healthcare, Shenzhen 518071, China
- Tsinghua Shenzhen International Graduate School, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen 518055, China
| | - Lu Si
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Center for Precision Medicine and Healthcare, Shenzhen 518071, China
- Tsinghua Shenzhen International Graduate School, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen 518055, China
| | - Yang Dong
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Center for Precision Medicine and Healthcare, Shenzhen 518071, China
- Tsinghua Shenzhen International Graduate School, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen 518055, China
| | - Yuanhuan Zhu
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Center for Precision Medicine and Healthcare, Shenzhen 518071, China
- Tsinghua Shenzhen International Graduate School, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen 518055, China
| | - Xiaolong Liu
- Tsinghua University, Department of Physics, Beijing 100084, China
| | - Lihong Chen
- Fujian Medical University, Department of Pathology and Institute of Oncology, School of Basic Medical Sciences, Fuzhou 350014, China
- Fujian Medical University, Diagnostic Pathology Center, Fuzhou 350014, China
- Fujian Medical University, Mengchao Hepatobiliary Hospital, Fuzhou 350014, China
| | - Hui Ma
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Center for Precision Medicine and Healthcare, Shenzhen 518071, China
- Tsinghua Shenzhen International Graduate School, Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Shenzhen 518055, China
- Tsinghua University, Department of Physics, Beijing 100084, China
| |
Collapse
|
18
|
Si L, Li N, Huang T, Du S, Dong Y, Yao Y, Ma H. Computational image translation from Mueller matrix polarimetry to bright-field microscopy. JOURNAL OF BIOPHOTONICS 2022; 15:e202100242. [PMID: 34775685 DOI: 10.1002/jbio.202100242] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Revised: 10/18/2021] [Accepted: 11/10/2021] [Indexed: 06/13/2023]
Abstract
Mueller matrix (MM) polarimetry can provide comprehensive information about the polarization properties that are closely related to the microstructural features and has demonstrated its potential in biomedical studies and clinical practices, and bright-field microscopy is widely used in pathological diagnosis as the golden standard. In this work, we improve the throughput of MM microscopy by learning a statistical transformation between these two imaging systems based on deep learning. Using this approach, the MM microscope can generate an image that is equivalent to a bright-field microscope image of the matching field of view. We add new transformative capability to the existing MM imaging system without requiring extra hardware. The translation model is based on conditional generative adversarial network with customized loss functions. We demonstrated the effectiveness of our approach on liver and breast tissues and evaluated the performance by four quantitative similarity assessment methods in pixel, image and distribution levels, respectively.
Collapse
Affiliation(s)
- Lu Si
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen, China
| | - Naiqi Li
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen, China
| | - Tongyu Huang
- Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Department of Biomedical Engineering, Tsinghua University, Beijing, China
| | - Shan Du
- Department of Pathology, University of Chinese Academy of Sciences Shenzhen Hospital, Shenzhen, China
| | - Yang Dong
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen, China
| | - Yue Yao
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen, China
| | - Hui Ma
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen, China
- Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
- Department of Biomedical Engineering, Tsinghua University, Beijing, China
- Department of Physics, Tsinghua University, Beijing, China
| |
Collapse
|
19
|
Trout RM, Gnanatheepam E, Gado A, Reik C, Ramella-Roman JC, Hunter M, Schnelldorfer T, Georgakoudi I. Polarization enhanced laparoscope for improved visualization of tissue structural changes associated with peritoneal cancer metastasis. BIOMEDICAL OPTICS EXPRESS 2022; 13:571-589. [PMID: 35284190 PMCID: PMC8884200 DOI: 10.1364/boe.443926] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/21/2021] [Revised: 12/17/2021] [Accepted: 12/20/2021] [Indexed: 06/03/2023]
Abstract
A polarization enhanced laparoscopy (PEL) imaging system was developed to examine the feasibility of utilizing PEL to augment conventional white light laparoscopy (WLL) in the visualization of peritoneal cancer metastases. The system includes a modified tip to illuminate tissue with linearly polarized light and elements in the detection path enabling recording of corresponding images linearly co- and cross-polarized relative to the incident light. WLL and PEL images from optical tissue phantoms with features of distinct scattering cross-section confirm the enhanced sensitivity of PEL to such characteristics. Additional comparisons based on images acquired from collagen gels with different levels of fiber alignment highlight another source of PEL contrast. Finally, PEL and WLL images of ex vivo human tissue illustrate the potential of PEL to improve visualization of cancerous tissue surrounded by healthy peritoneum. Given the simplicity of the approach and its potential for seamless integration with current clinical practice, our results provide motivation for clinical translation.
Collapse
Affiliation(s)
- Robert M. Trout
- Department of Biomedical Engineering, Tufts University, 200 College Ave, Medford, MA 01255, USA
| | - Einstein Gnanatheepam
- Department of Biomedical Engineering, Tufts University, 200 College Ave, Medford, MA 01255, USA
| | - Ahmed Gado
- Department of Biomedical Engineering, Tufts University, 200 College Ave, Medford, MA 01255, USA
| | - Christopher Reik
- Department of Biomedical Engineering, Tufts University, 200 College Ave, Medford, MA 01255, USA
| | | | - Martin Hunter
- Department of Biomedical Engineering, University of Massachusetts at Amherst, Amherst, MA, USA
| | - Thomas Schnelldorfer
- Department of Biomedical Engineering, Tufts University, 200 College Ave, Medford, MA 01255, USA
- Division of Surgical Oncology, Tufts Medical Center, 800 Washington St, Boston, MA 02111, USA
- Contributed equally
| | - Irene Georgakoudi
- Department of Biomedical Engineering, Tufts University, 200 College Ave, Medford, MA 01255, USA
- Contributed equally
| |
Collapse
|
20
|
Dong Y, Wan J, Wang X, Xue JH, Zou J, He H, Li P, Hou A, Ma H. A Polarization-Imaging-Based Machine Learning Framework for Quantitative Pathological Diagnosis of Cervical Precancerous Lesions. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:3728-3738. [PMID: 34260351 DOI: 10.1109/tmi.2021.3097200] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
Polarization images encode high resolution microstructural information even at low resolution. We propose a framework combining polarization imaging and traditional microscopy imaging, constructing a dual-modality machine learning framework that is not only accurate but also generalizable and interpretable. We demonstrate the viability of our proposed framework using the cervical intraepithelial neoplasia grading task, providing a polarimetry feature parameter to quantitatively characterize microstructural variations with lesion progression in hematoxylin-eosin-stained pathological sections of cervical precancerous tissues. By taking advantages of polarization imaging techniques and machine learning methods, the model enables interpretable and quantitative diagnosis of cervical precancerous lesion cases with improved sensitivity and accuracy in a low-resolution and wide-field system. The proposed framework applies routine image-analysis technology to identify the macro-structure and segment the target region in H&E-stained pathological images, and then employs emerging polarization method to extract the micro-structure information of the target region, which intends to expand the boundary of the current image-heavy digital pathology, bringing new possibilities for quantitative medical diagnosis.
Collapse
|
21
|
He C, He H, Chang J, Chen B, Ma H, Booth MJ. Polarisation optics for biomedical and clinical applications: a review. LIGHT, SCIENCE & APPLICATIONS 2021; 10:194. [PMID: 34552045 PMCID: PMC8458371 DOI: 10.1038/s41377-021-00639-x] [Citation(s) in RCA: 109] [Impact Index Per Article: 36.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2021] [Revised: 08/30/2021] [Accepted: 09/01/2021] [Indexed: 05/13/2023]
Abstract
Many polarisation techniques have been harnessed for decades in biological and clinical research, each based upon measurement of the vectorial properties of light or the vectorial transformations imposed on light by objects. Various advanced vector measurement/sensing techniques, physical interpretation methods, and approaches to analyse biomedically relevant information have been developed and harnessed. In this review, we focus mainly on summarising methodologies and applications related to tissue polarimetry, with an emphasis on the adoption of the Stokes-Mueller formalism. Several recent breakthroughs, development trends, and potential multimodal uses in conjunction with other techniques are also presented. The primary goal of the review is to give the reader a general overview in the use of vectorial information that can be obtained by polarisation optics for applications in biomedical and clinical research.
Collapse
Affiliation(s)
- Chao He
- Department of Engineering Science, University of Oxford, Parks Road, Oxford, OX1 3PJ, UK.
| | - Honghui He
- Guangdong Engineering Center of Polarisation Imaging and Sensing Technology, Tsinghua Shenzhen International Graduate School, Tsinghua University, 518055, Shenzhen, China.
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, 518055, Shenzhen, China.
| | - Jintao Chang
- Guangdong Engineering Center of Polarisation Imaging and Sensing Technology, Tsinghua Shenzhen International Graduate School, Tsinghua University, 518055, Shenzhen, China
- Department of Physics, Tsinghua University, 100084, Beijing, China
| | - Binguo Chen
- Guangdong Engineering Center of Polarisation Imaging and Sensing Technology, Tsinghua Shenzhen International Graduate School, Tsinghua University, 518055, Shenzhen, China
- Department of Biomedical Engineering, Tsinghua University, 100084, Beijing, China
| | - Hui Ma
- Guangdong Engineering Center of Polarisation Imaging and Sensing Technology, Tsinghua Shenzhen International Graduate School, Tsinghua University, 518055, Shenzhen, China
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, 518055, Shenzhen, China
- Department of Physics, Tsinghua University, 100084, Beijing, China
| | - Martin J Booth
- Department of Engineering Science, University of Oxford, Parks Road, Oxford, OX1 3PJ, UK.
| |
Collapse
|
22
|
Meng R, Shao C, Li P, Dong Y, Hou A, Li C, Lin L, He H, Ma H. Transmission Mueller matrix imaging with spatial filtering. OPTICS LETTERS 2021; 46:4009-4012. [PMID: 34388798 DOI: 10.1364/ol.435166] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Accepted: 07/11/2021] [Indexed: 06/13/2023]
Abstract
In this Letter, we report a study on the effects of spatial filtering for a transmission Mueller matrix imaging system. A spatial filter (SF) is placed on the back Fourier plane of the imaging lens in a dual-rotating-retarders Mueller matrix imaging system to select photons within a certain scattering angle. The system is then applied to three types of human cancerous tissues. When imaging with a small-aperture SF, some polarimetry basis parameters show sharp changes in contrast in the cancerous regions. Monte Carlo simulations using a simple sphere-cylinder scattering model also show that spatial filtering of the scattered photons provides extra information on the size and shape of the scattering particles. The results indicate that spatial filtering enhances the capability of polarization imaging as a powerful tool for biomedical diagnosis.
Collapse
|
23
|
Lee HR, Saytashev I, Du Le VN, Mahendroo M, Ramella-Roman J, Novikova T. Mueller matrix imaging for collagen scoring in mice model of pregnancy. Sci Rep 2021; 11:15621. [PMID: 34341418 PMCID: PMC8329204 DOI: 10.1038/s41598-021-95020-8] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2021] [Accepted: 07/19/2021] [Indexed: 11/16/2022] Open
Abstract
Preterm birth risk is associated with early softening of the uterine cervix in pregnancy due to the accelerated remodeling of collagen extracellular matrix. Studies of mice model of pregnancy were performed with an imaging Mueller polarimeter at different time points of pregnancy to find polarimetric parameters for collagen scoring. Mueller matrix images of the unstained sections of mice uterine cervices were taken at day 6 and day 18 of 19-days gestation period and at different spatial locations through the cervices. The logarithmic decomposition of the recorded Mueller matrices mapped the depolarization, linear retardance, and azimuth of the optical axis of cervical tissue. These images highlighted both the inner structure of cervix and the arrangement of cervical collagen fibers confirmed by the second harmonic generation microscopy. The statistical analysis and two-Gaussians fit of the distributions of linear retardance and linear depolarization in the entire images of cervical tissue (without manual selection of the specific regions of interest) quantified the randomization of collagen fibers alignment with gestation time. At day 18 the remodeling of cervical extracellular matrix of collagen was measurable at the external cervical os that is available for the direct optical observations in vivo. It supports the assumption that imaging Mueller polarimetry holds promise for the fast and accurate collagen scoring in pregnancy and the assessment of the preterm birth risk.
Collapse
Affiliation(s)
- Hee Ryung Lee
- LPICM, CNRS, Ecole polytechnique, IP Paris, 91128, Palaiseau, France
| | - Ilyas Saytashev
- Department of Biomedical Engineering, College of Engineering and Computing, Florida International University, 10555 West Flagler Street, Miami, FL, 33174, USA
| | - Vinh Nguyen Du Le
- Department of Biomedical Engineering, College of Engineering and Computing, Florida International University, 10555 West Flagler Street, Miami, FL, 33174, USA
| | - Mala Mahendroo
- Department of Obstetrics and Gynecology, University of Texas Southwestern Medical Center, Dallas, Texas, 75390, USA
| | - Jessica Ramella-Roman
- Department of Biomedical Engineering, College of Engineering and Computing, Florida International University, 10555 West Flagler Street, Miami, FL, 33174, USA.
- Department of Ophthalmology, Herbert Wertheim College of Medicine, Florida International University, 11200 SW 8th Street, Miami, FL, 33199, USA.
| | - Tatiana Novikova
- LPICM, CNRS, Ecole polytechnique, IP Paris, 91128, Palaiseau, France.
| |
Collapse
|
24
|
Ma D, Lu Z, Xia L, Liao Q, Yang W, Ma H, Liao R, Ma L, Liu Z. MuellerNet: a hybrid 3D-2D CNN for cell classification with Mueller matrix images. APPLIED OPTICS 2021; 60:6682-6694. [PMID: 34612912 DOI: 10.1364/ao.431076] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
Different from conventional microimaging techniques, polarization imaging can generate multiple polarization images in a single perspective by changing the polarization angle. However, how to efficiently fuse the information in these multiple polarization images by a convolutional neural network (CNN) is still a challenging problem. In this paper, we propose a hybrid 3D-2D convolutional neural network called MuellerNet, to classify biological cells with Mueller matrix images (MMIs). The MuellerNet includes a normal stream and a polarimetric stream, in which the first Mueller matrix image is taken as the input of normal stream, and the rest MMIs are stacked to form the input of a polarimetric stream. The normal stream is mainly constructed with a backbone network and, in the polarimetric stream, the attention mechanism is used to adaptively assign weights to different convolutional maps. To improve the network's discrimination, a loss function is introduced to simultaneously optimize parameters of the two streams. Two Mueller matrix image datasets are built, which include four types of breast cancer cells and three types of algal cells, respectively. Experiments are conducted on these two datasets with many well-known and recent networks. Results show that the proposed network efficiently improves the classification accuracy and helps to find discriminative features in MMIs.
Collapse
|
25
|
Zhu Y, Dong Y, Yao Y, Si L, Liu Y, He H, Ma H. Probing layered structures by multi-color backscattering polarimetry and machine learning. BIOMEDICAL OPTICS EXPRESS 2021; 12:4324-4339. [PMID: 34457417 PMCID: PMC8367275 DOI: 10.1364/boe.425614] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/19/2021] [Revised: 06/06/2021] [Accepted: 06/17/2021] [Indexed: 05/10/2023]
Abstract
Polarization imaging can quantitatively probe the characteristic microstructural features of biological tissues non-invasively. In biomedical tissues, layered structures are common. Superposition of two simple layers can result in a complex Mueller matrix, and multi-color backscattering polarimetry can help to probe layered structures. In this work, multi-color backscattering Mueller matrix images are measured for living nude mice skins. Preliminary analysis of anisotropy parameter A and linear polarizance parameter b show signs of a layered structure in the skin. For more detailed examinations on polarization features of layered samples, we generate Mueller matrices by experimenting with two-layered thick tissues and concentrically aligned silk submerged in milk. Then we use supervised machine learning to identify polarization parameters that are sensitive to layered structure and guide the synthesis of more parameters. Monte Carlo simulation is also adopted to explore the relationship between parameters and microstructures of media. We conclude that multi-color backscattering polarimetry combined with supervised machine learning can be applied to probe the characteristic microstructure in layered living tissue samples.
Collapse
Affiliation(s)
- Yuanhuan Zhu
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen 518055, China
| | - Yang Dong
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen 518055, China
| | - Yue Yao
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen 518055, China
| | - Lu Si
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen 518055, China
| | - Yudi Liu
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen 518055, China
| | - Honghui He
- Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
| | - Hui Ma
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen 518055, China
- Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
- Department of Physics, Tsinghua University, Beijing 100084, China
| |
Collapse
|
26
|
Liu Y, Dong Y, Si L, Meng R, Dong Y, Ma H. Comparison between image texture and polarization features in histopathology. BIOMEDICAL OPTICS EXPRESS 2021; 12:1593-1608. [PMID: 33796375 PMCID: PMC7984792 DOI: 10.1364/boe.416382] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Revised: 02/08/2021] [Accepted: 02/08/2021] [Indexed: 05/08/2023]
Abstract
Digital pathology has shown great importance for diagnostic purposes in the digital age by integrating basic image features into multi-modality information. We quantify the degree of correlation between the multiple texture features from H&E images and polarization parameter sets derived from Mueller matrix images of the same sample to provide more microstructural information for assisting diagnosis. The experimental result shows the correlations between texture feature and polarization parameter via Pearson coefficients. Polarization parameters t1 , DL and the depolarization parameter Δ correlated with image texture features Tamura_Fcon and Tamura_Frgh, and can be used as powerful tools to quantitatively characterize cell nuclei related with tumor progression in breast pathological tissues. Polarization parameters δ and rL associated with the image texture feature Tamura_Flin have great potential for the quantitative characterization of proliferative fibers produced by inflammation. Furthermore, polarization parameters have the advantages of stable recognition in low resolution images. This work validates the associations between image texture features and polarization parameters and the merit of polarization imaging methods in low-resolution situations.
Collapse
Affiliation(s)
- Yudi Liu
- Center for Precision Medicine and Healthcare, Tsinghua-Berkeley Shenzhen Institute, Shenzhen 518055, China
- These authors contributed equally to this work
| | - Yang Dong
- Center for Precision Medicine and Healthcare, Tsinghua-Berkeley Shenzhen Institute, Shenzhen 518055, China
- These authors contributed equally to this work
| | - Lu Si
- Center for Precision Medicine and Healthcare, Tsinghua-Berkeley Shenzhen Institute, Shenzhen 518055, China
| | - Ruoyu Meng
- Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Graduate School at Shenzhen Tsinghua University, Shenzhen 518055, China
| | - Yanmin Dong
- Shenzhen Hospital of Traditional Chinese Medicine, Shenzhen 518034, China
| | - Hui Ma
- Center for Precision Medicine and Healthcare, Tsinghua-Berkeley Shenzhen Institute, Shenzhen 518055, China
- Shenzhen Key Laboratory for Minimal Invasive Medical Technologies, Institute of Optical Imaging and Sensing, Graduate School at Shenzhen Tsinghua University, Shenzhen 518055, China
- Department of Physics, Tsinghua University, Beijing 100084, China
| |
Collapse
|