1
|
Cai S, Wu Y, Chen G. A Novel Elastomeric UNet for Medical Image Segmentation. Front Aging Neurosci 2022; 14:841297. [PMID: 35360219 PMCID: PMC8961507 DOI: 10.3389/fnagi.2022.841297] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 01/10/2022] [Indexed: 11/13/2022] Open
Abstract
Medical image segmentation is of important support for clinical medical applications. As most of the current medical image segmentation models are limited in the U-shaped structure, to some extent the deep convolutional neural network (CNN) structure design is hard to be accomplished. The design in this study mimics the way the wave is elastomeric propagating, extending the structure from both the horizontal and spatial dimensions for realizing the Elastomeric UNet (EUNet) structure. The EUNet can be divided into two types: horizontal EUNet and spatial EUNet, based on the propagation direction. The advantages of this design are threefold. First, the training structure can be deepened effectively. Second, the independence brought by each branch (a U-shaped design) makes the flexible design redundancy available. Finally, a horizontal and vertical series-parallel structure helps on feature accumulation and recursion. Researchers can adjust the design according to the requirements to achieve better segmentation performance for the independent structural design. The proposed networks were evaluated on two datasets: a self-built dataset (multi-photon microscopy, MPM) and publicly benchmark retinal datasets (DRIVE). The results of experiments demonstrated that the performance of EUNet outperformed the UNet and its variants.
Collapse
Affiliation(s)
- Sijing Cai
- Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, China
- School of Electronic & Electrical Engineering and Physics, Fujian University of Technology, Fuzhou, China
| | - Yi Wu
- Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, China
- Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou, China
| | - Guannan Chen
- Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, China
- Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou, China
| |
Collapse
|
2
|
Prinke P, Haueisen J, Klee S, Rizqie MQ, Supriyanto E, König K, Breunig HG, Piątek Ł. Automatic segmentation of skin cells in multiphoton data using multi-stage merging. Sci Rep 2021; 11:14534. [PMID: 34267247 PMCID: PMC8282875 DOI: 10.1038/s41598-021-93682-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2021] [Accepted: 06/27/2021] [Indexed: 01/10/2023] Open
Abstract
We propose a novel automatic segmentation algorithm that separates the components of human skin cells from the rest of the tissue in fluorescence data of three-dimensional scans using non-invasive multiphoton tomography. The algorithm encompasses a multi-stage merging on preprocessed superpixel images to ensure independence from a single empirical global threshold. This leads to a high robustness of the segmentation considering the depth-dependent data characteristics, which include variable contrasts and cell sizes. The subsequent classification of cell cytoplasm and nuclei are based on a cell model described by a set of four features. Two novel features, a relationship between outer cell and inner nucleus (OCIN) and a stability index, were derived. The OCIN feature describes the topology of the model, while the stability index indicates segment quality in the multi-stage merging process. These two new features, combined with the local gradient magnitude and compactness, are used for the model-based fuzzy evaluation of the cell segments. We exemplify our approach on an image stack with 200 × 200 × 100 μm3, including the skin layers of the stratum spinosum and the stratum basale of a healthy volunteer. Our image processing pipeline contributes to the fully automated classification of human skin cells in multiphoton data and provides a basis for the detection of skin cancer using non-invasive optical biopsy.
Collapse
Affiliation(s)
- Philipp Prinke
- Institute for Biomedical Engineering and Informatics, Technische Universität Ilmenau, 98693, Ilmenau, Germany.
| | - Jens Haueisen
- Institute for Biomedical Engineering and Informatics, Technische Universität Ilmenau, 98693, Ilmenau, Germany
| | - Sascha Klee
- Institute for Biomedical Engineering and Informatics, Technische Universität Ilmenau, 98693, Ilmenau, Germany.,Division Biostatistics and Data Science, Department of General Health Studies, Karl Landsteiner University of Health Sciences, Dr. Karl-Dorrek-Straße 30, 3500, Krems, Austria
| | - Muhammad Qurhanul Rizqie
- Institute for Biomedical Engineering and Informatics, Technische Universität Ilmenau, 98693, Ilmenau, Germany.,Informatics Engineering Program, Universitas Sriwijaya, Palembang, South Sumatera, Indonesia
| | - Eko Supriyanto
- Institute for Biomedical Engineering and Informatics, Technische Universität Ilmenau, 98693, Ilmenau, Germany.,IJN-UTM Cardiovascular Engineering Centre, Universiti Teknologi Malaysia, Johor Bahru, Johor, Malaysia
| | - Karsten König
- Department of Biophotonics and Laser Technology, Saarland University, Campus A5.1, 66123, Saarbrücken, Germany.,JenLab GmbH, Johann-Hittorf-Straße 8, 12489, Berlin, Germany
| | - Hans Georg Breunig
- Department of Biophotonics and Laser Technology, Saarland University, Campus A5.1, 66123, Saarbrücken, Germany.,JenLab GmbH, Johann-Hittorf-Straße 8, 12489, Berlin, Germany
| | - Łukasz Piątek
- Institute for Biomedical Engineering and Informatics, Technische Universität Ilmenau, 98693, Ilmenau, Germany.,Department of Artificial Intelligence, University of Information Technology and Management, H. Sucharskiego 2 Str, 35-225, Rzeszów, Poland
| |
Collapse
|
3
|
Cai S, Tian Y, Lui H, Zeng H, Wu Y, Chen G. Dense-UNet: a novel multiphoton in vivo cellular image segmentation model based on a convolutional neural network. Quant Imaging Med Surg 2020; 10:1275-1285. [PMID: 32550136 DOI: 10.21037/qims-19-1090] [Citation(s) in RCA: 47] [Impact Index Per Article: 11.8] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
Background Multiphoton microscopy (MPM) offers a feasible approach for the biopsy in clinical medicine, but it has not been used in clinical applications due to the lack of efficient image processing methods, especially the automatic segmentation technology. Segmentation technology is still one of the most challenging assignments of the MPM imaging technique. Methods The MPM imaging segmentation model based on deep learning is one of the most effective methods to address this problem. In this paper, the practicability of using a convolutional neural network (CNN) model to segment the MPM image of skin cells in vivo was explored. A set of MPM in vivo skin cells images with a resolution of 128×128 was successfully segmented under the Python environment with TensorFlow. A novel deep-learning segmentation model named Dense-UNet was proposed. The Dense-UNet, which is based on U-net structure, employed the dense concatenation to deepen the depth of the network architecture and achieve feature reuse. This model included four expansion modules (each module consisted of four down-sampling layers) to extract features. Results Sixty training images were taken from the dorsal forearm using a femtosecond Ti:Sa laser running at 735 nm. The resolution of the images is 128×128 pixels. Experimental results confirmed that the accuracy of Dense-UNet (92.54%) was higher than that of U-Net (88.59%), with a significantly lower loss value of 0.1681. The 90.60% Dice coefficient value of Dense-UNet outperformed U-Net by 11.07%. The F1-Score of Dense-UNet, U-Net, and Seg-Net was 93.35%, 90.02%, and 85.04%, respectively. Conclusions The deepened down-sampling path improved the ability of the model to capture cellular fined-detailed boundary features, while the symmetrical up-sampling path provided a more accurate location based on the test result. These results were the first time that the segmentation of MPM in vivo images had been adopted by introducing a deep CNN to bridge this gap in Dense-UNet technology. Dense-UNet has reached ultramodern performance for MPM images, especially for in vivo images with low resolution. This implementation supplies an automatic segmentation model based on deep learning for high-precision segmentation of MPM images in vivo.
Collapse
Affiliation(s)
- Sijing Cai
- Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, China.,School of Information Science and Engineering, Fujian University of Technology, Fuzhou, China
| | - Yunxian Tian
- Imaging Unit, Integrative Oncology Department, BC Cancer Research Centre, Vancouver, BC, Canada.,Photomedicine Institute, Department of Dermatology and Skin Science, University of British Columbia and Vancouver Coastal Health Research Institute, Vancouver, BC, Canada
| | - Harvey Lui
- Imaging Unit, Integrative Oncology Department, BC Cancer Research Centre, Vancouver, BC, Canada.,Photomedicine Institute, Department of Dermatology and Skin Science, University of British Columbia and Vancouver Coastal Health Research Institute, Vancouver, BC, Canada
| | - Haishan Zeng
- Imaging Unit, Integrative Oncology Department, BC Cancer Research Centre, Vancouver, BC, Canada.,Photomedicine Institute, Department of Dermatology and Skin Science, University of British Columbia and Vancouver Coastal Health Research Institute, Vancouver, BC, Canada
| | - Yi Wu
- Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, China.,Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou, China
| | - Guannan Chen
- Key Laboratory of Optoelectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, China.,Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou, China
| |
Collapse
|