A novel multiphoton microscopy images segmentation method based on superpixel and watershed

The Algorithm of Watershed Color Image Segmentation Based on Morphological Gradient

The traditional watershed algorithm has the disadvantage of over-segmentation and interference with an image by reflected light. We propose an improved watershed color image segmentation algorithm. It is based on a morphological gradient. This method obtains the component gradient of a color image in a new color space is not disturbed by the reflected light. The gradient image is reconstructed by opening and closing. Therefore, the final gradient image is obtained. The maximum inter-class variance algorithm is used to obtain the threshold automatically for the final gradient image. The original gradient image is forcibly calibrated with the obtained binary labeled image, and the modified gradient image is segmented by watershed. Experimental results show that the proposed method can obtain an accurate and continuous target contour. It will achieve the minimum number of segmentation regions following human vision. Compared with similar algorithms, this way can suppress the meaningless area generated by the reflected light. It will maintain the edge information of the object well. It will improve the robustness and applicability. From the experimental results, it can be seen that compared with the region-growing method and the automatic threshold method; the proposed algorithm has a great improvement in operation efficiency, which increased by 10%. The accuracy and recall rate of the proposed algorithm is more than 0.98. Through the experimental comparison, the advantages of the proposed algorithm in object segmentation can be more intuitively illustrated.

Automatic segmentation of skin cells in multiphoton data using multi-stage merging

We propose a novel automatic segmentation algorithm that separates the components of human skin cells from the rest of the tissue in fluorescence data of three-dimensional scans using non-invasive multiphoton tomography. The algorithm encompasses a multi-stage merging on preprocessed superpixel images to ensure independence from a single empirical global threshold. This leads to a high robustness of the segmentation considering the depth-dependent data characteristics, which include variable contrasts and cell sizes. The subsequent classification of cell cytoplasm and nuclei are based on a cell model described by a set of four features. Two novel features, a relationship between outer cell and inner nucleus (OCIN) and a stability index, were derived. The OCIN feature describes the topology of the model, while the stability index indicates segment quality in the multi-stage merging process. These two new features, combined with the local gradient magnitude and compactness, are used for the model-based fuzzy evaluation of the cell segments. We exemplify our approach on an image stack with 200 × 200 × 100  μm³, including the skin layers of the stratum spinosum and the stratum basale of a healthy volunteer. Our image processing pipeline contributes to the fully automated classification of human skin cells in multiphoton data and provides a basis for the detection of skin cancer using non-invasive optical biopsy.

Dense-UNet: a novel multiphoton in vivo cellular image segmentation model based on a convolutional neural network

Background

Multiphoton microscopy (MPM) offers a feasible approach for the biopsy in clinical medicine, but it has not been used in clinical applications due to the lack of efficient image processing methods, especially the automatic segmentation technology. Segmentation technology is still one of the most challenging assignments of the MPM imaging technique.

Methods

The MPM imaging segmentation model based on deep learning is one of the most effective methods to address this problem. In this paper, the practicability of using a convolutional neural network (CNN) model to segment the MPM image of skin cells in vivo was explored. A set of MPM in vivo skin cells images with a resolution of 128×128 was successfully segmented under the Python environment with TensorFlow. A novel deep-learning segmentation model named Dense-UNet was proposed. The Dense-UNet, which is based on U-net structure, employed the dense concatenation to deepen the depth of the network architecture and achieve feature reuse. This model included four expansion modules (each module consisted of four down-sampling layers) to extract features.

Results

Sixty training images were taken from the dorsal forearm using a femtosecond Ti:Sa laser running at 735 nm. The resolution of the images is 128×128 pixels. Experimental results confirmed that the accuracy of Dense-UNet (92.54%) was higher than that of U-Net (88.59%), with a significantly lower loss value of 0.1681. The 90.60% Dice coefficient value of Dense-UNet outperformed U-Net by 11.07%. The F1-Score of Dense-UNet, U-Net, and Seg-Net was 93.35%, 90.02%, and 85.04%, respectively.

Conclusions

The deepened down-sampling path improved the ability of the model to capture cellular fined-detailed boundary features, while the symmetrical up-sampling path provided a more accurate location based on the test result. These results were the first time that the segmentation of MPM in vivo images had been adopted by introducing a deep CNN to bridge this gap in Dense-UNet technology. Dense-UNet has reached ultramodern performance for MPM images, especially for in vivo images with low resolution. This implementation supplies an automatic segmentation model based on deep learning for high-precision segmentation of MPM images in vivo.

Evaluation of breast carcinoma regression after preoperative chemotherapy by label-free multiphoton imaging and image analysis

Neoadjuvant chemotherapy is increasingly being used in breast carcinoma as it significantly improves the prognosis and consistently leads to an increased rate of breast preservation. How to accurately assess tumor response after treatment is a crucial factor for developing reasonable therapeutic strategy. In this study, we were in an attempt to monitor tumor response by multimodal multiphoton imaging including two-photon excitation fluorescence and second-harmonic generation imaging. We found that multiphoton imaging can identify different degrees of tumor response such as a slight, significant, or complete response and can detect morphological alteration associated with extracellular matrix during the progression of breast carcinoma following preoperative chemotherapy. Two quantitative optical biomarkers including tumor cellularity and collagen content were extracted based on automatic image analysis to help monitor changes in tumor and its microenvironment. Furthermore, tumor regression grade diagnosis was tried to evaluate by multiphoton microscopy. These results may offer a basic framework for using multiphoton microscopic imaging techniques as a helpful diagnostic tool for assessing breast carcinoma response after presurgical treatment.

Multiphoton microscopy in surgical oncology- a systematic review and guide for clinical translatability

BACKGROUND

Multiphoton microscopy (MPM) facilitates three-dimensional, high-resolution functional imaging of unlabeled tissues in vivo and ex vivo. This systematic review discusses the diagnostic value, advantages and challenges in the practical use of MPM in surgical oncology.

METHOD AND FINDINGS

A Medline search was conducted in April 2019. Fifty-three original research papers investigating MPM compared to standard histology in human patients with solid tumors were identified. A qualitative synopsis and meta-analysis of 14 blinded studies was performed. Risk of bias and applicability were evaluated. MPM can image fresh, frozen or fixed tissues up to a depth 1000 μm in the z-plane. Best results including functional imaging and virtual histochemistry are obtained by in vivo imaging or scanning fresh tissue immediately after excision. Two-photon excited fluorescence by natural fluorophores of the cytoplasm and second harmonic generation signals by fluorophores of the extracellular matrix can be scanned simultaneously, providing high resolution optical histochemistry comparable to standard histology. Functional parameters like fluorescence lifetime imaging or optical redox ratio provide additional objective information. A major concern is inability to visualize the nucleus. However, in a subpopulation analysis of 440 specimens, MPM yielded a sensitivity of 94%, specificity of 96% and accuracy of 95% for the detection of malignant tissue.

CONCLUSION

MPM is a promising emerging technique in surgical oncology. Ex vivo imaging has high sensitivity, specificity and accuracy for the detection of tumor cells. For broad clinical application in vivo, technical challenges need to be resolved.