MuellerNet: a hybrid 3D-2D CNN for cell classification with Mueller matrix images

Dual Polarization Modality Fusion Network for Assisting Pathological Diagnosis

Polarization imaging is sensitive to sub-wavelength microstructures of various cancer tissues, providing abundant optical characteristics and microstructure information of complex pathological specimens. However, how to reasonably utilize polarization information to strengthen pathological diagnosis ability remains a challenging issue. In order to take full advantage of pathological image information and polarization features of samples, we propose a dual polarization modality fusion network (DPMFNet), which consists of a multi-stream CNN structure and a switched attention fusion module for complementarily aggregating the features from different modality images. Our proposed switched attention mechanism could obtain the joint feature embeddings by switching the attention map of different modality images to improve their semantic relatedness. By including a dual-polarization contrastive training scheme, our method can synthesize and align the interaction and representation of two polarization features. Experimental evaluations on three cancer datasets show the superiority of our method in assisting pathological diagnosis, especially in small datasets and low imaging resolution cases. Grad-CAM visualizes the important regions of the pathological images and the polarization images, indicating that the two modalities play different roles and allow us to give insightful corresponding explanations and analysis on cancer diagnosis conducted by the DPMFNet. This technique has potential to facilitate the performance of pathological aided diagnosis and broaden the current digital pathology boundary based on pathological image features.

Combined Mueller matrix imaging and artificial intelligence classification framework for Hepatitis B detection

SIGNIFICANCE

The combination of polarized imaging with artificial intelligence (AI) technology has provided a powerful tool for performing an objective and precise diagnosis in medicine.

AIM

An approach is proposed for the detection of hepatitis B (HB) virus using a combined Mueller matrix imaging technique and deep learning method.

APPROACH

In the proposed approach, Mueller matrix imaging polarimetry is applied to obtain 4 × 4 Mueller matrix images of 138 HBsAg-containing (positive) serum samples and 136 HBsAg-free (negative) serum samples. The kernel estimation density results show that, of the 16 Mueller matrix elements, elements M 22 and M 33 provide the best discriminatory power between the positive and negative samples.

RESULTS

As a result, M 22 and M 33 are taken as the inputs to five different deep learning models: Xception, VGG16, VGG19, ResNet 50, and ResNet150. It is shown that the optimal classification accuracy (94.5%) is obtained using the VGG19 model with element M 22 as the input.

CONCLUSIONS

Overall, the results confirm that the proposed hybrid Mueller matrix imaging and AI framework provides a simple and effective approach for HB virus detection.

Polarization-based probabilistic discriminative model for quantitative characterization of cancer cells

We propose a polarization-based probabilistic discriminative model for deriving a set of new sigmoid-transformed polarimetry feature parameters, which not only enables accurate and quantitative characterization of cancer cells at pixel level, but also accomplish the task with a simple and stable model. By taking advantages of polarization imaging techniques, these parameters enable a low-magnification and wide-field imaging system to separate the types of cells into more specific categories that previously were distinctive under high magnification. Instead of blindly choosing the model, the L0 regularization method is used to obtain the simplified and stable polarimetry feature parameter. We demonstrate the model viability by using the pathological tissues of breast cancer and liver cancer, in each of which there are two derived parameters that can characterize the cells and cancer cells respectively with satisfactory accuracy and sensitivity. The stability of the final model opens the possibility for physical interpretation and analysis. This technique may bypass the typically labor-intensive and subjective tumor evaluating system, and could be used as a blueprint for an objective and automated procedure for cancer cell screening.