Multispectral Imaging for Automated Tissue Identification of Normal Human Surgical Specimens

Signal to noise ratio quantifies the contribution of spectral channels to classification of human head and neck tissues ex vivo using deep learning and multispectral imaging

SIGNIFICANCE

Accurate identification of tissues is critical for performing safe surgery. Combining multispectral imaging (MSI) with deep learning is a promising approach to increasing tissue discrimination and classification. Evaluating the contributions of spectral channels to tissue discrimination is important for improving MSI systems.

AIM

Develop a metric to quantify the contributions of individual spectral channels to tissue classification in MSI.

APPROACH

MSI was integrated into a digital operating microscope with three sensors and seven illuminants. Two convolutional neural network (CNN) models were trained to classify 11 head and neck tissue types using white light (RGB) or MSI images. The signal to noise ratio (SNR) of spectral channels was compared with the impact of channels on tissue classification performance as determined using CNN visualization methods.

RESULTS

Overall tissue classification accuracy was higher with use of MSI images compared with RGB images, both for classification of all 11 tissue types and binary classification of nerve and parotid ( p < 0.001 ). Removing spectral channels with SNR > 20 reduced tissue classification accuracy.

CONCLUSIONS

The spectral channel SNR is a useful metric for both understanding CNN tissue classification and quantifying the contributions of different spectral channels in an MSI system.

Texture synthesis of ecological plant protection image based on convolution neural network

Texture synthesis technology is an important realistic rendering technology. Texture synthesis technology also has a good application prospect in image rendering and other fields. Convolutional neural network is a very popular technology in recent years. Convolutional neural network model can learn the features in data and realize intelligent processing through the feature learning in data. Later, with the rapid improvement of convolutional neural network, texture synthesis technology based on neural network came into being. The purpose of this paper is to study the texture synthesis method of ecological plant protection image based on convolutional neural network. By studying the context and research implications, the definition of textures as well as texture synthesis methods, convolutional neural networks, and based on convolutional neural network. In the experiment, the experimental environment is established, and the subjective evaluation and objective evaluation of the image texture synthesis method experiment are investigated and studied by using swap algorithm. The experimental results show that the method used in this paper is superior to other methods.

Multispectral Image under Tissue Classification Algorithm in Screening of Cervical Cancer

The objectives of this study were to improve the efficiency and accuracy of early clinical diagnosis of cervical cancer and to explore the application of tissue classification algorithm combined with multispectral imaging in screening of cervical cancer. 50 patients with suspected cervical cancer were selected. Firstly, the multispectral imaging technology was used to collect the multispectral images of the cervical tissues of 50 patients under the conventional white light waveband, the narrowband green light waveband, and the narrowband blue light waveband. Secondly, the collected multispectral images were fused, and then the tissue classification algorithm was used to segment the diseased area according to the difference between the cervical tissues without lesions and the cervical tissues with lesions. The difference in the contrast and other characteristics of the multiband spectrum fusion image would segment the diseased area, which was compared with the results of the disease examination. The average gradient, standard deviation (SD), and image entropy were adopted to evaluate the image quality, and the sensitivity and specificity were selected to evaluate the clinical application value of discussed method. The fused spectral image was compared with the image without lesions, it was found that there was a clear difference, and the fused multispectral image showed a contrast of 0.7549, which was also higher than that before fusion (0.4716), showing statistical difference (P < 0.05). The average gradient, SD, and image entropy of the multispectral image assisted by the tissue classification algorithm were 2.0765, 65.2579, and 4.974, respectively, showing statistical difference (P < 0.05). Compared with the three reported indicators, the values of the algorithm in this study were higher. The sensitivity and specificity of the multispectral image with the tissue classification algorithm were 85.3% and 70.8%, respectively, which were both greater than those of the image without the algorithm. It showed that the multispectral image assisted by tissue classification algorithm can effectively screen the cervical cancer and can quickly, efficiently, and safely segment the cervical tissue from the lesion area and the nonlesion area. The segmentation result was the same as that of the doctor's disease examination, indicating that it showed high clinical application value. This provided an effective reference for the clinical application of multispectral imaging technology assisted by tissue classification algorithm in the early screening and diagnosis of cervical cancer.

Deep learning classification of inverted papilloma malignant transformation using 3D convolutional neural networks and magnetic resonance imaging

BACKGROUND

Distinguishing benign inverted papilloma (IP) tumors from those that have undergone malignant transformation to squamous cell carcinoma (IP-SCC) is important but challenging to do preoperatively. Magnetic resonance imaging (MRI) can help differentiate these 2 entities, but no established method exists that can automatically synthesize all potentially relevant MRI image features to distinguish IP and IP-SCC. We explored a deep learning approach, using 3-dimensional convolutional neural networks (CNNs), to address this challenge.

METHODS

Retrospective chart reviews were performed at 2 institutions to create a data set of preoperative MRIs with corresponding surgical pathology reports. The MRI data set included all available MRI sequences in the axial plane, which were used to train, validate, and test 3 CNN models. Saliency maps were generated to visualize areas of MRIs with greatest influence on predictions.

RESULTS

A total of 90 patients with IP (n = 64) or IP-SCC (n = 26) tumors were identified, with a total of 446 images of distinct MRI sequences for IP (n = 329) or IP-SCC (n = 117). The best CNN model, All-Net, demonstrated a sensitivity of 66.7%, specificity of 81.5%, overall accuracy of 77.9%, and receiver-operating characteristic area under the curve of 0.80 (95% confidence interval, 0.682-0.898) for test classification performance. The other 2 models, Small-All-Net and Elastic-All-Net, showed similar performance levels.

CONCLUSION

A deep learning approach with 3-dimensional CNNs can distinguish IP and IP-SCC with moderate test classification performance. Although CNNs demonstrate promise to enhance the prediction of IP-SCC using MRIs, more data are needed before they can reach the predictive value already established by human MRI evaluation.