Comparison of Fine-Tuned Deep Convolutional Neural Networks for the Automated Classification of Lung Cancer Cytology Images with Integration of Additional Classifiers

OBJECTIVE

It is essential to accurately diagnose and classify histological subtypes into adenocarcinoma (ADC), squamous cell carcinoma (SCC), and small cell lung carcinoma (SCLC) for the appropriate treatment of lung cancer patients. However, improving the accuracy and stability of diagnosis is challenging, especially for non-small cell carcinomas. The purpose of this study was to compare multiple deep convolutional neural network (DCNN) technique with subsequent additional classifiers in terms of accuracy and characteristics in each histology.

METHODS

Lung cancer cytological images were classified into ADC, SCC, and SCLC with four fine-tuned DCNN models consisting of AlexNet, GoogLeNet (Inception V3), VGG16 and ResNet50 pretrained by natural images in ImageNet database. For more precise classification, the figures of 3 histological probabilities were further applied to subsequent machine learning classifiers using Naïve Bayes (NB), Support vector machine (SVM), Random forest (RF), and Neural network (NN).

RESULTS

The classification accuracies of the AlexNet, GoogLeNet, VGG16 and ResNet50 were 74.0%, 66.8%, 76.8% and 74.0%, respectively. Well differentiated typical morphologies were tended to be correctly judged by all four architectures. However, poorly differentiated non-small cell carcinomas lacking typical structures were inclined to be misrecognized in some DCNNs. Regarding the histological types, ADC were best judged by AlexNet and SCC by VGG16. Subsequent machine learning classifiers of NB, SVV, RF, and NN improved overall accuracies to 75.1%, 77.5%, 78.2%, and 78.9%, respectively.

CONCLUSION

Fine-tuning DCNNs in combination with additional classifiers improved classification of cytological diagnosis of lung cancer, although classification bias could be indicated among DCNN architectures.

γH2AX, a DNA Double-Strand Break Marker, Correlates with PD-L1 Expression in Smoking-Related Lung Adenocarcinoma

In recent years, the choice of immune checkpoint inhibitors (ICIs) as a treatment based on high expression of programmed death-ligand 1 (PD-L1) in lung cancers has been increasing in prevalence. The high expression of PD-L1 could be a predictor of ICI efficacy as well as high tumor mutation burden (TMB), which is determined using next-generation sequencing (NGS). However, a great deal of effort is required to perform NGS to determine TMB. The present study focused on γH2AX, a double-strand DNA break marker, and the suspected positive relation between TMB and γH2AX was investigated. We assessed the possibility of γH2AX being an alternative marker of TMB or PD-L1. One hundred formalin-fixed, paraffin-embedded specimens of lung cancer were examined. All of the patients in the study received thoracic surgery, having been diagnosed with lung adenocarcinoma or squamous cell carcinoma. The expressions of γH2AX and PD-L1 (clone: SP142) were evaluated immunohistochemically. Other immunohistochemical indicators, p53 and Ki-67, were also used to estimate the relationships of γH2AX. Positive relationships between γH2AX and PD-L1 were proven, especially in lung adenocarcinoma. Tobacco consumption was associated with higher expression of γH2AX, PD-L1, Ki-67, and p53. In conclusion, the immunoexpression of γH2AX could be a predictor for the adaptation of ICIs as well of as PD-L1 and TMB.

Mutual stain conversion between Giemsa and Papanicolaou in cytological images using cycle generative adversarial network

Objective

Papanicolaou and Giemsa stains used in cytology have different characteristics and complementary roles. In this study, we focused on cycle-consistent generative adversarial network (CycleGAN), which is an image translation technique using deep learning, and we conducted mutual stain conversion between Giemsa and Papanicolaou in cytological images using CycleGAN.

Methods

A total of 191 Giemsa-stained images and 209 Papanicolaou-stained images were collected from 63 patients with lung cancer. From those images, 67 images from nine cases were used for testing and the remaining images were used for training. For data augmentation, the number of training images was increased by rotation and inversion, and the images were pipelined to CycleGAN to train the mutual conversion process involving Giemsa- and Papanicolaou-stained images. Three pathologists and three cytotechnologists performed visual evaluations of the authenticity of cell nuclei, cytoplasm, and cell layouts of the test images translated using CycleGAN.

Results

As a result of converting Giemsa-stained images into Papanicolaou-stained images, the background red blood cell patterns present in Giemsa-stained images disappeared, and cell patterns that reproduced the shape and staining of the cell nuclei and cytoplasm peculiar to Papanicolaou staining were obtained. Regarding the reverse-translated results, nuclei became larger, and red blood cells that were not evident in Papanicolaou-stained images appeared. After visual evaluation, although actual images exhibited better results than converted images, the results were promising for various applications.

Discussion

The stain translation technique investigated in this paper can complement specimens under conditions where only single stained specimens are available; it also has potential applications in the massive training of artificial intelligence systems for cell classification, and can also be used for training cytotechnologist and pathologists.

Early detection of genotoxic hepatocarcinogens in rats using γH2AX and Ki-67: prediction by machine learning

Direct DNA double-strand breaks result in phosphorylation of H2AX, a variant of the histone H2 protein. Phosphorylated H2AX (γH2AX) may be a potential indicator in the evaluation of genotoxicity and hepatocarcinogenicity. In this study, γH2AX and Ki-67 were detected in the short-term responses (24 h after chemical administration) to classify genotoxic hepatocarcinogens (GHs) from non-GH chemicals. One hundred and thirty-five 6-week-old Crl: CD(SD) (SPF) male rats were treated with 22 chemicals including 11 GH and 11 non-GH, sacrificed 24 h later, and immunostained with γH2AX and Ki-67. Positivity rates of these markers were measured in the 3 liver ZONEs 1-3; portal, lobular, and central venous regions. These values were input into 3 machine learning models-Naïve Bayes, Random Forest, and k-Nearest Neighbor to classify GH and non-GH using a 10-fold cross-validation method. All 11 and 10 out of 11 GH caused significant increase in γH2AX and Ki-67 levels, respectively (P < .05). Of the 3 machine learning models, Random Forest performed the best. GH were identified with 95.0% sensitivity (76/80 GH-treated rats), 90.9% specificity (50/55 non-GH-treated rats), and 90.0% overall correct response rate using γH2AX staining, and 96.2% sensitivity (77/80), 81.8% specificity (45/55), and 90.4% overall correct response rate using Ki-67 labeling. Random Forest model using γH2AX and Ki-67 could independently predict GH in the early stage with high accuracy.

A case of chronic gastric anisakiasis coexisting with early gastric cancer

Background

Anisakiasis is a parasitic disease caused by the consumption of raw or undercooked fish that is infected with Anisakis third-stage larvae. In countries, such as Japan, Italy, and Spain, where people have a custom of eating raw or marinated fish, anisakiasis is a common infection. Although anisakiasis has been reported in the gastrointestinal tract in several countries, reports of anisakiasis accompanied by cancer are rare.

Case presentation

We present the rare case of a 40-year-old male patient with anisakiasis coexisting with mucosal gastric cancer. Submucosal gastric cancer was suspected on gastric endoscopy and endoscopic ultrasonography. After laparoscopic distal gastrectomy, granulomatous inflammation with Anisakis larvae in the submucosa was pathologically revealed beneath mucosal tubular adenocarcinoma. Histological and immunohistochemical investigation showed cancer cells as intestinal absorptive-type cells that did not produce mucin.

Conclusion

Anisakis larvae could have invaded the cancer cells selectively because of the lack of mucin in the cancerous epithelium. Anisakiasis coexisting with cancer is considered reasonable rather than coincidental. In cancer with anisakiasis, preoperative diagnosis may be difficult because anisakiasis leads to morphological changes in the cancer.

Morphologic Analysis of M2 Macrophage in Glioblastoma: Involvement of Macrophage Extracellular Traps (METs)

Macrophages are classified into two phenotypes, M1 and M2, based on their roles. M2 macrophages suppress inflammation and increase in proportion to the malignancy of brain tumors. Recently, macrophage extracellular traps (METs), which change into a network, have been reported as a unique form of macrophage cell death. In this study, immunohistochemical analysis of macrophages in METs in human glioblastoma was performed. To distinguish between M1 and M2 macrophages, multiple immunostainings with Iba1 combined with CD163 or CD204 were performed. M2 macrophages were present in small amounts in normal and borderline areas but showed an increasing trend as they shifted to tumor areas, and most of them were the activated- or phagocytic-type. We also successfully detected METs coexisting with fibrin and lactoferrin near the border between the tumor and necrotic area. M2 macrophages not only suppressed inflammation but also were involved in the formation of METs. This study found that M2 macrophages play various roles in unstable situations.

Automated Classification of Idiopathic Pulmonary Fibrosis in Pathological Images Using Convolutional Neural Network and Generative Adversarial Networks

Interstitial pneumonia of uncertain cause is referred to as idiopathic interstitial pneumonia (IIP). Among the various types of IIPs, the prognosis of cases of idiopathic pulmonary fibrosis (IPF) is extremely poor, and accurate differentiation between IPF and non-IPF pneumonia is critical. In this study, we consider deep learning (DL) methods owing to their excellent image classification capabilities. Although DL models require large quantities of training data, collecting a large number of pathological specimens is difficult for rare diseases. In this study, we propose an end-to-end scheme to automatically classify IIPs using a convolutional neural network (CNN) model. To compensate for the lack of data on rare diseases, we introduce a two-step training method to generate pathological images of IIPs using a generative adversarial network (GAN). Tissue specimens from 24 patients with IIPs were scanned using a whole slide scanner, and the resulting images were divided into patch images with a size of 224 × 224 pixels. A progressive growth GAN (PGGAN) model was trained using 23,142 IPF images and 7817 non-IPF images to generate 10,000 images for each of the two categories. The images generated by the PGGAN were used along with real images to train the CNN model. An evaluation of the images generated by the PGGAN showed that cells and their locations were well-expressed. We also obtained the best classification performance with a detection sensitivity of 97.2% and a specificity of 69.4% for IPF using DenseNet. The classification performance was also improved by using PGGAN-generated images. These results indicate that the proposed method may be considered effective for the diagnosis of IPF.

Automated Description Generation of Cytologic Findings for Lung Cytological Images Using a Pretrained Vision Model and Dual Text Decoders: Preliminary Study

OBJECTIVE

Cytology plays a crucial role in lung cancer diagnosis. Pulmonary cytology involves cell morphological characterisation in the specimen and reporting the corresponding findings, which are extremely burdensome tasks. In this study, we propose a technique to generate cytologic findings from for cytologic images to assist in the reporting of pulmonary cytology.

METHODS

For this study, 801 patch images were retrieved using cytology specimens collected from 206 patients; the findings were assigned to each image as a dataset for generating cytologic findings. The proposed method consists of a vision model and dual text decoders. In the former, a convolutional neural network (CNN) is used to classify a given image as benign or malignant, and the features related to the image are extracted from the intermediate layer. Independent text decoders for benign and malignant cells are prepared for text generation, and the text decoder switches according to the CNN classification results. The text decoder is configured using a transformer that uses the features obtained from the CNN for generating findings.

RESULTS

The sensitivity and specificity were 100% and 96.4%, respectively, for automated benign and malignant case classification, and the saliency map indicated characteristic benign and malignant areas. The grammar and style of the generated texts were confirmed correct, achieving a BLEU-4 score of 0.828, reflecting high degree of agreement with the gold standard, outperforming existing LLM-based image-captioning methods and single-text-decoder ablation model.

CONCLUSION

Experimental results indicate that the proposed method is useful for pulmonary cytology classification and generation of cytologic findings.