PDBL: Improving Histopathological Tissue Classification With Plug-and-Play Pyramidal Deep-Broad Learning

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Learning From Incorrectness: Active Learning With Negative Pre-Training and Curriculum Querying for Histological Tissue Classification

Patch-level histological tissue classification is an effective pre-processing method for histological slide analysis. However, the classification of tissue with deep learning requires expensive annotation costs. To alleviate the limitations of annotation budgets, the application of active learning (AL) to histological tissue classification is a promising solution. Nevertheless, there is a large imbalance in performance between categories during application, and the tissue corresponding to the categories with relatively insufficient performance are equally important for cancer diagnosis. In this paper, we propose an active learning framework called ICAL, which contains Incorrectness Negative Pre-training (INP) and Category-wise Curriculum Querying (CCQ) to address the above problem from the perspective of category-to-category and from the perspective of categories themselves, respectively. In particular, INP incorporates the unique mechanism of active learning to treat the incorrect prediction results that obtained from CCQ as complementary labels for negative pre-training, in order to better distinguish similar categories during the training process. CCQ adjusts the query weights based on the learning status on each category by the model trained by INP, and utilizes uncertainty to evaluate and compensate for query bias caused by inadequate category performance. Experimental results on two histological tissue classification datasets demonstrate that ICAL achieves performance approaching that of fully supervised learning with less than 16% of the labeled data. In comparison to the state-of-the-art active learning algorithms, ICAL achieved better and more balanced performance in all categories and maintained robustness with extremely low annotation budgets. The source code will be released at https://github.com/LactorHwt/ICAL.

TGMIL: A hybrid multi-instance learning model based on the Transformer and the Graph Attention Network for whole-slide images classification of renal cell carcinoma

BACKGROUND AND OBJECTIVES

The pathological diagnosis of renal cell carcinoma is crucial for treatment. Currently, the multi-instance learning method is commonly used for whole-slide image classification of renal cell carcinoma, which is mainly based on the assumption of independent identical distribution. But this is inconsistent with the need to consider the correlation between different instances in the diagnosis process. Furthermore, the problem of high resource consumption of pathology images is still urgent to be solved. Therefore, we propose a new multi-instance learning method to solve this problem.

METHODS

In this study, we proposed a hybrid multi-instance learning model based on the Transformer and the Graph Attention Network, called TGMIL, to achieve whole-slide image of renal cell carcinoma classification without pixel-level annotation or region of interest extraction. Our approach is divided into three steps. First, we designed a feature pyramid with the multiple low magnifications of whole-slide image named MMFP. It makes the model incorporates richer information, and reduces memory consumption as well as training time compared to the highest magnification. Second, TGMIL amalgamates the Transformer and the Graph Attention's capabilities, adeptly addressing the loss of instance contextual and spatial. Within the Graph Attention network stream, an easy and efficient approach employing max pooling and mean pooling yields the graph adjacency matrix, devoid of extra memory consumption. Finally, the outputs of two streams of TGMIL are aggregated to achieve the classification of renal cell carcinoma.

RESULTS

On the TCGA-RCC validation set, a public dataset for renal cell carcinoma, the area under a receiver operating characteristic (ROC) curve (AUC) and accuracy of TGMIL were 0.98±0.0015,0.9191±0.0062, respectively. It showcased remarkable proficiency on the private validation set of renal cell carcinoma pathology images, attaining AUC of 0.9386±0.0162 and ACC of 0.9197±0.0124. Furthermore, on the public breast cancer whole-slide image test dataset, CAMELYON 16, our model showed good classification performance with an accuracy of 0.8792.

CONCLUSIONS

TGMIL models the diagnostic process of pathologists and shows good classification performance on multiple datasets. Concurrently, the MMFP module efficiently diminishes resource requirements, offering a novel angle for exploring computational pathology images.

CaMeL-Net: Centroid-aware metric learning for efficient multi-class cancer classification in pathology images

BACKGROUND AND OBJECTIVE

Cancer grading in pathology image analysis is a major task due to its importance in patient care, treatment, and management. The recent developments in artificial neural networks for computational pathology have demonstrated great potential to improve the accuracy and quality of cancer diagnosis. These improvements are generally ascribable to the advance in the architecture of the networks, often leading to increase in the computation and resources. In this work, we propose an efficient convolutional neural network that is designed to conduct multi-class cancer classification in an accurate and robust manner via metric learning.

METHODS

We propose a centroid-aware metric learning network for an improved cancer grading in pathology images. The proposed network utilizes centroids of different classes within the feature embedding space to optimize the relative distances between pathology images, which manifest the innate similarities/dissimilarities between them. For improved optimization, we introduce a new loss function and a training strategy that are tailored to the proposed network and metric learning.

RESULTS

We evaluated the proposed approach on multiple datasets of colorectal and gastric cancers. For the colorectal cancer, two different datasets were employed that were collected from different acquisition settings. the proposed method achieved an accuracy, F1-score, quadratic weighted kappa of 88.7%, 0.849, and 0.946 for the first dataset and 83.3%, 0.764, and 0.907 for the second dataset, respectively. For the gastric cancer, the proposed method obtained an accuracy of 85.9%, F1-score of 0.793, and quadratic weighted kappa of 0.939. We also found that the proposed method outperforms other competing models and is computationally efficient.

CONCLUSIONS

The experimental results demonstrate that the prediction results by the proposed network are both accurate and reliable. The proposed network not only outperformed other related methods in cancer classification but also achieved superior computational efficiency during training and inference. The future study will entail further development of the proposed method and the application of the method to other problems and domains.

Prognostic prediction based on histopathologic features of tumor microenvironment in colorectal cancer

Purpose

To automatically quantify colorectal tumor microenvironment (TME) in hematoxylin and eosin stained whole slide images (WSIs), and to develop a TME signature for prognostic prediction in colorectal cancer (CRC).

Methods

A deep learning model based on VGG19 architecture and transfer learning strategy was trained to recognize nine different tissue types in whole slide images of patients with CRC. Seven of the nine tissue types were defined as TME components besides background and debris. Then 13 TME features were calculated based on the areas of TME components. A total of 562 patients with gene expression data, survival information and WSIs were collected from The Cancer Genome Atlas project for further analysis. A TME signature for prognostic prediction was developed and validated using Cox regression method. A prognostic prediction model combined the TME signature and clinical variables was also established. At last, gene-set enrichment analysis was performed to identify the significant TME signature associated pathways by querying Gene Ontology database and Kyoto Encyclopedia of Genes and Genomes database.

Results

The deep learning model achieved an accuracy of 94.2% for tissue type recognition. The developed TME signature was found significantly associated to progression-free survival. The clinical combined model achieved a concordance index of 0.714. Gene-set enrichment analysis revealed the TME signature associated genes were enriched in neuroactive ligand-receptor interaction pathway.

Conclusion

The TME signature was proved to be a prognostic factor and the associated biologic pathways would be beneficial to a better understanding of TME in CRC patients.

An Evolutionary Attention-Based Network for Medical Image Classification

Deep learning has become a primary choice in medical image analysis due to its powerful representation capability. However, most existing deep learning models designed for medical image classification can only perform well on a specific disease. The performance drops dramatically when it comes to other diseases. Generalizability remains a challenging problem. In this paper, we propose an evolutionary attention-based network (EDCA-Net), which is an effective and robust network for medical image classification tasks. To extract task-related features from a given medical dataset, we first propose the densely connected attentional network (DCA-Net) where feature maps are automatically channel-wise weighted, and the dense connectivity pattern is introduced to improve the efficiency of information flow. To improve the model capability and generalizability, we introduce two types of evolution: intra- and inter-evolution. The intra-evolution optimizes the weights of DCA-Net, while the inter-evolution allows two instances of DCA-Net to exchange training experience during training. The evolutionary DCA-Net is referred to as EDCA-Net. The EDCA-Net is evaluated on four publicly accessible medical datasets of different diseases. Experiments showed that the EDCA-Net outperforms the state-of-the-art methods on three datasets and achieves comparable performance on the last dataset, demonstrating good generalizability for medical image classification.

Multi-layer pseudo-supervision for histopathology tissue semantic segmentation using patch-level classification labels

Tissue-level semantic segmentation is a vital step in computational pathology. Fully-supervised models have already achieved outstanding performance with dense pixel-level annotations. However, drawing such labels on the giga-pixel whole slide images is extremely expensive and time-consuming. In this paper, we use only patch-level classification labels to achieve tissue semantic segmentation on histopathology images, finally reducing the annotation efforts. We propose a two-step model including a classification and a segmentation phases. In the classification phase, we propose a CAM-based model to generate pseudo masks by patch-level labels. In the segmentation phase, we achieve tissue semantic segmentation by our propose Multi-Layer Pseudo-Supervision. Several technical novelties have been proposed to reduce the information gap between pixel-level and patch-level annotations. As a part of this paper, we introduce a new weakly-supervised semantic segmentation (WSSS) dataset for lung adenocarcinoma (LUAD-HistoSeg). We conduct several experiments to evaluate our proposed model on two datasets. Our proposed model outperforms five state-of-the-art WSSS approaches. Note that we can achieve comparable quantitative and qualitative results with the fully-supervised model, with only around a 2% gap for MIoU and FwIoU. By comparing with manual labeling on a randomly sampled 100 patches dataset, patch-level labeling can greatly reduce the annotation time from hours to minutes. The source code and the released datasets are available at: https://github.com/ChuHan89/WSSS-Tissue.