A graph neural network framework for mapping histological topology in oral mucosal tissue

Combining graph neural networks and computer vision methods for cell nuclei classification in lung tissue

The detection of tumoural cells from whole slide images is an essential task in medical diagnosis and research. In this article, we propose and analyse a novel approach that combines computer vision-based models with graph neural networks to improve the accuracy of automated tumoural cell detection in lung tissue. Our proposal leverages the inherent structure and relationships between cells in the tissue. Experimental results on our own curated dataset show that modelling the problem with graphs gives the model a clear advantage over just working at pixel level. This change in perspective provides extra information that makes it possible to improve the performance. The reduction of dimensionality that comes from working with the graph also allows us to increase the field of view with low computational requirements. Code is available at https://github.com/Jerry-Master/lung-tumour-study, models are uploaded to https://huggingface.co/Jerry-Master/Hovernet-plus-Graphs, and the dataset is published on Zenodo https://zenodo.org/doi/10.5281/zenodo.8368122.

HSG-MGAF Net: Heterogeneous subgraph-guided multiscale graph attention fusion network for interpretable prediction of whole-slide image

BACKGROUND AND OBJECTIVE

Pathological whole slide image (WSI) prediction and region of interest (ROI) localization are important issues in computer-aided diagnosis and postoperative analysis in clinical applications. Existing computer-aided methods for predicting WSI are mainly based on multiple instance learning (MIL) and its variants. However, most of the methods are based on instance independence and identical distribution assumption and performed at a single scale, which not fully exploit the hierarchical multiscale heterogeneous information contained in WSI.

METHODS

Heterogeneous Subgraph-Guided Multiscale Graph Attention Fusion Network (HSG-MGAF Net) is proposed to build the topology of critical image patches at two scales for adaptive WSI prediction and lesion localization. The HSG-MGAF Net simulates the hierarchical heterogeneous information of WSI through graph and hypergraph at two scales, respectively. This framework not only fully exploits the low-order and potential high-order correlations of image patches at each scale, but also leverages the heterogeneous information of the two scales for adaptive WSI prediction.

RESULTS

We validate the superiority of the proposed method on the CAMELYON16 and the TCGA- NSCLC, and the results show that HSG-MGAF Net outperforms the state-of-the-art method on both datasets. The average ACC, AUC and F1 score of HSG-MGAF Net can reach 92.7 %/0.951/0.892 and 92.2 %/0.957/0.919, respectively. The obtained heatmaps can also localize the positive regions more accurately, which have great consistency with the pixel-level labels.

CONCLUSIONS

The results demonstrate that HSG-MGAF Net outperforms existing weakly supervised learning methods by introducing critical heterogeneous information between the two scales. This approach paves the way for further research on light weighted heterogeneous graph-based WSI prediction and ROI localization.

Prediction of Tuberculosis From Lung Tissue Images of Diversity Outbred Mice Using Jump Knowledge Based Cell Graph Neural Network

Tuberculosis (TB), primarily affecting the lungs, is caused by the bacterium Mycobacterium tuberculosis and poses a significant health risk. Detecting acid-fast bacilli (AFB) in stained samples is critical for TB diagnosis. Whole Slide (WS) Imaging allows for digitally examining these stained samples. However, current deep-learning approaches to analyzing large-sized whole slide images (WSIs) often employ patch-wise analysis, potentially missing the complex spatial patterns observed in the granuloma essential for accurate TB classification. To address this limitation, we propose an approach that models cell characteristics and interactions as a graph, capturing both cell-level information and the overall tissue micro-architecture. This method differs from the strategies in related cell graph-based works that rely on edge thresholds based on sparsity/density in cell graph construction, emphasizing a biologically informed threshold determination instead. We introduce a cell graph-based jumping knowledge neural network (CG-JKNN) that operates on the cell graphs where the edge thresholds are selected based on the length of the mycobacteria's cords and the activated macrophage nucleus's size to reflect the actual biological interactions observed in the tissue. The primary process involves training a Convolutional Neural Network (CNN) to segment AFBs and macrophage nuclei, followed by converting large (42831*41159 pixels) lung histology images into cell graphs where an activated macrophage nucleus/AFB represents each node within the graph and their interactions are denoted as edges. To enhance the interpretability of our model, we employ Integrated Gradients and Shapely Additive Explanations (SHAP). Our analysis incorporated a combination of 33 graph metrics and 20 cell morphology features. In terms of traditional machine learning models, Extreme Gradient Boosting (XGBoost) was the best performer, achieving an F1 score of 0.9813 and an Area under the Precision-Recall Curve (AUPRC) of 0.9848 on the test set. Among graph-based models, our CG-JKNN was the top performer, attaining an F1 score of 0.9549 and an AUPRC of 0.9846 on the held-out test set. The integration of graph-based and morphological features proved highly effective, with CG-JKNN and XGBoost showing promising results in classifying instances into AFB and activated macrophage nucleus. The features identified as significant by our models closely align with the criteria used by pathologists in practice, highlighting the clinical applicability of our approach. Future work will explore knowledge distillation techniques and graph-level classification into distinct TB progression categories.

Spatial Morphoproteomic Features Predict Uniqueness of Immune Microarchitectures and Responses in Lymphoid Follicles

Multiplex imaging technologies allow the characterization of single cells in their cellular environments. Understanding the organization of single cells within their microenvironment and quantifying disease-status related biomarkers is essential for multiplex datasets. Here we proposed SNOWFLAKE, a graph neural network framework pipeline for the prediction of disease-status from combined multiplex cell expression and morphology in human B-cell follicles. We applied SNOWFLAKE to a multiplex dataset related to COVID-19 infection in humans and showed better predictive power of the SNOWFLAKE pipeline compared to other machine learning and deep learning methods. Moreover, we combined morphological features inside graph edge features to utilize attribution methods for extracting disease-relevant motifs from single-cell spatial graphs. The underlying subgraphs were further analyzed and associated with disease status across the dataset. We showed that SNOWFLAKE successfully extracted significant low dimensional embedding from subgraphs with a clear separation between disease status and helped characterize unique cellular interactions in the subgraphs. SNOWFLAKE is a generalizable pipeline for the analysis of multiplex imaging data modality by extracting disease-relevant subgraphs guided by graph-level prediction.