Deep learning-based image analysis methods for brightfield-acquired multiplex immunohistochemistry images

Digital analysis of the prostate tumor microenvironment with high-order chromogenic multiplexing

As our understanding of the tumor microenvironment grows, the pathology field is increasingly utilizing multianalyte diagnostic assays to understand important characteristics of tumor growth. In clinical settings, brightfield chromogenic assays represent the gold-standard and have developed significant trust as the first-line diagnostic method. However, conventional brightfield tests have been limited to low-order assays that are visually interrogated. We have developed a hybrid method of brightfield chromogenic multiplexing that overcomes these limitations and enables high-order multiplex assays. However, how compatible high-order brightfield multiplexed images are with advanced analytical algorithms has not been extensively evaluated. In the present study, we address this gap by developing a novel 6-marker prostate cancer assay that targets diverse aspects of the tumor microenvironment such as prostate-specific biomarkers (PSMA and p504s), immune biomarkers (CD8 and PD-L1), a prognostic biomarker (Ki-67), as well as an adjunctive diagnostic biomarker (basal cell cocktail) and apply the assay to 143 differentially graded adenocarcinoma prostate tissues. The tissues were then imaged on our spectroscopic multiplexing imaging platform and mined for proteomic and spatial features that were correlated with cancer presence and disease grade. Extracted features were used to train a UMAP model that differentiated healthy from cancerous tissue with an accuracy of 89% and identified clusters of cells based on cancer grade. For spatial analysis, cell-to-cell distances were calculated for all biomarkers and differences between healthy and adenocarcinoma tissues were studied. We report that p504s positive cells were at least 2× closer to cells expressing PD-L1, CD8, Ki-67, and basal cell in adenocarcinoma tissues relative to the healthy control tissues. These findings offer a powerful insight to understand the fingerprint of the prostate tumor microenvironment and indicate that high-order chromogenic multiplexing is compatible with digital analysis. Thus, the presented chromogenic multiplexing system combines the clinical applicability of brightfield assays with the emerging diagnostic power of high-order multiplexing in a digital pathology friendly format that is well-suited for translational studies to better understand mechanisms of tumor development and growth.

Automated detection of Kaposi sarcoma-associated herpesvirus infected cells in immunohistochemical images of skin biopsies

Immunohistochemical (IHC) staining for the antigen of Kaposi sarcoma-associated herpesvirus (KSHV), latency-associated nuclear antigen (LANA), is helpful in diagnosing Kaposi sarcoma (KS). A challenge, however, lies in distinguishing anti-LANA-positive cells from morphologically similar brown counterparts. In this work, we demonstrate a framework for automated localization and quantification of LANA positivity in whole slide images (WSI) of skin biopsies, leveraging weakly supervised multiple instance learning (MIL) while reducing false positive predictions by introducing a novel morphology-based slide aggregation method. Our framework generates interpretable heatmaps, offering insights into precise anti-LANA-positive cell localization within WSIs and a quantitative value for the percentage of positive tiles, which may assist with histological subtyping. We trained and tested our framework with an anti-LANA-stained KS pathology dataset prepared by pathologists in the United States from skin biopsies of KS-suspected patients investigated in Uganda. We achieved an area under the receiver operating characteristic curve (AUC) of 0.99 with a sensitivity and specificity of 98.15% and 96.00% in predicting anti-LANA-positive WSIs in a test dataset. We believe that the framework can provide promise for automated detection of LANA in skin biopsies, which may be especially impactful in resource-limited areas that lack trained pathologists.

Recent developments in immunotherapy for gastrointestinal tract cancers

The past few decades have witnessed the rise of immunotherapy for Gastrointestinal (GI) tract cancers. The role of immune checkpoint inhibitors (ICIs), particularly programmed death protein 1 (PD-1) and PD ligand-1 antibodies, has become increasingly pivotal in the treatment of advanced and perioperative GI tract cancers. Currently, anti-PD-1 plus chemotherapy is considered as first-line regimen for unselected advanced gastric/gastroesophageal junction adenocarcinoma (G/GEJC), mismatch repair deficient (dMMR)/microsatellite instability-high (MSI-H) colorectal cancer (CRC), and advanced esophageal cancer (EC). In addition, the encouraging performance of claudin18.2-redirected chimeric antigen receptor T-cell (CAR-T) therapy in later-line GI tract cancers brings new hope for cell therapy in solid tumour treatment. Nevertheless, immunotherapy for GI tumour remains yet precise, and researchers are dedicated to further maximising and optimising the efficacy. This review summarises the important research, latest progress, and future directions of immunotherapy for GI tract cancers including EC, G/GEJC, and CRC.

Revolutionizing Pathology with Artificial Intelligence: Innovations in Immunohistochemistry

Artificial intelligence (AI) is a reality of our times, and it has been successfully implemented in all fields, including medicine. As a relatively new domain, all efforts are directed towards creating algorithms applicable in most medical specialties. Pathology, as one of the most important areas of interest for precision medicine, has received significant attention in the development and implementation of AI algorithms. This focus is especially important for achieving accurate diagnoses. Moreover, immunohistochemistry (IHC) serves as a complementary diagnostic tool in pathology. It can be further augmented through the application of deep learning (DL) and machine learning (ML) algorithms for assessing and analyzing immunohistochemical markers. Such advancements can aid in delineating targeted therapeutic approaches and prognostic stratification. This article explores the applications and integration of various AI software programs and platforms used in immunohistochemical analysis. It concludes by highlighting the application of these technologies to pathologies such as breast, prostate, lung, melanocytic proliferations, and hematologic conditions. Additionally, it underscores the necessity for further innovative diagnostic algorithms to assist physicians in the diagnostic process.

Keratin 17 modulates the immune topography of pancreatic cancer

BACKGROUND

The immune microenvironment impacts tumor growth, invasion, metastasis, and patient survival and may provide opportunities for therapeutic intervention in pancreatic ductal adenocarcinoma (PDAC). Although never studied as a potential modulator of the immune response in most cancers, Keratin 17 (K17), a biomarker of the most aggressive (basal) molecular subtype of PDAC, is intimately involved in the histogenesis of the immune response in psoriasis, basal cell carcinoma, and cervical squamous cell carcinoma. Thus, we hypothesized that K17 expression could also impact the immune cell response in PDAC, and that uncovering this relationship could provide insight to guide the development of immunotherapeutic opportunities to extend patient survival.

METHODS

Multiplex immunohistochemistry (mIHC) and automated image analysis based on novel computational imaging technology were used to decipher the abundance and spatial distribution of T cells, macrophages, and tumor cells, relative to K17 expression in 235 PDACs.

RESULTS

K17 expression had profound effects on the exclusion of intratumoral CD8+ T cells and was also associated with decreased numbers of peritumoral CD8+ T cells, CD16+ macrophages, and CD163+ macrophages (p < 0.0001). The differences in the intratumor and peritumoral CD8+ T cell abundance were not impacted by neoadjuvant therapy, tumor stage, grade, lymph node status, histologic subtype, nor KRAS, p53, SMAD4, or CDKN2A mutations.

CONCLUSIONS

Thus, K17 expression correlates with major differences in the immune microenvironment that are independent of any tested clinicopathologic or tumor intrinsic variables, suggesting that targeting K17-mediated immune effects on the immune system could restore the innate immunologic response to PDAC and might provide novel opportunities to restore immunotherapeutic approaches for this most deadly form of cancer.

Immunocytochemical markers, molecular testing and digital cytopathology for aspiration cytology of metastatic breast carcinoma

Fine-needle aspiration cytology (FNAC) is a versatile diagnostic procedure uniquely suited for tissue biopsy of breast carcinomas and axillary metastases and/or recurrences. With the expanding treatment options and accompanying theragnostic tests, it is crucial to recognize the developments on ancillary testing and digital cytopathology techniques related to aspiration cytology of metastatic breast carcinoma. In this review, we aim to summarize and update the evidence of immunocytochemistry, for the detection of carcinoma cells (epithelial markers), confirmation of breast primary (breast-specific markers), assessment of surrogate immunostains (hormone receptors, ki-67 proliferative index and HER2) and theragnostic biomarkers, with discussion on potential diagnostic pitfalls, followed by the application of molecular tests, and digital cytopathologic techniques for assessing metastatic breast carcinoma in cytology.

Image-based multiplex immune profiling of cancer tissues: translational implications. A report of the International Immuno-oncology Biomarker Working Group on Breast Cancer

Recent advances in the field of immuno-oncology have brought transformative changes in the management of cancer patients. The immune profile of tumours has been found to have key value in predicting disease prognosis and treatment response in various cancers. Multiplex immunohistochemistry and immunofluorescence have emerged as potent tools for the simultaneous detection of multiple protein biomarkers in a single tissue section, thereby expanding opportunities for molecular and immune profiling while preserving tissue samples. By establishing the phenotype of individual tumour cells when distributed within a mixed cell population, the identification of clinically relevant biomarkers with high-throughput multiplex immunophenotyping of tumour samples has great potential to guide appropriate treatment choices. Moreover, the emergence of novel multi-marker imaging approaches can now provide unprecedented insights into the tumour microenvironment, including the potential interplay between various cell types. However, there are significant challenges to widespread integration of these technologies in daily research and clinical practice. This review addresses the challenges and potential solutions within a structured framework of action from a regulatory and clinical trial perspective. New developments within the field of immunophenotyping using multiplexed tissue imaging platforms and associated digital pathology are also described, with a specific focus on translational implications across different subtypes of cancer. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Self-supervised deep learning for highly efficient spatial immunophenotyping

BACKGROUND

Efficient biomarker discovery and clinical translation depend on the fast and accurate analytical output from crucial technologies such as multiplex imaging. However, reliable cell classification often requires extensive annotations. Label-efficient strategies are urgently needed to reveal diverse cell distribution and spatial interactions in large-scale multiplex datasets.

METHODS

This study proposed Self-supervised Learning for Antigen Detection (SANDI) for accurate cell phenotyping while mitigating the annotation burden. The model first learns intrinsic pairwise similarities in unlabelled cell images, followed by a classification step to map learnt features to cell labels using a small set of annotated references. We acquired four multiplex immunohistochemistry datasets and one imaging mass cytometry dataset, comprising 2825 to 15,258 single-cell images to train and test the model.

FINDINGS

With 1% annotations (18-114 cells), SANDI achieved weighted F1-scores ranging from 0.82 to 0.98 across the five datasets, which was comparable to the fully supervised classifier trained on 1828-11,459 annotated cells (-0.002 to -0.053 of averaged weighted F1-score, Wilcoxon rank-sum test, P = 0.31). Leveraging the immune checkpoint markers stained in ovarian cancer slides, SANDI-based cell identification reveals spatial expulsion between PD1-expressing T helper cells and T regulatory cells, suggesting an interplay between PD1 expression and T regulatory cell-mediated immunosuppression.

INTERPRETATION

By striking a fine balance between minimal expert guidance and the power of deep learning to learn similarity within abundant data, SANDI presents new opportunities for efficient, large-scale learning for histology multiplex imaging data.

FUNDING

This study was funded by the Royal Marsden/ICR National Institute of Health Research Biomedical Research Centre.

Unsupervised Stain Decomposition via Inversion Regulation for Multiplex Immunohistochemistry Images

Multiplex Immunohistochemistry (mIHC) is a cost-effective and accessible method for in situ labeling of multiple protein biomarkers in a tissue sample. By assigning a different stain to each biomarker, it allows the visualization of different types of cells within the tumor vicinity for downstream analysis. However, to detect different types of stains in a given mIHC image is a challenging problem, especially when the number of stains is high. Previous deep-learning-based methods mostly assume full supervision; yet the annotation can be costly. In this paper, we propose a novel unsupervised stain decomposition method to detect different stains simultaneously. Our method does not require any supervision, except for color samples of different stains. A main technical challenge is that the problem is underdetermined and can have multiple solutions. To conquer this issue, we propose a novel inversion regulation technique, which eliminates most undesirable solutions. On a 7-plexed IHC images dataset, the proposed method achieves high quality stain decomposition results without human annotation.

Topology-Guided Multi-Class Cell Context Generation for Digital Pathology

In digital pathology, the spatial context of cells is important for cell classification, cancer diagnosis and prognosis. To model such complex cell context, however, is challenging. Cells form different mixtures, lineages, clusters and holes. To model such structural patterns in a learnable fashion, we introduce several mathematical tools from spatial statistics and topological data analysis. We incorporate such structural descriptors into a deep generative model as both conditional inputs and a differentiable loss. This way, we are able to generate high quality multi-class cell layouts for the first time. We show that the topology-rich cell layouts can be used for data augmentation and improve the performance of downstream tasks such as cell classification.

Towards the characterization of the tumor microenvironment through dictionary learning-based interpretable classification of multiplexed immunofluorescence images

Objective.Histology image analysis is a crucial diagnostic step in staging and treatment planning, especially for cancerous lesions. With the increasing adoption of computational methods for image analysis, significant strides are being made to improve the performance metrics of image segmentation and classification frameworks. However, many developed frameworks effectively function as black boxes, granting minimal context to the decision-making process. Thus, there is a need to develop methods that offer reasonable discriminatory power and a biologically-informed intuition to the decision-making process.Approach.In this study, we utilized and modified a discriminative feature-based dictionary learning (DFDL) paradigm to generate a classification framework that allows for discrimination between two distinct clinical histologies. This framework allows us (i) to discriminate between 2 clinically distinct diseases or histologies and (ii) provides interpretable group-specific representative dictionary image patches, or 'atoms', generated during classifier training. This implementation is performed on multiplexed immunofluorescence images from two separate patient cohorts- a pancreatic cohort consisting of cancerous and non-cancerous tissues and a metastatic non-small cell lung cancer (mNSCLC) cohort of responders and non-responders to an immunotherapeutic treatment regimen. The analysis was done at both the image-level and subject-level. Five cell types were selected, namely, epithelial cells, cytotoxic lymphocytes, antigen presenting cells, HelperT cells, and T-regulatory cells, as our phenotypes of interest.Results.We showed that DFDL had significant discriminant capabilities for both the pancreatic pathologies cohort (subject-level AUC-0.8878) and the mNSCLC immunotherapy response cohort (subject-level AUC-0.7221). The secondary analysis also showed that more than 50% of the obtained dictionary atoms from the classifier contained biologically relevant information.Significance.Our method shows that the generated dictionary features can help distinguish patients presenting two different histologies with strong sensitivity and specificity metrics. These features allow for an additional layer of model interpretability, a highly desirable element in clinical applications for identifying novel biological phenomena.

An Approach toward Automatic Specifics Diagnosis of Breast Cancer Based on an Immunohistochemical Image

The paper explored the problem of automatic diagnosis based on immunohistochemical image analysis. The issue of automated diagnosis is a preliminary and advisory statement for a diagnostician. The authors studied breast cancer histological and immunohistochemical images using the following biomarkers progesterone, estrogen, oncoprotein, and a cell proliferation biomarker. The authors developed a breast cancer diagnosis method based on immunohistochemical image analysis. The proposed method consists of algorithms for image preprocessing, segmentation, and the determination of informative indicators (relative area and intensity of cells) and an algorithm for determining the molecular genetic breast cancer subtype. An adaptive algorithm for image preprocessing was developed to improve the quality of the images. It includes median filtering and image brightness equalization techniques. In addition, the authors developed a software module part of the HIAMS software package based on the Java programming language and the OpenCV computer vision library. Four molecular genetic breast cancer subtypes could be identified using this solution: subtype Luminal A, subtype Luminal B, subtype HER2/neu amplified, and basalt-like subtype. The developed algorithm for the quantitative characteristics of the immunohistochemical images showed sufficient accuracy in determining the cancer subtype "Luminal A". It was experimentally established that the relative area of the nuclei of cells covered with biomarkers of progesterone, estrogen, and oncoprotein was more than 85%. The given approach allows for automating and accelerating the process of diagnosis. Developed algorithms for calculating the quantitative characteristics of cells on immunohistochemical images can increase the accuracy of diagnosis.

Cell2Grid: an efficient, spatial, and convolutional neural network-ready representation of cell segmentation data

Purpose

Cell segmentation algorithms are commonly used to analyze large histologic images as they facilitate interpretation, but on the other hand they complicate hypothesis-free spatial analysis. Therefore, many applications train convolutional neural networks (CNNs) on high-resolution images that resolve individual cells instead, but their practical application is severely limited by computational resources. In this work, we propose and investigate an alternative spatial data representation based on cell segmentation data for direct training of CNNs.

Approach

We introduce and analyze the properties of Cell2Grid, an algorithm that generates compact images from cell segmentation data by placing individual cells into a low-resolution grid and resolves possible cell conflicts. For evaluation, we present a case study on colorectal cancer relapse prediction using fluorescent multiplex immunohistochemistry images.

Results

We could generate Cell2Grid images at 5 - μ m resolution that were 100 times smaller than the original ones. Cell features, such as phenotype counts and nearest-neighbor cell distances, remain similar to those of original cell segmentation tables ( p < 0.0001 ). These images could be directly fed to a CNN for predicting colon cancer relapse. Our experiments showed that test set error rate was reduced by 25% compared with CNNs trained on images rescaled to 5 μ m with bilinear interpolation. Compared with images at 1 - μ m resolution (bilinear rescaling), our method reduced CNN training time by 85%.

Conclusions

Cell2Grid is an efficient spatial data representation algorithm that enables the use of conventional CNNs on cell segmentation data. Its cell-based representation additionally opens a door for simplified model interpretation and synthetic image generation.

Bright-Field Multiplex Immunohistochemistry Assay for Tumor Microenvironment Evaluation in Melanoma Tissues

Simple Summary

Bright-field (BF) immunohistochemistry (IHC) remains the gold standard for histopathological evaluations. The development of new BF multiplex IHC could be very useful for the study and characterization of the tumor microenvironment (TME) in melanoma samples. We herein compared different BF IHC multiplex protocols for the study of TME in primary cutaneous melanoma tissues and offered the best optimized protocol for visualization and evaluation. These methodologies are studied to maximize the quality of staining considering the tissue characteristics under examination, maintaining a high level of standardization and reproducibility.

Abstract

The tumor microenvironment (TME) plays a crucial role in melanoma development, progression and response to treatment. As many of the most relevant TME cell phenotypes are defined by the simultaneous detection of more than two markers, the bright-field (BF) multiplex immunohistochemistry (IHC) technique has been introduced for the quantitative assessment and evaluation of the relative spatial distances between immune cells and melanoma cells. In the current study, we aimed to validate BF multiplex IHC techniques in the Ventana Discovery Ultra Immunostainer to be applied to the evaluation of the TME in variably pigmented melanoma tissues. The BF multiplex IHC staining was performed using different combinations of six immune-cell markers—CD3, CD4, CD8, CD20, CD68 and CD163—and the melanoma cell marker SOX10. Our results show that the BF double IHC Yellow/Purple protocol guarantees the maximum contrast in all the cell populations tested and the combination SOX10 (Green), CD8 (Yellow) and CD163 (Purple) of the BF triple IHC protocol ensures the best contrast and discrimination between the three stained cell populations. Furthermore, the labeled cells were clearly distinct and easily identifiable using the image analysis software. Our standardized BF IHC multiplex protocols can be used to better assess the immune contexts of melanoma patients with potential applications to drive therapeutic decisions within clinical trials.

Deep Learning-Inferred Multiplex ImmunoFluorescence for Immunohistochemical Image Quantification

Reporting biomarkers assessed by routine immunohistochemical (IHC) staining of tissue is broadly used in diagnostic pathology laboratories for patient care. To date, clinical reporting is predominantly qualitative or semi-quantitative. By creating a multitask deep learning framework referred to as DeepLIIF, we present a single-step solution to stain deconvolution/separation, cell segmentation, and quantitative single-cell IHC scoring. Leveraging a unique de novo dataset of co-registered IHC and multiplex immunofluorescence (mpIF) staining of the same slides, we segment and translate low-cost and prevalent IHC slides to more expensive-yet-informative mpIF images, while simultaneously providing the essential ground truth for the superimposed brightfield IHC channels. Moreover, a new nuclear-envelop stain, LAP2beta, with high (>95%) cell coverage is introduced to improve cell delineation/segmentation and protein expression quantification on IHC slides. By simultaneously translating input IHC images to clean/separated mpIF channels and performing cell segmentation/classification, we show that our model trained on clean IHC Ki67 data can generalize to more noisy and artifact-ridden images as well as other nuclear and non-nuclear markers such as CD3, CD8, BCL2, BCL6, MYC, MUM1, CD10, and TP53. We thoroughly evaluate our method on publicly available benchmark datasets as well as against pathologists' semi-quantitative scoring. The code, the pre-trained models, along with easy-to-run containerized docker files as well as Google CoLab project are available at https://github.com/nadeemlab/deepliif.

The Role of Artificial Intelligence in Early Cancer Diagnosis

Improving the proportion of patients diagnosed with early-stage cancer is a key priority of the World Health Organisation. In many tumour groups, screening programmes have led to improvements in survival, but patient selection and risk stratification are key challenges. In addition, there are concerns about limited diagnostic workforces, particularly in light of the COVID-19 pandemic, placing a strain on pathology and radiology services. In this review, we discuss how artificial intelligence algorithms could assist clinicians in (1) screening asymptomatic patients at risk of cancer, (2) investigating and triaging symptomatic patients, and (3) more effectively diagnosing cancer recurrence. We provide an overview of the main artificial intelligence approaches, including historical models such as logistic regression, as well as deep learning and neural networks, and highlight their early diagnosis applications. Many data types are suitable for computational analysis, including electronic healthcare records, diagnostic images, pathology slides and peripheral blood, and we provide examples of how these data can be utilised to diagnose cancer. We also discuss the potential clinical implications for artificial intelligence algorithms, including an overview of models currently used in clinical practice. Finally, we discuss the potential limitations and pitfalls, including ethical concerns, resource demands, data security and reporting standards.

Immunohistochemistry as a detection tool for ion channels involved in dental pain signaling

Background

Despite advances in pain detection, diagnosis, and management, the prevalence of dental pain is still on the rise. Although dental pain is not directly related to fatal outcomes, the two most common types of dental pain—dental caries and dentin hypersensitivity—have a significant impact on an individual’s quality of life. Understanding the mechanism of the pain pathway is one of the crucial steps in providing better treatment for these patients. Ion channels are critical biomolecules that have been the subject of dental study owing to their roles in the transmission and transduction of external stimuli, as well as in the control and perception of pain. Numerous immunohistochemical (IHC) staining approaches have also been used to identify the many ion channels implicated in peripheral pain signaling in dental pulp.

Highlight

This review highlights the critical steps in IHC and its role in the detection of ion channels involved in the dental pain signaling pathway.

Conclusion

The key ion channels identified using IHC and whose functions have been widely researched in dental tissues are addressed in this review article.

Zozus M, Jiang X, Brimhall BB, Windham AK, Schmidt S, Visweswaran S, Ye Y, Goodrum H, Ling Y, Barapatre S, Becich MJ. Artificial intelligence in clinical and translational science: Successes, challenges and opportunities

Artificial intelligence (AI) is transforming many domains, including finance, agriculture, defense, and biomedicine. In this paper, we focus on the role of AI in clinical and translational research (CTR), including preclinical research (T1), clinical research (T2), clinical implementation (T3), and public (or population) health (T4). Given the rapid evolution of AI in CTR, we present three complementary perspectives: (1) scoping literature review, (2) survey, and (3) analysis of federally funded projects. For each CTR phase, we addressed challenges, successes, failures, and opportunities for AI. We surveyed Clinical and Translational Science Award (CTSA) hubs regarding AI projects at their institutions. Nineteen of 63 CTSA hubs (30%) responded to the survey. The most common funding source (48.5%) was the federal government. The most common translational phase was T2 (clinical research, 40.2%). Clinicians were the intended users in 44.6% of projects and researchers in 32.3% of projects. The most common computational approaches were supervised machine learning (38.6%) and deep learning (34.2%). The number of projects steadily increased from 2012 to 2020. Finally, we analyzed 2604 AI projects at CTSA hubs using the National Institutes of Health Research Portfolio Online Reporting Tools (RePORTER) database for 2011-2019. We mapped available abstracts to medical subject headings and found that nervous system (16.3%) and mental disorders (16.2) were the most common topics addressed. From a computational perspective, big data (32.3%) and deep learning (30.0%) were most common. This work represents a snapshot in time of the role of AI in the CTSA program.

How the variability between computer-assisted analysis procedures evaluating immune markers can influence patients' outcome prediction

Differences between computer-assisted image analysis (CAI) algorithms may cause discrepancies in the identification of immunohistochemically stained immune biomarkers in biopsies of breast cancer patients. These discrepancies have implications for their association with disease outcome. This study aims to compare three CAI procedures (A, B and C) to measure positive marker areas in post-neoadjuvant chemotherapy biopsies of patients with triple-negative breast cancer (TNBC) and to explore the differences in their performance in determining the potential association with relapse in these patients. A total of 3304 digital images of biopsy tissue obtained from 118 TNBC patients were stained for seven immune markers using immunohistochemistry (CD4, CD8, FOXP3, CD21, CD1a, CD83, HLA-DR) and were analyzed with procedures A, B and C. The three methods measure the positive pixel markers in the total tissue areas. The extent of agreement between paired CAI procedures, a principal component analysis (PCA) and Cox multivariate analysis was assessed. Comparisons of paired procedures showed close agreement for most of the immune markers at low concentration. The probability of differences between the paired procedures B/C and B/A was generally higher than those observed in C/A. The principal component analysis, largely based on data from CD8, CD1a and HLA-DR, identified two groups of patients with a significantly lower probability of relapse than the others. The multivariate regression models showed similarities in the factors associated with relapse for procedures A and C, as opposed to those obtained with procedure B. General agreement among the results of CAI procedures would not guarantee that the same predictive breast cancer markers were consistently identified. These results highlight the importance of developing additional strategies to improve the sensitivity of CAI procedures.

A Spatial Quantitative Systems Pharmacology Platform spQSP-IO for Simulations of Tumor-Immune Interactions and Effects of Checkpoint Inhibitor Immunotherapy

Quantitative systems pharmacology (QSP) models have become increasingly common in fundamental mechanistic studies and drug discovery in both academic and industrial environments. With imaging techniques widely adopted and other spatial quantification of tumor such as spatial transcriptomics gaining traction, it is crucial that these data reflecting tumor spatial heterogeneity be utilized to inform the QSP models to enhance their predictive power. We developed a hybrid computational model platform, spQSP-IO, to extend QSP models of immuno-oncology with spatially resolved agent-based models (ABM), combining their powers to track whole patient-scale dynamics and recapitulate the emergent spatial heterogeneity in the tumor. Using a model of non-small-cell lung cancer developed based on this platform, we studied the role of the tumor microenvironment and cancer-immune cell interactions in tumor development and applied anti-PD-1 treatment to virtual patients and studied how the spatial distribution of cells changes during tumor growth in response to the immune checkpoint inhibition treatment. Using parameter sensitivity analysis and biomarker analysis, we are able to identify mechanisms and pretreatment measurements correlated with treatment efficacy. By incorporating spatial data that highlight both heterogeneity in tumors and variability among individual patients, spQSP-IO models can extend the QSP framework and further advance virtual clinical trials.

Challenges and Opportunities in the Statistical Analysis of Multiplex Immunofluorescence Data

Simple Summary

Immune modulation is considered a hallmark of cancer initiation and progression, and has offered promising opportunities for therapeutic manipulation. Multiplex immunofluorescence (mIF) technology has enabled the tumor immune microenvironment (TIME) to be studied at an increased scale, in terms of both the number of markers and the number of samples. Another benefit of mIF technology is the ability to measure not only the abundance but also the spatial location of multiple cells types within a tissue sample simultaneously, allowing for assessment of the co-localization of different types of immune markers. Thus, the use of mIF technologies have enable researchers to characterize patient, clinical, and tumor characteristics in the hope of identifying patients whom might benefit from immunotherapy treatments. In this review we outline some of the challenges and opportunities in the statistical analyses of mIF data to study the TIME.

Abstract

Immune modulation is considered a hallmark of cancer initiation and progression. The recent development of immunotherapies has ushered in a new era of cancer treatment. These therapeutics have led to revolutionary breakthroughs; however, the efficacy of immunotherapy has been modest and is often restricted to a subset of patients. Hence, identification of which cancer patients will benefit from immunotherapy is essential. Multiplex immunofluorescence (mIF) microscopy allows for the assessment and visualization of the tumor immune microenvironment (TIME). The data output following image and machine learning analyses for cell segmenting and phenotyping consists of the following information for each tumor sample: the number of positive cells for each marker and phenotype(s) of interest, number of total cells, percent of positive cells for each marker, and spatial locations for all measured cells. There are many challenges in the analysis of mIF data, including many tissue samples with zero positive cells or “zero-inflated” data, repeated measurements from multiple TMA cores or tissue slides per subject, and spatial analyses to determine the level of clustering and co-localization between the cell types in the TIME. In this review paper, we will discuss the challenges in the statistical analysis of mIF data and opportunities for further research.

Inter-Metastatic Heterogeneity of Tumor Marker Expression and Microenvironment Architecture in a Preclinical Cancer Model

BACKGROUND

Preclinical drug development studies rarely consider the impact of a candidate drug on established metastatic disease. This may explain why agents that are successful in subcutaneous and even orthotopic preclinical models often fail to demonstrate efficacy in clinical trials. It is reasonable to anticipate that sites of metastasis will be phenotypically unique, as each tumor will have evolved heterogeneously with respect to gene expression as well as the associated phenotypic outcome of that expression. The objective for the studies described here was to gain an understanding of the tumor heterogeneity that exists in established metastatic disease and use this information to define a preclinical model that is more predictive of treatment outcome when testing novel drug candidates clinically.

METHODS

Female NCr nude mice were inoculated with fluorescent (mKate), Her2/neu-positive human breast cancer cells (JIMT-mKate), either in the mammary fat pad (orthotopic; OT) to replicate a primary tumor, or directly into the left ventricle (intracardiac; IC), where cells eventually localize in multiple sites to create a model of established metastasis. Tumor development was monitored by in vivo fluorescence imaging (IVFI). Subsequently, animals were sacrificed, and tumor tissues were isolated and imaged ex vivo. Tumors within organ tissues were further analyzed via multiplex immunohistochemistry (mIHC) for Her2/neu expression, blood vessels (CD31), as well as a nuclear marker (Hoechst) and fluorescence (mKate) expressed by the tumor cells.

RESULTS

Following IC injection, JIMT-1mKate cells consistently formed tumors in the lung, liver, brain, kidney, ovaries, and adrenal glands. Disseminated tumors were highly variable when assessing vessel density (CD31) and tumor marker expression (mkate, Her2/neu). Interestingly, tumors which developed within an organ did not adopt a vessel microarchitecture that mimicked the organ where growth occurred, nor did the vessel microarchitecture appear comparable to the primary tumor. Rather, metastatic lesions showed considerable variability, suggesting that each secondary tumor is a distinct disease entity from a microenvironmental perspective.

CONCLUSIONS

The data indicate that more phenotypic heterogeneity in the tumor microenvironment exists in models of metastatic disease than has been previously appreciated, and this heterogeneity may better reflect the metastatic cancer in patients typically enrolled in early-stage Phase I/II clinical trials. Similar to the suggestion of others in the past, the use of models of established metastasis preclinically should be required as part of the anticancer drug candidate development process, and this may be particularly important for targeted therapeutics and/or nanotherapeutics.

State of machine and deep learning in histopathological applications in digestive diseases

Machine learning (ML)- and deep learning (DL)-based imaging modalities have exhibited the capacity to handle extremely high dimensional data for a number of computer vision tasks. While these approaches have been applied to numerous data types, this capacity can be especially leveraged by application on histopathological images, which capture cellular and structural features with their high-resolution, microscopic perspectives. Already, these methodologies have demonstrated promising performance in a variety of applications like disease classification, cancer grading, structure and cellular localizations, and prognostic predictions. A wide range of pathologies requiring histopathological evaluation exist in gastroenterology and hepatology, indicating these as disciplines highly targetable for integration of these technologies. Gastroenterologists have also already been primed to consider the impact of these algorithms, as development of real-time endoscopic video analysis software has been an active and popular field of research. This heightened clinical awareness will likely be important for future integration of these methods and to drive interdisciplinary collaborations on emerging studies. To provide an overview on the application of these methodologies for gastrointestinal and hepatological histopathological slides, this review will discuss general ML and DL concepts, introduce recent and emerging literature using these methods, and cover challenges moving forward to further advance the field.

Artificial intelligence for pancreatic cancer detection: Recent development and future direction

Artificial intelligence (AI) has been increasingly utilized in medical applications, especially in the field of gastroenterology. AI can assist gastroenterologists in imaging-based testing and prediction of clinical diagnosis, for examples, detecting polyps during colonoscopy, identifying small bowel lesions using capsule endoscopy images, and predicting liver diseases based on clinical parameters. With its high mortality rate, pancreatic cancer can highly benefit from AI since the early detection of small lesion is difficult with conventional imaging techniques and current biomarkers. Endoscopic ultrasound (EUS) is a main diagnostic tool with high sensitivity for pancreatic adenocarcinoma and pancreatic cystic lesion. The standard tumor markers have not been effective for diagnosis. There have been recent research studies in AI application in EUS and novel biomarkers to early detect and differentiate malignant pancreatic lesions. The findings are impressive compared to the available traditional methods. Herein, we aim to explore the utility of AI in EUS and novel serum and cyst fluid biomarkers for pancreatic cancer detection.

Artificial Intelligence in Cancer Research and Precision Medicine

Artificial intelligence (AI) is rapidly reshaping cancer research and personalized clinical care. Availability of high-dimensionality datasets coupled with advances in high-performance computing, as well as innovative deep learning architectures, has led to an explosion of AI use in various aspects of oncology research. These applications range from detection and classification of cancer, to molecular characterization of tumors and their microenvironment, to drug discovery and repurposing, to predicting treatment outcomes for patients. As these advances start penetrating the clinic, we foresee a shifting paradigm in cancer care becoming strongly driven by AI. SIGNIFICANCE: AI has the potential to dramatically affect nearly all aspects of oncology-from enhancing diagnosis to personalizing treatment and discovering novel anticancer drugs. Here, we review the recent enormous progress in the application of AI to oncology, highlight limitations and pitfalls, and chart a path for adoption of AI in the cancer clinic.

In Situ Characterization of Follicular Helper CD4 T Cells Using Multiplexed Imaging

Follicular helper CD4 T (Tfh) cells play an essential role in the formation of germinal centers (GCs), where mature B cells proliferate, differentiate, and provide long-term protective humoral responses. Despite the extensive phenotypic characterization and identification of human Tfh cell subsets, their spatial positioning at tissue level is not well understood. Here, we describe a quantitative multiplexed immunofluorescence approach allowing for the comprehensive in situ characterization of Tfh cells in human tonsils and lymph nodes (LNs) from individuals with angioimmunoblastic T-cell lymphoma (AITL). We have developed eight multiplexed panels comprising a spectrum of Tfh cell markers, like PD-1, CXCR5, and ICOS, along with transcription factors (Bcl6, Tbet, GATA3), to assess their expression, frequencies, spatial distribution and co-localization in a quantitative manner. Combined analysis of relevant markers revealed the presence of several Tfh cell subsets at tissue level based on the differential expression of surface receptors, nuclear factors as well as their distinct localization within the follicular areas. Interestingly, we found a considerable amount of tonsillar Tfh cells expressing high levels of the Th2 regulator GATA3. The co-expression of GATA3, CXCR5, and BCL6, points to an important role of GATA3 for the generation of effector human Tfh cells. Furthermore, our data revealed significantly different Tfh cell profile signatures between health and disease. Therefore, our imaging platform generates meaningful information for the in situ characterization of human Tfh cells and could provide the base for future studies aiming to a comprehensive understanding of Tfh cell tissue heterogeneity.

Deep learning in cancer pathology: a new generation of clinical biomarkers

Clinical workflows in oncology rely on predictive and prognostic molecular biomarkers. However, the growing number of these complex biomarkers tends to increase the cost and time for decision-making in routine daily oncology practice; furthermore, biomarkers often require tumour tissue on top of routine diagnostic material. Nevertheless, routinely available tumour tissue contains an abundance of clinically relevant information that is currently not fully exploited. Advances in deep learning (DL), an artificial intelligence (AI) technology, have enabled the extraction of previously hidden information directly from routine histology images of cancer, providing potentially clinically useful information. Here, we outline emerging concepts of how DL can extract biomarkers directly from histology images and summarise studies of basic and advanced image analysis for cancer histology. Basic image analysis tasks include detection, grading and subtyping of tumour tissue in histology images; they are aimed at automating pathology workflows and consequently do not immediately translate into clinical decisions. Exceeding such basic approaches, DL has also been used for advanced image analysis tasks, which have the potential of directly affecting clinical decision-making processes. These advanced approaches include inference of molecular features, prediction of survival and end-to-end prediction of therapy response. Predictions made by such DL systems could simplify and enrich clinical decision-making, but require rigorous external validation in clinical settings.

Sequential Colocalization of ERa, PR, and AR Hormone Receptors Using Confocal Microscopy Enables New Insights into Normal Breast and Prostate Tissue and Cancers

Simple Summary

At present, platforms for multiplex immunohistochemistry (e.g., Opal) identify markers in distinct cell populations within a tissue section using multispectral fluorescence and optic microscopy. However, the optic resolution is not enough to colocalize markers at the subcellular level in the main epithelial or cancer population. We use confocal microscopy in multiplex detection of nuclear hormone receptors since they are an important part of the diagnosis and treatment of breast and prostate cancer. Moreover, we increased the quantitative dynamic range and resolution through increasing the signal/noise ration through reducing autofluorescence and increased longer antibody incubations. ColNu mIHCF identified distinct patterns of nuclear receptor colocalization in breast cancers. Furthermore, in prostate cancer all cancer epithelium was positive for ERa at the plasma membrane; and in normal prostate a small ERa+/p63+/AR− basal population suggest stem cell commitment to differentiation. ColNu mIHCF could be used for improving diagnosis and treatment in cancer.

Abstract

Multiplex immunohistochemistry (mIHC) use markers staining different cell populations applying widefield optical microscopy. Resolution is low not resolving subcellular co-localization. We sought to colocalize markers at subcellular level with antibodies validated for clinical diagnosis, including the single secondary antibody (combination of anti-rabbit/mouse-antibodies) used for diagnostic IHC with any primary antibody, and confocal microscopy. We explore colocalization in the nucleus (ColNu) of nuclear hormone receptors (ERa, PR, and AR) along with the baseline marker p63 in paired samples of breast and prostate tissues. We established ColNu mIHCF as a reliable technique easily implemented in a hospital setting. In ERa+ breast cancer, we identified different colocalization patterns (nuclear or cytoplasmatic) with PR and AR on the luminal epithelium. A triple-negative breast-cancer case expressed membrane-only ERa. A PR-only case was double positive PR/p63. In normal prostate, we identified an ERa+/p63+/AR-negative distinct population. All prostate cancer cases characteristically expressed ERa on the apical membrane of the AR+ epithelium. We confirmed this using ERa IHC and needle-core biopsies. ColNu mIHCF is feasible and already revealed a new marker for prostate cancer and identified sub-patterns in breast cancer. It could be useful for pathology as well as for functional studies in normal prostate and breast tissues.