Spatial Characterization of Tumor-Infiltrating Lymphocytes and Breast Cancer Progression

Publicly available datasets of breast histopathology H&E whole-slide images: A scoping review

Advancements in digital pathology and computing resources have made a significant impact in the field of computational pathology for breast cancer diagnosis and treatment. However, access to high-quality labeled histopathological images of breast cancer is a big challenge that limits the development of accurate and robust deep learning models. In this scoping review, we identified the publicly available datasets of breast H&E-stained whole-slide images (WSIs) that can be used to develop deep learning algorithms. We systematically searched 9 scientific literature databases and 9 research data repositories and found 17 publicly available datasets containing 10 385 H&E WSIs of breast cancer. Moreover, we reported image metadata and characteristics for each dataset to assist researchers in selecting proper datasets for specific tasks in breast cancer computational pathology. In addition, we compiled 2 lists of breast H&E patches and private datasets as supplementary resources for researchers. Notably, only 28% of the included articles utilized multiple datasets, and only 14% used an external validation set, suggesting that the performance of other developed models may be susceptible to overestimation. The TCGA-BRCA was used in 52% of the selected studies. This dataset has a considerable selection bias that can impact the robustness and generalizability of the trained algorithms. There is also a lack of consistent metadata reporting of breast WSI datasets that can be an issue in developing accurate deep learning models, indicating the necessity of establishing explicit guidelines for documenting breast WSI dataset characteristics and metadata.

Automated scoring methods for quantitative interpretation of Tumour infiltrating lymphocytes (TILs) in breast cancer: a systematic review

Tumour microenvironment (TME) of breast cancer mainly comprises malignant, stromal, immune, and tumour infiltrating lymphocyte (TILs). Assessment of TILs is crucial for determining the disease's prognosis. Manual TIL assessments are hampered by multiple limitations, including low precision, poor inter-observer reproducibility, and time consumption. In response to these challenges, automated scoring emerges as a promising approach. The aim of this systematic review is to assess the evidence on the approaches and performance of automated scoring methods for TILs assessment in breast cancer. This review presents a comprehensive compilation of studies related to automated scoring of TILs, sourced from four databases (Web of Science, Scopus, Science Direct, and PubMed), employing three primary keywords (artificial intelligence, breast cancer, and tumor-infiltrating lymphocytes). The PICOS framework was employed for study eligibility, and reporting adhered to the PRISMA guidelines. The initial search yielded a total of 1910 articles. Following screening and examination, 27 studies met the inclusion criteria and data were extracted for the review. The findings indicate a concentration of studies on automated TILs assessment in developed countries, specifically the United States and the United Kingdom. From the analysis, a combination of sematic segmentation and object detection (n = 10, 37%) and convolutional neural network (CNN) (n = 11, 41%), become the most frequent automated task and ML approaches applied for model development respectively. All models developed their own ground truth datasets for training and validation, and 59% of the studies assessed the prognostic value of TILs. In conclusion, this analysis contends that automated scoring methods for TILs assessment of breast cancer show significant promise for commodification and application within clinical settings.

Disrupting prostate cancer research: Challenge accepted; report from the 2023 Coffey-Holden Prostate Cancer Academy Meeting

INTRODUCTION

The 2023 Coffey-Holden Prostate Cancer Academy (CHPCA) Meeting, themed "Disrupting Prostate Cancer Research: Challenge Accepted," was convened at the University of California, Los Angeles, Luskin Conference Center, in Los Angeles, CA, from June 22 to 25, 2023.

METHODS

The 2023 marked the 10th Annual CHPCA Meeting, a discussion-oriented scientific think-tank conference convened annually by the Prostate Cancer Foundation, which centers on innovative and emerging research topics deemed pivotal for advancing critical unmet needs in prostate cancer research and clinical care. The 2023 CHPCA Meeting was attended by 81 academic investigators and included 40 talks across 8 sessions.

RESULTS

The central topic areas covered at the meeting included: targeting transcription factor neo-enhancesomes in cancer, AR as a pro-differentiation and oncogenic transcription factor, why few are cured with androgen deprivation therapy and how to change dogma to cure metastatic prostate cancer without castration, reducing prostate cancer morbidity and mortality with genetics, opportunities for radiation to enhance therapeutic benefit in oligometastatic prostate cancer, novel immunotherapeutic approaches, and the new era of artificial intelligence-driven precision medicine.

DISCUSSION

This article provides an overview of the scientific presentations delivered at the 2023 CHPCA Meeting, such that this knowledge can help in facilitating the advancement of prostate cancer research worldwide.

Visual Intratumor Heterogeneity and Breast Tumor Progression

High intratumoral heterogeneity is thought to be a poor prognostic indicator. However, the source of heterogeneity may also be important, as genomic heterogeneity is not always reflected in histologic or 'visual' heterogeneity. We aimed to develop a predictor of histologic heterogeneity and evaluate its association with outcomes and molecular heterogeneity. We used VGG16 to train an image classifier to identify unique, patient-specific visual features in 1655 breast tumors (5907 core images) from the Carolina Breast Cancer Study (CBCS). Extracted features for images, as well as the epithelial and stromal image components, were hierarchically clustered, and visual heterogeneity was defined as a greater distance between images from the same patient. We assessed the association between visual heterogeneity, clinical features, and DNA-based molecular heterogeneity using generalized linear models, and we used Cox models to estimate the association between visual heterogeneity and tumor recurrence. Basal-like and ER-negative tumors were more likely to have low visual heterogeneity, as were the tumors from younger and Black women. Less heterogeneous tumors had a higher risk of recurrence (hazard ratio = 1.62, 95% confidence interval = 1.22-2.16), and were more likely to come from patients whose tumors were comprised of only one subclone or had a TP53 mutation. Associations were similar regardless of whether the image was based on stroma, epithelium, or both. Histologic heterogeneity adds complementary information to commonly used molecular indicators, with low heterogeneity predicting worse outcomes. Future work integrating multiple sources of heterogeneity may provide a more comprehensive understanding of tumor progression.

Spatial distance between tumor and lymphocyte can predict the survival of patients with resectable lung adenocarcinoma

Background and objective

Spatial interaction between tumor-infiltrating lymphocytes (TILs) and tumor cells is valuable in predicting the effectiveness of immune response and prognosis amongst patients with lung adenocarcinoma (LUAD). Recent evidence suggests that the spatial distance between tumor cells and lymphocytes also influences the immune responses, but the distance analysis based on Hematoxylin and Eosin (H&E) -stained whole-slide images (WSIs) remains insufficient. To address this issue, we aim to explore the relationship between distance and prognosis prediction of patients with LUAD in this study.

Methods

We recruited patients with resectable LUAD from three independent cohorts in this multi-center study. We proposed a simple but effective deep learning-driven workflow to automatically segment different cell types in the tumor region using the HoVer-Net model, and quantified the spatial distance (DIST) between tumor cells and lymphocytes based on H&E-stained WSIs. The association of DIST with disease-free survival (DFS) was explored in the discovery set (D1, n = 276) and the two validation sets (V1, n = 139; V2, n = 115).

Results

In multivariable analysis, the low DIST group was associated with significantly better DFS in the discovery set (D1, HR, 0.61; 95 % CI, 0.40-0.94; p = 0.027) and the two validation sets (V1, HR, 0.54; 95 % CI, 0.32-0.91; p = 0.022; V2, HR, 0.44; 95 % CI, 0.24-0.81; p = 0.009). By integrating the DIST with clinicopathological factors, the integrated model (full model) had better discrimination for DFS in the discovery set (C-index, D1, 0.745 vs. 0.723) and the two validation sets (V1, 0.621 vs. 0.596; V2, 0.671 vs. 0.650). Furthermore, the computerized DIST was associated with immune phenotypes such as immune-desert and inflamed phenotypes.

Conclusions

The integration of DIST with clinicopathological factors could improve the stratification performance of patients with resectable LUAD, was beneficial for the prognosis prediction of LUAD patients, and was also expected to assist physicians in individualized treatment.

Enhancing AI Research for Breast Cancer: A Comprehensive Review of Tumor-Infiltrating Lymphocyte Datasets

The field of immunology is fundamental to our understanding of the intricate dynamics of the tumor microenvironment. In particular, tumor-infiltrating lymphocyte (TIL) assessment emerges as essential aspect in breast cancer cases. To gain comprehensive insights, the quantification of TILs through computer-assisted pathology (CAP) tools has become a prominent approach, employing advanced artificial intelligence models based on deep learning techniques. The successful recognition of TILs requires the models to be trained, a process that demands access to annotated datasets. Unfortunately, this task is hampered not only by the scarcity of such datasets, but also by the time-consuming nature of the annotation phase required to create them. Our review endeavors to examine publicly accessible datasets pertaining to the TIL domain and thereby become a valuable resource for the TIL community. The overall aim of the present review is thus to make it easier to train and validate current and upcoming CAP tools for TIL assessment by inspecting and evaluating existing publicly available online datasets.

Tumor-infiltrating lymphocytes in HER2-positive breast cancer treated with neoadjuvant chemotherapy and dual HER2-blockade

Tumor-infiltrating lymphocytes (TILs) have been associated with outcomes in HER2-positive breast cancer patients treated with neoadjuvant chemotherapy and trastuzumab. However, it remains unclear if TILs could be a prognostic and/or predictive biomarker in the context of dual HER2-targeting treatment. In this study, we evaluated the association between TILs and pathological response (pCR) and invasive-disease free survival (IDFS) in 389 patients with stage II-III HER2 positive breast cancer who received neoadjuvant anthracycline-containing or anthracycline-free chemotherapy combined with trastuzumab and pertuzumab in the TRAIN-2 trial. Although no significant association was seen between TILs and pCR, patients with TIL scores ≥60% demonstrated an excellent 3-year IDFS of 100% (95% CI 100-100), regardless of hormone receptor status, nodal stage and attainment of pCR. Additionally, in patients with hormone receptor positive disease, TILs as a continuous variable showed a trend to a positive association with pCR (adjusted Odds Ratio per 10% increase in TILs 1.15, 95% CI 0.99-1.34, p = 0.070) and IDFS (adjusted Hazard Ratio per 10% increase in TILs 0.71, 95% CI 0.50-1.01, p = 0.058). We found no interactions between TILs and anthracycline treatment. Our results suggest that high TIL scores might be able to identify stage II-III HER2-positive breast cancer patients with a favorable prognosis.

Attention De-sparsification Matters: Inducing diversity in digital pathology representation learning

We propose DiRL, a Diversity-inducing Representation Learning technique for histopathology imaging. Self-supervised learning (SSL) techniques, such as contrastive and non-contrastive approaches, have been shown to learn rich and effective representations of digitized tissue samples with limited pathologist supervision. Our analysis of vanilla SSL-pretrained models' attention distribution reveals an insightful observation: sparsity in attention, i.e, models tends to localize most of their attention to some prominent patterns in the image. Although attention sparsity can be beneficial in natural images due to these prominent patterns being the object of interest itself, this can be sub-optimal in digital pathology; this is because, unlike natural images, digital pathology scans are not object-centric, but rather a complex phenotype of various spatially intermixed biological components. Inadequate diversification of attention in these complex images could result in crucial information loss. To address this, we leverage cell segmentation to densely extract multiple histopathology-specific representations, and then propose a prior-guided dense pretext task, designed to match the multiple corresponding representations between the views. Through this, the model learns to attend to various components more closely and evenly, thus inducing adequate diversification in attention for capturing context-rich representations. Through quantitative and qualitative analysis on multiple tasks across cancer types, we demonstrate the efficacy of our method and observe that the attention is more globally distributed.

Immune Cytolytic Activity and Strategies for Therapeutic Treatment

Intratumoral immune cytolytic activity (CYT), calculated as the geometric mean of granzyme-A (GZMA) and perforin-1 (PRF1) expression, has emerged as a critical factor in cancer immunotherapy, with significant implications for patient prognosis and treatment outcomes. Immune checkpoint pathways, the composition of the tumor microenvironment (TME), antigen presentation, and metabolic pathways regulate CYT. Here, we describe the various methods with which we can assess CYT. The detection and analysis of tumor-infiltrating lymphocytes (TILs) using flow cytometry or immunohistochemistry provide important information about immune cell populations within the TME. Gene expression profiling and spatial analysis techniques, such as multiplex immunofluorescence and imaging mass cytometry allow the study of CYT in the context of the TME. We discuss the significant clinical implications that CYT has, as its increased levels are associated with positive clinical outcomes and a favorable prognosis. Moreover, CYT can be used as a prognostic biomarker and aid in patient stratification. Altering CYT through the different methods targeting it, offers promising paths for improving treatment responses. Overall, understanding and modulating CYT is critical for improving cancer immunotherapy. Research into CYT and the factors that influence it has the potential to transform cancer treatment and improve patient outcomes.

Open and reusable deep learning for pathology with WSInfer and QuPath

Digital pathology has seen a proliferation of deep learning models in recent years, but many models are not readily reusable. To address this challenge, we developed WSInfer: an open-source software ecosystem designed to streamline the sharing and reuse of deep learning models for digital pathology. The increased access to trained models can augment research on the diagnostic, prognostic, and predictive capabilities of digital pathology.

Spatial Distribution of Immune Cells Drives Resistance to Neoadjuvant Chemotherapy in Triple-Negative Breast Cancer

Neoadjuvant chemotherapy (NAC) alone or combined with target therapies represents the standard of care for localized triple-negative breast cancer (TNBC). However, only a fraction of patients have a response, necessitating better understanding of the complex elements in the TNBC ecosystem that establish continuous and multidimensional interactions. Resolving such complexity requires new spatially-defined approaches. Here, we used spatial transcriptomics to investigate the multidimensional organization of TNBC at diagnosis and explore the contribution of each cell component to response to NAC. Starting from a consecutive retrospective series of TNBC cases, we designed a case-control study including 24 patients with TNBC of which 12 experienced a pathologic complete response (pCR) and 12 no-response or progression (pNR) after NAC. Over 200 regions of interest (ROI) were profiled. Our computational approaches described a model that recapitulates clinical response to therapy. The data were validated in an independent cohort of patients. Differences in the transcriptional program were detected in the tumor, stroma, and immune infiltrate comparing patients with a pCR with those with pNR. In pCR, spatial contamination between the tumor mass and the infiltrating lymphocytes was observed, sustained by a massive activation of IFN-signaling. Conversely, pNR lesions displayed increased pro-angiogenetic signaling and oxygen-based metabolism. Only modest differences were observed in the stroma, revealing a topology-based functional heterogeneity of the immune infiltrate. Thus, spatial transcriptomics provides fundamental information on the multidimensionality of TNBC and allows an effective prediction of tumor behavior. These results open new perspectives for the improvement and personalization of therapeutic approaches to TNBCs.

Predicting early breast cancer recurrence from histopathological images in the Carolina Breast Cancer Study

Approaches for rapidly identifying patients at high risk of early breast cancer recurrence are needed. Image-based methods for prescreening hematoxylin and eosin (H&E) stained tumor slides could offer temporal and financial efficiency. We evaluated a data set of 704 1-mm tumor core H&E images (2-4 cores per case), corresponding to 202 participants (101 who recurred; 101 non-recurrent matched on age and follow-up time) from breast cancers diagnosed between 2008-2012 in the Carolina Breast Cancer Study. We leveraged deep learning to extract image information and trained a model to identify recurrence. Cross-validation accuracy for predicting recurrence was 62.4% [95% CI: 55.7, 69.1], similar to grade (65.8% [95% CI: 59.3, 72.3]) and ER status (66.3% [95% CI: 59.8, 72.8]). Interestingly, 70% (19/27) of early-recurrent low-intermediate grade tumors were identified by our image model. Relative to existing markers, image-based analyses provide complementary information for predicting early recurrence.

Clinicopathological and prognostic value of TIL and PD L1 in triple negative breast carcinomas

Triple negative breast cancer (TNBC), a highly aggressive subtype of breast cancer, accounts for 15 % of all diagnosed breast cancers. This group, which has the worst clinical outcome, high recurrence rate and poor prognosis, does not benefit from specific treatment. Therefore, there is a need to develop more effective biomarker and therapeutic strategies especially for this group. A positive level of immunity has been found to be associated with patient survival in various organ cancers. More specifically, tumor infiltrating lymphocytes (TIL) have been documented to have strong prognostic value. The programmed cell death 1 (PD 1) protein on the surface of T lymphocytes is activated by the Programmed cell death ligand 1 (PD-L1) protein on the cancer cell surface. PD- L1 is thought to form a pathway that results in suppression of antitumor responses when activated. Patients with breast cancer (BC) who underwent resection without neoadjuvant chemotherapy between 2010 and 2020 were included in this study. Of the 302 BCs examined, 21 constitute the group with TNBC. In our study, the mean age of the Triple positive breast cancer (TPBC) and TNBC groups was similar (55.67 ± 12.61 vs. 53.23 ± 8.21, p = 0.384). There was no significant correlation between TPBC and TNBC and tumor size, lymph node, histological grade, and PD-L1 positivity in the center of the tumor (all p-value >.05). It was observed that tumor stage was higher in patients with TNBC than in patients with TPBC (19 % vs. 1.1 %, p = .002). The Ki 67 proliferation index was found to be higher in patients with TNBC than in patients with TPBC (90.5 % vs. 41.8 %, p .001). Although not statistically significant, clinically, CD 3 and CD 8 immune scores with high tumor margin were higher in patients with TNBC than in patients with TPBC (90.4 % vs, 9.6 % and 85.7 % vs. 14.3 %, respectively). Positive expression of PD-L1 at the tumor margin was significantly higher in patients with TNBC than patients with TPBC (20.3 % vs, 52.4 %, p = .002). By Kaplan-Meier analysis, the survival distribution of CD 3 and CD 8 immunoscore, tumor central and margin PD-L1 values were compared. Mean follow-up was 136.18 months (range, 1 - 144 months); and the 10-year Overall Survival (OS) estimate for the population was 90.9 % (95 % CI, 85.5 - 96.7). In this study, this difference was not statistically significant according to the log-rank test. In this study, we aimed to evaluate the relationship between CD 3, CD 8 T lymphocyte immune score and PD-L1 expression at the tumor center and margin in TNBC, the prognostic value and clinicopathological significance of this relationship.

Advances in the application of computational pathology in diagnosis, immunomicroenvironment recognition, and immunotherapy evaluation of breast cancer: a narrative review

BACKGROUND

Breast cancer (BC) is a prevalent and highly lethal malignancy affecting women worldwide. Immunotherapy has emerged as a promising therapeutic strategy for BC, offering potential improvements in patient survival. Neoadjuvant therapy (NAT) has also gained significant clinical traction. With the advancement of computer technology, Artificial Intelligence (AI) has been increasingly applied in pathology research, expanding and redefining the scope of the field. This narrative review aims to provide a comprehensive overview of the current literature on the application of computational pathology in BC, specifically focusing on diagnosis, immune microenvironment recognition, and the evaluation of immunotherapy and NAT response.

METHODS

A thorough examination of relevant literature was conducted, focusing on studies investigating the role of computational pathology in BC diagnosis, immune microenvironment recognition, and immunotherapy and NAT assessment.

RESULTS

The application of computational pathology has shown significant potential in BC management. AI-based techniques enable improved diagnosis and classification of BC subtypes, enhance the identification and characterization of the immune microenvironment, and facilitate the evaluation of immunotherapy and NAT response. However, challenges related to data quality, standardization, and algorithm development still need to be addressed.

CONCLUSION

The integration of computational pathology and AI has transformative implications for BC patient care. By leveraging AI-based technologies, clinicians can make more informed decisions in diagnosis, treatment planning, and therapeutic response assessment. Future research should focus on refining AI algorithms, addressing technical challenges, and conducting large-scale clinical validation studies to facilitate the translation of computational pathology into routine clinical practice for BC patients.

Application of radiomics-based multiomics combinations in the tumor microenvironment and cancer prognosis

The advent of immunotherapy, a groundbreaking advancement in cancer treatment, has given rise to the prominence of the tumor microenvironment (TME) as a critical area of research. The clinical implications of an improved understanding of the TME are significant and far-reaching. Radiomics has been increasingly utilized in the comprehensive assessment of the TME and cancer prognosis. Similarly, the advancement of pathomics, which is based on pathological images, can offer additional insights into the panoramic view and microscopic information of tumors. The combination of pathomics and radiomics has revolutionized the concept of a "digital biopsy". As genomics and transcriptomics continue to evolve, integrating radiomics with genomic and transcriptomic datasets can offer further insights into tumor and microenvironment heterogeneity and establish correlations with biological significance. Therefore, the synergistic analysis of digital image features (radiomics, pathomics) and genetic phenotypes (genomics) can comprehensively decode and characterize the heterogeneity of the TME as well as predict cancer prognosis. This review presents a comprehensive summary of the research on important radiomics biomarkers for predicting the TME, emphasizing the interplay between radiomics, genomics, transcriptomics, and pathomics, as well as the application of multiomics in decoding the TME and predicting cancer prognosis. Finally, we discuss the challenges and opportunities in multiomics research. In conclusion, this review highlights the crucial role of radiomics and multiomics associations in the assessment of the TME and cancer prognosis. The combined analysis of radiomics, pathomics, genomics, and transcriptomics is a promising research direction with substantial research significance and value for comprehensive TME evaluation and cancer prognosis assessment.

ChampKit: A framework for rapid evaluation of deep neural networks for patch-based histopathology classification

BACKGROUND AND OBJECTIVE

Histopathology is the gold standard for diagnosis of many cancers. Recent advances in computer vision, specifically deep learning, have facilitated the analysis of histopathology images for many tasks, including the detection of immune cells and microsatellite instability. However, it remains difficult to identify optimal models and training configurations for different histopathology classification tasks due to the abundance of available architectures and the lack of systematic evaluations. Our objective in this work is to present a software tool that addresses this need and enables robust, systematic evaluation of neural network models for patch classification in histology in a light-weight, easy-to-use package for both algorithm developers and biomedical researchers.

METHODS

Here we present ChampKit (Comprehensive Histopathology Assessment of Model Predictions toolKit): an extensible, fully reproducible evaluation toolkit that is a one-stop-shop to train and evaluate deep neural networks for patch classification. ChampKit curates a broad range of public datasets. It enables training and evaluation of models supported by timm directly from the command line, without the need for users to write any code. External models are enabled through a straightforward API and minimal coding. As a result, Champkit facilitates the evaluation of existing and new models and deep learning architectures on pathology datasets, making it more accessible to the broader scientific community. To demonstrate the utility of ChampKit, we establish baseline performance for a subset of possible models that could be employed with ChampKit, focusing on several popular deep learning models, namely ResNet18, ResNet50, and R26-ViT, a hybrid vision transformer. In addition, we compare each model trained either from random weight initialization or with transfer learning from ImageNet pretrained models. For ResNet18, we also consider transfer learning from a self-supervised pretrained model.

RESULTS

The main result of this paper is the ChampKit software. Using ChampKit, we were able to systemically evaluate multiple neural networks across six datasets. We observed mixed results when evaluating the benefits of pretraining versus random intialization, with no clear benefit except in the low data regime, where transfer learning was found to be beneficial. Surprisingly, we found that transfer learning from self-supervised weights rarely improved performance, which is counter to other areas of computer vision.

CONCLUSIONS

Choosing the right model for a given digital pathology dataset is nontrivial. ChampKit provides a valuable tool to fill this gap by enabling the evaluation of hundreds of existing (or user-defined) deep learning models across a variety of pathology tasks. Source code and data for the tool are freely accessible at https://github.com/SBU-BMI/champkit.

Lymphocyte detection for cancer analysis using a novel fusion block based channel boosted CNN

Tumor-infiltrating lymphocytes, specialized immune cells, are considered an important biomarker in cancer analysis. Automated lymphocyte detection is challenging due to its heterogeneous morphology, variable distribution, and presence of artifacts. In this work, we propose a novel Boosted Channels Fusion-based CNN "BCF-Lym-Detector" for lymphocyte detection in multiple cancer histology images. The proposed network initially selects candidate lymphocytic regions at the tissue level and then detects lymphocytes at the cellular level. The proposed "BCF-Lym-Detector" generates diverse boosted channels by utilizing the feature learning capability of different CNN architectures. In this connection, a new adaptive fusion block is developed to combine and select the most relevant lymphocyte-specific features from the generated enriched feature space. Multi-level feature learning is used to retain lymphocytic spatial information and detect lymphocytes with variable appearances. The assessment of the proposed "BCF-Lym-Detector" show substantial improvement in terms of F-score (0.93 and 0.84 on LYSTO and NuClick, respectively), which suggests that the diverse feature extraction and dynamic feature selection enhanced the feature learning capacity of the proposed network. Moreover, the proposed technique's generalization on unseen test sets with a good recall (0.75) and F-score (0.73) shows its potential use for pathologists' assistance.

Pitfalls in machine learning-based assessment of tumor-infiltrating lymphocytes in breast cancer: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer

The clinical significance of the tumor-immune interaction in breast cancer is now established, and tumor-infiltrating lymphocytes (TILs) have emerged as predictive and prognostic biomarkers for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2-negative) breast cancer and HER2-positive breast cancer. How computational assessments of TILs might complement manual TIL assessment in trial and daily practices is currently debated. Recent efforts to use machine learning (ML) to automatically evaluate TILs have shown promising results. We review state-of-the-art approaches and identify pitfalls and challenges of automated TIL evaluation by studying the root cause of ML discordances in comparison to manual TIL quantification. We categorize our findings into four main topics: (1) technical slide issues, (2) ML and image analysis aspects, (3) data challenges, and (4) validation issues. The main reason for discordant assessments is the inclusion of false-positive areas or cells identified by performance on certain tissue patterns or design choices in the computational implementation. To aid the adoption of ML for TIL assessment, we provide an in-depth discussion of ML and image analysis, including validation issues that need to be considered before reliable computational reporting of TILs can be incorporated into the trial and routine clinical management of patients with triple-negative breast cancer. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Spatial analyses of immune cell infiltration in cancer: current methods and future directions: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer

Modern histologic imaging platforms coupled with machine learning methods have provided new opportunities to map the spatial distribution of immune cells in the tumor microenvironment. However, there exists no standardized method for describing or analyzing spatial immune cell data, and most reported spatial analyses are rudimentary. In this review, we provide an overview of two approaches for reporting and analyzing spatial data (raster versus vector-based). We then provide a compendium of spatial immune cell metrics that have been reported in the literature, summarizing prognostic associations in the context of a variety of cancers. We conclude by discussing two well-described clinical biomarkers, the breast cancer stromal tumor infiltrating lymphocytes score and the colon cancer Immunoscore, and describe investigative opportunities to improve clinical utility of these spatial biomarkers. © 2023 The Pathological Society of Great Britain and Ireland.

Comparison of Programmed Cell Death Ligand 1 Status between Core Needle Biopsy and Surgical Specimens of Triple-Negative Breast Cancer

PURPOSE

Pembrolizumab is currently used to treat advanced triple-negative breast cancer (TNBC) and high-risk early TNBC with neoadjuvant chemotherapy (NAC). The tumor-infiltrating lymphocyte (TIL) level and programmed cell death ligand 1 (PD-L1) status are predictors of response to NAC and immune checkpoint inhibitor treatment. We aimed to investigate whether the PD-L1 status in core needle biopsies (CNBs) could represent the whole tumor in TNBC.

MATERIALS AND METHODS

A total of 49 patients diagnosed with TNBC who received upfront surgery without NAC between January 2018 and March 2021 were included. The PD-L1 expression (SP142 and 22C3 clones) and TIL were evaluated in paired CNBs and resected specimens. The concordance PD-L1 status and TIL levels between CNBs and resected specimens were analyzed.

RESULTS

PD-L1 positivity was more frequently observed in resected specimens. The overall reliability of TIL level in the CNB was good [intraclass correlation coefficient (ICC)=0.847, p<0.001]. The agreements of PD-L1 status were good and fair, respectively (SP142, κ=0.503, p<0.001; 22C3, κ=0.380, p=0.010). As the core number of CNB increased, the reliability and agreement also improved, especially from five tumor cores (TIL, ICC=0.911, p<0.001; PD-L1 [22C3], κ=0.750, p=0.028). Regarding PD-L1 (SP142), no further improvement was observed with ≥5 tumor cores (κ=0.600, p=0.058).

CONCLUSION

CNBs with ≥5 tumor cores were sufficient to represent the TIL level and PD-L1 (22C3) status in TNBC.

Recent Advancements in Deep Learning Using Whole Slide Imaging for Cancer Prognosis

This review furnishes an exhaustive analysis of the latest advancements in deep learning techniques applied to whole slide images (WSIs) in the context of cancer prognosis, focusing specifically on publications from 2019 through 2023. The swiftly maturing field of deep learning, in combination with the burgeoning availability of WSIs, manifests significant potential in revolutionizing the predictive modeling of cancer prognosis. In light of the swift evolution and profound complexity of the field, it is essential to systematically review contemporary methodologies and critically appraise their ramifications. This review elucidates the prevailing landscape of this intersection, cataloging major developments, evaluating their strengths and weaknesses, and providing discerning insights into prospective directions. In this paper, a comprehensive overview of the field aims to be presented, which can serve as a critical resource for researchers and clinicians, ultimately enhancing the quality of cancer care outcomes. This review's findings accentuate the need for ongoing scrutiny of recent studies in this rapidly progressing field to discern patterns, understand breakthroughs, and navigate future research trajectories.

Automated Detection and Scoring of Tumor-Infiltrating Lymphocytes in Breast Cancer Histopathology Slides

Detection of tumor-infiltrating lymphocytes (TILs) in cancer images has gained significant importance as these lymphocytes can be used as a biomarker in cancer detection and treatment procedures. Our goal was to develop and apply a TILs detection tool that utilizes deep learning models, following two sequential steps. First, based on the guidelines from the International Immuno-Oncology Biomarker Working Group (IIOBWG) on Breast Cancer, we labeled 63 large pathology imaging slides and annotated the TILs in the stroma area to create the dataset required for model development. In the second step, various machine learning models were employed and trained to detect the stroma where U-Net deep learning structure was able to achieve 98% accuracy. After detecting the stroma area, a Mask R-CNN model was employed for the TILs detection task. The R-CNN model detected the TILs in various images and was used as the backbone analysis network for the GUI development of the TILs detection tool. This is the first study to combine two deep learning models for TILs detection at the cellular level in breast tumor histopathology slides. Our novel approach can be applied to scoring TILs in large cancer slides. Statistical analysis showed that the output of the implemented approach had 95% concordance with the scores assigned by the pathologists, with a p-value of 0.045 (n = 63). This demonstrated that the results from the developed software were statistically meaningful and highly accurate. The implemented approach in analyzing whole tumor histology slides and the newly developed TILs detection tool can be used for research purposes in biomedical and pathology applications and it can provide researchers and clinicians with the TIL score for various input images. Future research using additional breast cancer slides from various sources for further training and validation of the developed models is necessary for more inclusive, rigorous, and robust clinical applications.

Combining the guidelines and multiphoton imaging methods to improve the prognostic value of tumor-infiltrating lymphocytes in breast cancer

Multiphoton microscopy (MPM) was introduced to label-freely obtain tumor-infiltrating lymphocytes (TILs) images from a total of 611 patients, and the prognostic value of TILs in breast cancer was assessed by the MPM method (TILs-MPM) and guidelines method proposed by the International Immuno-Oncology Biomarker Working Group (TILs-WG), respectively. Moreover, the clinical (CLI) model, TILs-WG + TILs-MPM model, and full model (CLI + TILs-WG + TILs-MPM) were developed to investigate the prognostic value of TILs. The results show that TILs-WG performs better in estrogen receptor (ER)-negative subgroup, and TILs-MPM is comparable with TILs-WG in the ER-negative subgroup, but has the best performance in the ER-positive subgroup. Furthermore, the TILs-WG + TILs-MPM model can significantly improve the prognostic power compared with the TILs-WG model, and the full model has excellent performance, with high area under the curve (AUC) and hazard ratio (HR) in both ER-positive, ER-negative subgroups, and the complete cohort. Our results suggest that the combination of TILs-WG with TILs-MPM model can greatly improve the prognostic value of TILs.

Prognostic Significance of Tumor-Infiltrating Lymphocytes Determined Using LinkNet on Colorectal Cancer Pathology Images

PURPOSE

Tumor-infiltrating lymphocytes (TILs) have a significant prognostic value in cancers. However, very few automated, deep learning-based TIL scoring algorithms have been developed for colorectal cancer (CRC).

MATERIALS AND METHODS

We developed an automated, multiscale LinkNet workflow for quantifying TILs at the cellular level in CRC tumors using H&E-stained images from the Lizard data set with annotations of lymphocytes. The predictive performance of the automatic TIL scores (TILsLink) for disease progression and overall survival (OS) was evaluated using two international data sets, including 554 patients with CRC from The Cancer Genome Atlas (TCGA) and 1,130 patients with CRC from Molecular and Cellular Oncology (MCO).

RESULTS

The LinkNet model provided outstanding precision (0.9508), recall (0.9185), and overall F1 score (0.9347). Clear continuous TIL-hazard relationships were observed between TILsLink and the risk of disease progression or death in both TCGA and MCO cohorts. Both univariate and multivariate Cox regression analyses for the TCGA data demonstrated that patients with high TIL abundance had a significant (approximately 75%) reduction in risk for disease progression. In both the MCO and TCGA cohorts, the TIL-high group was significantly associated with improved OS in univariate analysis (30% and 54% reduction in risk, respectively). The favorable effects of high TIL levels were consistently observed in different subgroups (classified according to known risk factors).

CONCLUSION

The proposed deep-learning workflow for automatic TIL quantification on the basis of LinkNet can be a useful tool for CRC. TILsLink is likely an independent risk factor for disease progression and carries predictive information of disease progression beyond the current clinical risk factors and biomarkers. The prognostic significance of TILsLink for OS is also evident.

Artificial Intelligence-Powered Whole-Slide Image Analyzer Reveals a Distinctive Distribution of Tumor-Infiltrating Lymphocytes in Neuroendocrine Neoplasms

Despite the importance of tumor-infiltrating lymphocytes (TIL) and PD-L1 expression to the immune checkpoint inhibitor (ICI) response, a comprehensive assessment of these biomarkers has not yet been conducted in neuroendocrine neoplasm (NEN). We collected 218 NENs from multiple organs, including 190 low/intermediate-grade NENs and 28 high-grade NENs. TIL distribution was derived from Lunit SCOPE IO, an artificial intelligence (AI)-powered hematoxylin and eosin (H&E) analyzer, as developed from 17,849 whole slide images. The proportion of intra-tumoral TIL-high cases was significantly higher in high-grade NEN (75.0% vs. 46.3%, p = 0.008). The proportion of PD-L1 combined positive score (CPS) ≥ 1 case was higher in high-grade NEN (85.7% vs. 33.2%, p < 0.001). The PD-L1 CPS ≥ 1 group showed higher intra-tumoral, stromal, and combined TIL densities, compared to the CPS < 1 group (7.13 vs. 2.95, p < 0.001; 200.9 vs. 120.5, p < 0.001; 86.7 vs. 56.1, p = 0.004). A significant correlation was observed between TIL density and PD-L1 CPS (r = 0.37, p < 0.001 for intra-tumoral TIL; r = 0.24, p = 0.002 for stromal TIL and combined TIL). AI-powered TIL analysis reveals that intra-tumoral TIL density is significantly higher in high-grade NEN, and PD-L1 CPS has a positive correlation with TIL densities, thus showing its value as predictive biomarkers for ICI response in NEN.

From Immunohistochemistry to New Digital Ecosystems: A State-of-the-Art Biomarker Review for Precision Breast Cancer Medicine

Breast cancers represent complex ecosystem-like networks of malignant cells and their associated microenvironment. Estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) are biomarkers ubiquitous to clinical practice in evaluating prognosis and predicting response to therapy. Recent feats in breast cancer have led to a new digital era, and advanced clinical trials have resulted in a growing number of personalized therapies with corresponding biomarkers. In this state-of-the-art review, we included the latest 10-year updated recommendations for ER, PR, and HER2, along with the most salient information on tumor-infiltrating lymphocytes (TILs), Ki-67, PD-L1, and several prognostic/predictive biomarkers at genomic, transcriptomic, and proteomic levels recently developed for selection and optimization of breast cancer treatment. Looking forward, the multi-omic landscape of the tumor ecosystem could be integrated with computational findings from whole slide images and radiomics in predictive machine learning (ML) models. These are new digital ecosystems on the road to precision breast cancer medicine.

Quantifying Spatial Heterogeneity of Tumor-Infiltrating Lymphocytes to Predict Survival of Individual Cancer Patients

Tumor-infiltrating lymphocytes (TILs), identified on HE-stained histopathological images in the cancer area, are indicators of the adaptive immune response against cancers and play a major role in personalized cancer immunotherapy. Recent works indicate that the spatial organization of TILs may be prognostic of disease-specific survival and recurrence. However, there are a limited number of methods that were proposed and tested in analyses of the spatial structure of TILs. In this work, we evaluated 14 different spatial measures, including the one developed for other omics data, on 10,532 TIL maps from 23 cancer types in terms of reproducibility, uniqueness, and impact on patient survival. For each spatial measure, 16 different scenarios for the definition of prognostic factor were tested. We found no difference in survival prediction when TIL maps were stored as binary images or continuous TIL probability scores. When spatial measures were discretized into a low and high category, a higher correlation with survival was observed. Three measures with the highest cancer prognosis capability were spatial autocorrelation, GLCM M1, and closeness centrality. Most of the tested measures could be further tuned to increase prediction performance.

Development of Training Materials for Pathologists to Provide Machine Learning Validation Data of Tumor-Infiltrating Lymphocytes in Breast Cancer

Simple Summary

The High Throughput Truthing project aims to develop a dataset of stromal tumor-infiltrating lymphocytes (sTILs) density evaluations in hematoxylin and eosin-stained invasive breast cancer specimens fit for a regulatory purpose. After completion of the pilot study, the analysis demonstrated inconsistencies and gaps in the provided training to pathologists. Select regions of interest (ROIs) were reviewed by an expert panel, who provided annotations and commentary on the challenges of the sTILs assessment. We used these annotations to develop a training document and reference standard for new training materials. These materials will train crowd-sourced pathologists to help create an algorithm validation dataset and contribute to sTILs evaluations in clinical practice.

Abstract

The High Throughput Truthing project aims to develop a dataset for validating artificial intelligence and machine learning models (AI/ML) fit for regulatory purposes. The context of this AI/ML validation dataset is the reporting of stromal tumor-infiltrating lymphocytes (sTILs) density evaluations in hematoxylin and eosin-stained invasive breast cancer biopsy specimens. After completing the pilot study, we found notable variability in the sTILs estimates as well as inconsistencies and gaps in the provided training to pathologists. Using the pilot study data and an expert panel, we created custom training materials to improve pathologist annotation quality for the pivotal study. We categorized regions of interest (ROIs) based on their mean sTILs density and selected ROIs with the highest and lowest sTILs variability. In a series of eight one-hour sessions, the expert panel reviewed each ROI and provided verbal density estimates and comments on features that confounded the sTILs evaluation. We aggregated and shaped the comments to identify pitfalls and instructions to improve our training materials. From these selected ROIs, we created a training set and proficiency test set to improve pathologist training with the goal to improve data collection for the pivotal study. We are not exploring AI/ML performance in this paper. Instead, we are creating materials that will train crowd-sourced pathologists to be the reference standard in a pivotal study to create an AI/ML model validation dataset. The issues discussed here are also important for clinicians to understand about the evaluation of sTILs in clinical practice and can provide insight to developers of AI/ML models.