Automated mitotic spindle hotspot counts are highly associated with clinical outcomes in systemically untreated early-stage triple-negative breast cancer

Operable triple-negative breast cancer (TNBC) has a higher risk of recurrence and death compared to other subtypes. Tumor size and nodal status are the primary clinical factors used to guide systemic treatment, while biomarkers of proliferation have not demonstrated value. Recent studies suggest that subsets of TNBC have a favorable prognosis, even without systemic therapy. We evaluated the association of fully automated mitotic spindle hotspot (AMSH) counts with recurrence-free (RFS) and overall survival (OS) in two separate cohorts of patients with early-stage TNBC who did not receive systemic therapy. AMSH counts were obtained from areas with the highest mitotic density in digitized whole slide images processed with a convolutional neural network trained to detect mitoses. In 140 patients from the Mayo Clinic TNBC cohort, AMSH counts were significantly associated with RFS and OS in a multivariable model controlling for nodal status, tumor size, and tumor-infiltrating lymphocytes (TILs) (p < 0.0001). For every 10-point increase in AMSH counts, there was a 16% increase in the risk of an RFS event (HR 1.16, 95% CI 1.08-1.25), and a 7% increase in the risk of death (HR 1.07, 95% CI 1.00-1.14). We corroborated these findings in a separate cohort of systemically untreated TNBC patients from Radboud UMC in the Netherlands. Our findings suggest that AMSH counts offer valuable prognostic information in patients with early-stage TNBC who did not receive systemic therapy, independent of tumor size, nodal status, and TILs. If further validated, AMSH counts could help inform future systemic therapy de-escalation strategies.

Comparing deep learning and pathologist quantification of cell-level PD-L1 expression in non-small cell lung cancer whole-slide images

Programmed death-ligand 1 (PD-L1) expression is currently used in the clinic to assess eligibility for immune-checkpoint inhibitors via the tumor proportion score (TPS), but its efficacy is limited by high interobserver variability. Multiple papers have presented systems for the automatic quantification of TPS, but none report on the task of determining cell-level PD-L1 expression and often reserve their evaluation to a single PD-L1 monoclonal antibody or clinical center. In this paper, we report on a deep learning algorithm for detecting PD-L1 negative and positive tumor cells at a cellular level and evaluate it on a cell-level reference standard established by six readers on a multi-centric, multi PD-L1 assay dataset. This reference standard also provides for the first time a benchmark for computer vision algorithms. In addition, in line with other papers, we also evaluate our algorithm at slide-level by measuring the agreement between the algorithm and six pathologists on TPS quantification. We find a moderately low interobserver agreement at cell-level level (mean reader-reader F1 score = 0.68) which our algorithm sits slightly under (mean reader-AI F1 score = 0.55), especially for cases from the clinical center not included in the training set. Despite this, we find good AI-pathologist agreement on quantifying TPS compared to the interobserver agreement (mean reader-reader Cohen's kappa = 0.54, 95% CI 0.26-0.81, mean reader-AI kappa = 0.49, 95% CI 0.27-0.72). In conclusion, our deep learning algorithm demonstrates promise in detecting PD-L1 expression at a cellular level and exhibits favorable agreement with pathologists in quantifying the tumor proportion score (TPS). We publicly release our models for use via the Grand-Challenge platform.

DEPICTER: Deep representation clustering for histology annotation

Automatic segmentation of histopathology whole-slide images (WSI) usually involves supervised training of deep learning models with pixel-level labels to classify each pixel of the WSI into tissue regions such as benign or cancerous. However, fully supervised segmentation requires large-scale data manually annotated by experts, which can be expensive and time-consuming to obtain. Non-fully supervised methods, ranging from semi-supervised to unsupervised, have been proposed to address this issue and have been successful in WSI segmentation tasks. But these methods have mainly been focused on technical advancements in algorithmic performance rather than on the development of practical tools that could be used by pathologists or researchers in real-world scenarios. In contrast, we present DEPICTER (Deep rEPresentatIon ClusTERing), an interactive segmentation tool for histopathology annotation that produces a patch-wise dense segmentation map at WSI level. The interactive nature of DEPICTER leverages self- and semi-supervised learning approaches to allow the user to participate in the segmentation producing reliable results while reducing the workload. DEPICTER consists of three steps: first, a pretrained model is used to compute embeddings from image patches. Next, the user selects a number of benign and cancerous patches from the multi-resolution image. Finally, guided by the deep representations, label propagation is achieved using our novel seeded iterative clustering method or by directly interacting with the embedding space via feature space gating. We report both real-time interaction results with three pathologists and evaluate the performance on three public cancer classification dataset benchmarks through simulations. The code and demos of DEPICTER are publicly available at https://github.com/eduardchelebian/depicter.

Biological background of colorectal polyps and carcinomas with heterotopic ossification: A national study and literature review

The biological mechanisms and potential clinical impact of heterotopic ossification (HO) in colorectal neoplasms are not fully understood. This study investigates the clinicopathological characteristics of colorectal neoplasms associated with HO and examines the potential role of the bone morphogenetic protein (BMP) pathway in development of HO. An artificial intelligence (AI) based classification of colorectal cancers (CRC) exhibiting HO and their association with consensus molecular subtypes (CMS) is performed. The study included 77 cases via the Dutch nationwide Pathology databank. Immunohistochemistry for BMP2, SMAD4, and Osterix was performed. An AI algorithm assessed the tumour-stroma ratio to approximate the CMS. A literature search yielded 96 case reports, which were analysed and compared with our cases for clinicopathological parameters. HO was more frequently observed in our cohort in traditional serrated adenomas (25%), tubulovillous adenomas (25%) and juvenile polyps (25%), while in the literature it was most often seen in juvenile polyps (38.2%) and inflammatory polyps (29.4%). In both cohorts, carcinomas were mostly conventional (>60%) followed by mucinous and serrated adenocarcinomas. Higher expression of BMP2, SMAD4, and Osterix was observed in tumour and/or stromal cells directly surrounding bone, indicating activation of the BMP pathway. The tumour-stroma analysis appointed >50% of the cases to the mesenchymal subtype (CMS4) (59%). HO has a predilection for serrated and juvenile/inflammatory polyps, mucinous and serrated adenocarcinomas. BMP signalling is activated and seems to play a role in formation of HO in colorectal neoplasms. In line with TGFβ/BMP pathway activation associated with CMS4 CRC, HO seems associated with CMS4.

LYSTO: The Lymphocyte Assessment Hackathon and Benchmark Dataset

We introduce LYSTO, the Lymphocyte Assessment Hackathon, which was held in conjunction with the MICCAI 2019 Conference in Shenzhen (China). The competition required participants to automatically assess the number of lymphocytes, in particular T-cells, in images of colon, breast, and prostate cancer stained with CD3 and CD8 immunohistochemistry. Differently from other challenges setup in medical image analysis, LYSTO participants were solely given a few hours to address this problem. In this paper, we describe the goal and the multi-phase organization of the hackathon; we describe the proposed methods and the on-site results. Additionally, we present post-competition results where we show how the presented methods perform on an independent set of lung cancer slides, which was not part of the initial competition, as well as a comparison on lymphocyte assessment between presented methods and a panel of pathologists. We show that some of the participants were capable to achieve pathologist-level performance at lymphocyte assessment. After the hackathon, LYSTO was left as a lightweight plug-and-play benchmark dataset on grand-challenge website, together with an automatic evaluation platform.

Image-based multiplex immune profiling of cancer tissues: translational implications. A report of the International Immuno-oncology Biomarker Working Group on Breast Cancer

Recent advances in the field of immuno-oncology have brought transformative changes in the management of cancer patients. The immune profile of tumours has been found to have key value in predicting disease prognosis and treatment response in various cancers. Multiplex immunohistochemistry and immunofluorescence have emerged as potent tools for the simultaneous detection of multiple protein biomarkers in a single tissue section, thereby expanding opportunities for molecular and immune profiling while preserving tissue samples. By establishing the phenotype of individual tumour cells when distributed within a mixed cell population, the identification of clinically relevant biomarkers with high-throughput multiplex immunophenotyping of tumour samples has great potential to guide appropriate treatment choices. Moreover, the emergence of novel multi-marker imaging approaches can now provide unprecedented insights into the tumour microenvironment, including the potential interplay between various cell types. However, there are significant challenges to widespread integration of these technologies in daily research and clinical practice. This review addresses the challenges and potential solutions within a structured framework of action from a regulatory and clinical trial perspective. New developments within the field of immunophenotyping using multiplexed tissue imaging platforms and associated digital pathology are also described, with a specific focus on translational implications across different subtypes of cancer. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Endometrial Pipelle Biopsy Computer-Aided Diagnosis: A Feasibility Study

Endometrial biopsies are important in the diagnostic workup of women who present with abnormal uterine bleeding or hereditary risk of endometrial cancer. In general, approximately 10% of all endometrial biopsies demonstrate endometrial (pre)malignancy that requires specific treatment. As the diagnostic evaluation of mostly benign cases results in a substantial workload for pathologists, artificial intelligence (AI)-assisted preselection of biopsies could optimize the workflow. This study aimed to assess the feasibility of AI-assisted diagnosis for endometrial biopsies (endometrial Pipelle biopsy computer-aided diagnosis), trained on daily-practice whole-slide images instead of highly selected images. Endometrial biopsies were classified into 6 clinically relevant categories defined as follows: nonrepresentative, normal, nonneoplastic, hyperplasia without atypia, hyperplasia with atypia, and malignant. The agreement among 15 pathologists, within these classifications, was evaluated in 91 endometrial biopsies. Next, an algorithm (trained on a total of 2819 endometrial biopsies) rated the same 91 cases, and we compared its performance using the pathologist's classification as the reference standard. The interrater reliability among pathologists was moderate with a mean Cohen's kappa of 0.51, whereas for a binary classification into benign vs (pre)malignant, the agreement was substantial with a mean Cohen's kappa of 0.66. The AI algorithm performed slightly worse for the 6 categories with a moderate Cohen's kappa of 0.43 but was comparable for the binary classification with a substantial Cohen's kappa of 0.65. AI-assisted diagnosis of endometrial biopsies was demonstrated to be feasible in discriminating between benign and (pre)malignant endometrial tissues, even when trained on unselected cases. Endometrial premalignancies remain challenging for both pathologists and AI algorithms. Future steps to improve reliability of the diagnosis are needed to achieve a more refined AI-assisted diagnostic solution for endometrial biopsies that covers both premalignant and malignant diagnoses.

Multi-resolution deep learning characterizes tertiary lymphoid structures and their prognostic relevance in solid tumors

BACKGROUND

Tertiary lymphoid structures (TLSs) are dense accumulations of lymphocytes in inflamed peripheral tissues, including cancer, and are associated with improved survival and response to immunotherapy in various solid tumors. Histological TLS quantification has been proposed as a novel predictive and prognostic biomarker, but lack of standardized methods of TLS characterization hampers assessment of TLS densities across different patients, diseases, and clinical centers.

METHODS

We introduce an approach based on HookNet-TLS, a multi-resolution deep learning model, for automated and unbiased TLS quantification and identification of germinal centers in routine hematoxylin and eosin stained digital pathology slides. We developed HookNet-TLS using n = 1019 manually annotated TCGA slides from clear cell renal cell carcinoma, muscle-invasive bladder cancer, and lung squamous cell carcinoma.

RESULTS

Here we show that HookNet-TLS automates TLS quantification across multiple cancer types achieving human-level performance and demonstrates prognostic associations similar to visual assessment.

CONCLUSIONS

HookNet-TLS has the potential to be used as a tool for objective quantification of TLS in routine H&E digital pathology slides. We make HookNet-TLS publicly available to promote its use in research.

The Complexity of Shapes: How the Circularity of Tumor Nodules Affects Prognosis in Colorectal Cancer

The current stratification of tumor nodules in colorectal cancer (CRC) staging is subjective and leads to high interobserver variability. In this study, the objective assessment of the shape of lymph node metastases (LNMs), extranodal extension (ENE), and tumor deposits (TDs) was correlated with outcomes. A test cohort and a validation cohort were included from 2 different institutions. The test cohort consisted of 190 cases of stage III CRC. Slides with LNMs and TDs were annotated and processed using a segmentation algorithm to determine their shape. The complexity ratio was calculated for every shape and correlated with outcomes. A cohort of 160 stage III CRC cases was used to validate findings. TDs showed significantly more complex shapes than LNMs with ENE, which were more complex than LNMs without ENE (P < .001). In the test cohort, patients with the highest sum of complexity ratios had significantly lower disease-free survival (P < .01). When only the nodule with the highest complexity was considered, this effect was even stronger (P < .001). This maximum complexity ratio per patient was identified as an independent prognostic factor in the multivariate analysis (hazard ratio, 2.47; P < .05). The trends in the validation cohort confirmed the results. More complex nodules in stage III CRC were correlated with significantly worse disease-free survival, even if only based on the most complex nodule. These results suggest that more complex nodules reflect more invasive tumor biology. As most of the more complex nodules were diagnosed as TDs, we suggest providing a more prominent role for TDs in the nodal stage and include an objective complexity measure in their definition.

PROACTING: predicting pathological complete response to neoadjuvant chemotherapy in breast cancer from routine diagnostic histopathology biopsies with deep learning

BACKGROUND

Invasive breast cancer patients are increasingly being treated with neoadjuvant chemotherapy; however, only a fraction of the patients respond to it completely. To prevent overtreatment, there is an urgent need for biomarkers to predict treatment response before administering the therapy.

METHODS

In this retrospective study, we developed hypothesis-driven interpretable biomarkers based on deep learning, to predict the pathological complete response (pCR, i.e., the absence of tumor cells in the surgical resection specimens) to neoadjuvant chemotherapy solely using digital pathology H&E images of pre-treatment breast biopsies. Our approach consists of two steps: First, we use deep learning to characterize aspects of the tumor micro-environment by detecting mitoses and segmenting tissue into several morphology compartments including tumor, lymphocytes and stroma. Second, we derive computational biomarkers from the segmentation and detection output to encode slide-level relationships of components of the tumor microenvironment, such as tumor and mitoses, stroma, and tumor infiltrating lymphocytes (TILs).

RESULTS

We developed and evaluated our method on slides from n = 721 patients from three European medical centers with triple-negative and Luminal B breast cancers and performed external independent validation on n = 126 patients from a public dataset. We report the predictive value of the investigated biomarkers for predicting pCR with areas under the receiver operating characteristic curve between 0.66 and 0.88 across the tested cohorts.

CONCLUSION

The proposed computational biomarkers predict pCR, but will require more evaluation and finetuning for clinical application. Our results further corroborate the potential role of deep learning to automate TILs quantification, and their predictive value in breast cancer neoadjuvant treatment planning, along with automated mitoses quantification. We made our method publicly available to extract segmentation-based biomarkers for research purposes.

Fully Automated Tumor Bud Assessment in Hematoxylin and Eosin-Stained Whole Slide Images of Colorectal Cancer

Tumor budding (TB), the presence of single cells or small clusters of up to 4 tumor cells at the invasive front of colorectal cancer (CRC), is a proven risk factor for adverse outcomes. International definitions are necessary to reduce interobserver variability. According to the current international guidelines, hotspots at the invasive front should be counted in hematoxylin and eosin (H&E)-stained slides. This is time-consuming and prone to interobserver variability; therefore, there is a need for computer-aided diagnosis solutions. In this study, we report an artificial intelligence-based method for detecting TB in H&E-stained whole slide images. We propose a fully automated pipeline to identify the tumor border, detect tumor buds, characterize them based on the number of tumor cells, and produce a TB density map to identify the TB hotspot. The method outputs the TB count in the hotspot as a computational biomarker. We show that the proposed automated TB detection workflow performs on par with a panel of 5 pathologists at detecting tumor buds and that the hotspot-based TB count is an independent prognosticator in both the univariate and the multivariate analysis, validated on a cohort of n = 981 patients with CRC. Computer-aided detection of tumor buds based on deep learning can perform on par with expert pathologists for the detection and quantification of tumor buds in H&E-stained CRC histopathology slides, strongly facilitating the introduction of budding as an independent prognosticator in clinical routine and clinical trials.

Modelling digital health data: The ExaMode ontology for computational pathology

Computational pathology can significantly benefit from ontologies to standardize the employed nomenclature and help with knowledge extraction processes for high-quality annotated image datasets. The end goal is to reach a shared model for digital pathology to overcome data variability and integration problems. Indeed, data annotation in such a specific domain is still an unsolved challenge and datasets cannot be steadily reused in diverse contexts due to heterogeneity issues of the adopted labels, multilingualism, and different clinical practices.

Material and methods

This paper presents the ExaMode ontology, modeling the histopathology process by considering 3 key cancer diseases (colon, cervical, and lung tumors) and celiac disease. The ExaMode ontology has been designed bottom-up in an iterative fashion with continuous feedback and validation from pathologists and clinicians. The ontology is organized into 5 semantic areas that defines an ontological template to model any disease of interest in histopathology.

Results

The ExaMode ontology is currently being used as a common semantic layer in: (i) an entity linking tool for the automatic annotation of medical records; (ii) a web-based collaborative annotation tool for histopathology text reports; and (iii) a software platform for building holistic solutions integrating multimodal histopathology data.

Discussion

The ontology ExaMode is a key means to store data in a graph database according to the RDF data model. The creation of an RDF dataset can help develop more accurate algorithms for image analysis, especially in the field of digital pathology. This approach allows for seamless data integration and a unified query access point, from which we can extract relevant clinical insights about the considered diseases using SPARQL queries.

Pitfalls in machine learning-based assessment of tumor-infiltrating lymphocytes in breast cancer: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer

The clinical significance of the tumor-immune interaction in breast cancer is now established, and tumor-infiltrating lymphocytes (TILs) have emerged as predictive and prognostic biomarkers for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2-negative) breast cancer and HER2-positive breast cancer. How computational assessments of TILs might complement manual TIL assessment in trial and daily practices is currently debated. Recent efforts to use machine learning (ML) to automatically evaluate TILs have shown promising results. We review state-of-the-art approaches and identify pitfalls and challenges of automated TIL evaluation by studying the root cause of ML discordances in comparison to manual TIL quantification. We categorize our findings into four main topics: (1) technical slide issues, (2) ML and image analysis aspects, (3) data challenges, and (4) validation issues. The main reason for discordant assessments is the inclusion of false-positive areas or cells identified by performance on certain tissue patterns or design choices in the computational implementation. To aid the adoption of ML for TIL assessment, we provide an in-depth discussion of ML and image analysis, including validation issues that need to be considered before reliable computational reporting of TILs can be incorporated into the trial and routine clinical management of patients with triple-negative breast cancer. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Spatial analyses of immune cell infiltration in cancer: current methods and future directions: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer

Modern histologic imaging platforms coupled with machine learning methods have provided new opportunities to map the spatial distribution of immune cells in the tumor microenvironment. However, there exists no standardized method for describing or analyzing spatial immune cell data, and most reported spatial analyses are rudimentary. In this review, we provide an overview of two approaches for reporting and analyzing spatial data (raster versus vector-based). We then provide a compendium of spatial immune cell metrics that have been reported in the literature, summarizing prognostic associations in the context of a variety of cancers. We conclude by discussing two well-described clinical biomarkers, the breast cancer stromal tumor infiltrating lymphocytes score and the colon cancer Immunoscore, and describe investigative opportunities to improve clinical utility of these spatial biomarkers. © 2023 The Pathological Society of Great Britain and Ireland.

Deep learning for multi-class semantic segmentation enables colorectal cancer detection and classification in digital pathology images

In colorectal cancer (CRC), artificial intelligence (AI) can alleviate the laborious task of characterization and reporting on resected biopsies, including polyps, the numbers of which are increasing as a result of CRC population screening programs ongoing in many countries all around the globe. Here, we present an approach to address two major challenges in the automated assessment of CRC histopathology whole-slide images. We present an AI-based method to segment multiple ([Formula: see text]) tissue compartments in the H &E-stained whole-slide image, which provides a different, more perceptible picture of tissue morphology and composition. We test and compare a panel of state-of-the-art loss functions available for segmentation models, and provide indications about their use in histopathology image segmentation, based on the analysis of (a) a multi-centric cohort of CRC cases from five medical centers in the Netherlands and Germany, and (b) two publicly available datasets on segmentation in CRC. We used the best performing AI model as the basis for a computer-aided diagnosis system that classifies colon biopsies into four main categories that are relevant pathologically. We report the performance of this system on an independent cohort of more than 1000 patients. The results show that with a good segmentation network as a base, a tool can be developed which can support pathologists in the risk stratification of colorectal cancer patients, among other possible uses. We have made the segmentation model available for research use on https://grand-challenge.org/algorithms/colon-tissue-segmentation/ .

Semi-Supervised Learning to Automate Tumor Bud Detection in Cytokeratin-Stained Whole-Slide Images of Colorectal Cancer

Tumor budding is a histopathological biomarker associated with metastases and adverse survival outcomes in colorectal carcinoma (CRC) patients. It is characterized by the presence of single tumor cells or small clusters of cells within the tumor or at the tumor-invasion front. In order to obtain a tumor budding score for a patient, the region with the highest tumor bud density must first be visually identified by a pathologist, after which buds will be counted in the chosen hotspot field. The automation of this process will expectedly increase efficiency and reproducibility. Here, we present a deep learning convolutional neural network model that automates the above procedure. For model training, we used a semi-supervised learning method, to maximize the detection performance despite the limited amount of labeled training data. The model was tested on an independent dataset in which human- and machine-selected hotspots were mapped in relation to each other and manual and machine detected tumor bud numbers in the manually selected fields were compared. We report the results of the proposed method in comparison with visual assessment by pathologists. We show that the automated tumor bud count achieves a prognostic value comparable with visual estimation, while based on an objective and reproducible quantification. We also explore novel metrics to quantify buds such as density and dispersion and report their prognostic value. We have made the model available for research use on the grand-challenge platform.

Abstract P2-11-34: Mitotic spindle hotspot counting using deep learning networks is highly associated with clinical outcomes in patients with early-stage triple-negative breast cancer who did not receive systemic therapy

Background: Triple-negative breast cancers (TNBC) exhibit high rates of recurrence and mortality. However, recent studies suggest that a subset of patients (pts) with early-stage TNBC enriched in tumor-infiltrating lymphocytes (TILs) have excellent clinical outcomes even in the absence of systemic therapy. Additional histological biomarkers that could identify pts for future systemic therapy escalation/de-escalation strategies are of great interest. TNBC are frequently highly proliferative with abundant mitoses. However, classic markers of proliferation (manual mitosis counting and Ki-67) appear to offer no prognostic value. Here, we evaluated the prognostic effects of automated mitotic spindle hotspot (AMSH) counting on RFS in independent cohorts of systemically untreated early-stage TNBC.

Methods: AMSH counting was conducted with a state-of-the-art deep learning algorithm trained on the detection of mitoses within 2 mm2 areas with the highest mitotic density (i.e. hotspots) in digital H&E images. Details of the development, training and validation of the algorithm were published previously [1] in a cohort of unselected TNBC. We obtained AMSH counts in a centrally confirmed TNBC cohort from Mayo Clinic [2] and focused our analysis on pts who received locoregional therapy but no systemic therapy. Using a fractional polynomial analysis with a multivariable proportional hazards regression model, we confirmed the assumption of linearity in the log hazard for the continuous variable AMSH and evaluated whether AMSH counts were prognostic of RFS. We corroborated our findings in an independent cohort of systemically untreated TNBC pts from the Radboud University Medical Center in the Netherlands (Radboud Cohort). Results are reported at a median follow-up of 8.1 and 6.7 years for the Mayo and Netherlands cohorts, respectively.

Results: Among 182 pts with who did not receive systemic therapy in the Mayo Cohort, 140 (77%) with available AMSH counts were included. The mean age was 61 (range: 31-94), 71% were postmenopausal, 67% had tumors ≤ 2cm, and 83% were node-negative. As expected, most tumors were Nottingham grade 3 (84%) and had a high Ki-67 proliferation index (54% with Ki-67 &gt;30%). Most tumors (73%) had stromal TILs ≤ 30%. The median AMSH count was 18 (IQR: 8, 42). AMSH counts were linearly associated with grade and tumor size, with the proportion of pts with grade 3 tumors and size &gt; 2 cm increasing as the AMSH counts increased (p=0.007 and p=0.059, respectively). In a multivariate model controlling for nodal status, tumor size, and stromal TILs, AMSH counts were independently associated with RFS (p&lt; 0.0001). For every 10-point increase in the AMSH count, we observed a 17% increase in the risk of experiencing an RFS event (HR 1.17, 95% CI 1.08-1.26). We corroborated our findings in the Radboud Cohort (n=126). The mean age was 68 (range: 40-96), and 81% were node-negative. While the median AMSH count was 36 (IQR: 16-63), higher than in the Mayo Cohort (p=0.004), the prognostic impact was similar, with a significant association between AMSH count and RFS (p=0.028) in a multivariate model corrected for nodal status, tumor size, and stromal TILs. For every 10-point increase in the AMSH count in the Netherlands cohort, we observed a 9% increase in the risk of experiencing an RFS event (HR 1.09, 95% CI 1.01-1.17). RFS rates according to AMSH counts for both cohorts are shown in the Table.

Conclusions: AMSH counting is a new proliferation biomarker that provides prognostic value independent of nodal status, tumor size, and stromal TILs in systemically untreated early-stage TNBC. Plans are underway to evaluate AMSH counts in additional cohorts of systemically untreated TNBC, and in other disease settings such as prior to neoadjuvant systemic therapy. If validated, this biomarker should be prospectively evaluated as a potential selection biomarker in clinical trials of systemic therapy de-escalation.

References:

1. PMID: 29994086

2. PMID: 28913760

Table RFS according to AMSH counts in the Mayo and Radboud Cohorts

Citation Format: Roberto A. Leon-Ferre, Jodi M. Carter, David Zahrieh, Jason P. Sinnwell, Roberto Salgado, Vera Suman, David Hillman, Judy C. Boughey, Krishna R. Kalari, Fergus J. Couch, James N. Ingle, Maschenka Balkenkohl, Francesco Ciompi, Jeroen van der Laak, Matthew P. Goetz. Mitotic spindle hotspot counting using deep learning networks is highly associated with clinical outcomes in patients with early-stage triple-negative breast cancer who did not receive systemic therapy [abstract]. In: Proceedings of the 2022 San Antonio Breast Cancer Symposium; 2022 Dec 6-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2023;83(5 Suppl):Abstract nr P2-11-34.

Mitosis domain generalization in histopathology images - The MIDOG challenge

The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor proliferation and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate when applied in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MIDOG 2021 challenge was the creation of scanner-agnostic MF detection algorithms. The challenge used a training set of 200 cases, split across four scanning systems. As test set, an additional 100 cases split across four scanning systems, including two previously unseen scanners, were provided. In this paper, we evaluate and compare the approaches that were submitted to the challenge and identify methodological factors contributing to better performance. The winning algorithm yielded an F₁ score of 0.748 (CI95: 0.704-0.781), exceeding the performance of six experts on the same task.

Unleashing the potential of digital pathology data by training computer-aided diagnosis models without human annotations

The digitalization of clinical workflows and the increasing performance of deep learning algorithms are paving the way towards new methods for tackling cancer diagnosis. However, the availability of medical specialists to annotate digitized images and free-text diagnostic reports does not scale with the need for large datasets required to train robust computer-aided diagnosis methods that can target the high variability of clinical cases and data produced. This work proposes and evaluates an approach to eliminate the need for manual annotations to train computer-aided diagnosis tools in digital pathology. The approach includes two components, to automatically extract semantically meaningful concepts from diagnostic reports and use them as weak labels to train convolutional neural networks (CNNs) for histopathology diagnosis. The approach is trained (through 10-fold cross-validation) on 3’769 clinical images and reports, provided by two hospitals and tested on over 11’000 images from private and publicly available datasets. The CNN, trained with automatically generated labels, is compared with the same architecture trained with manual labels. Results show that combining text analysis and end-to-end deep neural networks allows building computer-aided diagnosis tools that reach solid performance (micro-accuracy = 0.908 at image-level) based only on existing clinical data without the need for manual annotations.

Deep learning for fully-automated nuclear pleomorphism scoring in breast cancer

To guide the choice of treatment, every new breast cancer is assessed for aggressiveness (i.e., graded) by an experienced histopathologist. Typically, this tumor grade consists of three components, one of which is the nuclear pleomorphism score (the extent of abnormalities in the overall appearance of tumor nuclei). The degree of nuclear pleomorphism is subjectively classified from 1 to 3, where a score of 1 most closely resembles epithelial cells of normal breast epithelium and 3 shows the greatest abnormalities. Establishing numerical criteria for grading nuclear pleomorphism is challenging, and inter-observer agreement is poor. Therefore, we studied the use of deep learning to develop fully automated nuclear pleomorphism scoring in breast cancer. The reference standard used for training the algorithm consisted of the collective knowledge of an international panel of 10 pathologists on a curated set of regions of interest covering the entire spectrum of tumor morphology in breast cancer. To fully exploit the information provided by the pathologists, a first-of-its-kind deep regression model was trained to yield a continuous scoring rather than limiting the pleomorphism scoring to the standard three-tiered system. Our approach preserves the continuum of nuclear pleomorphism without necessitating a large data set with explicit annotations of tumor nuclei. Once translated to the traditional system, our approach achieves top pathologist-level performance in multiple experiments on regions of interest and whole-slide images, compared to a panel of 10 and 4 pathologists, respectively.

Convolutional Neural Networks for the Evaluation of Chronic and Inflammatory Lesions in Kidney Transplant Biopsies

In kidney transplant biopsies, both inflammation and chronic changes are important features that predict long-term graft survival. Quantitative scoring of these features is important for transplant diagnostics and kidney research. However, visual scoring is poorly reproducible and labor intensive. The goal of this study was to investigate the potential of convolutional neural networks (CNNs) to quantify inflammation and chronic features in kidney transplant biopsies. A structure segmentation CNN and a lymphocyte detection CNN were applied on 125 whole-slide image pairs of periodic acid-Schiff- and CD3-stained slides. The CNN results were used to quantify healthy and sclerotic glomeruli, interstitial fibrosis, tubular atrophy, and inflammation within both nonatrophic and atrophic tubuli, and in areas of interstitial fibrosis. The computed tissue features showed high correlation with Banff lesion scores of five pathologists (A.A., A.Dend., J.H.B., J.K., and T.N.). Analyses on a small subset showed a moderate correlation toward higher CD3⁺ cell density within scarred regions and higher CD3⁺ cell count inside atrophic tubuli correlated with long-term change of estimated glomerular filtration rate. The presented CNNs are valid tools to yield objective quantitative information on glomeruli number, fibrotic tissue, and inflammation within scarred and non-scarred kidney parenchyma in a reproducible manner. CNNs have the potential to improve kidney transplant diagnostics and will benefit the community as a novel method to generate surrogate end points for large-scale clinical studies.

Comparison of three validated PD-L1 immunohistochemical assays in urothelial carcinoma of the bladder: interchangeability and issues related to patient selection

Different programmed cell death-ligand 1 (PD-L1) assays and scoring algorithms are being used in the evaluation of PD-L1 expression for the selection of patients for immunotherapy in specific settings of advanced urothelial carcinoma (UC). In this paper, we sought to investigate three approved assays (Ventana SP142 and SP263, and Dako 22C3) in UC with emphasis on implications for patient selection for atezolizumab/pembrolizumab as the first line of treatment. Tumors from 124 patients with invasive UC of the bladder were analyzed using tissue microarrays (TMA). Serial sections were stained with SP263 and SP142 on Ventana Benchmark Ultra and with 22C3 on Dako Autostainer Link 48. Stains were evaluated independently by two observers and scored using the combined positive score (CPS) and tumor infiltrating immune cells (IC) algorithms. Differences in proportions (DP), overall percent agreement (OPA), positive percent agreement (PPA), negative percent agreement (NPA), and Cohen κ were calculated for all comparable cases. Good overall concordance in analytic performance was observed for 22C3 and SP263 with both scoring algorithms; specifically, the highest OPA was observed between 22C3 and SP263 (89.6%) when using CPS. On the other hand, SP142 consistently showed lower positivity rates with high differences in proportions (DP) compared with 22C3 and SP263 with both CPS and IC, and with a low PPA, especially when using the CPS algorithm. In conclusion, 22C3 and SP263 assays show comparable analytical performance while SP142 shows divergent staining results, with important implications for the selection of patients for both pembrolizumab and atezolizumab.

A Decade of GigaScience: The Challenges of Gigapixel Pathology Images

In the last decade, the field of computational pathology has advanced at a rapid pace because of the availability of deep neural networks, which achieved their first successes in computer vision tasks in 2012. An important driver for the progress of the field were public competitions, so called 'Grand Challenges', in which increasingly large data sets were offered to the public to solve clinically relevant tasks. Going from the first Pathology challenges, which had data obtained from 23 patients, to current challenges sharing data of thousands of patients, performance of developed deep learning solutions has reached (and sometimes surpassed) the level of experienced pathologists for specific tasks. We expect future challenges to broaden the horizon, for instance by combining data from radiology, pathology and tumor genetics, and to extract prognostic and predictive information independent of currently used grading schemes.

Program death ligand-1 immunocytochemistry in lung cancer cytological samples: A systematic review

In this era of personalized medicine, targeted immunotherapies like immune checkpoint inhibitors (ICI) blocking the programmed death‐1 (PD‐1)/program death ligand‐1 (PD‐L1) axis have become an integral part of treating advanced stage non‐small cell lung carcinoma (NSCLC) and many other cancer types. Multiple monoclonal antibodies are available commercially to detect PD‐L1 expression in tumor cells by immunohistochemistry (IHC). As most clinical trials initially required tumor biopsy for PD‐L1 detection by IHC, many of the currently available PD‐1/PD‐L1 assays have been developed and validated on formalin fixed tissue specimens. The majority (>50%) of lung cancer cases do not have a surgical biopsy or resection specimen available for ancillary testing and instead must rely primarily on fine needle aspiration biopsy specimens for diagnosis, staging and ancillary tests. Review of the literature shows multiple studies exploring the feasibility of PD‐L1 IHC on cytological samples. In addition, there are studies addressing various aspects of IHC validation on cytology preparations including pre‐analytical (e.g., different fixatives), analytical (e.g., antibody clone, staining platforms, inter and intra‐observer agreement, cytology‐histology concordance) and post‐analytical (e.g., clinical outcome) issues. Although promising results in this field have emerged utilizing cytology samples, many important questions still need to be addressed. This review summarizes the literature of PD‐L1 IHC in lung cytology specimens and provides practical tips for optimizing analysis.

Ultra-high-frequency ultrasound and machine learning approaches for the differential diagnosis of melanocytic lesions

Malignant melanoma (MM) is one of the most dangerous skin cancers. The aim of this study was to present a potential new method for the differential diagnosis of MM from melanocytic naevi (MN). We examined 20 MM and 19 MN with a new ultra-high-frequency ultrasound (UHFUS) equipped with a 70 MHz linear probe. Ultrasonographic images were processed for calculating 8 morphological parameters (area, perimeter, circularity, area ratio, standard deviation of normalized radial range, roughness index, overlap ratio and normalized residual mean square value) and 122 texture parameters. Colour Doppler images were used to evaluate the vascularization. Features reduction was implemented by means of principal component analysis (PCA), and 23 classification algorithms were tested on the reduced features using histological response as ground-truth. Best results were obtained using only the first component of the PCA and the weighted k-nearest neighbour classifier; this combination led to an accuracy of 76.9%, area under the ROC curve of 83%, sensitivity of 84% and specificity of 70%. The histological analysis still remains the gold-standard, but the UHFUS images processing using a machine learning approach could represent a new non-invasive approach.

A Partial Label-Based Machine Learning Approach For Cervical Whole-Slide Image Classification: The Winning TissueNet Solution. Annu Int Conf IEEE Eng Med Biol Soc 2021;2021:2127-2131. [PMID: 34891709 DOI: 10.1109/embc46164.2021.9631009]

Cervical cancer is the fourth most common cancer in women worldwide. To determine early treatment for patients, it is critical to accurately classify the cervical intraepithelial lesion status based on a microscopic biopsy. Lesion classification is a 4-class problem, with biopsies being designated as benign or increasingly malignant as class 1-3, with 3 being invasive cancer. Unfortunately, traditional biopsy analysis by a pathologist is time-consuming and subject to intra- and inter-observer variability. For this reason, it is of interest to develop automatic analysis pipelines to classify lesion status directly from a digitalized whole slide image (WSI). The recent TissueNet Challenge was organized to find the best automatic detection pipeline for this task, using a dataset of 1015 annotated WSI slides. In this work, we present our winning end-to-end solution for cervical slide classification composed of a two-step classification model: First, we classify individual slide patches using an ensemble CNN, followed by an SVM-based slide classification using statistical features of the aggregated patch-level predictions. Importantly, we present the key innovation of our approach, which is a novel partial label-based loss function that allows us to supplement the supervised WSI patch annotations with weakly supervised patches based on the WSI class. This led to us not requiring additional expert tissue annotation, while still reaching the winning score of 94.7%. Our approach is a step towards the clinical inclusion of automatic pipelines for cervical cancer treatment planning.Clinical relevance- The explanation of the winning Tis-sueNet AI algorithm for automated cervical cancer classification, which may provide insights for the next generation of computer assisted tools in digital pathology.

Multi_Scale_Tools: A Python Library to Exploit Multi-Scale Whole Slide Images

Algorithms proposed in computational pathology can allow to automatically analyze digitized tissue samples of histopathological images to help diagnosing diseases. Tissue samples are scanned at a high-resolution and usually saved as images with several magnification levels, namely whole slide images (WSIs). Convolutional neural networks (CNNs) represent the state-of-the-art computer vision methods targeting the analysis of histopathology images, aiming for detection, classification and segmentation. However, the development of CNNs that work with multi-scale images such as WSIs is still an open challenge. The image characteristics and the CNN properties impose architecture designs that are not trivial. Therefore, single scale CNN architectures are still often used. This paper presents Multi_Scale_Tools, a library aiming to facilitate exploiting the multi-scale structure of WSIs. Multi_Scale_Tools currently include four components: a pre-processing component, a scale detector, a multi-scale CNN for classification and a multi-scale CNN for segmentation of the images. The pre-processing component includes methods to extract patches at several magnification levels. The scale detector allows to identify the magnification level of images that do not contain this information, such as images from the scientific literature. The multi-scale CNNs are trained combining features and predictions that originate from different magnification levels. The components are developed using private datasets, including colon and breast cancer tissue samples. They are tested on private and public external data sources, such as The Cancer Genome Atlas (TCGA). The results of the library demonstrate its effectiveness and applicability. The scale detector accurately predicts multiple levels of image magnification and generalizes well to independent external data. The multi-scale CNNs outperform the single-magnification CNN for both classification and segmentation tasks. The code is developed in Python and it will be made publicly available upon publication. It aims to be easy to use and easy to be improved with additional functions.

Impact of PD-L1 and PD-1 Expression on the Prognostic Significance of CD8+ Tumor-Infiltrating Lymphocytes in Non-Small Cell Lung Cancer

The immune infiltrate within tumors has proved to be very powerful in the prognostic stratification of patients and much attention is also being paid towards its predictive value. In this work we therefore aimed at clarifying the significance and impact of PD-L1 and PD-1 expression on the prognostic value of CD8⁺ tumor infiltrating lymphocytes (TILs) in a cohort of consecutive patients with primary resected non-small cell lung cancer (NSCLC). Tissue microarrays (TMA) were built using one representative formalin fixed paraffin embedded block for every case, with 5 cores for each block. TMA sections were stained with PD-L1 (clone SP263), PD-1 (clone NAT105) and CD8 (clone SP57). Number of CD8⁺ cells per mm² were automatically counted; median, 25^th and 75^th percentiles of CD8⁺ cells were used as threshold for statistical clinical outcome analysis and evaluated in patients subgroups defined by expression of PD-L1 and PD-1 within tumors. We found an overall strong prognostic value of CD8⁺ cells in our cohort of 314 resected NSCLC, especially in PD-L1 negative tumors lacking PD-1⁺ TILs, and demonstrated that in PD-L1 positive tumors a higher density of CD8⁺ lymphocytes is necessary to improve the prognosis. Our data strengthen the concept of the importance of the assessment and quantification of the immune contexture in cancer and, similarly to what has been carried on in colorectal cancer, promote the efforts for the establishment of an Immunoscore for NSCLC for prognostic and possibly predictive purposes.

PD-1/PD-L1 in Cancer: Pathophysiological, Diagnostic and Therapeutic Aspects

Immune evasion is a key strategy adopted by tumor cells to escape the immune system while promoting their survival and metastatic spreading. Indeed, several mechanisms have been developed by tumors to inhibit immune responses. PD-1 is a cell surface inhibitory receptor, which plays a major physiological role in the maintenance of peripheral tolerance. In pathological conditions, activation of the PD-1/PD-Ls signaling pathway may block immune cell activation, a mechanism exploited by tumor cells to evade the antitumor immune control. Targeting the PD-1/PD-L1 axis has represented a major breakthrough in cancer treatment. Indeed, the success of PD-1 blockade immunotherapies represents an unprecedented success in the treatment of different cancer types. To improve the therapeutic efficacy, a deeper understanding of the mechanisms regulating PD-1 expression and signaling in the tumor context is required. We provide an overview of the current knowledge of PD-1 expression on both tumor-infiltrating T and NK cells, summarizing the recent evidence on the stimuli regulating its expression. We also highlight perspectives and limitations of the role of PD-L1 expression as a predictive marker, discuss well-established and novel potential approaches to improve patient selection and clinical outcome and summarize current indications for anti-PD1/PD-L1 immunotherapy.

Deep learning in histopathology: the path to the clinic

Machine learning techniques have great potential to improve medical diagnostics, offering ways to improve accuracy, reproducibility and speed, and to ease workloads for clinicians. In the field of histopathology, deep learning algorithms have been developed that perform similarly to trained pathologists for tasks such as tumor detection and grading. However, despite these promising results, very few algorithms have reached clinical implementation, challenging the balance between hope and hype for these new techniques. This Review provides an overview of the current state of the field, as well as describing the challenges that still need to be addressed before artificial intelligence in histopathology can achieve clinical value.

Optimized tumour infiltrating lymphocyte assessment for triple negative breast cancer prognostics

The tumour microenvironment has been shown to be a valuable source of prognostic information for different cancer types. This holds in particular for triple negative breast cancer (TNBC), a breast cancer subtype for which currently no prognostic biomarkers are established. Although different methods to assess tumour infiltrating lymphocytes (TILs) have been published, it remains unclear which method (marker, region) yields the most optimal prognostic information. In addition, to date, no objective TILs assessment methods are available. For this proof of concept study, a subset of our previously described TNBC cohort (n = 94) was stained for CD3, CD8 and FOXP3 using multiplex immunohistochemistry and subsequently imaged by a multispectral imaging system. Advanced whole-slide image analysis algorithms, including convolutional neural networks (CNN) were used to register unmixed multispectral images and corresponding H&E sections, to segment the different tissue compartments (tumour, stroma) and to detect all individual positive lymphocytes. Densities of positive lymphocytes were analysed in different regions within the tumour and its neighbouring environment and correlated to relapse free survival (RFS) and overall survival (OS). We found that for all TILs markers the presence of a high density of positive cells correlated with an improved survival. None of the TILs markers was superior to the others. The results of TILs assessment in the various regions did not show marked differences between each other. The negative correlation between TILs and survival in our cohort are in line with previous studies. Our results provide directions for optimizing TILs assessment methodology.

Neural Image Compression for Gigapixel Histopathology Image Analysis

We propose Neural Image Compression (NIC), a two-step method to build convolutional neural networks for gigapixel image analysis solely using weak image-level labels. First, gigapixel images are compressed using a neural network trained in an unsupervised fashion, retaining high-level information while suppressing pixel-level noise. Second, a convolutional neural network (CNN) is trained on these compressed image representations to predict image-level labels, avoiding the need for fine-grained manual annotations. We compared several encoding strategies, namely reconstruction error minimization, contrastive training and adversarial feature learning, and evaluated NIC on a synthetic task and two public histopathology datasets. We found that NIC can exploit visual cues associated with image-level labels successfully, integrating both global and local visual information. Furthermore, we visualized the regions of the input gigapixel images where the CNN attended to, and confirmed that they overlapped with annotations from human experts.

Pitfalls in assessing stromal tumor infiltrating lymphocytes (sTILs) in breast cancer

Stromal tumor-infiltrating lymphocytes (sTILs) are important prognostic and predictive biomarkers in triple-negative (TNBC) and HER2-positive breast cancer. Incorporating sTILs into clinical practice necessitates reproducible assessment. Previously developed standardized scoring guidelines have been widely embraced by the clinical and research communities. We evaluated sources of variability in sTIL assessment by pathologists in three previous sTIL ring studies. We identify common challenges and evaluate impact of discrepancies on outcome estimates in early TNBC using a newly-developed prognostic tool. Discordant sTIL assessment is driven by heterogeneity in lymphocyte distribution. Additional factors include: technical slide-related issues; scoring outside the tumor boundary; tumors with minimal assessable stroma; including lymphocytes associated with other structures; and including other inflammatory cells. Small variations in sTIL assessment modestly alter risk estimation in early TNBC but have the potential to affect treatment selection if cutpoints are employed. Scoring and averaging multiple areas, as well as use of reference images, improve consistency of sTIL evaluation. Moreover, to assist in avoiding the pitfalls identified in this analysis, we developed an educational resource available at www.tilsinbreastcancer.org/pitfalls.

Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group

Assessment of tumor-infiltrating lymphocytes (TILs) is increasingly recognized as an integral part of the prognostic workflow in triple-negative (TNBC) and HER2-positive breast cancer, as well as many other solid tumors. This recognition has come about thanks to standardized visual reporting guidelines, which helped to reduce inter-reader variability. Now, there are ripe opportunities to employ computational methods that extract spatio-morphologic predictive features, enabling computer-aided diagnostics. We detail the benefits of computational TILs assessment, the readiness of TILs scoring for computational assessment, and outline considerations for overcoming key barriers to clinical translation in this arena. Specifically, we discuss: 1. ensuring computational workflows closely capture visual guidelines and standards; 2. challenges and thoughts standards for assessment of algorithms including training, preanalytical, analytical, and clinical validation; 3. perspectives on how to realize the potential of machine learning models and to overcome the perceptual and practical limits of visual scoring.

Assessment of individual tumor buds using keratin immunohistochemistry: moderate interobserver agreement suggests a role for machine learning

Tumor budding is a promising and cost-effective biomarker with strong prognostic value in colorectal cancer. However, challenges related to interobserver variability persist. Such variability may be reduced by immunohistochemistry and computer-aided tumor bud selection. Development of computer algorithms for this purpose requires unequivocal examples of individual tumor buds. As such, we undertook a large-scale, international, and digital observer study on individual tumor bud assessment. From a pool of 46 colorectal cancer cases with tumor budding, 3000 tumor bud candidates were selected, largely based on digital image analysis algorithms. For each candidate bud, an image patch (size 256 × 256 µm) was extracted from a pan cytokeratin-stained whole-slide image. Members of an International Tumor Budding Consortium (n = 7) were asked to categorize each candidate as either (1) tumor bud, (2) poorly differentiated cluster, or (3) neither, based on current definitions. Agreement was assessed with Cohen's and Fleiss Kappa statistics. Fleiss Kappa showed moderate overall agreement between observers (0.42 and 0.51), while Cohen's Kappas ranged from 0.25 to 0.63. Complete agreement by all seven observers was present for only 34% of the 3000 tumor bud candidates, while 59% of the candidates were agreed on by at least five of the seven observers. Despite reports of moderate-to-substantial agreement with respect to tumor budding grade, agreement with respect to individual pan cytokeratin-stained tumor buds is moderate at most. A machine learning approach may prove especially useful for a more robust assessment of individual tumor buds.

Learning to detect lymphocytes in immunohistochemistry with deep learning

The immune system is of critical importance in the development of cancer. The evasion of destruction by the immune system is one of the emerging hallmarks of cancer. We have built a dataset of 171,166 manually annotated CD3⁺ and CD8⁺ cells, which we used to train deep learning algorithms for automatic detection of lymphocytes in histopathology images to better quantify immune response. Moreover, we investigate the effectiveness of four deep learning based methods when different subcompartments of the whole-slide image are considered: normal tissue areas, areas with immune cell clusters, and areas containing artifacts. We have compared the proposed methods in breast, colon and prostate cancer tissue slides collected from nine different medical centers. Finally, we report the results of an observer study on lymphocyte quantification, which involved four pathologists from different medical centers, and compare their performance with the automatic detection. The results give insights on the applicability of the proposed methods for clinical use. U-Net obtained the highest performance with an F1-score of 0.78 and the highest agreement with manual evaluation (κ=0.72), whereas the average pathologists agreement with reference standard was κ=0.64. The test set and the automatic evaluation procedure are publicly available at lyon19.grand-challenge.org.

Assessment of intracoronary stent location and extension in intravascular ultrasound sequences

PURPOSE

An intraluminal coronary stent is a metal scaffold deployed in a stenotic artery during percutaneous coronary intervention (PCI). In order to have an effective deployment, a stent should be optimally placed with regard to anatomical structures such as bifurcations and stenoses. Intravascular ultrasound (IVUS) is a catheter-based imaging technique generally used for PCI guiding and assessing the correct placement of the stent. A novel approach that automatically detects the boundaries and the position of the stent along the IVUS pullback is presented. Such a technique aims at optimizing the stent deployment.

METHODS

The method requires the identification of the stable frames of the sequence and the reliable detection of stent struts. Using these data, a measure of likelihood for a frame to contain a stent is computed. Then, a robust binary representation of the presence of the stent in the pullback is obtained applying an iterative and multiscale quantization of the signal to symbols using the Symbolic Aggregate approXimation algorithm.

RESULTS

The technique was extensively validated on a set of 103 IVUS of sequences of in vivo coronary arteries containing metallic and bioabsorbable stents acquired through an international multicentric collaboration across five clinical centers. The method was able to detect the stent position with an overall F-measure of 86.4%, a Jaccard index score of 75% and a mean distance of 2.5 mm from manually annotated stent boundaries, and in bioabsorbable stents with an overall F-measure of 88.6%, a Jaccard score of 77.7 and a mean distance of 1.5 mm from manually annotated stent boundaries. Additionally, a map indicating the distance between the lumen and the stent along the pullback is created in order to show the angular sectors of the sequence in which the malapposition is present.

CONCLUSIONS

Results obtained comparing the automatic results vs the manual annotation of two observers shows that the method approaches the interobserver variability. Similar performances are obtained on both metallic and bioabsorbable stents, showing the flexibility and robustness of the method.

Whole-Slide Mitosis Detection in H&E Breast Histology Using PHH3 as a Reference to Train Distilled Stain-Invariant Convolutional Networks

Manual counting of mitotic tumor cells in tissue sections constitutes one of the strongest prognostic markers for breast cancer. This procedure, however, is time-consuming and error-prone. We developed a method to automatically detect mitotic figures in breast cancer tissue sections based on convolutional neural networks (CNNs). Application of CNNs to hematoxylin and eosin (H&E) stained histological tissue sections is hampered by noisy and expensive reference standards established by pathologists, lack of generalization due to staining variation across laboratories, and high computational requirements needed to process gigapixel whole-slide images (WSIs). In this paper, we present a method to train and evaluate CNNs to specifically solve these issues in the context of mitosis detection in breast cancer WSIs. First, by combining image analysis of mitotic activity in phosphohistone-H3 restained slides and registration, we built a reference standard for mitosis detection in entire H&E WSIs requiring minimal manual annotation effort. Second, we designed a data augmentation strategy that creates diverse and realistic H&E stain variations by modifying H&E color channels directly. Using it during training combined with network ensembling resulted in a stain invariant mitosis detector. Third, we applied knowledge distillation to reduce the computational requirements of the mitosis detection ensemble with a negligible loss of performance. The system was trained in a single-center cohort and evaluated in an independent multicenter cohort from the cancer genome atlas on the three tasks of the tumor proliferation assessment challenge. We obtained a performance within the top three best methods for most of the tasks of the challenge.

Long-Term Active Surveillance of Screening Detected Subsolid Nodules is a Safe Strategy to Reduce Overtreatment

INTRODUCTION

Lung cancer presenting as subsolid nodule (SSN) can show slow growth, hence treating SSN is controversial. Our aim was to determine the long-term outcome of subjects with unresected SSNs in lung cancer screening.

METHODS

Since 2005, the Multicenter Italian Lung Detection (MILD) screening trial implemented active surveillance for persistent SSN, as opposed to early resection. Presence of SSNs was related to diagnosis of cancer at the site of SSN, elsewhere in the lung, or in the body. The risk of overall mortality and lung cancer mortality was tested by Cox proportional hazards model.

RESULTS

SSNs were found in 16.9% (389 of 2303) of screenees. During 9.3 ± 1.2 years of follow-up, the hazard ratio of lung cancer diagnosis in subjects with SSN was 6.77 (95% confidence interval: 3.39-13.54), with 73% (22 of 30) of cancers not arising from SSN (median time to diagnosis 52 months from SSN). Lung cancer-specific mortality in subjects with SSN was significantly increased (hazard ratio = 3.80; 95% confidence interval: 1.24-11.65) compared to subjects without lung nodules. Lung cancer arising from SSN did not lead to death within the follow-up period.

CONCLUSIONS

Subjects with SSN in the MILD cohort showed a high risk of developing lung cancer elsewhere in the lung, with only a minority of cases arising from SSN, and never representing the cause of death. These results show the safety of active surveillance for conservative management of SSN until signs of solid component growth and the need for prolonged follow-up because of high risk of other cancers.

Visual discrimination of screen-detected persistent from transient subsolid nodules: An observer study

Purpose

To evaluate whether, and to which extent, experienced radiologists are able to visually correctly differentiate transient from persistent subsolid nodules from a single CT examination alone and to determine CT morphological features to make this differentiation.

Materials and methods

We selected 86 transient and 135 persistent subsolid nodules from the National Lung Screening Trial (NLST) database. Four experienced radiologists visually assessed a predefined list of morphological features and gave a final judgment on a continuous scale (0–100). To assess observer performance, area under the receiver operating characteristic (ROC) curve was calculated. Statistical differences of morphological features between transient and persistent lesions were calculated using Chi-square. Inter-observer agreement of morphological features was evaluated by percentage agreement.

Results

Forty-nine lesions were excluded by at least 2 observers, leaving 172 lesions for analysis. On average observers were able to differentiate transient from persistent subsolid nodules ≥ 10 mm with an area under the curve of 0.75 (95% CI 0.67–0.82). Nodule type, lesion margin, presence of a well-defined border, and pleural retraction showed significant differences between transient and persistent lesions in two observers. Average pair-wise percentage agreement for these features was 81%, 64%, 47% and 89% respectively. Agreement for other morphological features varied from 53% to 95%.

Conclusion

The visual capacity of experienced radiologists to differentiate persistent and transient subsolid nodules is moderate in subsolid nodules larger than 10 mm. Performance of the visual assessment of CT morphology alone is not sufficient to generally abandon a short-term follow-up for subsolid nodules.

Automatic segmentation of the solid core and enclosed vessels in subsolid pulmonary nodules

Subsolid pulmonary nodules are commonly encountered in lung cancer screening and clinical routine. Compared to other nodule types, subsolid nodules are associated with a higher malignancy probability for which the size and mass of the nodule and solid core are important indicators. However, reliably measuring these characteristics on computed tomography (CT) can be hampered by the presence of vessels encompassed by the nodule, since vessels have similar CT attenuation as solid cores. This can affect treatment decisions and patient management. We present a method based on voxel classification to automatically identify vessels and solid cores in given subsolid nodules on CT. Three experts validated our method on 170 screen-detected subsolid nodules from the Multicentric Italian Lung Disease trial. The agreement between the proposed method and the observers was substantial for vessel detection and moderate for solid core detection, which was similar to the inter-observer agreement. We found a relatively high variability in the inter-observer agreement and low method-observer agreements for delineating the borders of vessels and solid cores, illustrating the difficulty of this task. However, 92.4% of the proposed vessel and 80.6% of the proposed solid core segmentations were labeled as usable in clinical practice by the majority of experts.