Resolution-agnostic tissue segmentation in whole-slide histopathology images with convolutional neural networks

Computational pathology: A survey review and the way forward

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Histology-Based Radiomics for [18F]FDG PET Identifies Tissue Heterogeneity in Pancreatic Cancer

Radiomics features can reveal hidden patterns in a tumor but usually lack an underlying biologic rationale. In this work, we aimed to investigate whether there is a correlation between radiomics features extracted from [18F]FDG PET images and histologic expression patterns of a glycolytic marker, monocarboxylate transporter-4 (MCT4), in pancreatic cancer. Methods: A cohort of pancreatic ductal adenocarcinoma patients (n = 29) for whom both tumor cross sections and [18F]FDG PET/CT scans were available was used to develop an [18F]FDG PET radiomics signature. By using immunohistochemistry for MCT4, we computed density maps of MCT4 expression and extracted pathomics features. Cluster analysis identified 2 subgroups with distinct MCT4 expression patterns. From corresponding [18F]FDG PET scans, radiomics features that associate with the predefined MCT4 subgroups were identified. Results: Complex heat map visualization showed that the MCT4-high/heterogeneous subgroup was correlating with a higher MCT4 expression level and local variation. This pattern linked to a specific [18F]FDG PET signature, characterized by a higher SUVmean and SUVmax and second-order radiomics features, correlating with local variation. This MCT4-based [18F]FDG PET signature of 7 radiomics features demonstrated prognostic value in an independent cohort of pancreatic cancer patients (n = 71) and identified patients with worse survival. Conclusion: Our cross-modal pipeline allows the development of PET scan signatures based on immunohistochemical analysis of markers of a particular biologic feature, here demonstrated on pancreatic cancer using intratumoral MCT4 expression levels to select [18F]FDG PET radiomics features. This study demonstrated the potential of radiomics scores to noninvasively capture intratumoral marker heterogeneity and identify a subset of pancreatic ductal adenocarcinoma patients with a poor prognosis.

RegWSI: Whole slide image registration using combined deep feature- and intensity-based methods: Winner of the ACROBAT 2023 challenge

BACKGROUND AND OBJECTIVE

The automatic registration of differently stained whole slide images (WSIs) is crucial for improving diagnosis and prognosis by fusing complementary information emerging from different visible structures. It is also useful to quickly transfer annotations between consecutive or restained slides, thus significantly reducing the annotation time and associated costs. Nevertheless, the slide preparation is different for each stain and the tissue undergoes complex and large deformations. Therefore, a robust, efficient, and accurate registration method is highly desired by the scientific community and hospitals specializing in digital pathology.

METHODS

We propose a two-step hybrid method consisting of (i) deep learning- and feature-based initial alignment algorithm, and (ii) intensity-based nonrigid registration using the instance optimization. The proposed method does not require any fine-tuning to a particular dataset and can be used directly for any desired tissue type and stain. The registration time is low, allowing one to perform efficient registration even for large datasets. The method was proposed for the ACROBAT 2023 challenge organized during the MICCAI 2023 conference and scored 1st place. The method is released as open-source software.

RESULTS

The proposed method is evaluated using three open datasets: (i) Automatic Nonrigid Histological Image Registration Dataset (ANHIR), (ii) Automatic Registration of Breast Cancer Tissue Dataset (ACROBAT), and (iii) Hybrid Restained and Consecutive Histological Serial Sections Dataset (HyReCo). The target registration error (TRE) is used as the evaluation metric. We compare the proposed algorithm to other state-of-the-art solutions, showing considerable improvement. Additionally, we perform several ablation studies concerning the resolution used for registration and the initial alignment robustness and stability. The method achieves the most accurate results for the ACROBAT dataset, the cell-level registration accuracy for the restained slides from the HyReCo dataset, and is among the best methods evaluated on the ANHIR dataset.

CONCLUSIONS

The article presents an automatic and robust registration method that outperforms other state-of-the-art solutions. The method does not require any fine-tuning to a particular dataset and can be used out-of-the-box for numerous types of microscopic images. The method is incorporated into the DeeperHistReg framework, allowing others to directly use it to register, transform, and save the WSIs at any desired pyramid level (resolution up to 220k x 220k). We provide free access to the software. The results are fully and easily reproducible. The proposed method is a significant contribution to improving the WSI registration quality, thus advancing the field of digital pathology.

Automatic quantification of tumor-stroma ratio as a prognostic marker for pancreatic cancer

PURPOSE

This study aims to introduce an innovative multi-step pipeline for automatic tumor-stroma ratio (TSR) quantification as a potential prognostic marker for pancreatic cancer, addressing the limitations of existing staging systems and the lack of commonly used prognostic biomarkers.

METHODS

The proposed approach involves a deep-learning-based method for the automatic segmentation of tumor epithelial cells, tumor bulk, and stroma from whole-slide images (WSIs). Models were trained using five-fold cross-validation and evaluated on an independent external test set. TSR was computed based on the segmented components. Additionally, TSR's predictive value for six-month survival on the independent external dataset was assessed.

RESULTS

Median Dice (inter-quartile range (IQR)) of 0.751(0.15) and 0.726(0.25) for tumor epithelium segmentation on internal and external test sets, respectively. Median Dice of 0.76(0.11) and 0.863(0.17) for tumor bulk segmentation on internal and external test sets, respectively. TSR was evaluated as an independent prognostic marker, demonstrating a cross-validation AUC of 0.61±0.12 for predicting six-month survival on the external dataset.

CONCLUSION

Our pipeline for automatic TSR quantification offers promising potential as a prognostic marker for pancreatic cancer. The results underscore the feasibility of computational biomarker discovery in enhancing patient outcome prediction, thus contributing to personalized patient management.

Multi-resolution deep learning characterizes tertiary lymphoid structures and their prognostic relevance in solid tumors

BACKGROUND

Tertiary lymphoid structures (TLSs) are dense accumulations of lymphocytes in inflamed peripheral tissues, including cancer, and are associated with improved survival and response to immunotherapy in various solid tumors. Histological TLS quantification has been proposed as a novel predictive and prognostic biomarker, but lack of standardized methods of TLS characterization hampers assessment of TLS densities across different patients, diseases, and clinical centers.

METHODS

We introduce an approach based on HookNet-TLS, a multi-resolution deep learning model, for automated and unbiased TLS quantification and identification of germinal centers in routine hematoxylin and eosin stained digital pathology slides. We developed HookNet-TLS using n = 1019 manually annotated TCGA slides from clear cell renal cell carcinoma, muscle-invasive bladder cancer, and lung squamous cell carcinoma.

RESULTS

Here we show that HookNet-TLS automates TLS quantification across multiple cancer types achieving human-level performance and demonstrates prognostic associations similar to visual assessment.

CONCLUSIONS

HookNet-TLS has the potential to be used as a tool for objective quantification of TLS in routine H&E digital pathology slides. We make HookNet-TLS publicly available to promote its use in research.

PROACTING: predicting pathological complete response to neoadjuvant chemotherapy in breast cancer from routine diagnostic histopathology biopsies with deep learning

BACKGROUND

Invasive breast cancer patients are increasingly being treated with neoadjuvant chemotherapy; however, only a fraction of the patients respond to it completely. To prevent overtreatment, there is an urgent need for biomarkers to predict treatment response before administering the therapy.

METHODS

In this retrospective study, we developed hypothesis-driven interpretable biomarkers based on deep learning, to predict the pathological complete response (pCR, i.e., the absence of tumor cells in the surgical resection specimens) to neoadjuvant chemotherapy solely using digital pathology H&E images of pre-treatment breast biopsies. Our approach consists of two steps: First, we use deep learning to characterize aspects of the tumor micro-environment by detecting mitoses and segmenting tissue into several morphology compartments including tumor, lymphocytes and stroma. Second, we derive computational biomarkers from the segmentation and detection output to encode slide-level relationships of components of the tumor microenvironment, such as tumor and mitoses, stroma, and tumor infiltrating lymphocytes (TILs).

RESULTS

We developed and evaluated our method on slides from n = 721 patients from three European medical centers with triple-negative and Luminal B breast cancers and performed external independent validation on n = 126 patients from a public dataset. We report the predictive value of the investigated biomarkers for predicting pCR with areas under the receiver operating characteristic curve between 0.66 and 0.88 across the tested cohorts.

CONCLUSION

The proposed computational biomarkers predict pCR, but will require more evaluation and finetuning for clinical application. Our results further corroborate the potential role of deep learning to automate TILs quantification, and their predictive value in breast cancer neoadjuvant treatment planning, along with automated mitoses quantification. We made our method publicly available to extract segmentation-based biomarkers for research purposes.

Artifact Augmentation for Learning-based Quality Control of Whole Slide Images

The acquisition of whole slide images is prone to artifacts that can require human control and re-scanning, both in clinical workflows and in research-oriented settings. Quality control algorithms are a first step to overcome this challenge, as they limit the use of low quality images. Developing quality control systems in histopathology is not straightforward, also due to the limited availability of data related to this topic. We address the problem by proposing a tool to augment data with artifacts. The proposed method seamlessly generates and blends artifacts from an external library to a given histopathology dataset. The datasets augmented by the blended artifacts are then used to train an artifact detection network in a supervised way. We use the YOLOv5 model for the artifact detection with a slightly modified training pipeline. The proposed tool can be extended into a complete framework for the quality assessment of whole slide images.Clinical relevance- The proposed method may be useful for the initial quality screening of whole slide images. Each year, millions of whole slide images are acquired and digitized worldwide. Numerous of them contain artifacts affecting the following AI-oriented analysis. Therefore, a tool operating at the acquisition phase and improving the initial quality assessment is crucial to increase the performance of digital pathology algorithms, e.g., early cancer diagnosis.

Gigapixel end-to-end training using streaming and attention

Current hardware limitations make it impossible to train convolutional neural networks on gigapixel image inputs directly. Recent developments in weakly supervised learning, such as attention-gated multiple instance learning, have shown promising results, but often use multi-stage or patch-wise training strategies risking suboptimal feature extraction, which can negatively impact performance. In this paper, we propose to train a ResNet-34 encoder with an attention-gated classification head in an end-to-end fashion, which we call StreamingCLAM, using a streaming implementation of convolutional layers. This allows us to train end-to-end on 4-gigapixel microscopic images using only slide-level labels. We achieve a mean area under the receiver operating characteristic curve of 0.9757 for metastatic breast cancer detection (CAMELYON16), close to fully supervised approaches using pixel-level annotations. Our model can also detect MYC-gene translocation in histologic slides of diffuse large B-cell lymphoma, achieving a mean area under the ROC curve of 0.8259. Furthermore, we show that our model offers a degree of interpretability through the attention mechanism.

Continual learning strategies for cancer-independent detection of lymph node metastases

Recently, large, high-quality public datasets have led to the development of convolutional neural networks that can detect lymph node metastases of breast cancer at the level of expert pathologists. Many cancers, regardless of the site of origin, can metastasize to lymph nodes. However, collecting and annotating high-volume, high-quality datasets for every cancer type is challenging. In this paper we investigate how to leverage existing high-quality datasets most efficiently in multi-task settings for closely related tasks. Specifically, we will explore different training and domain adaptation strategies, including prevention of catastrophic forgetting, for breast, colon and head-and-neck cancer metastasis detection in lymph nodes. Our results show state-of-the-art performance on colon and head-and-neck cancer metastasis detection tasks. We show the effectiveness of adaptation of networks from one cancer type to another to obtain multi-task metastasis detection networks. Furthermore, we show that leveraging existing high-quality datasets can significantly boost performance on new target tasks and that catastrophic forgetting can be effectively mitigated.Last, we compare different mitigation strategies.

An automatic entropy method to efficiently mask histology whole-slide images

Tissue segmentation of histology whole-slide images (WSI) remains a critical task in automated digital pathology workflows for both accurate disease diagnosis and deep phenotyping for research purposes. This is especially challenging when the tissue structure of biospecimens is relatively porous and heterogeneous, such as for atherosclerotic plaques. In this study, we developed a unique approach called 'EntropyMasker' based on image entropy to tackle the fore- and background segmentation (masking) task in histology WSI. We evaluated our method on 97 high-resolution WSI of human carotid atherosclerotic plaques in the Athero-Express Biobank Study, constituting hematoxylin and eosin and 8 other staining types. Using multiple benchmarking metrics, we compared our method with four widely used segmentation methods: Otsu's method, Adaptive mean, Adaptive Gaussian and slideMask and observed that our method had the highest sensitivity and Jaccard similarity index. We envision EntropyMasker to fill an important gap in WSI preprocessing, machine learning image analysis pipelines, and enable disease phenotyping beyond the field of atherosclerosis.

Predictive uncertainty estimation for out-of-distribution detection in digital pathology

Machine learning model deployment in clinical practice demands real-time risk assessment to identify situations in which the model is uncertain. Once deployed, models should be accurate for classes seen during training while providing informative estimates of uncertainty to flag abnormalities and unseen classes for further analysis. Although recent developments in uncertainty estimation have resulted in an increasing number of methods, a rigorous empirical evaluation of their performance on large-scale digital pathology datasets is lacking. This work provides a benchmark for evaluating prevalent methods on multiple datasets by comparing the uncertainty estimates on both in-distribution and realistic near and far out-of-distribution (OOD) data on a whole-slide level. To this end, we aggregate uncertainty values from patch-based classifiers to whole-slide level uncertainty scores. We show that results found in classical computer vision benchmarks do not always translate to the medical imaging setting. Specifically, we demonstrate that deep ensembles perform best at detecting far-OOD data but can be outperformed on a more challenging near-OOD detection task by multi-head ensembles trained for optimal ensemble diversity. Furthermore, we demonstrate the harmful impact OOD data can have on the performance of deployed machine learning models. Overall, we show that uncertainty estimates can be used to discriminate in-distribution from OOD data with high AUC scores. Still, model deployment might require careful tuning based on prior knowledge of prospective OOD data.

[Digitalization in dermatopathology]

The histomorphological analysis of tissue sections by specially trained dermatopathologists is a central component for making the dermatological diagnosis. It is the foundation for the understanding of clinical aspects, pathophysiology and not least the treatment of skin diseases and is therefore an essential part of modern dermatology. New technological developments in recent years offer a variety of possibilities to digitalize dermatopathology, which could significantly change and even revolutionize the work of dermatopathologists in the coming years; however, like any new development there are limiting factors and open questions that need to be discussed. This article is intended to provide an overview of the current state of the art and to highlight the corresponding opportunities and risks on the road to digital dermatopathology.

Convolutional Neural Networks for the Evaluation of Chronic and Inflammatory Lesions in Kidney Transplant Biopsies

In kidney transplant biopsies, both inflammation and chronic changes are important features that predict long-term graft survival. Quantitative scoring of these features is important for transplant diagnostics and kidney research. However, visual scoring is poorly reproducible and labor intensive. The goal of this study was to investigate the potential of convolutional neural networks (CNNs) to quantify inflammation and chronic features in kidney transplant biopsies. A structure segmentation CNN and a lymphocyte detection CNN were applied on 125 whole-slide image pairs of periodic acid-Schiff- and CD3-stained slides. The CNN results were used to quantify healthy and sclerotic glomeruli, interstitial fibrosis, tubular atrophy, and inflammation within both nonatrophic and atrophic tubuli, and in areas of interstitial fibrosis. The computed tissue features showed high correlation with Banff lesion scores of five pathologists (A.A., A.Dend., J.H.B., J.K., and T.N.). Analyses on a small subset showed a moderate correlation toward higher CD3⁺ cell density within scarred regions and higher CD3⁺ cell count inside atrophic tubuli correlated with long-term change of estimated glomerular filtration rate. The presented CNNs are valid tools to yield objective quantitative information on glomeruli number, fibrotic tissue, and inflammation within scarred and non-scarred kidney parenchyma in a reproducible manner. CNNs have the potential to improve kidney transplant diagnostics and will benefit the community as a novel method to generate surrogate end points for large-scale clinical studies.

H2G-Net: A multi-resolution refinement approach for segmentation of breast cancer region in gigapixel histopathological images

Over the past decades, histopathological cancer diagnostics has become more complex, and the increasing number of biopsies is a challenge for most pathology laboratories. Thus, development of automatic methods for evaluation of histopathological cancer sections would be of value. In this study, we used 624 whole slide images (WSIs) of breast cancer from a Norwegian cohort. We propose a cascaded convolutional neural network design, called H2G-Net, for segmentation of breast cancer region from gigapixel histopathological images. The design involves a detection stage using a patch-wise method, and a refinement stage using a convolutional autoencoder. To validate the design, we conducted an ablation study to assess the impact of selected components in the pipeline on tumor segmentation. Guiding segmentation, using hierarchical sampling and deep heatmap refinement, proved to be beneficial when segmenting the histopathological images. We found a significant improvement when using a refinement network for post-processing the generated tumor segmentation heatmaps. The overall best design achieved a Dice similarity coefficient of 0.933±0.069 on an independent test set of 90 WSIs. The design outperformed single-resolution approaches, such as cluster-guided, patch-wise high-resolution classification using MobileNetV2 (0.872±0.092) and a low-resolution U-Net (0.874±0.128). In addition, the design performed consistently on WSIs across all histological grades and segmentation on a representative × 400 WSI took ~ 58 s, using only the central processing unit. The findings demonstrate the potential of utilizing a refinement network to improve patch-wise predictions. The solution is efficient and does not require overlapping patch inference or ensembling. Furthermore, we showed that deep neural networks can be trained using a random sampling scheme that balances on multiple different labels simultaneously, without the need of storing patches on disk. Future work should involve more efficient patch generation and sampling, as well as improved clustering.

Multiplex Immunofluorescence and the Digital Image Analysis Workflow for Evaluation of the Tumor Immune Environment in Translational Research

A robust understanding of the tumor immune environment has important implications for cancer diagnosis, prognosis, research, and immunotherapy. Traditionally, immunohistochemistry (IHC) has been regarded as the standard method for detecting proteins in situ, but this technique allows for the evaluation of only one cell marker per tissue sample at a time. However, multiplexed imaging technologies enable the multiparametric analysis of a tissue section at the same time. Also, through the curation of specific antibody panels, these technologies enable researchers to study the cell subpopulations within a single immunological cell group. Thus, multiplexed imaging gives investigators the opportunity to better understand tumor cells, immune cells, and the interactions between them. In the multiplexed imaging technology workflow, once the protocol for a tumor immune micro environment study has been defined, histological slides are digitized to produce high-resolution images in which regions of interest are selected for the interrogation of simultaneously expressed immunomarkers (including those co-expressed by the same cell) by using an image analysis software and algorithm. Most currently available image analysis software packages use similar machine learning approaches in which tissue segmentation first defines the different components that make up the regions of interest and cell segmentation, then defines the different parameters, such as the nucleus and cytoplasm, that the software must utilize to segment single cells. Image analysis tools have driven dramatic evolution in the field of digital pathology over the past several decades and provided the data necessary for translational research and the discovery of new therapeutic targets. The next step in the growth of digital pathology is optimization and standardization of the different tasks in cancer research, including image analysis algorithm creation, to increase the amount of data generated and their accuracy in a short time as described herein. The aim of this review is to describe this process, including an image analysis algorithm creation for multiplex immunofluorescence analysis, as an essential part of the optimization and standardization of the different processes in cancer research, to increase the amount of data generated and their accuracy in a short time.

High throughput whole-slide scanning to enable large-scale data repository building

Digital pathology and artificial intelligence (AI) rely on digitization of patient material as a necessary first step. AI development benefits from large sample sizes and diverse cohorts, and therefore efforts to digitize glass slides must meet these needs in an efficient and cost-effective manner. Technical innovation in whole-slide imaging has enabled high throughput slide scanning through the coordinated increase in scanner capacity, speed, and automation. Combining these hardware innovations with automated informatics approaches has enabled more efficient workflows and the opportunity to provide higher quality imaging data using fewer personnel. Here we review several practical considerations for deploying high throughput scanning and we present strategies to increase efficiency with a focus on quality. Finally, we review remaining challenges and issue a call to vendors to innovate in the areas of automation and quality control in order to make high throughput scanning realizable to laboratories with limited resources. This article is protected by copyright. All rights reserved.

Detection of Prostate Cancer in Whole-Slide Images Through End-to-End Training With Image-Level Labels

Prostate cancer is the most prevalent cancer among men in Western countries, with 1.1 million new diagnoses every year. The gold standard for the diagnosis of prostate cancer is a pathologists' evaluation of prostate tissue. To potentially assist pathologists deep / learning / based cancer detection systems have been developed. Many of the state-of-the-art models are patch / based convolutional neural networks, as the use of entire scanned slides is hampered by memory limitations on accelerator cards. Patch-based systems typically require detailed, pixel-level annotations for effective training. However, such annotations are seldom readily available, in contrast to the clinical reports of pathologists, which contain slide-level labels. As such, developing algorithms which do not require manual pixel-wise annotations, but can learn using only the clinical report would be a significant advancement for the field. In this paper, we propose to use a streaming implementation of convolutional layers, to train a modern CNN (ResNet / 34) with 21 million parameters end-to-end on 4712 prostate biopsies. The method enables the use of entire biopsy images at high-resolution directly by reducing the GPU memory requirements by 2.4 TB. We show that modern CNNs, trained using our streaming approach, can extract meaningful features from high-resolution images without additional heuristics, reaching similar performance as state-of-the-art patch-based and multiple-instance learning methods. By circumventing the need for manual annotations, this approach can function as a blueprint for other tasks in histopathological diagnosis. The source code to reproduce the streaming models is available at https://github.com/DIAGNijmegen/ pathology-streaming-pipeline.

Overcoming an Annotation Hurdle: Digitizing Pen Annotations from Whole Slide Images

Background:

The development of artificial intelligence (AI) in pathology frequently relies on digitally annotated whole slide images (WSI). The creation of these annotations – manually drawn by pathologists in digital slide viewers – is time consuming and expensive. At the same time, pathologists routinely annotate glass slides with a pen to outline cancerous regions, for example, for molecular assessment of the tissue. These pen annotations are currently considered artifacts and excluded from computational modeling.

Methods:

We propose a novel method to segment and fill hand-drawn pen annotations and convert them into a digital format to make them accessible for computational models. Our method is implemented in Python as an open source, publicly available software tool.

Results:

Our method is able to extract pen annotations from WSI and save them as annotation masks. On a data set of 319 WSI with pen markers, we validate our algorithm segmenting the annotations with an overall Dice metric of 0.942, Precision of 0.955, and Recall of 0.943. Processing all images takes 15 min in contrast to 5 h manual digital annotation time. Further, the approach is robust against text annotations.

Conclusions:

We envision that our method can take advantage of already pen-annotated slides in scenarios in which the annotations would be helpful for training computational models. We conclude that, considering the large archives of many pathology departments that are currently being digitized, our method will help to collect large numbers of training samples from those data.

Image-to-image translation for automatic ink removal in whole slide images

Purpose: Deep learning models are showing promise in digital pathology to aid diagnoses. Training complex models requires a significant amount and diversity of well-annotated data, typically housed in institutional archives. These slides often contain clinically meaningful markings to indicate regions of interest. If slides are scanned with the ink present, then the downstream model may end up looking for regions with ink before making a classification. If scanned without the markings, the information regarding where the relevant regions are located is lost. A compromise solution is to scan the slide with the annotations present but digitally remove them. Approach: We proposed a straightforward framework to digitally remove ink markings from whole slide images using a conditional generative adversarial network based on Pix2Pix. Results: The peak signal-to-noise ratio increased 30%, structural similarity index increased 20%, and visual information fidelity increased 200% relative to previous methods. Conclusions: When comparing our digital removal of marked images with rescans of clean slides, our method qualitatively and quantitatively exceeds current benchmarks, opening the possibility of using archived clinical samples as resources to fuel the next generation of deep learning models for digital pathology.

Developing image analysis pipelines of whole-slide images: Pre- and post-processing

Deep learning has pushed the scope of digital pathology beyond simple digitization and telemedicine. The incorporation of these algorithms in routine workflow is on the horizon and maybe a disruptive technology, reducing processing time, and increasing detection of anomalies. While the newest computational methods enjoy much of the press, incorporating deep learning into standard laboratory workflow requires many more steps than simply training and testing a model. Image analysis using deep learning methods often requires substantial pre- and post-processing order to improve interpretation and prediction. Similar to any data processing pipeline, images must be prepared for modeling and the resultant predictions need further processing for interpretation. Examples include artifact detection, color normalization, image subsampling or tiling, removal of errant predictions, etc. Once processed, predictions are complicated by image file size - typically several gigabytes when unpacked. This forces images to be tiled, meaning that a series of subsamples from the whole-slide image (WSI) are used in modeling. Herein, we review many of these methods as they pertain to the analysis of biopsy slides and discuss the multitude of unique issues that are part of the analysis of very large images.

Deep computational pathology in breast cancer

Deep Learning (DL) algorithms are a set of techniques that exploit large and/or complex real-world datasets for cross-domain and cross-discipline prediction and classification tasks. DL architectures excel in computer vision tasks, and in particular image processing and interpretation. This has prompted a wave of disruptingly innovative applications in medical imaging, where DL strategies have the potential to vastly outperform human experts. This is particularly relevant in the context of histopathology, where whole slide imaging (WSI) of stained tissue in conjuction with DL algorithms for their interpretation, selection and cancer staging are beginning to play an ever increasing role in supporting human operators in visual assessments. This has the potential to reduce everyday workload as well as to increase precision and reproducibility across observers, centers, staining techniques and even pathologies. In this paper we introduce the most common DL architectures used in image analysis, with a focus on histopathological image analysis in general and in breast histology in particular. We briefly review how, state-of-art DL architectures compare to human performance on across a number of critical tasks such as mitotic count, tubules analysis and nuclear pleomorphism analysis. Also, the development of DL algorithms specialized to pathology images have been enormously fueled by a number of world-wide challenges based on large, multicentric image databases which are now publicly available. In turn, this has allowed most recent efforts to shift more and more towards semi-supervised learning methods, which provide greater flexibility and applicability. We also review all major repositories of manually labelled pathology images in breast cancer and provide an in-depth discussion of the challenges specific to training DL architectures to interpret WSI data, as well as a review of the state-of-the-art methods for interpretation of images generated from immunohistochemical analysis of breast lesions. We finally discuss the future challenges and opportunities which the adoption of DL paradigms is most likely to pose in the field of pathology for breast cancer detection, diagnosis, staging and prognosis. This review is intended as a comprehensive stepping stone into the field of modern computational pathology for a transdisciplinary readership across technical and medical disciplines.