Computer-assisted mitotic count using a deep learning–based algorithm improves interobserver reproducibility and accuracy

Automated Nuclear Morphometry: A Deep Learning Approach for Prognostication in Canine Pulmonary Carcinoma to Enhance Reproducibility

The integration of deep learning-based tools into diagnostic workflows is increasingly prevalent due to their efficiency and reproducibility in various settings. We investigated the utility of automated nuclear morphometry for assessing nuclear pleomorphism (NP), a criterion of malignancy in the current grading system in canine pulmonary carcinoma (cPC), and its prognostic implications. We developed a deep learning-based algorithm for evaluating NP (variation in size, i.e., anisokaryosis and/or shape) using a segmentation model. Its performance was evaluated on 46 cPC cases with comprehensive follow-up data regarding its accuracy in nuclear segmentation and its prognostic ability. Its assessment of NP was compared to manual morphometry and established prognostic tests (pathologists' NP estimates (n = 11), mitotic count, histological grading, and TNM-stage). The standard deviation (SD) of the nuclear area, indicative of anisokaryosis, exhibited good discriminatory ability for tumor-specific survival, with an area under the curve (AUC) of 0.80 and a hazard ratio (HR) of 3.38. The algorithm achieved values comparable to manual morphometry. In contrast, the pathologists' estimates of anisokaryosis resulted in HR values ranging from 0.86 to 34.8, with slight inter-observer reproducibility (k = 0.204). Other conventional tests had no significant prognostic value in our study cohort. Fully automated morphometry promises a time-efficient and reproducible assessment of NP with a high prognostic value. Further refinement of the algorithm, particularly to address undersegmentation, and application to a larger study population are required.

Role of artificial intelligence in colorectal cancer

The sphere of artificial intelligence (AI) is ever expanding. Applications for clinical practice have been emerging over recent years. Although its uptake has been most prominent in endoscopy, this represents only one aspect of holistic patient care. There are a multitude of other potential avenues in which gastrointestinal care may be involved. We aim to review the role of AI in colorectal cancer as a whole. We performed broad scoping and focused searches of the applications of AI in the field of colorectal cancer. All trials including qualitative research were included from the year 2000 onwards. Studies were grouped into pre-operative, intra-operative and post-operative aspects. Pre-operatively, the major use is with endoscopic recognition. Colonoscopy has embraced the use for human derived classifications such as Narrow-band Imaging International Colorectal Endoscopic, Japan Narrow-band Imaging Expert Team, Paris and Kudo. However, novel detection and diagnostic methods have arisen from advances in AI classification. Intra-operatively, adjuncts such as image enhanced identification of structures and assessment of perfusion have led to improvements in clinical outcomes. Post-operatively, monitoring and surveillance have taken strides with potential socioeconomic and environmental savings. The uses of AI within the umbrella of colorectal surgery are multiple. We have identified existing technologies which are already augmenting cancer care. The future applications are exciting and could at least match, if not surpass human standards.

Mitotic activity: A systematic literature review of the assessment methodology and prognostic value in canine tumors

One of the most relevant prognostic indices for tumors is cellular proliferation, which is most commonly measured by the mitotic activity in routine tumor sections. The goal of this systematic review was to analyze the methods and prognostic relevance of histologically measuring mitotic activity that have been reported for canine tumors in the literature. A total of 137 articles that correlated the mitotic activity in canine tumors with patient outcome were identified through a systematic (PubMed and Scopus) and nonsystematic (Google Scholar) literature search and eligibility screening process. Mitotic activity methods encompassed the mitotic count (MC, number of mitotic figures per tumor area) in 126 studies, presumably the MC (method not specified) in 6 studies, and the mitotic index (MI, number of mitotic figures per number of tumor cells) in 5 studies. A particularly high risk of bias was identified based on the available details of the MC methods and statistical analyses, which often did not quantify the prognostic discriminative ability of the MC and only reported P values. A significant association of the MC with survival was found in 72 of 109 (66%) studies. However, survival was evaluated by at least 3 studies in only 7 tumor types/groups, of which a prognostic relevance is apparent for mast cell tumors of the skin, cutaneous melanoma, and soft tissue tumor of the skin and subcutis. None of the studies using the MI found a prognostic relevance. This review highlights the need for more studies with standardized methods and appropriate analysis of the discriminative ability to prove the prognostic value of the MC and MI in various tumor types. Future studies are needed to evaluate the influence of the performance of individual pathologists on the appropriateness of prognostic thresholds and investigate methods to improve interobserver reproducibility.

Mitotic activity: A systematic literature review of the assessment methodology and prognostic value in feline tumors

Increased proliferation is a driver of tumorigenesis, and quantification of mitotic activity is a standard task for prognostication. This systematic review is an analysis of all available references on mitotic activity in feline tumors to provide an overview of the assessment methods and prognostic value. A systematic literature search in PubMed and Scopus and a nonsystematic search in Google Scholar were conducted. All articles on feline tumors that correlated mitotic activity with patient outcome were identified. Data analysis revealed that of the 42 eligible articles, mitotic count (MC, mitotic figures/tumor area) was evaluated in 39 studies, and mitotic index (MI, mitotic figures/tumor cells) in 3 studies. The risk of bias was considered high for most studies (26/42, 62%) based on small study populations, insufficient details of the MC/MI methods, and lack of statistical measures for diagnostic accuracy or effect on outcome. The MC/MI methods varied between studies. A significant association of MC with survival was determined in 20 of 28 (71%) studies (10 studies evaluated other outcome metrics or provided individual patient data), while 1 study found an inverse effect. Three tumor types had at least 4 studies, and a prognostic association with survival was found in 5 of 6 studies on mast cell tumors, 5 of 5 on mammary tumors, and 3 of 4 on soft-tissue sarcomas. MI was shown to correlate with survival for mammary tumors by 2 research groups; however, comparisons to MC were not conducted. Further studies with standardized mitotic activity methods and appropriate statistical analysis for discriminant ability of patient outcome are needed to infer the prognostic value of MC and MI.

Enhancing mitosis quantification and detection in meningiomas with computational digital pathology

Mitosis is a critical criterion for meningioma grading. However, pathologists' assessment of mitoses is subject to significant inter-observer variation due to challenges in locating mitosis hotspots and accurately detecting mitotic figures. To address this issue, we leverage digital pathology and propose a computational strategy to enhance pathologists' mitosis assessment. The strategy has two components: (1) A depth-first search algorithm that quantifies the mathematically maximum mitotic count in 10 consecutive high-power fields, which can enhance the preciseness, especially in cases with borderline mitotic count. (2) Implementing a collaborative sphere to group a set of pathologists to detect mitoses under each high-power field, which can mitigate subjective random errors in mitosis detection originating from individual detection errors. By depth-first search algorithm (1) , we analyzed 19 meningioma slides and discovered that the proposed algorithm upgraded two borderline cases verified at consensus conferences. This improvement is attributed to the algorithm's ability to quantify the mitotic count more comprehensively compared to other conventional methods of counting mitoses. In implementing a collaborative sphere (2) , we evaluated the correctness of mitosis detection from grouped pathologists and/or pathology residents, where each member of the group annotated a set of 48 high-power field images for mitotic figures independently. We report that groups with sizes of three can achieve an average precision of 0.897 and sensitivity of 0.699 in mitosis detection, which is higher than an average pathologist in this study (precision: 0.750, sensitivity: 0.667). The proposed computational strategy can be integrated with artificial intelligence workflow, which envisions the future of achieving a rapid and robust mitosis assessment by interactive assisting algorithms that can ultimately benefit patient management.

Atypical Mitotic Figures Are Prognostically Meaningful for Canine Cutaneous Mast Cell Tumors

Cell division through mitosis (microscopically visible as mitotic figures, MFs) is a highly regulated process. However, neoplastic cells may exhibit errors in chromosome segregation (microscopically visible as atypical mitotic figures, AMFs) resulting in aberrant chromosome structures. AMFs have been shown to be of prognostic relevance for some neoplasms in humans but not in animals. In this study, the prognostic relevance of AMFs was evaluated for canine cutaneous mast cell tumors (ccMCT). Histological examination was conducted by one pathologist in whole slide images of 96 cases of ccMCT with a known survival time. Tumor-related death occurred in 11/18 high-grade and 2/78 low-grade cases (2011 two-tier system). The area under the curve (AUC) was 0.859 for the AMF count and 0.880 for the AMF to MF ratio with regard to tumor-related mortality. In comparison, the AUC for the mitotic count was 0.885. Based on our data, a prognostically meaningful threshold of ≥3 per 2.37 mm2 for the AMF count (sensitivity: 76.9%, specificity: 98.8%) and >7.5% for the AMF:MF ratio (sensitivity: 76.9%, specificity: 100%) is suggested. While the mitotic count of ≥ 6 resulted in six false positive cases, these could be eliminated when combined with the AMF to MF ratio. In conclusion, the results of this study suggests that AMF enumeration is a prognostically valuable test, particularly due to its high specificity with regard to tumor-related mortality. Additional validation and reproducibility studies are needed to further evaluate AMFs as a prognostic criterion for ccMCT and other tumor types.

Histological classification of canine and feline lymphoma using a modular approach based on deep learning and advanced image processing

Histopathological examination of tissue samples is essential for identifying tumor malignancy and the diagnosis of different types of tumor. In the case of lymphoma classification, nuclear size of the neoplastic lymphocytes is one of the key features to differentiate the different subtypes. Based on the combination of artificial intelligence and advanced image processing, we provide a workflow for the classification of lymphoma with regards to their nuclear size (small, intermediate, and large). As the baseline for our workflow testing, we use a Unet++ model trained on histological images of canine lymphoma with individually labeled nuclei. As an alternative to the Unet++, we also used a publicly available pre-trained and unmodified instance segmentation model called Stardist to demonstrate that our modular classification workflow can be combined with different types of segmentation models if they can provide proper nuclei segmentation. Subsequent to nuclear segmentation, we optimize algorithmic parameters for accurate classification of nuclear size using a newly derived reference size and final image classification based on a pathologists-derived ground truth. Our image classification module achieves a classification accuracy of up to 92% on canine lymphoma data. Compared to the accuracy ranging from 66.67 to 84% achieved using measurements provided by three individual pathologists, our algorithm provides a higher accuracy level and reproducible results. Our workflow also demonstrates a high transferability to feline lymphoma, as shown by its accuracy of up to 84.21%, even though our workflow was not optimized for feline lymphoma images. By determining the nuclear size distribution in tumor areas, our workflow can assist pathologists in subtyping lymphoma based on the nuclei size and potentially improve reproducibility. Our proposed approach is modular and comprehensible, thus allowing adaptation for specific tasks and increasing the users' trust in computer-assisted image classification.

Automated diagnosis of 7 canine skin tumors using machine learning on H&E-stained whole slide images

Microscopic evaluation of hematoxylin and eosin-stained slides is still the diagnostic gold standard for a variety of diseases, including neoplasms. Nevertheless, intra- and interrater variability are well documented among pathologists. So far, computer assistance via automated image analysis has shown potential to support pathologists in improving accuracy and reproducibility of quantitative tasks. In this proof of principle study, we describe a machine-learning-based algorithm for the automated diagnosis of 7 of the most common canine skin tumors: trichoblastoma, squamous cell carcinoma, peripheral nerve sheath tumor, melanoma, histiocytoma, mast cell tumor, and plasmacytoma. We selected, digitized, and annotated 350 hematoxylin and eosin-stained slides (50 per tumor type) to create a database divided into training, n = 245 whole-slide images (WSIs), validation (n = 35 WSIs), and test sets (n = 70 WSIs). Full annotations included the 7 tumor classes and 6 normal skin structures. The data set was used to train a convolutional neural network (CNN) for the automatic segmentation of tumor and nontumor classes. Subsequently, the detected tumor regions were classified patch-wise into 1 of the 7 tumor classes. A majority of patches-approach led to a tumor classification accuracy of the network on the slide-level of 95% (133/140 WSIs), with a patch-level precision of 85%. The same 140 WSIs were provided to 6 experienced pathologists for diagnosis, who achieved a similar slide-level accuracy of 98% (137/140 correct majority votes). Our results highlight the feasibility of artificial intelligence-based methods as a support tool in diagnostic oncologic pathology with future applications in other species and tumor types.

How to use AI in pathology

AI plays an important role in pathology, both in clinical practice supporting pathologists in their daily work, and in research discovering novel biomarkers for improved patient care. Still, AI is in its starting phase, and many pathology labs still need to transition to a digital workflow to be able to enjoy the benefits of AI. In this perspective, we explain the major benefits of AI in pathology, highlight key requirements that need to be met and example how to use it in a typical workflow.

A comprehensive multi-domain dataset for mitotic figure detection

The prognostic value of mitotic figures in tumor tissue is well-established for many tumor types and automating this task is of high research interest. However, especially deep learning-based methods face performance deterioration in the presence of domain shifts, which may arise from different tumor types, slide preparation and digitization devices. We introduce the MIDOG++ dataset, an extension of the MIDOG 2021 and 2022 challenge datasets. We provide region of interest images from 503 histological specimens of seven different tumor types with variable morphology with in total labels for 11,937 mitotic figures: breast carcinoma, lung carcinoma, lymphosarcoma, neuroendocrine tumor, cutaneous mast cell tumor, cutaneous melanoma, and (sub)cutaneous soft tissue sarcoma. The specimens were processed in several laboratories utilizing diverse scanners. We evaluated the extent of the domain shift by using state-of-the-art approaches, observing notable differences in single-domain training. In a leave-one-domain-out setting, generalizability improved considerably. This mitotic figure dataset is the first that incorporates a wide domain shift based on different tumor types, laboratories, whole slide image scanners, and species.

Artificial Intelligence to Predict the BRAF V595E Mutation in Canine Urinary Bladder Urothelial Carcinomas

In dogs, the BRAF mutation (V595E) is common in bladder and prostate cancer and represents a specific diagnostic marker. Recent advantages in artificial intelligence (AI) offer new opportunities in the field of tumour marker detection. While AI histology studies have been conducted in humans to detect BRAF mutation in cancer, comparable studies in animals are lacking. In this study, we used commercially available AI histology software to predict BRAF mutation in whole slide images (WSI) of bladder urothelial carcinomas (UC) stained with haematoxylin and eosin (HE), based on a training (n = 81) and a validation set (n = 96). Among 96 WSI, 57 showed identical PCR and AI-based BRAF predictions, resulting in a sensitivity of 58% and a specificity of 63%. The sensitivity increased substantially to 89% when excluding small or poor-quality tissue sections. Test reliability depended on tumour differentiation (p < 0.01), presence of inflammation (p < 0.01), slide quality (p < 0.02) and sample size (p < 0.02). Based on a small subset of cases with available adjacent non-neoplastic urothelium, AI was able to distinguish malignant from benign epithelium. This is the first study to demonstrate the use of AI histology to predict BRAF mutation status in canine UC. Despite certain limitations, the results highlight the potential of AI in predicting molecular alterations in routine tissue sections.

Deep learning predicts patients outcome and mutations from digitized histology slides in gastrointestinal stromal tumor

Risk assessment of gastrointestinal stromal tumor (GIST) according to the AFIP/Miettinen classification and mutational profiling are major tools for patient management. However, the AFIP/Miettinen classification depends heavily on mitotic counts, which is laborious and sometimes inconsistent between pathologists. It has also been shown to be imperfect in stratifying patients. Molecular testing is costly and time-consuming, therefore, not systematically performed in all countries. New methods to improve risk and molecular predictions are hence crucial to improve the tailoring of adjuvant therapy. We have built deep learning (DL) models on digitized HES-stained whole slide images (WSI) to predict patients' outcome and mutations. Models were trained with a cohort of 1233 GIST and validated on an independent cohort of 286 GIST. DL models yielded comparable results to the Miettinen classification for relapse-free-survival prediction in localized GIST without adjuvant Imatinib (C-index=0.83 in cross-validation and 0.72 for independent testing). DL splitted Miettinen intermediate risk GIST into high/low-risk groups (p value = 0.002 in the training set and p value = 0.29 in the testing set). DL models achieved an area under the receiver operating characteristic curve (AUC) of 0.81, 0.91, and 0.71 for predicting mutations in KIT, PDGFRA and wild type, respectively, in cross-validation and 0.76, 0.90, and 0.55 in independent testing. Notably, PDGFRA exon18 D842V mutation, which is resistant to Imatinib, was predicted with an AUC of 0.87 and 0.90 in cross-validation and independent testing, respectively. Additionally, novel histological criteria predictive of patients' outcome and mutations were identified by reviewing the tiles selected by the models. As a proof of concept, our study showed the possibility of implementing DL with digitized WSI and may represent a reproducible way to improve tailoring therapy and precision medicine for patients with GIST.

Deep learning supported mitoses counting on whole slide images: A pilot study for validating breast cancer grading in the clinical workflow

Introduction

Breast cancer (BC) prognosis is largely influenced by histopathological grade, assessed according to the Nottingham modification of Bloom-Richardson (BR). Mitotic count (MC) is a component of histopathological grading but is prone to subjectivity. This study investigated whether mitoses counting in BC using digital whole slide images (WSI) compares better to light microscopy (LM) when assisted by artificial intelligence (AI), and to which extent differences in digital MC (AI assisted or not) result in BR grade variations.

Methods

Fifty BC patients with paired core biopsies and resections were randomly selected. Component scores for BR grade were extracted from pathology reports. MC was assessed using LM, WSI, and AI. Different modalities (LM-MC, WSI-MC, and AI-MC) were analyzed for correlation with scatterplots and linear regression, and for agreement in final BR with Cohen's κ.

Results

MC modalities strongly correlated in both biopsies and resections: LM-MC and WSI-MC (R² 0.85 and 0.83, respectively), LM-MC and AI-MC (R² 0.85 and 0.95), and WSI-MC and AI-MC (R² 0.77 and 0.83). Agreement in BR between modalities was high in both biopsies and resections: LM-MC and WSI-MC (κ 0.93 and 0.83, respectively), LM-MC and AI-MC (κ 0.89 and 0.83), and WSI-MC and AI-MC (κ 0.96 and 0.73).

Conclusion

This first validation study shows that WSI-MC may compare better to LM-MC when using AI. Agreement between BR grade based on the different mitoses counting modalities was high. These results suggest that mitoses counting on WSI can well be done, and validate the presented AI algorithm for pathologist supervised use in daily practice. Further research is required to advance our knowledge of AI-MC, but it appears at least non-inferior to LM-MC.

Mitosis domain generalization in histopathology images - The MIDOG challenge

The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor proliferation and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate when applied in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MIDOG 2021 challenge was the creation of scanner-agnostic MF detection algorithms. The challenge used a training set of 200 cases, split across four scanning systems. As test set, an additional 100 cases split across four scanning systems, including two previously unseen scanners, were provided. In this paper, we evaluate and compare the approaches that were submitted to the challenge and identify methodological factors contributing to better performance. The winning algorithm yielded an F₁ score of 0.748 (CI95: 0.704-0.781), exceeding the performance of six experts on the same task.

ReCasNet: Improving consistency within the two-stage mitosis detection framework

Mitotic count (MC) is an important histological parameter for cancer diagnosis and grading, but the manual process for obtaining MC from whole-slide histopathological images is very time-consuming and prone to error. Therefore, deep learning models have been proposed to facilitate this process. Existing approaches utilize a two-stage pipeline: the detection stage for identifying the locations of potential mitotic cells and the classification stage for refining prediction confidences. However, this pipeline formulation can lead to inconsistencies in the classification stage due to the poor prediction quality of the detection stage and the mismatches in training data distributions between the two stages. In this study, we propose a Refine Cascade Network (ReCasNet), an enhanced deep learning pipeline that mitigates the aforementioned problems with three improvements. First, window relocation was used to reduce the number of poor quality false positives generated during the detection stage. Second, object re-cropping was performed with another deep learning model to adjust poorly centered objects. Third, improved data selection strategies were introduced during the classification stage to reduce the mismatches in training data distributions. ReCasNet was evaluated on two large-scale mitotic figure recognition datasets, canine cutaneous mast cell tumor (CCMCT) and canine mammary carcinoma (CMC), which resulted in up to 4.8% percentage point improvements in the F1 scores for mitotic cell detection and 44.1% reductions in mean absolute percentage error (MAPE) for MC prediction. Techniques that underlie ReCasNet can be generalized to other two-stage object detection pipeline and should contribute to improving the performances of deep learning models in broad digital pathology applications.

Cytologic scoring of equine exercise-induced pulmonary hemorrhage: Performance of human experts and a deep learning-based algorithm

Exercise-induced pulmonary hemorrhage (EIPH) is a relevant respiratory disease in sport horses, which can be diagnosed by examination of bronchoalveolar lavage fluid (BALF) cells using the total hemosiderin score (THS). The aim of this study was to evaluate the diagnostic accuracy and reproducibility of annotators and to validate a deep learning-based algorithm for the THS. Digitized cytological specimens stained for iron were prepared from 52 equine BALF samples. Ten annotators produced a THS for each slide according to published methods. The reference methods for comparing annotator's and algorithmic performance included a ground truth dataset, the mean annotators' THSs, and chemical iron measurements. Results of the study showed that annotators had marked interobserver variability of the THS, which was mostly due to a systematic error between annotators in grading the intracytoplasmatic hemosiderin content of individual macrophages. Regarding overall measurement error between the annotators, 87.7% of the variance could be reduced by using standardized grades based on the ground truth. The algorithm was highly consistent with the ground truth in assigning hemosiderin grades. Compared with the ground truth THS, annotators had an accuracy of diagnosing EIPH (THS of < or ≥ 75) of 75.7%, whereas, the algorithm had an accuracy of 92.3% with no relevant differences in correlation with chemical iron measurements. The results show that deep learning-based algorithms are useful for improving reproducibility and routine applicability of the THS. For THS by experts, a diagnostic uncertainty interval of 40 to 110 is proposed. THSs within this interval have insufficient reproducibility regarding the EIPH diagnosis.

Feline Soft Tissue Sarcomas: A Review of the Classification and Histological Grading, with Comparison to Human and Canine

Simple Summary

Soft tissue sarcomas are a common form of cancer arising in the skin and connective tissues of domestic cats. Soft tissue sarcomas encompass a group of different histological subtypes of tumours, which can behave in a range of different ways in the patient. In dogs and in humans, this group of tumours can be given a histological score (“grade”) at the time of diagnosis, which is prognostic, but there is no equivalent, well-established grading system for these tumours in cats. This review looks at soft tissue sarcomas in terms of which histological subtypes of tumour should be included in this group, and how pathologists approach their grading, comparing feline tumours with their human and canine counterparts.

Abstract

Soft tissue sarcomas are one of the most commonly diagnosed tumours arising in the skin and subcutis of our domestic cats, and are malignant neoplasms with a range of histological presentations and potential biological behaviours. However, unlike their canine and human counterparts, there is no well-established histological grading system for pathologists to apply to these tumours, in order to provide a more accurate and refined prognosis. The situation is further complicated by the presence of feline injection site sarcomas as an entity, as well as confusion over terminology for this group of tumours and which histological types should be included. There is also an absence of large scale studies. This review looks at these tumours in domestic cats, their classification and histological grading, with comparisons to the human and canine grading system.

Diagnosis, Prognosis and Treatment of Canine Cutaneous and Subcutaneous Mast Cell Tumors

Mast cell tumors (MCTs) are hematopoietic neoplasms composed of mast cells. It is highly common in dogs and is extremely important in the veterinary oncology field. It represents the third most common tumor subtype, and is the most common malignant skin tumor in dogs, corresponding to 11% of skin cancer cases. The objective of this critical review was to present the report of the 2nd Consensus meeting on the Diagnosis, Prognosis, and Treatment of Canine Cutaneous and Subcutaneous Mast Cell Tumors, which was organized by the Brazilian Association of Veterinary Oncology (ABROVET) in August 2021. The most recent information on cutaneous and subcutaneous mast cell tumors in dogs is presented and discussed.