Validation of digital microscopy: Review of validation methods and sources of bias

A Minimal Approach to Demonstrate Concordance of Digital and Conventional Microscopy in Toxicologic Pathology

Digitalization of pathology workflows has undergone a rapid evolution and has been widely established in the diagnostic field but remains a challenge in the nonclinical safety context due to lack of regulatory guidance and validation experience for good laboratory practice (GLP) use. One means to demonstrate that digital slides are fit for purpose, that is, provide sufficient quality for pathologists to reach a diagnosis, is conduction of comparison studies, which have been published both, for veterinary and human diagnostic pathology, but not for toxicologic pathology. Here, we present an approach that uses study material from nonclinical safety studies and that allows for the statistical comparison of concordance rates for glass and digital slide evaluation while minimizing time and effort for the involved personnel. Using a benchmark study design, we demonstrate that evaluation of digital slides fits the purpose of nonclinical safety evaluation. These results add to reports of successful workflow validations and support the full adaptation of digital pathology in the regulatory field.

Mitotic activity: A systematic literature review of the assessment methodology and prognostic value in canine tumors

One of the most relevant prognostic indices for tumors is cellular proliferation, which is most commonly measured by the mitotic activity in routine tumor sections. The goal of this systematic review was to analyze the methods and prognostic relevance of histologically measuring mitotic activity that have been reported for canine tumors in the literature. A total of 137 articles that correlated the mitotic activity in canine tumors with patient outcome were identified through a systematic (PubMed and Scopus) and nonsystematic (Google Scholar) literature search and eligibility screening process. Mitotic activity methods encompassed the mitotic count (MC, number of mitotic figures per tumor area) in 126 studies, presumably the MC (method not specified) in 6 studies, and the mitotic index (MI, number of mitotic figures per number of tumor cells) in 5 studies. A particularly high risk of bias was identified based on the available details of the MC methods and statistical analyses, which often did not quantify the prognostic discriminative ability of the MC and only reported P values. A significant association of the MC with survival was found in 72 of 109 (66%) studies. However, survival was evaluated by at least 3 studies in only 7 tumor types/groups, of which a prognostic relevance is apparent for mast cell tumors of the skin, cutaneous melanoma, and soft tissue tumor of the skin and subcutis. None of the studies using the MI found a prognostic relevance. This review highlights the need for more studies with standardized methods and appropriate analysis of the discriminative ability to prove the prognostic value of the MC and MI in various tumor types. Future studies are needed to evaluate the influence of the performance of individual pathologists on the appropriateness of prognostic thresholds and investigate methods to improve interobserver reproducibility.

Mitotic activity: A systematic literature review of the assessment methodology and prognostic value in feline tumors

Increased proliferation is a driver of tumorigenesis, and quantification of mitotic activity is a standard task for prognostication. This systematic review is an analysis of all available references on mitotic activity in feline tumors to provide an overview of the assessment methods and prognostic value. A systematic literature search in PubMed and Scopus and a nonsystematic search in Google Scholar were conducted. All articles on feline tumors that correlated mitotic activity with patient outcome were identified. Data analysis revealed that of the 42 eligible articles, mitotic count (MC, mitotic figures/tumor area) was evaluated in 39 studies, and mitotic index (MI, mitotic figures/tumor cells) in 3 studies. The risk of bias was considered high for most studies (26/42, 62%) based on small study populations, insufficient details of the MC/MI methods, and lack of statistical measures for diagnostic accuracy or effect on outcome. The MC/MI methods varied between studies. A significant association of MC with survival was determined in 20 of 28 (71%) studies (10 studies evaluated other outcome metrics or provided individual patient data), while 1 study found an inverse effect. Three tumor types had at least 4 studies, and a prognostic association with survival was found in 5 of 6 studies on mast cell tumors, 5 of 5 on mammary tumors, and 3 of 4 on soft-tissue sarcomas. MI was shown to correlate with survival for mammary tumors by 2 research groups; however, comparisons to MC were not conducted. Further studies with standardized mitotic activity methods and appropriate statistical analysis for discriminant ability of patient outcome are needed to infer the prognostic value of MC and MI.

Cytologic scoring of equine exercise-induced pulmonary hemorrhage: Performance of human experts and a deep learning-based algorithm

Exercise-induced pulmonary hemorrhage (EIPH) is a relevant respiratory disease in sport horses, which can be diagnosed by examination of bronchoalveolar lavage fluid (BALF) cells using the total hemosiderin score (THS). The aim of this study was to evaluate the diagnostic accuracy and reproducibility of annotators and to validate a deep learning-based algorithm for the THS. Digitized cytological specimens stained for iron were prepared from 52 equine BALF samples. Ten annotators produced a THS for each slide according to published methods. The reference methods for comparing annotator's and algorithmic performance included a ground truth dataset, the mean annotators' THSs, and chemical iron measurements. Results of the study showed that annotators had marked interobserver variability of the THS, which was mostly due to a systematic error between annotators in grading the intracytoplasmatic hemosiderin content of individual macrophages. Regarding overall measurement error between the annotators, 87.7% of the variance could be reduced by using standardized grades based on the ground truth. The algorithm was highly consistent with the ground truth in assigning hemosiderin grades. Compared with the ground truth THS, annotators had an accuracy of diagnosing EIPH (THS of < or ≥ 75) of 75.7%, whereas, the algorithm had an accuracy of 92.3% with no relevant differences in correlation with chemical iron measurements. The results show that deep learning-based algorithms are useful for improving reproducibility and routine applicability of the THS. For THS by experts, a diagnostic uncertainty interval of 40 to 110 is proposed. THSs within this interval have insufficient reproducibility regarding the EIPH diagnosis.

Standardisation of canine meningioma grading: Inter-observer agreement and recommendations for reproducible histopathologic criteria

The human grading system is currently applied to canine meningioma, although it has not been validated in dogs. The present study focused on standardising the human grading system applied to canine meningioma. Four veterinary neuropathologists graded 186 canine meningiomas as follows: Grade I tumour, with <4 mitoses/2.37 mm2 ; Grade II tumour, with ≥4 mitoses/2.37 mm2 , brain invasion or at least three of the following criteria: sheeting architecture, hypercellularity, small cells, macronucleoli, necrosis; Grade III tumour, with ≥20 mitoses/2.37 mm2 or anaplasia. Slides with grading disagreement were reviewed to define a consensus diagnosis and to assess reproducible criteria. Concordance between histologic grade and the consensus diagnosis, as well as intra- and inter-observer agreements for each criterion, were statistically analysed. Concordance between histologic grade and consensus diagnosis ranged from 59% to 100%, with lower concordance for Grade I and II tumours. The lowest inter-observer agreement was recorded for macronucleoli, small cells, hypercellularity and sheeting architecture. Tumour invasion and necrosis displayed fair agreement, while moderate agreement was reached for mitotic grade and anaplasia. The following recommendations were issued to improve the reproducibility of canine meningioma grading: (1) Assess mitotic grade in consecutive HPFs within the most mitotically active area; (2) Define invasion as neoplastic protrusions within central nervous tissue without pial lining; (3) Report spontaneous necrosis; (4) Report prominent nucleoli when visible at ×100; (5) Report pattern loss when visible at ×100 in >50% of the tumour; (6) Report necrosis, small cells, hypercellularity and macronucleoli, even when focal; (7) Report anaplasia if multifocal.

Canine Vaginal Cytology: A Revised Definition of Exfoliated Vaginal Cells

Vaginal cytology is an important examination method in the context of gynecological disorders and cycle staging in the bitch. While collection and preparation of samples are easy, the evaluation appears to be challenging. Inconsistent definitions of cell attributes such as size, cornification and the appearance of the nucleus have been published. The aim of the project was to develop a tutorial for vaginal cell determination. To get a deeper insight into the use of cytology in practice, an online survey was distributed to veterinarians interested in small animal reproduction. Participants were asked to define eight cells and answer questions. The agreement of the 16 participants, working in eight different countries, determining the cells was poor (κ = 0.412). Eleven respondents stated that vaginal cytology has a low reliability. Nevertheless, 13 participants use this tool regularly. The tutorial was developed as a flowchart based on the survey results, scientific literature and own measurements. It guides the user systematically through the evaluation of specific cell characteristics. An evaluation of the results of five raters with difference experience levels led to a high agreement (κ = 0.858). Vaginal cytology is a useful diagnostic tool, but it seems helpful to standardize the determination of cell types.