The cytopathologist's role in developing and evaluating artificial intelligence in cytopathology practice

Prediction of tumor origin in cancers of unknown primary origin with cytology-based deep learning

Cancer of unknown primary (CUP) site poses diagnostic challenges due to its elusive nature. Many cases of CUP manifest as pleural and peritoneal serous effusions. Leveraging cytological images from 57,220 cases at four tertiary hospitals, we developed a deep-learning method for tumor origin differentiation using cytological histology (TORCH) that can identify malignancy and predict tumor origin in both hydrothorax and ascites. We examined its performance on three internal (n = 12,799) and two external (n = 14,538) testing sets. In both internal and external testing sets, TORCH achieved area under the receiver operating curve values ranging from 0.953 to 0.991 for cancer diagnosis and 0.953 to 0.979 for tumor origin localization. TORCH accurately predicted primary tumor origins, with a top-1 accuracy of 82.6% and top-3 accuracy of 98.9%. Compared with results derived from pathologists, TORCH showed better prediction efficacy (1.677 versus 1.265, P < 0.001), enhancing junior pathologists' diagnostic scores significantly (1.326 versus 1.101, P < 0.001). Patients with CUP whose initial treatment protocol was concordant with TORCH-predicted origins had better overall survival than those who were administrated discordant treatment (27 versus 17 months, P = 0.006). Our study underscores the potential of TORCH as a valuable ancillary tool in clinical practice, although further validation in randomized trials is warranted.

Rapid on-site evaluation (ROSE) of image-guided FNA specimens improves subsequent core biopsy adequacy in clinical trial patients: The impact of preanalytical factors and its correlation with survival

BACKGROUND

Sufficient tumor collection has become of utmost importance in therapeutic experimental protocols. Rapid on-site evaluation (ROSE) ensures adequate sampling for quantification of biomarkers, molecular analyses, and other ancillary studies. The objectives of this study were to evaluate the role of ROSE in trial-associated fine-needle aspiration (FNA) and to analyze predictors of adequacy and cumulative survival from in-house FNA cases used in clinical trials.

METHODS

Clinical trial FNA biopsies performed at a large academic institution were analyzed over 10 months using a comprehensive chart review of the electronic medical records. SPSS version 28 was used for statistical analysis.

RESULTS

Three hundred twenty-five FNAs were collected for 57 clinical trials. In total, 225 individual patients had an average of 1.4 FNA procedures each as a result of a multidepartmental collaborative effort. ROSE was performed for all patients, and adequacy was evaluated by cytotechnologists. Seventy-eight percent of samples were considered adequate, 14% were considered less than optimal, and 8% were considered inadequate, with the latter two categories designated together as less than adequate. The imaging modalities were mainly ultrasound-guided (n = 267; 82%) and computed tomography-guided (n = 58; 18%). There was a statistically significant association between adequate sampling and ultrasound-guided biopsies (83%) compared with computed tomography-guided biopsies (59%; p < .01). The effect of body mass index (BMI) on mortality was also a significant finding. The authors observed a survival benefit in patients who had elevated BMIs (range, 25.0-34.9 kg/m2 ) compared with those who were underweight (BMI, <18.5 kg/m2 ) or class III obese (BMI, >35.0 kg/m2 ; p < .01). Therefore, the best predictors of adequacy and mortality were imaging modality and BMI, respectively.

CONCLUSIONS

Ultrasound-guided modalities are recommended for obtaining adequate FNA sampling for clinical trials. In addition, patients with cancer who had slightly elevated BMIs (25.0-34.0 kg/m2 ) had increased overall survival in this cohort.

Application of multiple-finding segmentation utilizing Mask R-CNN-based deep learning in a rat model of drug-induced liver injury

Drug-induced liver injury (DILI) presents significant diagnostic challenges, and recently artificial intelligence-based deep learning technology has been used to predict various hepatic findings. In this study, we trained a set of Mask R-CNN-based deep algorithms to learn and quantify typical toxicant induced-histopathological lesions, portal area, and connective tissue in Sprague Dawley rats. We compared a set of single-finding models (SFMs) and a combined multiple-finding model (MFM) for their ability to simultaneously detect, classify, and quantify multiple hepatic findings on rat liver slide images. All of the SFMs yielded mean average precision (mAP) values above 85%, suggesting that the models had been successfully established. The MFM showed better performance than the SFMs, with a total mAP value of 92.46%. We compared the model predictions for slide images with ground-truth annotations generated by an accredited pathologist. For the MFM, the overall and individual finding predictions were highly correlated with the annotated areas, with R-squared values of 0.852, 0.952, 0.999, 0.990, and 0.958 being obtained for portal area, infiltration, necrosis, vacuolation, and connective tissue (including fibrosis), respectively. Our results indicate that the proposed MFM could be a useful tool for detecting and predicting multiple hepatic findings in basic non-clinical study settings.

Examining longitudinal markers of bladder cancer recurrence through a semiautonomous machine learning system for quantifying specimen atypia from urine cytology

BACKGROUND

Urine cytology is generally considered the primary approach for screening for recurrence of bladder cancer. However, it is currently unclear how best to use cytological examinations for assessment and early detection of recurrence, beyond identifying a positive finding that requires more invasive methods to confirm recurrence and decide on therapeutic options. Because screening programs are frequent, and can be burdensome, finding quantitative means to reduce this burden for patients, cytopathologists, and urologists is an important endeavor and can improve both the efficiency and reliability of findings. Additionally, identifying ways to risk-stratify patients is crucial for improving quality of life while reducing the risk of future recurrence or progression of the cancer.

METHODS

In this study, a computational machine learning tool, AutoParis-X, was leveraged to extract imaging features from urine cytology examinations longitudinally to study the predictive potential of urine cytology for assessing recurrence risk. This study examined how the significance of imaging predictors changes over time before and after surgery to determine which predictors and time periods are most relevant for assessing recurrence risk.

RESULTS

Results indicate that imaging predictors extracted using AutoParis-X can predict recurrence as well or better than traditional cytological/histological assessments alone and that the predictiveness of these features is variable across time, with key differences in overall specimen atypia identified immediately before tumor recurrence.

CONCLUSIONS

Further research will clarify how computational methods can be effectively used in high-volume screening programs to improve recurrence detection and complement traditional modes of assessment.

Towards Artificial Intelligence Applications in Next Generation Cytopathology

Over the last 20 years we have seen an increase in techniques in the field of computational pathology and machine learning, improving our ability to analyze and interpret imaging. Neural networks, in particular, have been used for more than thirty years, starting with the computer assisted smear test using early generation models. Today, advanced machine learning, working on large image data sets, has been shown to perform classification, detection, and segmentation with remarkable accuracy and generalization in several domains. Deep learning algorithms, as a branch of machine learning, are thus attracting attention in digital pathology and cytopathology, providing feasible solutions for accurate and efficient cytological diagnoses, ranging from efficient cell counts to automatic classification of anomalous cells and queries over large clinical databases. The integration of machine learning with related next-generation technologies powered by AI, such as augmented/virtual reality, metaverse, and computational linguistic models are a focus of interest in health care digitalization, to support education, diagnosis, and therapy. In this work we will consider how all these innovations can help cytopathology to go beyond the microscope and to undergo a hyper-digitalized transformation. We also discuss specific challenges to their applications in the field, notably, the requirement for large-scale cytopathology datasets, the necessity of new protocols for sharing information, and the need for further technological training for pathologists.

An emerging era of computational cytology

BACKGROUND

The significant advancement in digital imaging, data management, advanced computational power, and artificial neural network have an immense impact on the field of cytology. The amalgamation of these areas has generated a newer discipline known as computational cytology.

AIMS AND OBJECTIVE

In To discuss the various important aspects of computational cytology.

MATERIALS AND METHODS

We reviewed the different studies published in English during the last few years on computational cytology.

RESULT

Computational cytology is a newer and emerging discipline in pathology that deals with the patient's meta-data and digital image data to make a mathematical model to produce diagnostic interpretations and predictions. The role of the cytologist is now changing from a simple observational scientist and slide interpreter to a dynamic and integrated multi-parametric prediction-based scientist.

CONCLUSION

In the current stage, the cytologist must understand the situation and should have a vision of the complete scenario on computational cytology.

Ethical and legal considerations influencing human involvement in the implementation of artificial intelligence in a clinical pathway: A multi-stakeholder perspective

Introduction

Ethical and legal factors will have an important bearing on when and whether automation is appropriate in healthcare. There is a developing literature on the ethics of artificial intelligence (AI) in health, including specific legal or regulatory questions such as whether there is a right to an explanation of AI decision-making. However, there has been limited consideration of the specific ethical and legal factors that influence when, and in what form, human involvement may be required in the implementation of AI in a clinical pathway, and the views of the wide range of stakeholders involved. To address this question, we chose the exemplar of the pathway for the early detection of Barrett's Oesophagus (BE) and oesophageal adenocarcinoma, where Gehrung and colleagues have developed a “semi-automated”, deep-learning system to analyse samples from the CytospongeTM TFF3 test (a minimally invasive alternative to endoscopy), where AI promises to mitigate increasing demands for pathologists' time and input.

Methods

We gathered a multidisciplinary group of stakeholders, including developers, patients, healthcare professionals and regulators, to obtain their perspectives on the ethical and legal issues that may arise using this exemplar.

Results

The findings are grouped under six general themes: risk and potential harms; impacts on human experts; equity and bias; transparency and oversight; patient information and choice; accountability, moral responsibility and liability for error. Within these themes, a range of subtle and context-specific elements emerged, highlighting the importance of pre-implementation, interdisciplinary discussions and appreciation of pathway specific considerations.

Discussion

To evaluate these findings, we draw on the well-established principles of biomedical ethics identified by Beauchamp and Childress as a lens through which to view these results and their implications for personalised medicine. Our findings are not only relevant to this context but have implications for AI in digital pathology and healthcare more broadly.

Intraoperative cytological diagnosis of brain tumours: A preliminary study using a deep learning model

BACKGROUND

Intraoperative pathological diagnosis of central nervous system (CNS) tumours is essential to planning patient management in neuro-oncology. Frozen section slides and cytological preparations provide architectural and cellular information that is analysed by pathologists to reach an intraoperative diagnosis. Progress in the fields of artificial intelligence and machine learning means that AI systems have significant potential for the provision of highly accurate real-time diagnosis in cytopathology.

OBJECTIVE

To investigate the efficiency of machine-learning models in the intraoperative cytological diagnosis of CNS tumours.

MATERIALS AND METHODS

We trained a deep neural network to classify biopsy material for intraoperative tissue diagnosis of four major brain lesions. Overall, 205 medical images were obtained from squash smear slides of histologically correlated cases, with 18 high-grade and 11 low-grade gliomas, 17 metastatic carcinomas, and 9 non-neoplastic pathological brain tissue samples. The neural network model was trained and evaluated using 5-fold cross-validation.

RESULTS

The model achieved 95% and 97% diagnostic accuracy in the patch-level classification and patient-level classification tasks, respectively.

CONCLUSIONS

We conclude that deep learning-based classification of cytological preparations may be a promising complementary method for the rapid and accurate intraoperative diagnosis of CNS tumours.

"SMART" cytology: The next generation cytology for precision diagnosis

Cytology plays an important role in diagnosing and managing human diseases, especially cancer, as it is often a simple, low cost yet effective, and non-invasive or minimally invasive diagnostic tool. However, traditional morphology-based cytology practice has limitations, especially in the era of precision diagnosis. Recently there have been tremendous efforts devoted to apply computational tools and to perform molecular analysis on cytological samples for a variety of clinical purposes. Now is probably the appropriate juncture to integrate morphology, machine learning, and molecular analysis together and transform cytology from a morphology-driven practice to the next level - "SMART" Cytology. In this article we will provide a rather brief review of the relevant works for computational analysis on cytology samples, focusing on single-cell-based multiplex quantitative analysis of biomarkers, and introduce the conceptual framework of "SMART (Single cell, Multiplex, AI-driven, and Real Time)" Cytology.

BestCyte® primary screening of 500 ThinPrep Pap Test thin-layers: 3 Cytologists' Interobserver diagnostic concordance with predicate manual microscopy relative to Truth Reference diagnoses defining NILM, ASCUS+, LSIL+, and ASCH+ thresholds for specificity, sensitivity, and equivalency grading

Background

The BestCyte® Cell Sorter Imaging System (BestCyte) facilitates algorithmic discrimination of clinically relevant cells in Pap test cytopathology by classifying and projecting images of cells in galleries based on cytomorphology. Warranted is awareness of potential BestCyte advantages as measured through 3 cytologists' interobserver diagnostic concordance, specificity and sensitivity differentials, and equivalency grading relative to manual microscopy (MM).

Objectives

Using 500 MM-reported ThinPrep thin-layers, analyze: (1) cytologists' blinded BestCyte screening to raise Bethesda diagnoses; (2) correlate BestCyte and MM diagnoses (i.e., predicate) to establish Truth Reference Diagnoses (TRDx) from concordance between 4 possible diagnoses; (3) analyze cytologists' and MM predicate diagnoses through 4 diagnostic thresholds defined by TRDx: NILM (Negative) for specificity, and ASCUS+, LSIL+, and ASCH+ (Positive) for graded sensitivity (with abnormal cells decreasing in size with increasing dysplasia); and, (4) statistically determine cytologists' equivalency grading to MM using 95% Confidence Interval (CI) ranges.

Results

500 TRDx breakdown (n/%): NILM (241/48.2), ASCUS (79/15.8), ASCH (9/1.80), AGUS (2/0.40), LSIL (86/17.2), HSIL (68/13.6), CA (2/0.40), UNSAT (13/2.60). TRDx breakdown (n/%) per 4 of 4, 3 of 4, 2 of 4 diagnostic concordances: 264 (52.8%), 182 (36.4%), 54 (10.8%), respectively. No cases of discordant diagnoses were recorded. HSIL TRDx were established from 66.2% of 4 of 4 concordances, followed by NILM (59.3%), LSIL (46.5%), ASCUS (41.8%); antithetically, from 4.40% of 2 of 4 concordances. Specificity for MM predicate (NILM): 67.08%; for Cytologists 1, 2, and 3: 89.71%, 82.30%, 97.53%, respectively. For NILM threshold, cytologists revealed Significantly Superior equivalency to MM. Sensitivity for ASCUS+, LSIL+, and ASCH+ thresholds: MM (91.36%, 86.67%, 74.36%); Cytologist 1 (95.88%, 96.97%, 94.87%); Cytologist 2 (95.47%, 95.76%, 93.59%), Cytologist 3 (94.65%, 95.15%, 98.72%), respectively. Cytologists revealed Significantly Superior equivalency to MM for graded Positive thresholds; with Cytologist 3 for ASCUS+ being: Superior.

Conclusions

BestCyte detects and efficiently displays abnormal cells in strategic galleries standardizing objectivity by systematizing mosaics of cell-types for cytologists' consideration. BestCyte fosters consistent, enhanced cytologists' sensitivity values for the ASCUS+, LSIL+, and ASCH+ Positive thresholds relative to MM. Also, BestCyte facilitates improved specificity and superior equivalency grading to MM reflecting efficient screening, and reduced labor. Confident interpretations of small dysplastic epithelial cells characteristic of HSIL led to exceptional interobserver diagnostic concordance inferring BestCyte is primed for effective cervical cancer screening practice.

Cytologic scoring of equine exercise-induced pulmonary hemorrhage: Performance of human experts and a deep learning-based algorithm

Exercise-induced pulmonary hemorrhage (EIPH) is a relevant respiratory disease in sport horses, which can be diagnosed by examination of bronchoalveolar lavage fluid (BALF) cells using the total hemosiderin score (THS). The aim of this study was to evaluate the diagnostic accuracy and reproducibility of annotators and to validate a deep learning-based algorithm for the THS. Digitized cytological specimens stained for iron were prepared from 52 equine BALF samples. Ten annotators produced a THS for each slide according to published methods. The reference methods for comparing annotator's and algorithmic performance included a ground truth dataset, the mean annotators' THSs, and chemical iron measurements. Results of the study showed that annotators had marked interobserver variability of the THS, which was mostly due to a systematic error between annotators in grading the intracytoplasmatic hemosiderin content of individual macrophages. Regarding overall measurement error between the annotators, 87.7% of the variance could be reduced by using standardized grades based on the ground truth. The algorithm was highly consistent with the ground truth in assigning hemosiderin grades. Compared with the ground truth THS, annotators had an accuracy of diagnosing EIPH (THS of < or ≥ 75) of 75.7%, whereas, the algorithm had an accuracy of 92.3% with no relevant differences in correlation with chemical iron measurements. The results show that deep learning-based algorithms are useful for improving reproducibility and routine applicability of the THS. For THS by experts, a diagnostic uncertainty interval of 40 to 110 is proposed. THSs within this interval have insufficient reproducibility regarding the EIPH diagnosis.

The Utility of Unsupervised Machine Learning in Anatomic Pathology

OBJECTIVES

Developing accurate supervised machine learning algorithms is hampered by the lack of representative annotated datasets. Most data in anatomic pathology are unlabeled and creating large, annotated datasets is a time consuming and laborious process. Unsupervised learning, which does not require annotated data, possesses the potential to assist with this challenge. This review aims to introduce the concept of unsupervised learning and illustrate how clustering, generative adversarial networks (GANs) and autoencoders have the potential to address the lack of annotated data in anatomic pathology.

METHODS

A review of unsupervised learning with examples from the literature was carried out.

RESULTS

Clustering can be used as part of semisupervised learning where labels are propagated from a subset of annotated data points to remaining unlabeled data points in a dataset. GANs may assist by generating large amounts of synthetic data and performing color normalization. Autoencoders allow training of a network on a large, unlabeled dataset and transferring learned representations to a classifier using a smaller, labeled subset (unsupervised pretraining).

CONCLUSIONS

Unsupervised machine learning techniques such as clustering, GANs, and autoencoders, used individually or in combination, may help address the lack of annotated data in pathology and improve the process of developing supervised learning models.

Computer-assisted mitotic count using a deep learning–based algorithm improves interobserver reproducibility and accuracy

The mitotic count (MC) is an important histological parameter for prognostication of malignant neoplasms. However, it has inter- and intraobserver discrepancies due to difficulties in selecting the region of interest (MC-ROI) and in identifying or classifying mitotic figures (MFs). Recent progress in the field of artificial intelligence has allowed the development of high-performance algorithms that may improve standardization of the MC. As algorithmic predictions are not flawless, computer-assisted review by pathologists may ensure reliability. In the present study, we compared partial (MC-ROI preselection) and full (additional visualization of MF candidates and display of algorithmic confidence values) computer-assisted MC analysis to the routine (unaided) MC analysis by 23 pathologists for whole-slide images of 50 canine cutaneous mast cell tumors (ccMCTs). Algorithmic predictions aimed to assist pathologists in detecting mitotic hotspot locations, reducing omission of MFs, and improving classification against imposters. The interobserver consistency for the MC significantly increased with computer assistance (interobserver correlation coefficient, ICC = 0.92) compared to the unaided approach (ICC = 0.70). Classification into prognostic stratifications had a higher accuracy with computer assistance. The algorithmically preselected hotspot MC-ROIs had a consistently higher MCs than the manually selected MC-ROIs. Compared to a ground truth (developed with immunohistochemistry for phosphohistone H3), pathologist performance in detecting individual MF was augmented when using computer assistance (F1-score of 0.68 increased to 0.79) with a reduction in false negatives by 38%. The results of this study demonstrate that computer assistance may lead to more reproducible and accurate MCs in ccMCTs.

The Dynamics of Pathology Dataset Creation Using Urine Cytology as an Example

INTRODUCTION

Dataset creation is one of the first tasks required for training AI algorithms but is underestimated in pathology. High-quality data are essential for training algorithms and data should be labelled accurately and include sufficient morphological diversity. The dynamics and challenges of labelling a urine cytology dataset using The Paris System (TPS) criteria are presented.

METHODS

2,454 images were labelled by pathologist consensus via video conferencing over a 14-day period. During the labelling sessions, the dynamics of the labelling process were recorded. Quality assurance images were randomly selected from images labelled in previous sessions within this study and randomly distributed throughout new labelling sessions. To assess the effect of time on the labelling process, the labelled set of images was split into 2 groups according to the median relative label time and the time taken to label images and intersession agreement were assessed.

RESULTS

Labelling sessions ranged from 24 m 11 s to 41 m 06 s in length, with a median of 33 m 47 s. The majority of the 2,454 images were labelled as benign urothelial cells, with atypical and malignant urothelial cells more sparsely represented. The time taken to label individual images ranged from 1 s to 42 s with a median of 2.9 s. Labelling times differed significantly among categories, with the median label time for the atypical urothelial category being 7.2 s, followed by the malignant urothelial category at 3.8 s and the benign urothelial category at 2.9 s. The overall intersession agreement for quality assurance images was substantial. The level of agreement differed among classes of urothelial cells - benign and malignant urothelial cell classes showed almost perfect agreement and the atypical urothelial cell class showed moderate agreement. Image labelling times seemed to speed up, and there was no evidence of worsening of intersession agreement with session time.

DISCUSSION/CONCLUSION

Important aspects of pathology dataset creation are presented, illustrating the significant resources required for labelling a large dataset. We present evidence that the time taken to categorise urine cytology images varies by diagnosis/class. The known challenges relating to the reproducibility of the AUC (atypical) category in TPS when compared to the NHGUC (benign) or HGUC (malignant) categories is also confirmed.

Artificial intelligence and computational pathology

Data processing and learning has become a spearhead for the advancement of medicine, with pathology and laboratory medicine has no exception. The incorporation of scientific research through clinical informatics, including genomics, proteomics, bioinformatics, and biostatistics, into clinical practice unlocks innovative approaches for patient care. Computational pathology is burgeoning subspecialty in pathology that promises a better-integrated solution to whole-slide images, multi-omics data, and clinical informatics. However, computational pathology faces several challenges, including the ability to integrate raw data from different sources, limitation of hardware processing capacity, and a lack of specific training programs, as well as issues on ethics and larger societal acceptable practices that are still solidifying. The establishment of the entire industry of computational pathology requires far-reaching changes of the three essential elements connecting patients and doctors: the local laboratory, the scan center, and the central cloud hub/portal for data processing and retrieval. Computational pathology, unlocked through information integration and advanced digital communication networks, has the potential to improve clinical workflow efficiency, diagnostic quality, and ultimately create personalized diagnosis and treatment plans for patients. This review describes clinical perspectives and discusses the statistical methods, clinical applications, potential obstacles, and future directions of computational pathology.

Developing and Preliminary Validating an Automatic Cell Classification System for Bone Marrow Smears: a Pilot Study

Bone marrow smear examination is an indispensable diagnostic tool in the evaluation of hematological diseases, but the process of manual differential count is labor extensive. In this study, we developed an automatic system with integrated scanning hardware and machine learning-based software to perform differential cell count on bone marrow smears to assist diagnosis. The initial development of the artificial neural network was based on 3000 marrow smear samples retrospectively archived from Sir Run Run Shaw Hospital affiliated to Zhejiang University School of Medicine between June 2016 and December 2018. The preliminary field validating test of the system was based on 124 marrow smears newly collected from the Second Affiliated Hospital of Harbin Medical University between April 2019 and November 2019. The study was performed in parallel of machine automatic recognition with conventional manual differential count by pathologists using the microscope. We selected representative 600,000 marrow cell images as training set of the algorithm, followed by random captured 30,867 cell images for validation. In validation, the overall accuracy of automatic cell classification was 90.1% (95% CI, 89.8-90.5%). In a preliminary field validating test, the reliability coefficient (ICC) of cell series proportion between the two analysis methods were high (ICC ≥ 0.883, P < 0.0001) and the results by the two analysis methods were consistent for granulocytes and erythrocytes. The system was effective in cell classification and differential cell count on marrow smears. It provides a useful digital tool in the screening and evaluation of various hematological disorders.