BestCyte® Cell Sorter Imaging System: Primary and Adjudicative Whole Slide Image Rescreening Review Times of 500 ThinPrep Pap Test Thin-layers - An Intra-observer, Time-Surrogate Analysis of Diagnostic Confidence Potentialities

Improved Diagnostic Accuracy of Thyroid Fine-Needle Aspiration Cytology with Artificial Intelligence Technology

Background: Artificial intelligence (AI) is increasingly being applied in pathology and cytology, showing promising results. We collected a large dataset of whole slide images (WSIs) of thyroid fine-needle aspiration cytology (FNA), incorporating z-stacking, from institutions across the nation to develop an AI model. Methods: We conducted a multicenter retrospective diagnostic accuracy study using thyroid FNA dataset from the Open AI Dataset Project that consists of digitalized images samples collected from 3 university hospitals and 215 Korean institutions through extensive quality check during the case selection, scanning, labeling, and reviewing process. Multiple z-layer images were captured using three different scanners and image patches were extracted from WSIs and resized after focus fusion and color normalization. We pretested six AI models, determining Inception ResNet v2 as the best model using a subset of dataset, and subsequently tested the final model with total datasets. Additionally, we compared the performance of AI and cytopathologists using randomly selected 1031 image patches and reevaluated the cytopathologists' performance after reference to AI results. Results: A total of 10,332 image patches from 306 thyroid FNAs, comprising 78 malignant (papillary thyroid carcinoma) and 228 benign from 86 institutions were used for the AI training. Inception ResNet v2 achieved highest accuracy of 99.7%, 97.7%, and 94.9% for training, validation, and test dataset, respectively (sensitivity 99.9%, 99.6%, and 100% and specificity 99.6%, 96.4%, and 90.4% for training, validation, and test dataset, respectively). In the comparison between AI and human, AI model showed higher accuracy and specificity than the average expert cytopathologists beyond the two-standard deviation (accuracy 99.71% [95% confidence interval (CI), 99.38-100.00%] vs. 88.91% [95% CI, 86.99-90.83%], sensitivity 99.81% [95% CI, 99.54-100.00%] vs. 87.26% [95% CI, 85.22-89.30%], and specificity 99.61% [95% CI, 99.23-99.99%] vs. 90.58% [95% CI, 88.80-92.36%]). Moreover, after referring to the AI results, the performance of all the experts (accuracy 96%, 95%, and 96%, respectively) and the diagnostic agreement (from 0.64 to 0.84) increased. Conclusions: These results suggest that the application of AI technology to thyroid FNA cytology may improve the diagnostic accuracy as well as intra- and inter-observer variability among pathologists. Further confirmatory research is needed.

Digital cytology part 2: artificial intelligence in cytology: a concept paper with review and recommendations from the American Society of Cytopathology Digital Cytology Task Force

Digital cytology and artificial intelligence (AI) are gaining greater adoption in the cytology laboratory. However, peer-reviewed real-world data and literature are lacking in regard to the current clinical landscape. The American Society of Cytopathology in conjunction with the International Academy of Cytology and the Digital Pathology Association established a special task force comprising 20 members with expertise and/or interest in digital cytology. The aim of the group was to investigate the feasibility of incorporating digital cytology, specifically cytology whole slide scanning and AI applications, into the workflow of the laboratory. In turn, the impact on cytopathologists, cytologists (cytotechnologists), and cytology departments were also assessed. The task force reviewed existing literature on digital cytology, conducted a worldwide survey, and held a virtual roundtable discussion on digital cytology and AI with multiple industry corporate representatives. This white paper, presented in 2 parts, summarizes the current state of digital cytology and AI practice in global cytology practice. Part 1 of the white paper is presented as a separate paper which details a review and best practice recommendations for incorporating digital cytology into practice. Part 2 of the white paper presented here provides a comprehensive review of AI in cytology practice along with best practice recommendations and legal considerations. Additionally, the cytology global survey results highlighting current AI practices by various laboratories, as well as current attitudes, are reported.

Large-scale validation study of an improved semiautonomous urine cytology assessment tool: AutoParis-X

BACKGROUND

Adopting a computational approach for the assessment of urine cytology specimens has the potential to improve the efficiency, accuracy, and reliability of bladder cancer screening, which has heretofore relied on semisubjective manual assessment methods. As rigorous, quantitative criteria and guidelines have been introduced for improving screening practices (e.g., The Paris System for Reporting Urinary Cytology), algorithms to emulate semiautonomous diagnostic decision-making have lagged behind, in part because of the complex and nuanced nature of urine cytology reporting.

METHODS

In this study, the authors report on the development and large-scale validation of a deep-learning tool, AutoParis-X, which can facilitate rapid, semiautonomous examination of urine cytology specimens.

RESULTS

The results of this large-scale, retrospective validation study indicate that AutoParis-X can accurately determine urothelial cell atypia and aggregate a wide variety of cell-related and cluster-related information across a slide to yield an atypia burden score, which correlates closely with overall specimen atypia and is predictive of Paris system diagnostic categories. Importantly, this approach accounts for challenges associated with the assessment of overlapping cell cluster borders, which improve the ability to predict specimen atypia and accurately estimate the nuclear-to-cytoplasm ratio for cells in these clusters.

CONCLUSIONS

The authors developed a publicly available, open-source, interactive web application that features a simple, easy-to-use display for examining urine cytology whole-slide images and determining the level of atypia in specific cells, flagging the most abnormal cells for pathologist review. The accuracy of AutoParis-X (and other semiautomated digital pathology systems) indicates that these technologies are approaching clinical readiness and necessitates full evaluation of these algorithms in head-to-head clinical trials.

BestCyte® primary screening of 500 ThinPrep Pap Test thin-layers: 3 Cytologists' Interobserver diagnostic concordance with predicate manual microscopy relative to Truth Reference diagnoses defining NILM, ASCUS+, LSIL+, and ASCH+ thresholds for specificity, sensitivity, and equivalency grading

Background

The BestCyte® Cell Sorter Imaging System (BestCyte) facilitates algorithmic discrimination of clinically relevant cells in Pap test cytopathology by classifying and projecting images of cells in galleries based on cytomorphology. Warranted is awareness of potential BestCyte advantages as measured through 3 cytologists' interobserver diagnostic concordance, specificity and sensitivity differentials, and equivalency grading relative to manual microscopy (MM).

Objectives

Using 500 MM-reported ThinPrep thin-layers, analyze: (1) cytologists' blinded BestCyte screening to raise Bethesda diagnoses; (2) correlate BestCyte and MM diagnoses (i.e., predicate) to establish Truth Reference Diagnoses (TRDx) from concordance between 4 possible diagnoses; (3) analyze cytologists' and MM predicate diagnoses through 4 diagnostic thresholds defined by TRDx: NILM (Negative) for specificity, and ASCUS+, LSIL+, and ASCH+ (Positive) for graded sensitivity (with abnormal cells decreasing in size with increasing dysplasia); and, (4) statistically determine cytologists' equivalency grading to MM using 95% Confidence Interval (CI) ranges.

Results

500 TRDx breakdown (n/%): NILM (241/48.2), ASCUS (79/15.8), ASCH (9/1.80), AGUS (2/0.40), LSIL (86/17.2), HSIL (68/13.6), CA (2/0.40), UNSAT (13/2.60). TRDx breakdown (n/%) per 4 of 4, 3 of 4, 2 of 4 diagnostic concordances: 264 (52.8%), 182 (36.4%), 54 (10.8%), respectively. No cases of discordant diagnoses were recorded. HSIL TRDx were established from 66.2% of 4 of 4 concordances, followed by NILM (59.3%), LSIL (46.5%), ASCUS (41.8%); antithetically, from 4.40% of 2 of 4 concordances. Specificity for MM predicate (NILM): 67.08%; for Cytologists 1, 2, and 3: 89.71%, 82.30%, 97.53%, respectively. For NILM threshold, cytologists revealed Significantly Superior equivalency to MM. Sensitivity for ASCUS+, LSIL+, and ASCH+ thresholds: MM (91.36%, 86.67%, 74.36%); Cytologist 1 (95.88%, 96.97%, 94.87%); Cytologist 2 (95.47%, 95.76%, 93.59%), Cytologist 3 (94.65%, 95.15%, 98.72%), respectively. Cytologists revealed Significantly Superior equivalency to MM for graded Positive thresholds; with Cytologist 3 for ASCUS+ being: Superior.

Conclusions

BestCyte detects and efficiently displays abnormal cells in strategic galleries standardizing objectivity by systematizing mosaics of cell-types for cytologists' consideration. BestCyte fosters consistent, enhanced cytologists' sensitivity values for the ASCUS+, LSIL+, and ASCH+ Positive thresholds relative to MM. Also, BestCyte facilitates improved specificity and superior equivalency grading to MM reflecting efficient screening, and reduced labor. Confident interpretations of small dysplastic epithelial cells characteristic of HSIL led to exceptional interobserver diagnostic concordance inferring BestCyte is primed for effective cervical cancer screening practice.