1
|
Irmakci I, Nateghi R, Zhou R, Vescovo M, Saft M, Ross AE, Yang XJ, Cooper LAD, Goldstein JA. Tissue Contamination Challenges the Credibility of Machine Learning Models in Real World Digital Pathology. Mod Pathol 2024; 37:100422. [PMID: 38185250 PMCID: PMC10960671 DOI: 10.1016/j.modpat.2024.100422] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Revised: 11/13/2023] [Accepted: 12/15/2023] [Indexed: 01/09/2024]
Abstract
Machine learning (ML) models are poised to transform surgical pathology practice. The most successful use attention mechanisms to examine whole slides, identify which areas of tissue are diagnostic, and use them to guide diagnosis. Tissue contaminants, such as floaters, represent unexpected tissue. Although human pathologists are extensively trained to consider and detect tissue contaminants, we examined their impact on ML models. We trained 4 whole-slide models. Three operate in placenta for the following functions: (1) detection of decidual arteriopathy, (2) estimation of gestational age, and (3) classification of macroscopic placental lesions. We also developed a model to detect prostate cancer in needle biopsies. We designed experiments wherein patches of contaminant tissue are randomly sampled from known slides and digitally added to patient slides and measured model performance. We measured the proportion of attention given to contaminants and examined the impact of contaminants in the t-distributed stochastic neighbor embedding feature space. Every model showed performance degradation in response to one or more tissue contaminants. Decidual arteriopathy detection--balanced accuracy decreased from 0.74 to 0.69 ± 0.01 with addition of 1 patch of prostate tissue for every 100 patches of placenta (1% contaminant). Bladder, added at 10% contaminant, raised the mean absolute error in estimating gestational age from 1.626 weeks to 2.371 ± 0.003 weeks. Blood, incorporated into placental sections, induced false-negative diagnoses of intervillous thrombi. Addition of bladder to prostate cancer needle biopsies induced false positives, a selection of high-attention patches, representing 0.033 mm2, and resulted in a 97% false-positive rate when added to needle biopsies. Contaminant patches received attention at or above the rate of the average patch of patient tissue. Tissue contaminants induce errors in modern ML models. The high level of attention given to contaminants indicates a failure to encode biological phenomena. Practitioners should move to quantify and ameliorate this problem.
Collapse
Affiliation(s)
- Ismail Irmakci
- Department of Pathology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois
| | - Ramin Nateghi
- Department of Pathology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois
| | - Rujoi Zhou
- Department of Pathology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois
| | - Mariavittoria Vescovo
- Department of Pathology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois
| | - Madeline Saft
- Department of Pathology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois
| | - Ashley E Ross
- Department of Pathology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois
| | - Ximing J Yang
- Department of Pathology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois
| | - Lee A D Cooper
- Department of Pathology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois
| | - Jeffery A Goldstein
- Department of Pathology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois.
| |
Collapse
|
2
|
Hodgson AJ, Shang YM, Boulianne P, Downes MR, Hwang D, Slodkowska E. A Practical Approach to Investigating Cross-Contaminants in the Anatomical Pathology Laboratory. Int J Surg Pathol 2020; 28:700-710. [PMID: 32188330 DOI: 10.1177/1066896920913110] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Tissue contaminants in anatomical pathology are not uncommon. While issues related to the presence of extraneous tissue on glass slides are often easily resolved, this is not always the case and several factors may contribute to diagnostic difficulty. Because of this, familiarity with the different steps involved in handling specimens in the anatomical pathology laboratory is essential when troubleshooting possible cross-contaminants. Most commonly, the specimen constituting the source of cross-contamination is handled before the actual contaminated case; however, this is not always so. In this article, we review the steps involved in processing pathology specimens as they pertain to cross-contamination; share an approach covering how to troubleshoot and prevent tissue contaminants in a systematic and practical manner; present some examples from our own experiences; and compare our experience to what is reported in the literature. The information included in this article will be of use to all members of the anatomical pathology team including medical laboratory technologists, laboratory managers and supervisors, pathologist assistants, trainees in pathology, and pathologists.
Collapse
Affiliation(s)
- Anjelica J Hodgson
- Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.,University of Toronto, Toronto, Ontario, Canada
| | - Yan Ming Shang
- Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
| | | | - Michelle R Downes
- Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.,University of Toronto, Toronto, Ontario, Canada
| | - David Hwang
- Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.,University of Toronto, Toronto, Ontario, Canada
| | - Elzbieta Slodkowska
- Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.,University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|