1
|
DeGrave AJ, Cai ZR, Janizek JD, Daneshjou R, Lee SI. Auditing the inference processes of medical-image classifiers by leveraging generative AI and the expertise of physicians. Nat Biomed Eng 2023:10.1038/s41551-023-01160-9. [PMID: 38155295 DOI: 10.1038/s41551-023-01160-9] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 10/30/2023] [Indexed: 12/30/2023]
Abstract
The inferences of most machine-learning models powering medical artificial intelligence are difficult to interpret. Here we report a general framework for model auditing that combines insights from medical experts with a highly expressive form of explainable artificial intelligence. Specifically, we leveraged the expertise of dermatologists for the clinical task of differentiating melanomas from melanoma 'lookalikes' on the basis of dermoscopic and clinical images of the skin, and the power of generative models to render 'counterfactual' images to understand the 'reasoning' processes of five medical-image classifiers. By altering image attributes to produce analogous images that elicit a different prediction by the classifiers, and by asking physicians to identify medically meaningful features in the images, the counterfactual images revealed that the classifiers rely both on features used by human dermatologists, such as lesional pigmentation patterns, and on undesirable features, such as background skin texture and colour balance. The framework can be applied to any specialized medical domain to make the powerful inference processes of machine-learning models medically understandable.
Collapse
Affiliation(s)
- Alex J DeGrave
- Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA
- Medical Scientist Training Program, University of Washington, Seattle, WA, USA
| | - Zhuo Ran Cai
- Program for Clinical Research and Technology, Department of Dermatology, Stanford University School of Medicine, Stanford, CA, USA
| | - Joseph D Janizek
- Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA
- Medical Scientist Training Program, University of Washington, Seattle, WA, USA
| | - Roxana Daneshjou
- Department of Dermatology, Stanford University School of Medicine, Stanford, CA, USA.
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA.
| | - Su-In Lee
- Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA.
| |
Collapse
|
2
|
DeGrave AJ, Cai ZR, Janizek JD, Daneshjou R, Lee SI. Dissection of medical AI reasoning processes via physician and generative-AI collaboration. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.05.12.23289878. [PMID: 37292705 PMCID: PMC10246034 DOI: 10.1101/2023.05.12.23289878] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Despite the proliferation and clinical deployment of artificial intelligence (AI)-based medical software devices, most remain black boxes that are uninterpretable to key stakeholders including patients, physicians, and even the developers of the devices. Here, we present a general model auditing framework that combines insights from medical experts with a highly expressive form of explainable AI that leverages generative models, to understand the reasoning processes of AI devices. We then apply this framework to generate the first thorough, medically interpretable picture of the reasoning processes of machine-learning-based medical image AI. In our synergistic framework, a generative model first renders "counterfactual" medical images, which in essence visually represent the reasoning process of a medical AI device, and then physicians translate these counterfactual images to medically meaningful features. As our use case, we audit five high-profile AI devices in dermatology, an area of particular interest since dermatology AI devices are beginning to achieve deployment globally. We reveal how dermatology AI devices rely both on features used by human dermatologists, such as lesional pigmentation patterns, as well as multiple, previously unreported, potentially undesirable features, such as background skin texture and image color balance. Our study also sets a precedent for the rigorous application of explainable AI to understand AI in any specialized domain and provides a means for practitioners, clinicians, and regulators to uncloak AI's powerful but previously enigmatic reasoning processes in a medically understandable way.
Collapse
Affiliation(s)
- Alex J DeGrave
- Paul G. Allen School of Computer Science and Engineering, University of Washington
- Medical Scientist Training Program, University of Washington
| | - Zhuo Ran Cai
- Program for Clinical Research and Technology, Stanford University
| | - Joseph D Janizek
- Paul G. Allen School of Computer Science and Engineering, University of Washington
- Medical Scientist Training Program, University of Washington
| | - Roxana Daneshjou
- Department of Dermatology, Stanford School of Medicine
- Department of Biomedical Data Science, Stanford School of Medicine
| | - Su-In Lee
- Paul G. Allen School of Computer Science and Engineering, University of Washington
| |
Collapse
|
3
|
Liu P, Su J, Zheng X, Chen M, Chen X, Li J, Peng C, Kuang Y, Zhu W. A Clinicopathological Analysis of Melanocytic Nevi: A Retrospective Series. Front Med (Lausanne) 2021; 8:681668. [PMID: 34447761 PMCID: PMC8383488 DOI: 10.3389/fmed.2021.681668] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2021] [Accepted: 06/17/2021] [Indexed: 12/22/2022] Open
Abstract
Purpose: Melanocytic nevi are common cutaneous lesions. This study aimed to demonstrate the concordance and discordance between clinical and histopathological diagnoses of melanocytic nevi and the importance of histological evaluation in differentiating malignant lesions from diseases with similar clinical manifestations. Patients and Methods: We studied 4,561 consecutive patients with a clinical diagnosis of melanocytic nevi from 2014 to 2019. We compared the clinical diagnosis with the histopathological diagnosis to establish a histopathological concordance rate and then investigated the effects of clinical characteristics and the reasons for removal on misclassification. Results: Among 4,561 patients who were clinically diagnosed with melanocytic nevi, the overall histopathological concordance rate was 82.11% (3,745 of 4,561 patients), while the histopathological discordance rate was 17.89% (816 of 4,561 patients). The histopathological concordance included 90.25% common acquired melanocytic nevi (3,380 of 3,745 patients) and 9.75% other benign melanocytic neoplasms (365 of 3,745 patients). The most common diagnostic change was to seborrheic keratosis (n = 470, 10.30%), followed by basal cell carcinoma (n = 64, 1.40%), vascular tumor (n = 53, 1.16%), fibroma (n = 43, 0.94%), epidermoid cyst (n = 34, 0.75%), wart (n = 30, 0.66%), melanoma (n = 24, 0.53%), Bowen's disease (n = 16, 0.35%), squamous cell carcinoma (n = 4, 0.09%), keratoacanthoma (n = 2, 0.04%), and other neoplasms (n = 76, 1.67%). Male sex, old age, location of the lesion, and the reasons for removal have a potential effect on misclassification. The percentages of misclassified lesions on the trunk and limbs and the perineum and buttocks were higher than those in lesions without a change in diagnosis. Importantly, locations of lesions on the head and neck were significantly related to a change in diagnosis to non-melanoma skin cancer, while locations on the hands and feet were significantly related to a change in diagnosis to melanoma. In addition to a typical clinical features, removal due to lesion changes or repeated stimulation was significantly associated with a change in diagnosis to melanoma. Conclusions: Our study emphasizes the clinical differential diagnosis of melanocytic nevi, especially the possibility of malignant tumors. The occurrence of clinical features associated with clinicopathological discordance should raise the clinical suspect and be carefully differentiated from malignant tumors.
Collapse
Affiliation(s)
- Panpan Liu
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - Juan Su
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - Xuanwei Zheng
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - Mingliang Chen
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - Xiang Chen
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - Jie Li
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - Cong Peng
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - Yehong Kuang
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - Wu Zhu
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| |
Collapse
|
4
|
Massone C, Hofman-Wellenhof R, Chiodi S, Sola S. Dermoscopic Criteria, Histopathological Correlates and Genetic Findings of Thin Melanoma on Non-Volar Skin. Genes (Basel) 2021; 12:1288. [PMID: 34440462 PMCID: PMC8391530 DOI: 10.3390/genes12081288] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Revised: 08/19/2021] [Accepted: 08/19/2021] [Indexed: 11/23/2022] Open
Abstract
Dermoscopy is a non-invasive, in vivo technique that allows the visualization of subsurface skin structures in the epidermis, at the dermoepidermal junction, and in the upper dermis. Dermoscopy brought a new dimension in evaluating melanocytic skin neoplasms (MSN) also representing a link between clinical and pathologic examination of any MSN. However, histopathology remains the gold standard in diagnosing MSN. Dermoscopic-pathologic correlation enhances the level of quality of MSN diagnosis and increases the level of confidence of pathologists. Melanoma is one of the most genetically predisposed among all cancers in humans. The genetic landscape of melanoma has been described in the last years but is still a field in continuous evolution. Melanoma genetic markers play a role not only in melanoma susceptibility, initiation, and progression but also in prognosis and therapeutic decisions. Several studies described the dermoscopic specific criteria and predictors for melanoma and their histopathologic correlates, but only a few studies investigated the correlation among dermoscopy, pathology, and genetic of MSN. The aim of this work is to review the published data about dermoscopic features of melanoma, their histopathological correlates with regards also to genetic alterations. Particularly, this review will focus on low-CSD (cumulative sun damage) melanoma or superficial spreading melanoma, high-CSD melanoma, and nevus-associated melanoma.
Collapse
Affiliation(s)
| | | | | | - Simona Sola
- Surgical Pathology, Galliera Hospital, 16128 Genoa, Italy;
| |
Collapse
|