1
|
Hekler A, Maron RC, Haggenmüller S, Schmitt M, Wies C, Utikal JS, Meier F, Hobelsberger S, Gellrich FF, Sergon M, Hauschild A, French LE, Heinzerling L, Schlager JG, Ghoreschi K, Schlaak M, Hilke FJ, Poch G, Korsing S, Berking C, Heppt MV, Erdmann M, Haferkamp S, Drexler K, Schadendorf D, Sondermann W, Goebeler M, Schilling B, Kather JN, Krieghoff-Henning E, Brinker TJ. Using multiple real-world dermoscopic photographs of one lesion improves melanoma classification via deep learning. J Am Acad Dermatol 2024; 90:1028-1031. [PMID: 38199280 DOI: 10.1016/j.jaad.2023.11.065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Revised: 10/22/2023] [Accepted: 11/27/2023] [Indexed: 01/12/2024]
Affiliation(s)
- Achim Hekler
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Roman C Maron
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Sarah Haggenmüller
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Max Schmitt
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Christoph Wies
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany; Medical Faculty, University Heidelberg, Heidelberg, Germany
| | - Jochen S Utikal
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karl University of Heidelberg, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany; DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Friedegund Meier
- Department of Dermatology, Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Sarah Hobelsberger
- Department of Dermatology, Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Frank F Gellrich
- Department of Dermatology, Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Mildred Sergon
- Department of Dermatology, Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Axel Hauschild
- Department of Dermatology, University Hospital (UKSH), Kiel, Germany
| | - Lars E French
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany; Dr. Phillip Frost Department of Dermatology and Cutaneous Surgery, University of Miami, Miller School of Medicine, Miami, Florida
| | - Lucie Heinzerling
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany; Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Justin G Schlager
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Kamran Ghoreschi
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Max Schlaak
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Franz J Hilke
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Gabriela Poch
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Sören Korsing
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Carola Berking
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Markus V Heppt
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Michael Erdmann
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Sebastian Haferkamp
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Konstantin Drexler
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Dirk Schadendorf
- Department of Dermatology, Venereology and Allergology, University Hospital Essen, Essen, Germany
| | - Wiebke Sondermann
- Department of Dermatology, Venereology and Allergology, University Hospital Essen, Essen, Germany
| | - Matthias Goebeler
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg and National Center for Tumor Diseases (NCT) WERA Würzburg, Würzburg, Germany
| | - Bastian Schilling
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg and National Center for Tumor Diseases (NCT) WERA Würzburg, Würzburg, Germany
| | - Jakob N Kather
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
2
|
Haggenmüller S, Maron RC, Hekler A, Krieghoff-Henning E, Utikal JS, Gaiser M, Müller V, Fabian S, Meier F, Hobelsberger S, Gellrich FF, Sergon M, Hauschild A, Weichenthal M, French LE, Heinzerling L, Schlager JG, Ghoreschi K, Schlaak M, Hilke FJ, Poch G, Korsing S, Berking C, Heppt MV, Erdmann M, Haferkamp S, Drexler K, Schadendorf D, Sondermann W, Goebeler M, Schilling B, Kather JN, Fröhling S, Kaminski K, Doppler A, Bucher T, Brinker TJ. Patients' and dermatologists' preferences in AI-driven skin cancer diagnostics: prospective multicentric survey study. J Am Acad Dermatol 2024:S0190-9622(24)00649-2. [PMID: 38670313 DOI: 10.1016/j.jaad.2024.04.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Revised: 03/12/2024] [Accepted: 04/03/2024] [Indexed: 04/28/2024]
Affiliation(s)
- Sarah Haggenmüller
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Roman C Maron
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Achim Hekler
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Jochen S Utikal
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karls University of Heidelberg, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany; DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Maria Gaiser
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karls University of Heidelberg, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany; DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Verena Müller
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karls University of Heidelberg, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany; DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Sascha Fabian
- Department of Economics, University of Applied Science Neu-Ulm, Neu-Ulm, Germany
| | - Friedegund Meier
- Department of Dermatology, Faculty of Medicine and University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany and Skin Cancer Center at the University Cancer Centre Dresden and National Center for Tumor Diseases, Dresden, Germany
| | - Sarah Hobelsberger
- Department of Dermatology, Faculty of Medicine and University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany and Skin Cancer Center at the University Cancer Centre Dresden and National Center for Tumor Diseases, Dresden, Germany
| | - Frank F Gellrich
- Department of Dermatology, Faculty of Medicine and University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany and Skin Cancer Center at the University Cancer Centre Dresden and National Center for Tumor Diseases, Dresden, Germany
| | - Mildred Sergon
- Department of Dermatology, Faculty of Medicine and University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany and Skin Cancer Center at the University Cancer Centre Dresden and National Center for Tumor Diseases, Dresden, Germany
| | - Axel Hauschild
- Department of Dermatology, University Hospital (UKSH), Kiel, Germany
| | | | - Lars E French
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany; Dr. Phillip Frost Department of Dermatology and Cutaneous Surgery, University of Miami, Miller School of Medicine, Miami, FL, USA
| | - Lucie Heinzerling
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany; Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Justin G Schlager
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Kamran Ghoreschi
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Max Schlaak
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Franz J Hilke
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Gabriela Poch
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Sören Korsing
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Carola Berking
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Markus V Heppt
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Michael Erdmann
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Sebastian Haferkamp
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Konstantin Drexler
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Dirk Schadendorf
- Department of Dermatology, University Hospital Essen, Essen and German Cancer Consortium, partner site Essen and National Center for Tumor Diseases (NCT), NCT-West, Campus Essen and University Alliance Ruhr, Research Center One Health, University Duisburg-Essen, Essen, Germany
| | - Wiebke Sondermann
- Department of Dermatology, University Hospital Essen, Essen and German Cancer Consortium, partner site Essen and National Center for Tumor Diseases (NCT), NCT-West, Campus Essen and University Alliance Ruhr, Research Center One Health, University Duisburg-Essen, Essen, Germany
| | - Matthias Goebeler
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg and National Center for Tumor Diseases (NCT) WERA Würzburg, Germany
| | - Bastian Schilling
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg and National Center for Tumor Diseases (NCT) WERA Würzburg, Germany
| | - Jakob N Kather
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
| | - Stefan Fröhling
- Department of Translational Medical Oncology, National Center for Tumor Diseases (NCT) Heidelberg and German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | | | - Tabea Bucher
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
3
|
Haggenmüller S, Schmitt M, Krieghoff-Henning E, Hekler A, Maron RC, Wies C, Utikal JS, Meier F, Hobelsberger S, Gellrich FF, Sergon M, Hauschild A, French LE, Heinzerling L, Schlager JG, Ghoreschi K, Schlaak M, Hilke FJ, Poch G, Korsing S, Berking C, Heppt MV, Erdmann M, Haferkamp S, Drexler K, Schadendorf D, Sondermann W, Goebeler M, Schilling B, Kather JN, Fröhling S, Brinker TJ. Federated Learning for Decentralized Artificial Intelligence in Melanoma Diagnostics. JAMA Dermatol 2024; 160:303-311. [PMID: 38324293 PMCID: PMC10851139 DOI: 10.1001/jamadermatol.2023.5550] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Accepted: 09/01/2023] [Indexed: 02/08/2024]
Abstract
Importance The development of artificial intelligence (AI)-based melanoma classifiers typically calls for large, centralized datasets, requiring hospitals to give away their patient data, which raises serious privacy concerns. To address this concern, decentralized federated learning has been proposed, where classifier development is distributed across hospitals. Objective To investigate whether a more privacy-preserving federated learning approach can achieve comparable diagnostic performance to a classical centralized (ie, single-model) and ensemble learning approach for AI-based melanoma diagnostics. Design, Setting, and Participants This multicentric, single-arm diagnostic study developed a federated model for melanoma-nevus classification using histopathological whole-slide images prospectively acquired at 6 German university hospitals between April 2021 and February 2023 and benchmarked it using both a holdout and an external test dataset. Data analysis was performed from February to April 2023. Exposures All whole-slide images were retrospectively analyzed by an AI-based classifier without influencing routine clinical care. Main Outcomes and Measures The area under the receiver operating characteristic curve (AUROC) served as the primary end point for evaluating the diagnostic performance. Secondary end points included balanced accuracy, sensitivity, and specificity. Results The study included 1025 whole-slide images of clinically melanoma-suspicious skin lesions from 923 patients, consisting of 388 histopathologically confirmed invasive melanomas and 637 nevi. The median (range) age at diagnosis was 58 (18-95) years for the training set, 57 (18-93) years for the holdout test dataset, and 61 (18-95) years for the external test dataset; the median (range) Breslow thickness was 0.70 (0.10-34.00) mm, 0.70 (0.20-14.40) mm, and 0.80 (0.30-20.00) mm, respectively. The federated approach (0.8579; 95% CI, 0.7693-0.9299) performed significantly worse than the classical centralized approach (0.9024; 95% CI, 0.8379-0.9565) in terms of AUROC on a holdout test dataset (pairwise Wilcoxon signed-rank, P < .001) but performed significantly better (0.9126; 95% CI, 0.8810-0.9412) than the classical centralized approach (0.9045; 95% CI, 0.8701-0.9331) on an external test dataset (pairwise Wilcoxon signed-rank, P < .001). Notably, the federated approach performed significantly worse than the ensemble approach on both the holdout (0.8867; 95% CI, 0.8103-0.9481) and external test dataset (0.9227; 95% CI, 0.8941-0.9479). Conclusions and Relevance The findings of this diagnostic study suggest that federated learning is a viable approach for the binary classification of invasive melanomas and nevi on a clinically representative distributed dataset. Federated learning can improve privacy protection in AI-based melanoma diagnostics while simultaneously promoting collaboration across institutions and countries. Moreover, it may have the potential to be extended to other image classification tasks in digital cancer histopathology and beyond.
Collapse
Affiliation(s)
- Sarah Haggenmüller
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Max Schmitt
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Achim Hekler
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Roman C. Maron
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Christoph Wies
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Jochen S. Utikal
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karls University of Heidelberg, Mannheim, Germany
- Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany
- DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Friedegund Meier
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Sarah Hobelsberger
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Frank F. Gellrich
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Mildred Sergon
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Axel Hauschild
- Department of Dermatology, University Hospital (UKSH), Kiel, Germany
| | - Lars E. French
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
- Dr Phillip Frost Department of Dermatology and Cutaneous Surgery, Miller School of Medicine, University of Miami, Miami, Florida
| | - Lucie Heinzerling
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen–European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Justin G. Schlager
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Kamran Ghoreschi
- Department of Dermatology, Venereology and Allergology, Charité–Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Max Schlaak
- Department of Dermatology, Venereology and Allergology, Charité–Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Franz J. Hilke
- Department of Dermatology, Venereology and Allergology, Charité–Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Gabriela Poch
- Department of Dermatology, Venereology and Allergology, Charité–Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Sören Korsing
- Department of Dermatology, Venereology and Allergology, Charité–Universitätsmedizin Berlin, Corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Carola Berking
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen–European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Markus V. Heppt
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen–European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Michael Erdmann
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen–European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Sebastian Haferkamp
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Konstantin Drexler
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Dirk Schadendorf
- Department of Dermatology, Venereology and Allergology, University Hospital Essen, Essen, Germany
| | - Wiebke Sondermann
- Department of Dermatology, Venereology and Allergology, University Hospital Essen, Essen, Germany
| | - Matthias Goebeler
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg and National Center for Tumor Diseases (NCT) WERA, Würzburg, Germany
| | - Bastian Schilling
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg and National Center for Tumor Diseases (NCT) WERA, Würzburg, Germany
| | - Jakob N. Kather
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
| | - Stefan Fröhling
- Department of Translational Medical Oncology, National Center for Tumor Diseases (NCT) Heidelberg and German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J. Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| |
Collapse
|
4
|
Kendziora B, Patzer K, French LE, Schlager JG, Hartmann D. Antibiotic Prophylaxis of Surgical Site Infections in Cutaneous Surgery: A Prospective Observational Study. Acta Derm Venereol 2023; 103:adv4469. [PMID: 37165683 DOI: 10.2340/actadv.v103.4469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2022] [Accepted: 01/24/2023] [Indexed: 05/12/2023] Open
Abstract
The use of perioperative antibiotic prophylaxis in cutaneous surgery is controversial due to unclear efficacy and, thus, potentially unnecessary side-effects. This prospective observational study analysed the efficacy of oral perioperative antibiotic prophylaxis in preventing surgical site infections. Adult patients undergoing cutaneous surgery between August 2020 and May 2021 at Ludwig-Maximilian University Hospital Munich, Germany, without prior signs of infection were eligible. Propensity score weighting was used for covariate adjustment to account for non-randomized treatment assignment. Of 758 included patients, 23 received perioperative antibiotic prophylaxis (3.0%). In this group, a surgical site infection occurred in 1 of 45 lesions (2.2%) compared with 76 of 1,189 lesions (6.5%) in the group without perioperative antibiotic prophylaxis (735 patients, 97.0%). With covariate adjustment, the odds ratio for the occurrence of a surgical site infection in patients receiving perioperative antibiotic prophylaxis was 0.114 (95% confidence interval 0.073-0.182; p <0.001) on a per lesion level. The number of lesions needed to treat to prevent 1 surgical site infection was 17.6 (95% confidence interval 16.8-19.2). This prospective observational study shows a reduction in the incidence of surgical site infection in cutaneous surgery performed with perioperative antibiotic prophylaxis. The large size difference between the 2 study groups limits the study.
Collapse
Affiliation(s)
- Benjamin Kendziora
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany.
| | - Kathrin Patzer
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Lars E French
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, German; Dr. Phillip Frost Department of Dermatology & Cutaneous Surgery, Miller School of Medicine, University of Miami, Miami, USA
| | - Justin G Schlager
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Daniela Hartmann
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| |
Collapse
|
5
|
Maron RC, Hekler A, Haggenmüller S, von Kalle C, Utikal JS, Müller V, Gaiser M, Meier F, Hobelsberger S, Gellrich FF, Sergon M, Hauschild A, French LE, Heinzerling L, Schlager JG, Ghoreschi K, Schlaak M, Hilke FJ, Poch G, Korsing S, Berking C, Heppt MV, Erdmann M, Haferkamp S, Schadendorf D, Sondermann W, Goebeler M, Schilling B, Kather JN, Fröhling S, Lipka DB, Krieghoff-Henning E, Brinker TJ. Model soups improve performance of dermoscopic skin cancer classifiers. Eur J Cancer 2022; 173:307-316. [PMID: 35973360 DOI: 10.1016/j.ejca.2022.07.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Accepted: 07/04/2022] [Indexed: 11/20/2022]
Abstract
BACKGROUND Image-based cancer classifiers suffer from a variety of problems which negatively affect their performance. For example, variation in image brightness or different cameras can already suffice to diminish performance. Ensemble solutions, where multiple model predictions are combined into one, can improve these problems. However, ensembles are computationally intensive and less transparent to practitioners than single model solutions. Constructing model soups, by averaging the weights of multiple models into a single model, could circumvent these limitations while still improving performance. OBJECTIVE To investigate the performance of model soups for a dermoscopic melanoma-nevus skin cancer classification task with respect to (1) generalisation to images from other clinics, (2) robustness against small image changes and (3) calibration such that the confidences correspond closely to the actual predictive uncertainties. METHODS We construct model soups by fine-tuning pre-trained models on seven different image resolutions and subsequently averaging their weights. Performance is evaluated on a multi-source dataset including holdout and external components. RESULTS We find that model soups improve generalisation and calibration on the external component while maintaining performance on the holdout component. For robustness, we observe performance improvements for pertubated test images, while the performance on corrupted test images remains on par. CONCLUSIONS Overall, souping for skin cancer classifiers has a positive effect on generalisation, robustness and calibration. It is easy for practitioners to implement and by combining multiple models into a single model, complexity is reduced. This could be an important factor in achieving clinical applicability, as less complexity generally means more transparency.
Collapse
Affiliation(s)
- Roman C Maron
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Achim Hekler
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Sarah Haggenmüller
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Christof von Kalle
- Department of Clinical-Translational Sciences, Charité University Medicine and Berlin Institute of Health (BIH), Berlin, Germany
| | - Jochen S Utikal
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karl University of Heidelberg, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany; DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Verena Müller
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karl University of Heidelberg, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany; DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Maria Gaiser
- Department of Dermatology, Venereology and Allergology, University Medical Center Mannheim, Ruprecht-Karl University of Heidelberg, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany; DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
| | - Friedegund Meier
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Sarah Hobelsberger
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Frank F Gellrich
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Mildred Sergon
- Skin Cancer Center at the University Cancer Center and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Axel Hauschild
- Department of Dermatology, University Hospital (UKSH), Kiel, Germany
| | - Lars E French
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany; Dr. Phillip Frost Department of Dermatology and Cutaneous Surgery, University of Miami, Miller School of Medicine, Miami, FL, USA
| | - Lucie Heinzerling
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Justin G Schlager
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Kamran Ghoreschi
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Max Schlaak
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Franz J Hilke
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Gabriela Poch
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Sören Korsing
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Carola Berking
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Markus V Heppt
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Michael Erdmann
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - European Metropolitan Region Nürnberg, CCC Alliance WERA, Erlangen, Germany
| | - Sebastian Haferkamp
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Dirk Schadendorf
- Department of Dermatology, Venereology and Allergology, University Hospital Essen, Essen, Germany
| | - Wiebke Sondermann
- Department of Dermatology, Venereology and Allergology, University Hospital Essen, Essen, Germany
| | - Matthias Goebeler
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg, Würzburg, Germany
| | - Bastian Schilling
- Department of Dermatology, Venereology and Allergology, University Hospital Würzburg, Würzburg, Germany
| | - Jakob N Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Stefan Fröhling
- Department of Translational Medical Oncology, National Center for Tumor Diseases (NCT) Heidelberg and German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Daniel B Lipka
- Department of Translational Medical Oncology, National Center for Tumor Diseases (NCT) Heidelberg and German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
6
|
Hauser K, Kurz A, Haggenmüller S, Maron RC, von Kalle C, Utikal JS, Meier F, Hobelsberger S, Gellrich FF, Sergon M, Hauschild A, French LE, Heinzerling L, Schlager JG, Ghoreschi K, Schlaak M, Hilke FJ, Poch G, Kutzner H, Berking C, Heppt MV, Erdmann M, Haferkamp S, Schadendorf D, Sondermann W, Goebeler M, Schilling B, Kather JN, Fröhling S, Lipka DB, Hekler A, Krieghoff-Henning E, Brinker TJ. Explainable artificial intelligence in skin cancer recognition: A systematic review. Eur J Cancer 2022; 167:54-69. [PMID: 35390650 DOI: 10.1016/j.ejca.2022.02.025] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Revised: 02/22/2022] [Accepted: 02/24/2022] [Indexed: 01/18/2023]
Abstract
BACKGROUND Due to their ability to solve complex problems, deep neural networks (DNNs) are becoming increasingly popular in medical applications. However, decision-making by such algorithms is essentially a black-box process that renders it difficult for physicians to judge whether the decisions are reliable. The use of explainable artificial intelligence (XAI) is often suggested as a solution to this problem. We investigate how XAI is used for skin cancer detection: how is it used during the development of new DNNs? What kinds of visualisations are commonly used? Are there systematic evaluations of XAI with dermatologists or dermatopathologists? METHODS Google Scholar, PubMed, IEEE Explore, Science Direct and Scopus were searched for peer-reviewed studies published between January 2017 and October 2021 applying XAI to dermatological images: the search terms histopathological image, whole-slide image, clinical image, dermoscopic image, skin, dermatology, explainable, interpretable and XAI were used in various combinations. Only studies concerned with skin cancer were included. RESULTS 37 publications fulfilled our inclusion criteria. Most studies (19/37) simply applied existing XAI methods to their classifier to interpret its decision-making. Some studies (4/37) proposed new XAI methods or improved upon existing techniques. 14/37 studies addressed specific questions such as bias detection and impact of XAI on man-machine-interactions. However, only three of them evaluated the performance and confidence of humans using CAD systems with XAI. CONCLUSION XAI is commonly applied during the development of DNNs for skin cancer detection. However, a systematic and rigorous evaluation of its usefulness in this scenario is lacking.
Collapse
Affiliation(s)
- Katja Hauser
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Alexander Kurz
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Sarah Haggenmüller
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Roman C Maron
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Christof von Kalle
- Department of Clinical-Translational Sciences, Charité University Medicine and Berlin Institute of Health (BIH), Berlin, Germany
| | - Jochen S Utikal
- Department of Dermatology, Heidelberg University, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Friedegund Meier
- Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Sarah Hobelsberger
- Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Frank F Gellrich
- Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Mildred Sergon
- Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | - Axel Hauschild
- Department of Dermatology, University Hospital (UKSH), Kiel, Germany
| | - Lars E French
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany; Dr. Phillip Frost Department of Dermatology and Cutaneous Surgery, University of Miami, Miller School of Medicine, Miami, FL, USA
| | - Lucie Heinzerling
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Justin G Schlager
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Kamran Ghoreschi
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Max Schlaak
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Franz J Hilke
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Gabriela Poch
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Heinz Kutzner
- Dermatopathology Laboratory, Friedrichshafen, Germany
| | - Carola Berking
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - EMN, Friedrich-Alexander University Erlangen, Nuremberg, Germany
| | - Markus V Heppt
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - EMN, Friedrich-Alexander University Erlangen, Nuremberg, Germany
| | - Michael Erdmann
- Department of Dermatology, University Hospital Erlangen, Comprehensive Cancer Center Erlangen - EMN, Friedrich-Alexander University Erlangen, Nuremberg, Germany
| | - Sebastian Haferkamp
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Dirk Schadendorf
- Department of Dermatology, University Hospital Essen, Essen, Germany
| | - Wiebke Sondermann
- Department of Dermatology, University Hospital Essen, Essen, Germany
| | - Matthias Goebeler
- Department of Dermatology, University Hospital Würzburg, Würzburg, Germany
| | - Bastian Schilling
- Department of Dermatology, University Hospital Würzburg, Würzburg, Germany
| | - Jakob N Kather
- Division of Translational Medical Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Stefan Fröhling
- National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Daniel B Lipka
- National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Achim Hekler
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
7
|
Höhn J, Krieghoff-Henning E, Jutzi TB, von Kalle C, Utikal JS, Meier F, Gellrich FF, Hobelsberger S, Hauschild A, Schlager JG, French L, Heinzerling L, Schlaak M, Ghoreschi K, Hilke FJ, Poch G, Kutzner H, Heppt MV, Haferkamp S, Sondermann W, Schadendorf D, Schilling B, Goebeler M, Hekler A, Fröhling S, Lipka DB, Kather JN, Krahl D, Ferrara G, Haggenmüller S, Brinker TJ. Combining CNN-based histologic whole slide image analysis and patient data to improve skin cancer classification. Eur J Cancer 2021; 149:94-101. [PMID: 33838393 DOI: 10.1016/j.ejca.2021.02.032] [Citation(s) in RCA: 27] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2021] [Revised: 02/12/2021] [Accepted: 02/13/2021] [Indexed: 02/07/2023]
Abstract
BACKGROUND Clinicians and pathologists traditionally use patient data in addition to clinical examination to support their diagnoses. OBJECTIVES We investigated whether a combination of histologic whole slides image (WSI) analysis based on convolutional neural networks (CNNs) and commonly available patient data (age, sex and anatomical site of the lesion) in a binary melanoma/nevus classification task could increase the performance compared with CNNs alone. METHODS We used 431 WSIs from two different laboratories and analysed the performance of classifiers that used the image or patient data individually or three common fusion techniques. Furthermore, we tested a naive combination of patient data and an image classifier: for cases interpreted as 'uncertain' (CNN output score <0.7), the decision of the CNN was replaced by the decision of the patient data classifier. RESULTS The CNN on its own achieved the best performance (mean ± standard deviation of five individual runs) with AUROC of 92.30% ± 0.23% and balanced accuracy of 83.17% ± 0.38%. While the classification performance was not significantly improved in general by any of the tested fusions, naive strategy of replacing the image classifier with the patient data classifier on slides with low output scores improved balanced accuracy to 86.72% ± 0.36%. CONCLUSION In most cases, the CNN on its own was so accurate that patient data integration did not provide any benefit. However, incorporating patient data for lesions that were classified by the CNN with low 'confidence' improved balanced accuracy.
Collapse
Affiliation(s)
- Julia Höhn
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Tanja B Jutzi
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Christof von Kalle
- Department of Clinical-Translational Sciences, Charité University Medicine and Berlin Institute of Health (BIH), Berlin, Germany
| | - Jochen S Utikal
- Department of Dermatology, Heidelberg University, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Friedegund Meier
- Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität, Dresden, Germany
| | - Frank F Gellrich
- Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität, Dresden, Germany
| | - Sarah Hobelsberger
- Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität, Dresden, Germany
| | - Axel Hauschild
- Department of Dermatology, University Hospital of Kiel, Germany
| | - Justin G Schlager
- Department of Dermatology and Allergology, Ludwig Maximilian University of Munich, Munich, Germany
| | - Lars French
- Department of Dermatology and Allergology, Ludwig Maximilian University of Munich, Munich, Germany
| | - Lucie Heinzerling
- Department of Dermatology and Allergology, Ludwig Maximilian University of Munich, Munich, Germany
| | - Max Schlaak
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Kamran Ghoreschi
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Franz J Hilke
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Gabriela Poch
- Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Heinz Kutzner
- Dermatopathology Laboratory, Friedrichshafen, Germany
| | - Markus V Heppt
- Department of Dermatology, University Hospital Erlangen, Erlangen, Germany
| | - Sebastian Haferkamp
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Wiebke Sondermann
- Department of Dermatology, University Hospital Essen, Essen, Germany
| | - Dirk Schadendorf
- Department of Dermatology, University Hospital Essen, Essen, Germany
| | - Bastian Schilling
- Department of Dermatology, University Hospital Würzburg, Würzburg, Germany
| | - Matthias Goebeler
- Department of Dermatology, University Hospital Würzburg, Würzburg, Germany
| | - Achim Hekler
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Stefan Fröhling
- Section Translational Cancer Epigenomics, Division Translational Medical Oncology, German Cancer Research Center (DKFZ) & National Center for Tumor Diseases (NCT), Heidelberg, 69120, Germany
| | - Daniel B Lipka
- Section Translational Cancer Epigenomics, Division Translational Medical Oncology, German Cancer Research Center (DKFZ) & National Center for Tumor Diseases (NCT), Heidelberg, 69120, Germany
| | - Jakob N Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Dieter Krahl
- Private Laboratory of Dermatohistopathology, Mönchhofstraße 52, Heidelberg, 69120, Germany
| | - Gerardo Ferrara
- Anatomic Pathology Unit, Macerata General Hospital, Macerata, Italy
| | - Sarah Haggenmüller
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany.
| |
Collapse
|
8
|
Maron RC, Hekler A, Krieghoff-Henning E, Schmitt M, Schlager JG, Utikal JS, Brinker TJ. Reducing the Impact of Confounding Factors on Skin Cancer Classification via Image Segmentation: Technical Model Study. J Med Internet Res 2021; 23:e21695. [PMID: 33764307 PMCID: PMC8074854 DOI: 10.2196/21695] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2020] [Revised: 11/13/2020] [Accepted: 02/08/2021] [Indexed: 01/23/2023] Open
Abstract
Background Studies have shown that artificial intelligence achieves similar or better performance than dermatologists in specific dermoscopic image classification tasks. However, artificial intelligence is susceptible to the influence of confounding factors within images (eg, skin markings), which can lead to false diagnoses of cancerous skin lesions. Image segmentation can remove lesion-adjacent confounding factors but greatly change the image representation. Objective The aim of this study was to compare the performance of 2 image classification workflows where images were either segmented or left unprocessed before the subsequent training and evaluation of a binary skin lesion classifier. Methods Separate binary skin lesion classifiers (nevus vs melanoma) were trained and evaluated on segmented and unsegmented dermoscopic images. For a more informative result, separate classifiers were trained on 2 distinct training data sets (human against machine [HAM] and International Skin Imaging Collaboration [ISIC]). Each training run was repeated 5 times. The mean performance of the 5 runs was evaluated on a multi-source test set (n=688) consisting of a holdout and an external component. Results Our findings showed that when trained on HAM, the segmented classifiers showed a higher overall balanced accuracy (75.6% [SD 1.1%]) than the unsegmented classifiers (66.7% [SD 3.2%]), which was significant in 4 out of 5 runs (P<.001). The overall balanced accuracy was numerically higher for the unsegmented ISIC classifiers (78.3% [SD 1.8%]) than for the segmented ISIC classifiers (77.4% [SD 1.5%]), which was significantly different in 1 out of 5 runs (P=.004). Conclusions Image segmentation does not result in overall performance decrease but it causes the beneficial removal of lesion-adjacent confounding factors. Thus, it is a viable option to address the negative impact that confounding factors have on deep learning models in dermatology. However, the segmentation step might introduce new pitfalls, which require further investigations.
Collapse
Affiliation(s)
- Roman C Maron
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Achim Hekler
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Max Schmitt
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Justin G Schlager
- Department of Dermatology and Allergy, University Hospital, LMU Munich, Munich, Germany
| | - Jochen S Utikal
- Department of Dermatology, Heidelberg University, Mannheim, Germany.,Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| |
Collapse
|
9
|
Hekler A, Kather JN, Krieghoff-Henning E, Utikal JS, Meier F, Gellrich FF, Upmeier Zu Belzen J, French L, Schlager JG, Ghoreschi K, Wilhelm T, Kutzner H, Berking C, Heppt MV, Haferkamp S, Sondermann W, Schadendorf D, Schilling B, Izar B, Maron R, Schmitt M, Fröhling S, Lipka DB, Brinker TJ. Effects of Label Noise on Deep Learning-Based Skin Cancer Classification. Front Med (Lausanne) 2020; 7:177. [PMID: 32435646 PMCID: PMC7218064 DOI: 10.3389/fmed.2020.00177] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2020] [Accepted: 04/16/2020] [Indexed: 11/19/2022] Open
Abstract
Recent studies have shown that deep learning is capable of classifying dermatoscopic images at least as well as dermatologists. However, many studies in skin cancer classification utilize non-biopsy-verified training images. This imperfect ground truth introduces a systematic error, but the effects on classifier performance are currently unknown. Here, we systematically examine the effects of label noise by training and evaluating convolutional neural networks (CNN) with 804 images of melanoma and nevi labeled either by dermatologists or by biopsy. The CNNs are evaluated on a test set of 384 images by means of 4-fold cross validation comparing the outputs with either the corresponding dermatological or the biopsy-verified diagnosis. With identical ground truths of training and test labels, high accuracies with 75.03% (95% CI: 74.39–75.66%) for dermatological and 73.80% (95% CI: 73.10–74.51%) for biopsy-verified labels can be achieved. However, if the CNN is trained and tested with different ground truths, accuracy drops significantly to 64.53% (95% CI: 63.12–65.94%, p < 0.01) on a non-biopsy-verified and to 64.24% (95% CI: 62.66–65.83%, p < 0.01) on a biopsy-verified test set. In conclusion, deep learning methods for skin cancer classification are highly sensitive to label noise and future work should use biopsy-verified training images to mitigate this problem.
Collapse
Affiliation(s)
- Achim Hekler
- National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Jakob N Kather
- National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany.,Department of Medicine III, RWTH University Hospital Aachen, Aachen, Germany
| | - Eva Krieghoff-Henning
- National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Jochen S Utikal
- Department of Dermatology, Heidelberg University, Mannheim, Germany.,Skin Cancer Unit, German Cancer Research Center, Heidelberg, Germany
| | - Friedegund Meier
- Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Dresden, Germany.,Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Frank F Gellrich
- Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Dresden, Germany.,Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | | | - Lars French
- Department of Dermatology and Allergology, Ludwig Maximilian University of Munich, Munich, Germany
| | - Justin G Schlager
- Department of Dermatology and Allergology, Ludwig Maximilian University of Munich, Munich, Germany
| | - Kamran Ghoreschi
- Department of Dermatology, Venereology and Allergology, Charité-Universitätsmedizin Berlin, Berlin, Germany
| | - Tabea Wilhelm
- Department of Dermatology, Venereology and Allergology, Charité-Universitätsmedizin Berlin, Berlin, Germany
| | - Heinz Kutzner
- Dermatopathology Laboratory, Friedrichshafen, Germany
| | - Carola Berking
- Department of Dermatology, University Hospital Erlangen, Erlangen, Germany
| | - Markus V Heppt
- Department of Dermatology, University Hospital Erlangen, Erlangen, Germany
| | - Sebastian Haferkamp
- Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
| | - Wiebke Sondermann
- Department of Dermatology, University Hospital Essen, Essen, Germany
| | - Dirk Schadendorf
- Department of Dermatology, University Hospital Essen, Essen, Germany
| | - Bastian Schilling
- Department of Dermatology, University Hospital Würzburg, Würzburg, Germany
| | - Benjamin Izar
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, United States
| | - Roman Maron
- National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Max Schmitt
- National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Stefan Fröhling
- National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany.,Translational Cancer Epigenomics, Division of Translational Medical Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Daniel B Lipka
- National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany.,Translational Cancer Epigenomics, Division of Translational Medical Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany.,Faculty of Medicine, Medical Center, Otto-von-Guericke-University, Magdeburg, Germany
| | - Titus J Brinker
- National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| |
Collapse
|
10
|
Schmitt J, Deckert S, Alam M, Apfelbacher C, Barbaric J, Bauer A, Chalmers J, Chosidow O, Delamere F, Doney E, Eleftheriadou V, Grainge M, Johannsen L, Kottner J, Le Cleach L, Mayer A, Pinart M, Prescott L, Prinsen CAC, Ratib S, Schlager JG, Sharma M, Thomas KS, Weberschock T, Weller K, Werner RN, Wild T, Wilkes SR, Williams HC. Report from the kick-off meeting of the Cochrane Skin Group Core Outcome Set Initiative (CSG-COUSIN). Br J Dermatol 2016; 174:287-95. [PMID: 26779929 DOI: 10.1111/bjd.14337] [Citation(s) in RCA: 39] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/20/2015] [Indexed: 11/29/2022]
Abstract
A major obstacle of evidence-based clinical decision making is the use of nonstandardized, partly untested outcome measurement instruments. Core Outcome Sets (COSs) are currently developed in different medical fields to standardize and improve the selection of outcomes and outcome measurement instruments in clinical trials, in order to pool results of trials or to allow indirect comparison between interventions. A COS is an agreed minimum set of outcomes that should be measured and reported in all clinical trials of a specific disease or trial population. The international, multidisciplinary Cochrane Skin Group Core Outcome Set Initiative (CSG-COUSIN) aims to develop and implement COSs in dermatology, thus making trial evidence comparable and, herewith, more useful for clinical decision making. The inaugural meeting of CSG-COUSIN was held on 17-18 March 2015 in Dresden, Germany, as the exclusive theme of the Annual Cochrane Skin Group Meeting. In total, 29 individuals representing a broad mix of different stakeholder groups, professions, skills and perspectives attended. This report provides a description of existing COS initiatives in dermatology, highlights current methodological challenges in COS development, and presents the concept, aims and structure of CSG-COUSIN.
Collapse
Affiliation(s)
- J Schmitt
- Centre for Evidence-Based Healthcare, Medizinische Fakultät Carl Gustav Carus, TU Dresden, Dresden, Germany.,Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, U.K
| | - S Deckert
- Centre for Evidence-Based Healthcare, Medizinische Fakultät Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - M Alam
- Department of Dermatology, Northwestern University, Chicago, IL, U.S.A
| | - C Apfelbacher
- Medical Sociology, Institute of Epidemiology and Preventive Medicine, University of Regensburg, Regensburg, Germany
| | - J Barbaric
- Department for Development, Research and Health Technology Assessment, Agency for Quality and Accreditation in Health Care and Social Welfare, Zagreb, Croatia
| | - A Bauer
- Department of Dermatology, University Hospital Carl Gustav Carus, Dresden, Germany
| | - J Chalmers
- Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, U.K
| | - O Chosidow
- Department of Dermatology, AP-HP, Hôpitaux Universitaires Henri Mondor, Créteil, France.,Université Paris-Est Créteil, Créteil, France.,Satellite Français du Cochrane Skin Group, France.,EA EpiDermE, INSERM Créteil, Créteil, France
| | - F Delamere
- Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, U.K
| | - E Doney
- Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, U.K
| | - V Eleftheriadou
- Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, U.K
| | - M Grainge
- Division of Epidemiology and Public Health, School of Medicine, University of Nottingham, Nottingham, U.K
| | - L Johannsen
- Centre for Evidence-Based Healthcare, Medizinische Fakultät Carl Gustav Carus, TU Dresden, Dresden, Germany
| | - J Kottner
- Department of Dermatology and Allergy, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - L Le Cleach
- Department of Dermatology, AP-HP, Hôpitaux Universitaires Henri Mondor, Créteil, France.,Université Paris-Est Créteil, Créteil, France.,Satellite Français du Cochrane Skin Group, France.,EA EpiDermE, INSERM Créteil, Créteil, France
| | - A Mayer
- German Center for Health Services Research in Dermatology (CVderm), Institute for Health Services Research in Dermatology and Nursing (IVDP), University Medical Center Hamburg-Eppendorf (UKE), Hamburg, Germany
| | - M Pinart
- Centre for Research in Environmental Epidemiology (CREAL), Barcelona, Spain.,IMIM (Hospital del Mar Medical Research Institute), Barcelona, Spain.,Universitat Pompeu Fabra (UPF), Barcelona, Spain.,CIBER Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain
| | - L Prescott
- Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, U.K
| | - C A C Prinsen
- VU University Medical Center, Department of Epidemiology and Biostatistics, the EMGO Institute for Health and Care Research, Amsterdam, the Netherlands
| | - S Ratib
- Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, U.K
| | - J G Schlager
- Department of Dermatology and Allergy, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - M Sharma
- Derby Teaching Hospitals NHS Foundation Trust, Derby, U.K
| | - K S Thomas
- Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, U.K
| | - T Weberschock
- Department of Dermatology, Venereology, and Allergology, J.W. Goethe-University, Hospital, Frankfurt, Germany.,Evidence-Based Medicine Frankfurt, Institute for General Practice, Goethe-University, Frankfurt, Germany
| | - K Weller
- Department of Dermatology, Venerology and Allergy, Allergie-Centrum-Charité, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - R N Werner
- Department of Dermatology, Venerology and Allergy, Division of Evidence Based Medicine (dEBM), Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - T Wild
- Centre of Wound Healing, Departments of Dermatology, Venereology, Allergology and Immunology, Dessau Medical Center, Dessau, Germany
| | - S R Wilkes
- Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, U.K
| | - H C Williams
- Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, U.K
| |
Collapse
|