1
|
Zhou J, He X, Sun L, Xu J, Chen X, Chu Y, Zhou L, Liao X, Zhang B, Afvari S, Gao X. Pre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4. Nat Commun 2024; 15:5649. [PMID: 38969632 PMCID: PMC11226626 DOI: 10.1038/s41467-024-50043-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Accepted: 06/26/2024] [Indexed: 07/07/2024] Open
Abstract
Large language models (LLMs) are seen to have tremendous potential in advancing medical diagnosis recently, particularly in dermatological diagnosis, which is a very important task as skin and subcutaneous diseases rank high among the leading contributors to the global burden of nonfatal diseases. Here we present SkinGPT-4, which is an interactive dermatology diagnostic system based on multimodal large language models. We have aligned a pre-trained vision transformer with an LLM named Llama-2-13b-chat by collecting an extensive collection of skin disease images (comprising 52,929 publicly available and proprietary images) along with clinical concepts and doctors' notes, and designing a two-step training strategy. We have quantitatively evaluated SkinGPT-4 on 150 real-life cases with board-certified dermatologists. With SkinGPT-4, users could upload their own skin photos for diagnosis, and the system could autonomously evaluate the images, identify the characteristics and categories of the skin conditions, perform in-depth analysis, and provide interactive treatment recommendations.
Collapse
Affiliation(s)
- Juexiao Zhou
- Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
- Computational Bioscience Research Center, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
- DermAssure, LLC, New York, NY, USA
| | - Xiaonan He
- Emergency Critical Care Center, Beijing AnZhen Hospital, Affiliated to Capital Medical University, Beijing, China.
| | - Liyuan Sun
- Department of Dermatology, Beijing AnZhen Hospital, Affiliated to Capital Medical University, Beijing, China
| | - Jiannan Xu
- Department of Dermatology, Beijing AnZhen Hospital, Affiliated to Capital Medical University, Beijing, China
| | - Xiuying Chen
- Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
- Computational Bioscience Research Center, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
| | - Yuetan Chu
- Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
- Computational Bioscience Research Center, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
| | - Longxi Zhou
- Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
- Computational Bioscience Research Center, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
| | - Xingyu Liao
- Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
- Computational Bioscience Research Center, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
| | - Bin Zhang
- Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
- Computational Bioscience Research Center, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia
| | - Shawn Afvari
- DermAssure, LLC, New York, NY, USA
- Department of Dermatology, Brigham and Women's Hospital, Harvard University, Boston, MA, USA
- School of Medicine, New York Medical College, Valhalla, NY, USA
| | - Xin Gao
- Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia.
- Computational Bioscience Research Center, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Kingdom of Saudi Arabia.
| |
Collapse
|
3
|
Lakdawala N, Gronbeck C, Feng H. Comparison of prescribing patterns of non-physician clinicians and dermatologists in the Medicare population. Arch Dermatol Res 2023; 315:2679-2681. [PMID: 37195299 DOI: 10.1007/s00403-023-02638-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 03/23/2023] [Accepted: 05/09/2023] [Indexed: 05/18/2023]
Abstract
Non-physician clinicians (NPCs) are playing an increasing role in dermatologic patient care. This study expands upon existing workforce assessments of dermatology NPCs through the use of publicly-available Medicare datasets to better clarify prescribing patterns among independently-billing dermatology NPCs. The findings demonstrate prescribing similarities between NPCs and dermatologists for most medications, including biologic and immunosuppressive medications, although with higher use of oral prednisone, gabapentin, and hydroxyzine among NPCs. Dermatologists more frequently utilized high-potency topical steroids. These data provide initial insights into NPC prescribing patterns and should motivate further study of the identified differences and potential implications for patient care.
Collapse
Affiliation(s)
- Nehal Lakdawala
- University of Connecticut School of Medicine, Farmington, CT, USA
| | - Christian Gronbeck
- Department of Dermatology, University of Connecticut Health Center, 21 South Road, 2nd Floor, Farmington, CT, 06032, USA
| | - Hao Feng
- Department of Dermatology, University of Connecticut Health Center, 21 South Road, 2nd Floor, Farmington, CT, 06032, USA.
| |
Collapse
|
4
|
Gronbeck C, Kodumudi V, Brodell RT, Grant-Kels JM, Mostow EN, Feng H. Dermatology Workforce in the United States - Part 1: Overview, Transformations, and Implications. J Am Acad Dermatol 2022:S0190-9622(22)02240-X. [PMID: 35787408 DOI: 10.1016/j.jaad.2022.06.1191] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Revised: 06/15/2022] [Accepted: 06/22/2022] [Indexed: 11/19/2022]
Abstract
The dermatology workforce continues to evolve to meet the growing and diversified demands of the United States population. Part 1 of this continuing medical education (CME) series is designed to provide an overview of the dermatology workforce as well as delineate the motivators and socio-economic implications of significant workforce transformations which are impacting dermatologic health care. Part 2 of the series will consider the impact of workforce challenges on patient outcomes and discuss potential actions that may help to optimize workforce organization and care delivery.
Collapse
Affiliation(s)
- Christian Gronbeck
- Department of Dermatology, University of Connecticut Health Center, Farmington, Connecticut
| | - Vijay Kodumudi
- Department of Dermatology, University of Connecticut Health Center, Farmington, Connecticut
| | - Robert T Brodell
- Department of Dermatology, University of Mississippi Medical Center, Jackson, Mississippi; Sonny Montgomery Veterans Hospital, Jackson, Mississippi
| | - Jane M Grant-Kels
- Department of Dermatology, University of Connecticut Health Center, Farmington, Connecticut; Department of Dermatology, University of Florida College of Medicine, Gainesville, Florida
| | - Eliot N Mostow
- Dermatology Division, Northeast Ohio Medical University, Cleveland, Ohio
| | - Hao Feng
- Department of Dermatology, University of Connecticut Health Center, Farmington, Connecticut.
| |
Collapse
|