51
|
Raman R, Kumar Nair V, Nedungadi P, Kumar Sahu A, Kowalski R, Ramanathan S, Achuthan K. Fake news research trends, linkages to generative artificial intelligence and sustainable development goals. Heliyon 2024; 10:e24727. [PMID: 38322879 PMCID: PMC10844021 DOI: 10.1016/j.heliyon.2024.e24727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Revised: 12/14/2023] [Accepted: 01/12/2024] [Indexed: 02/08/2024] Open
Abstract
In the digital age, where information is a cornerstone for decision-making, social media's not-so-regulated environment has intensified the prevalence of fake news, with significant implications for both individuals and societies. This study employs a bibliometric analysis of a large corpus of 9678 publications spanning 2013-2022 to scrutinize the evolution of fake news research, identifying leading authors, institutions, and nations. Three thematic clusters emerge: Disinformation in social media, COVID-19-induced infodemics, and techno-scientific advancements in auto-detection. This work introduces three novel contributions: 1) a pioneering mapping of fake news research to Sustainable Development Goals (SDGs), indicating its influence on areas like health (SDG 3), peace (SDG 16), and industry (SDG 9); 2) the utilization of Prominence percentile metrics to discern critical and economically prioritized research areas, such as misinformation and object detection in deep learning; and 3) an evaluation of generative AI's role in the propagation and realism of fake news, raising pressing ethical concerns. These contributions collectively provide a comprehensive overview of the current state and future trajectories of fake news research, offering valuable insights for academia, policymakers, and industry.
Collapse
Affiliation(s)
- Raghu Raman
- Amrita School of Business, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, 690525, India
| | - Vinith Kumar Nair
- Amrita School of Business, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, 690525, India
| | - Prema Nedungadi
- Amrita School of Computing, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, 690525, India
| | - Aditya Kumar Sahu
- Amrita School of Computing, Amrita Vishwa Vidyapeetham, Amaravati, Andhra Pradesh, 522503, India
| | - Robin Kowalski
- College of Behavioral, Social and Health Sciences, Clemson University, Clemson, SC, 29634, USA
| | - Sasangan Ramanathan
- Amrita School of Engineering, Amrita Vishwa Vidyapeetham, Coimbatore, Tamilnadu, 641112, India
| | - Krishnashree Achuthan
- Center for Cybersecurity Systems and Networks, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, 690525, India
| |
Collapse
|
52
|
Abi-Rafeh J, Xu HH, Kazan R, Tevlin R, Furnas H. Large Language Models and Artificial Intelligence: A Primer for Plastic Surgeons on the Demonstrated and Potential Applications, Promises, and Limitations of ChatGPT. Aesthet Surg J 2024; 44:329-343. [PMID: 37562022 DOI: 10.1093/asj/sjad260] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 08/02/2023] [Accepted: 08/04/2023] [Indexed: 08/12/2023] Open
Abstract
BACKGROUND The rapidly evolving field of artificial intelligence (AI) holds great potential for plastic surgeons. ChatGPT, a recently released AI large language model (LLM), promises applications across many disciplines, including healthcare. OBJECTIVES The aim of this article was to provide a primer for plastic surgeons on AI, LLM, and ChatGPT, including an analysis of current demonstrated and proposed clinical applications. METHODS A systematic review was performed identifying medical and surgical literature on ChatGPT's proposed clinical applications. Variables assessed included applications investigated, command tasks provided, user input information, AI-emulated human skills, output validation, and reported limitations. RESULTS The analysis included 175 articles reporting on 13 plastic surgery applications and 116 additional clinical applications, categorized by field and purpose. Thirty-four applications within plastic surgery are thus proposed, with relevance to different target audiences, including attending plastic surgeons (n = 17, 50%), trainees/educators (n = 8, 24.0%), researchers/scholars (n = 7, 21%), and patients (n = 2, 6%). The 15 identified limitations of ChatGPT were categorized by training data, algorithm, and ethical considerations. CONCLUSIONS Widespread use of ChatGPT in plastic surgery will depend on rigorous research of proposed applications to validate performance and address limitations. This systemic review aims to guide research, development, and regulation to safely adopt AI in plastic surgery.
Collapse
|
53
|
Sarma G, Kashyap H, Medhi PP. ChatGPT in Head and Neck Oncology-Opportunities and Challenges. Indian J Otolaryngol Head Neck Surg 2024; 76:1425-1429. [PMID: 38440617 PMCID: PMC10908741 DOI: 10.1007/s12070-023-04201-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Accepted: 08/28/2023] [Indexed: 03/06/2024] Open
Abstract
Head and neck oncology represents a complex and challenging field, encompassing the diagnosis, treatment and management of various malignancies affecting the intricate anatomical structures of the head and neck region. With advancements in artificial intelligence (AI), chatbot applications have emerged as a promising tool to revolutionize the field of Head and Neck oncology. ChatGPT is a cutting-edge language model developed by OpenAI that can help the oncologist in the clinic in scheduling appointments, establishing a clinical diagnosis, making a treatment plan and follow-up. ChatGPT also plays an essential role in telemedicine consultations, medical documentation, scientific writing and research. ChatGPT carries its inherent drawbacks too. ChatGPT raises significant ethical concerns related to authorship, accountability, transparency, bias, and the potential for misinformation. ChatGPT's training data is limited to September 2021; thus, regular updates are required to keep pace with the rapidly evolving medical research and advancements. Therefore, a judicial approach to using ChatGPT is of utmost importance. Head and Neck Oncologists can reap the maximum benefit of this technology in terms of patient care, education and research to improve clinical outcomes.
Collapse
Affiliation(s)
- Gautam Sarma
- Department of Radiation Oncology, All India Institute of Medical Sciences Guwahati, Changsari, Assam, 781101 India
| | - Hrishikesh Kashyap
- Department of Radiation Oncology, All India Institute of Medical Sciences Guwahati, Changsari, Assam, 781101 India
| | - Partha Pratim Medhi
- Department of Radiation Oncology, All India Institute of Medical Sciences Guwahati, Changsari, Assam, 781101 India
| |
Collapse
|
54
|
Wang WH, Wang SY, Huang JY, Liu XD, Yang J, Liao M, Lu Q, Wu Z. An investigation study on the interpretation of ultrasonic medical reports using OpenAI's GPT-3.5-turbo model. JOURNAL OF CLINICAL ULTRASOUND : JCU 2024; 52:105-111. [PMID: 37930057 DOI: 10.1002/jcu.23590] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 09/23/2023] [Accepted: 10/04/2023] [Indexed: 11/07/2023]
Abstract
OBJECTIVES Ultrasound medical reports are an important means of diagnosing diseases and assessing treatment effectiveness. However, their professional terms and complex sentences often make it difficult for ordinary people to understand. Therefore, this study explores the clinical value of using artificial intelligence systems based on ChatGPT to interpret ultrasound medical reports. METHODS In this study, a combination of online and offline questionnaires were used to survey both physicians and non-medical individuals. The questionnaires evaluated ChatGPT's interpretation of ultrasound reports from both professional and comprehensibility perspectives, and the results were analyzed using Excel spreadsheets. Additionally, a portion of the research content was evaluated using the Likert Scale 5-point method in the questionnaire. RESULTS According to survey results, 67.4% of surveyed doctors believe that using ChatGPT for interpreting ultrasound medical reports can help improve work efficiency. At the same time, 69.72% of non-medical professionals believe it is necessary to enhance their understanding of medical ultrasound reports through ChatGPT interpretation, and 62.58% support the application of ChatGPT to ultrasound medical reports. The non-medical group's understanding of ultrasound medical reports significantly improved (p < 0.01) after implementing ChatGPT, However, 67.49% of the general public are concerned about ChatGPT's imperfect functionality, which may cause misleading information. This reflects that the public's trust in new technology is not high enough, and they are also worried about possible privacy leaks and security issues with ChatGPT technology. CONCLUSIONS The higher acceptance and support of non-medical individuals for the interpretation of medical reports by ChatGPT might be due to the system's natural language processing abilities that allow them to better understand and evaluate report contents. However, the expertise and experience of physicians are still irreplaceable. This suggests that the ChatGPT-based ultrasound medical report interpretation system has certain clinical value and application prospects, but further optimization is necessary to address its shortcomings in data quality and professionalism. This study provides a reference and inspiration for promoting the application and development of ultrasound technology and artificial intelligence systems in the medical field.
Collapse
Affiliation(s)
- Wen Hui Wang
- Department of Ultrasound, West China Hospital of Sichuan University, Chengdu, China
| | - Shi Yu Wang
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Jia Yan Huang
- Department of Ultrasound, West China Hospital of Sichuan University, Chengdu, China
| | - Xiao di Liu
- Department of Ultrasound, West China Hospital of Sichuan University, Chengdu, China
| | - Jie Yang
- Department of Ultrasound, West China Hospital of Sichuan University, Chengdu, China
| | - Min Liao
- Department of Ultrasound, West China Hospital of Sichuan University, Chengdu, China
| | - Qiang Lu
- Department of Ultrasound, West China Hospital of Sichuan University, Chengdu, China
| | - Zhe Wu
- School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| |
Collapse
|
55
|
Zalzal HG, Abraham A, Cheng J, Shah RK. Can ChatGPT help patients answer their otolaryngology questions? Laryngoscope Investig Otolaryngol 2024; 9:e1193. [PMID: 38362184 PMCID: PMC10866598 DOI: 10.1002/lio2.1193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 11/14/2023] [Accepted: 11/22/2023] [Indexed: 02/17/2024] Open
Abstract
Background Over the past year, the world has been captivated by the potential of artificial intelligence (AI). The appetite for AI in science, specifically healthcare is huge. It is imperative to understand the credibility of large language models in assisting the public in medical queries. Objective To evaluate the ability of ChatGPT to provide reasonably accurate answers to public queries within the domain of Otolaryngology. Methods Two board-certified otolaryngologists (HZ, RS) inputted 30 text-based patient queries into the ChatGPT-3.5 model. ChatGPT responses were rated by physicians on a scale (accurate, partially accurate, incorrect), while a similar 3-point scale involving confidence was given to layperson reviewers. Demographic data involving gender and education level was recorded for the public reviewers. Inter-rater agreement percentage was based on binomial distribution for calculating the 95% confidence intervals and performing significance tests. Statistical significance was defined as p < .05 for two-sided tests. Results In testing patient queries, both Otolaryngology physicians found that ChatGPT answered 98.3% of questions correctly, but only 79.8% (range 51.7%-100%) of patients were confident that the AI model was accurate in its responses (corrected agreement = 0.682; p < .001). Among the layperson responses, the corrected coefficient was of moderate agreement (0.571; p < .001). No correlation was noted among age, gender, or education level for the layperson responses. Conclusion ChatGPT is highly accurate in responding to questions posed by the public with regards to Otolaryngology from a physician standpoint. Public reviewers were not fully confident in believing the AI model, with subjective concerns related to less trust in AI answers compared to physician explanation. Larger evaluations with a representative public sample and broader medical questions should immediately be conducted by appropriate organizations, governing bodies, and/or governmental agencies to instill public confidence in AI and ChatGPT as a medical resource. Level of Evidence 4.
Collapse
Affiliation(s)
- Habib G. Zalzal
- Division of Otolaryngology‐Head and Neck SurgeryChildren's National HospitalWashingtonDistrict of ColumbiaUSA
| | | | - Jenhao Cheng
- Quality, Safety, AnalyticsChildren's National HospitalWashingtonDistrict of ColumbiaUSA
| | - Rahul K. Shah
- Division of Otolaryngology‐Head and Neck SurgeryChildren's National HospitalWashingtonDistrict of ColumbiaUSA
| |
Collapse
|
56
|
Kavadella A, Dias da Silva MA, Kaklamanos EG, Stamatopoulos V, Giannakopoulos K. Evaluation of ChatGPT's Real-Life Implementation in Undergraduate Dental Education: Mixed Methods Study. JMIR MEDICAL EDUCATION 2024; 10:e51344. [PMID: 38111256 PMCID: PMC10867750 DOI: 10.2196/51344] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Revised: 10/28/2023] [Accepted: 12/11/2023] [Indexed: 12/20/2023]
Abstract
BACKGROUND The recent artificial intelligence tool ChatGPT seems to offer a range of benefits in academic education while also raising concerns. Relevant literature encompasses issues of plagiarism and academic dishonesty, as well as pedagogy and educational affordances; yet, no real-life implementation of ChatGPT in the educational process has been reported to our knowledge so far. OBJECTIVE This mixed methods study aimed to evaluate the implementation of ChatGPT in the educational process, both quantitatively and qualitatively. METHODS In March 2023, a total of 77 second-year dental students of the European University Cyprus were divided into 2 groups and asked to compose a learning assignment on "Radiation Biology and Radiation Protection in the Dental Office," working collaboratively in small subgroups, as part of the educational semester program of the Dentomaxillofacial Radiology module. Careful planning ensured a seamless integration of ChatGPT, addressing potential challenges. One group searched the internet for scientific resources to perform the task and the other group used ChatGPT for this purpose. Both groups developed a PowerPoint (Microsoft Corp) presentation based on their research and presented it in class. The ChatGPT group students additionally registered all interactions with the language model during the prompting process and evaluated the final outcome; they also answered an open-ended evaluation questionnaire, including questions on their learning experience. Finally, all students undertook a knowledge examination on the topic, and the grades between the 2 groups were compared statistically, whereas the free-text comments of the questionnaires were thematically analyzed. RESULTS Out of the 77 students, 39 were assigned to the ChatGPT group and 38 to the literature research group. Seventy students undertook the multiple choice question knowledge examination, and examination grades ranged from 5 to 10 on the 0-10 grading scale. The Mann-Whitney U test showed that students of the ChatGPT group performed significantly better (P=.045) than students of the literature research group. The evaluation questionnaires revealed the benefits (human-like interface, immediate response, and wide knowledge base), the limitations (need for rephrasing the prompts to get a relevant answer, general content, false citations, and incapability to provide images or videos), and the prospects (in education, clinical practice, continuing education, and research) of ChatGPT. CONCLUSIONS Students using ChatGPT for their learning assignments performed significantly better in the knowledge examination than their fellow students who used the literature research methodology. Students adapted quickly to the technological environment of the language model, recognized its opportunities and limitations, and used it creatively and efficiently. Implications for practice: the study underscores the adaptability of students to technological innovations including ChatGPT and its potential to enhance educational outcomes. Educators should consider integrating ChatGPT into curriculum design; awareness programs are warranted to educate both students and educators about the limitations of ChatGPT, encouraging critical engagement and responsible use.
Collapse
Affiliation(s)
- Argyro Kavadella
- School of Dentistry, European University Cyprus, Nicosia, Cyprus
| | - Marco Antonio Dias da Silva
- Research Group of Teleducation and Teledentistry, Federal University of Campina Grande, Campina Grande, Brazil
| | - Eleftherios G Kaklamanos
- School of Dentistry, European University Cyprus, Nicosia, Cyprus
- School of Dentistry, Aristotle University of Thessaloniki, Thessaloniki, Greece
- Mohammed Bin Rashid University of Medicine and Health Sciences, Dubai, United Arab Emirates
| | - Vasileios Stamatopoulos
- Information Management Systems Institute, ATHENA Research and Innovation Center, Athens, Greece
| | | |
Collapse
|
57
|
Jain N, Gottlich C, Fisher J, Campano D, Winston T. Assessing ChatGPT's orthopedic in-service training exam performance and applicability in the field. J Orthop Surg Res 2024; 19:27. [PMID: 38167093 PMCID: PMC10762835 DOI: 10.1186/s13018-023-04467-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Accepted: 12/12/2023] [Indexed: 01/05/2024] Open
Abstract
BACKGROUND ChatGPT has gained widespread attention for its ability to understand and provide human-like responses to inputs. However, few works have focused on its use in Orthopedics. This study assessed ChatGPT's performance on the Orthopedic In-Service Training Exam (OITE) and evaluated its decision-making process to determine whether adoption as a resource in the field is practical. METHODS ChatGPT's performance on three OITE exams was evaluated through inputting multiple choice questions. Questions were classified by their orthopedic subject area. Yearly, OITE technical reports were used to gauge scores against resident physicians. ChatGPT's rationales were compared with testmaker explanations using six different groups denoting answer accuracy and logic consistency. Variables were analyzed using contingency table construction and Chi-squared analyses. RESULTS Of 635 questions, 360 were useable as inputs (56.7%). ChatGPT-3.5 scored 55.8%, 47.7%, and 54% for the years 2020, 2021, and 2022, respectively. Of 190 correct outputs, 179 provided a consistent logic (94.2%). Of 170 incorrect outputs, 133 provided an inconsistent logic (78.2%). Significant associations were found between test topic and correct answer (p = 0.011), and type of logic used and tested topic (p = < 0.001). Basic Science and Sports had adjusted residuals greater than 1.96. Basic Science and correct, no logic; Basic Science and incorrect, inconsistent logic; Sports and correct, no logic; and Sports and incorrect, inconsistent logic; had adjusted residuals greater than 1.96. CONCLUSIONS Based on annual OITE technical reports for resident physicians, ChatGPT-3.5 performed around the PGY-1 level. When answering correctly, it displayed congruent reasoning with testmakers. When answering incorrectly, it exhibited some understanding of the correct answer. It outperformed in Basic Science and Sports, likely due to its ability to output rote facts. These findings suggest that it lacks the fundamental capabilities to be a comprehensive tool in Orthopedic Surgery in its current form. LEVEL OF EVIDENCE II.
Collapse
Affiliation(s)
- Neil Jain
- Department of Orthopedic Surgery, Texas Tech University Health Sciences Center Lubbock, 3601 4th St, Lubbock, TX, 79430, USA.
| | - Caleb Gottlich
- Department of Orthopedic Surgery, Texas Tech University Health Sciences Center Lubbock, 3601 4th St, Lubbock, TX, 79430, USA
| | - John Fisher
- Department of Orthopedic Surgery, Texas Tech University Health Sciences Center Lubbock, 3601 4th St, Lubbock, TX, 79430, USA
| | - Dominic Campano
- Department of Orthopedic Surgery, Texas Tech University Health Sciences Center Lubbock, 3601 4th St, Lubbock, TX, 79430, USA
| | - Travis Winston
- Department of Orthopedic Surgery, Texas Tech University Health Sciences Center Lubbock, 3601 4th St, Lubbock, TX, 79430, USA
| |
Collapse
|
58
|
Sumbal A, Sumbal R, Amir A. Can ChatGPT-3.5 Pass a Medical Exam? A Systematic Review of ChatGPT's Performance in Academic Testing. JOURNAL OF MEDICAL EDUCATION AND CURRICULAR DEVELOPMENT 2024; 11:23821205241238641. [PMID: 38487300 PMCID: PMC10938614 DOI: 10.1177/23821205241238641] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 02/25/2024] [Indexed: 03/17/2024]
Abstract
OBJECTIVE We, therefore, aim to conduct a systematic review to assess the academic potential of ChatGPT-3.5, along with its strengths and limitations when giving medical exams. METHOD Following PRISMA guidelines, a systemic search of the literature was performed using electronic databases PUBMED/MEDLINE, Google Scholar, and Cochrane. Articles from their inception till April 4, 2023, were queried. A formal narrative analysis was conducted by systematically arranging similarities and differences between individual findings together. RESULTS After rigorous screening, 12 articles underwent this review. All the selected papers assessed the academic performance of ChatGPT-3.5. One study compared the performance of ChatGPT-3.5 with the performance of ChatGPT-4 when giving a medical exam. Overall, ChatGPT performed well in 4 tests, averaged in 4 tests, and performed badly in 4 tests. ChatGPT's performance was directly proportional to the level of the questions' difficulty but was unremarkable on whether the questions were binary, descriptive, or MCQ-based. ChatGPT's explanation, reasoning, memory, and accuracy were remarkably good, whereas it failed to understand image-based questions, and lacked insight and critical thinking. CONCLUSION ChatGPT-3.5 performed satisfactorily in the exams it took as an examinee. However, there is a need for future related studies to fully explore the potential of ChatGPT in medical education.
Collapse
Affiliation(s)
- Anusha Sumbal
- Dow University of Health Sciences, Karachi, Pakistan
| | - Ramish Sumbal
- Dow University of Health Sciences, Karachi, Pakistan
| | - Alina Amir
- Dow University of Health Sciences, Karachi, Pakistan
| |
Collapse
|
59
|
Morales-Ramirez P, Mishek H, Dasgupta A. The Genie Is Out of the Bottle: What ChatGPT Can and Cannot Do for Medical Professionals. Obstet Gynecol 2024; 143:e1-e6. [PMID: 37944140 DOI: 10.1097/aog.0000000000005446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Accepted: 10/12/2023] [Indexed: 11/12/2023]
Abstract
ChatGPT is a cutting-edge artificial intelligence technology that was released for public use in November 2022. Its rapid adoption has raised questions about capabilities, limitations, and risks. This article presents an overview of ChatGPT, and it highlights the current state of this technology for the medical field. The article seeks to provide a balanced perspective on what the model can and cannot do in three specific domains: clinical practice, research, and medical education. It also provides suggestions on how to optimize the use of this tool.
Collapse
|
60
|
Huang X, Estau D, Liu X, Yu Y, Qin J, Li Z. Evaluating the performance of ChatGPT in clinical pharmacy: A comparative study of ChatGPT and clinical pharmacists. Br J Clin Pharmacol 2024; 90:232-238. [PMID: 37626010 DOI: 10.1111/bcp.15896] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 08/01/2023] [Accepted: 08/14/2023] [Indexed: 08/27/2023] Open
Abstract
AIMS To evaluate the performance of chat generative pretrained transformer (ChatGPT) in key domains of clinical pharmacy practice, including prescription review, patient medication education, adverse drug reaction (ADR) recognition, ADR causality assessment and drug counselling. METHODS Questions and clinical pharmacist's answers were collected from real clinical cases and clinical pharmacist competency assessment. ChatGPT's responses were generated by inputting the same question into the 'New Chat' box of ChatGPT Mar 23 Version. Five licensed clinical pharmacists independently rated these answers on a scale of 0 (Completely incorrect) to 10 (Completely correct). The mean scores of ChatGPT and clinical pharmacists were compared using a paired 2-tailed Student's t-test. The text content of the answers was also descriptively summarized together. RESULTS The quantitative results indicated that ChatGPT was excellent in drug counselling (ChatGPT: 8.77 vs. clinical pharmacist: 9.50, P = .0791) and weak in prescription review (5.23 vs. 9.90, P = .0089), patient medication education (6.20 vs. 9.07, P = .0032), ADR recognition (5.07 vs. 9.70, P = .0483) and ADR causality assessment (4.03 vs. 9.73, P = .023). The capabilities and limitations of ChatGPT in clinical pharmacy practice were summarized based on the completeness and accuracy of the answers. ChatGPT revealed robust retrieval, information integration and dialogue capabilities. It lacked medicine-specific datasets as well as the ability for handling advanced reasoning and complex instructions. CONCLUSIONS While ChatGPT holds promise in clinical pharmacy practice as a supplementary tool, the ability of ChatGPT to handle complex problems needs further improvement and refinement.
Collapse
Affiliation(s)
- Xiaoru Huang
- Department of Pharmacy, Peking University Third Hospital, Beijing, China
- Department of Pharmaceutical Management and Clinical Pharmacy, College of Pharmacy, Peking University, Beijing, China
| | - Dannya Estau
- Department of Pharmacy, Peking University Third Hospital, Beijing, China
- Department of Pharmaceutical Management and Clinical Pharmacy, College of Pharmacy, Peking University, Beijing, China
| | - Xuening Liu
- Department of Pharmacy, Peking University Third Hospital, Beijing, China
- Department of Pharmaceutical Management and Clinical Pharmacy, College of Pharmacy, Peking University, Beijing, China
| | - Yang Yu
- Department of Pharmacy, Peking University Third Hospital, Beijing, China
- Department of Pharmaceutical Management and Clinical Pharmacy, College of Pharmacy, Peking University, Beijing, China
| | - Jiguang Qin
- Department of Pharmacy, Peking University Third Hospital, Beijing, China
- Department of Pharmaceutical Management and Clinical Pharmacy, College of Pharmacy, Peking University, Beijing, China
| | - Zijian Li
- Department of Pharmacy, Peking University Third Hospital, Beijing, China
- Department of Pharmaceutical Management and Clinical Pharmacy, College of Pharmacy, Peking University, Beijing, China
- Department of Cardiology and Institute of Vascular Medicine, Peking University Third Hospital, Beijing Key Laboratory of Cardiovascular Receptors Research, Key Laboratory of Cardiovascular Molecular Biology and Regulatory Peptides, Ministry of Health, State Key Laboratory of Vascular Homeostasis and Remodeling, Peking University, Beijing, China
| |
Collapse
|
61
|
Zawiah M, Al-Ashwal FY, Gharaibeh L, Abu Farha R, Alzoubi KH, Abu Hammour K, Qasim QA, Abrah F. ChatGPT and Clinical Training: Perception, Concerns, and Practice of Pharm-D Students. J Multidiscip Healthc 2023; 16:4099-4110. [PMID: 38116306 PMCID: PMC10729768 DOI: 10.2147/jmdh.s439223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2023] [Accepted: 12/04/2023] [Indexed: 12/21/2023] Open
Abstract
Background The emergence of Chat-Generative Pre-trained Transformer (ChatGPT) by OpenAI has revolutionized AI technology, demonstrating significant potential in healthcare and pharmaceutical education, yet its real-world applicability in clinical training warrants further investigation. Methods A cross-sectional study was conducted between April and May 2023 to assess PharmD students' perceptions, concerns, and experiences regarding the integration of ChatGPT into clinical pharmacy education. The study utilized a convenient sampling method through online platforms and involved a questionnaire with sections on demographics, perceived benefits, concerns, and experience with ChatGPT. Statistical analysis was performed using SPSS, including descriptive and inferential analyses. Results The findings of the study involving 211 PharmD students revealed that the majority of participants were male (77.3%), and had prior experience with artificial intelligence (68.2%). Over two-thirds were aware of ChatGPT. Most students (n= 139, 65.9%) perceived potential benefits in using ChatGPT for various clinical tasks, with concerns including over-reliance, accuracy, and ethical considerations. Adoption of ChatGPT in clinical training varied, with some students not using it at all, while others utilized it for tasks like evaluating drug-drug interactions and developing care plans. Previous users tended to have higher perceived benefits and lower concerns, but the differences were not statistically significant. Conclusion Utilizing ChatGPT in clinical training offers opportunities, but students' lack of trust in it for clinical decisions highlights the need for collaborative human-ChatGPT decision-making. It should complement healthcare professionals' expertise and be used strategically to compensate for human limitations. Further research is essential to optimize ChatGPT's effective integration.
Collapse
Affiliation(s)
- Mohammed Zawiah
- Department of Clinical Pharmacy, College of Pharmacy, Northern Border University, Rafha, 91911, Saudi Arabia
- Department of Pharmacy Practice, College of Clinical Pharmacy, Hodeidah University, Al Hodeidah, Yemen
| | - Fahmi Y Al-Ashwal
- Department of Clinical Pharmacy, College of Pharmacy, Al-Ayen University, Thi-Qar, Iraq
| | - Lobna Gharaibeh
- Pharmacological and Diagnostic Research Center, Faculty of Pharmacy, Al-Ahliyya Amman University, Amman, Jordan
| | - Rana Abu Farha
- Clinical Pharmacy and Therapeutics Department, Faculty of Pharmacy, Applied Science Private University, Amman, Jordan
| | - Karem H Alzoubi
- Department of Pharmacy Practice and Pharmacotherapeutics, University of Sharjah, Sharjah, 27272, United Arab Emirates
- Department of Clinical Pharmacy, Faculty of Pharmacy, Jordan University of Science and Technology, Irbid, 22110, Jordan
| | - Khawla Abu Hammour
- Department of Clinical Pharmacy and Biopharmaceutics, Faculty of Pharmacy, University of Jordan, Amman, Jordan
| | - Qutaiba A Qasim
- Department of Clinical Pharmacy, College of Pharmacy, Al-Ayen University, Thi-Qar, Iraq
| | - Fahd Abrah
- Discipline of Social and Administrative Pharmacy, School of Pharmaceutical Sciences, Universiti Sains Malaysia, Penang, Malaysia
| |
Collapse
|
62
|
Riedel M, Kaefinger K, Stuehrenberg A, Ritter V, Amann N, Graf A, Recker F, Klein E, Kiechle M, Riedel F, Meyer B. ChatGPT's performance in German OB/GYN exams - paving the way for AI-enhanced medical education and clinical practice. Front Med (Lausanne) 2023; 10:1296615. [PMID: 38155661 PMCID: PMC10753765 DOI: 10.3389/fmed.2023.1296615] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Accepted: 11/27/2023] [Indexed: 12/30/2023] Open
Abstract
Background Chat Generative Pre-Trained Transformer (ChatGPT) is an artificial learning and large language model tool developed by OpenAI in 2022. It utilizes deep learning algorithms to process natural language and generate responses, which renders it suitable for conversational interfaces. ChatGPT's potential to transform medical education and clinical practice is currently being explored, but its capabilities and limitations in this domain remain incompletely investigated. The present study aimed to assess ChatGPT's performance in medical knowledge competency for problem assessment in obstetrics and gynecology (OB/GYN). Methods Two datasets were established for analysis: questions (1) from OB/GYN course exams at a German university hospital and (2) from the German medical state licensing exams. In order to assess ChatGPT's performance, questions were entered into the chat interface, and responses were documented. A quantitative analysis compared ChatGPT's accuracy with that of medical students for different levels of difficulty and types of questions. Additionally, a qualitative analysis assessed the quality of ChatGPT's responses regarding ease of understanding, conciseness, accuracy, completeness, and relevance. Non-obvious insights generated by ChatGPT were evaluated, and a density index of insights was established in order to quantify the tool's ability to provide students with relevant and concise medical knowledge. Results ChatGPT demonstrated consistent and comparable performance across both datasets. It provided correct responses at a rate comparable with that of medical students, thereby indicating its ability to handle a diverse spectrum of questions ranging from general knowledge to complex clinical case presentations. The tool's accuracy was partly affected by question difficulty in the medical state exam dataset. Our qualitative assessment revealed that ChatGPT provided mostly accurate, complete, and relevant answers. ChatGPT additionally provided many non-obvious insights, especially in correctly answered questions, which indicates its potential for enhancing autonomous medical learning. Conclusion ChatGPT has promise as a supplementary tool in medical education and clinical practice. Its ability to provide accurate and insightful responses showcases its adaptability to complex clinical scenarios. As AI technologies continue to evolve, ChatGPT and similar tools may contribute to more efficient and personalized learning experiences and assistance for health care providers.
Collapse
Affiliation(s)
- Maximilian Riedel
- Department of Gynecology and Obstetrics, Klinikum Rechts der Isar, Technical University Munich (TU), Munich, Germany
| | - Katharina Kaefinger
- Department of Gynecology and Obstetrics, Klinikum Rechts der Isar, Technical University Munich (TU), Munich, Germany
| | - Antonia Stuehrenberg
- Department of Gynecology and Obstetrics, Klinikum Rechts der Isar, Technical University Munich (TU), Munich, Germany
| | - Viktoria Ritter
- Department of Gynecology and Obstetrics, Klinikum Rechts der Isar, Technical University Munich (TU), Munich, Germany
| | - Niklas Amann
- Department of Gynecology and Obstetrics, Friedrich–Alexander-University Erlangen–Nuremberg (FAU), Erlangen, Germany
| | - Anna Graf
- Department of Gynecology and Obstetrics, Klinikum Rechts der Isar, Technical University Munich (TU), Munich, Germany
| | - Florian Recker
- Department of Gynecology and Obstetrics, Bonn University Hospital, Bonn, Germany
| | - Evelyn Klein
- Department of Gynecology and Obstetrics, Klinikum Rechts der Isar, Technical University Munich (TU), Munich, Germany
| | - Marion Kiechle
- Department of Gynecology and Obstetrics, Klinikum Rechts der Isar, Technical University Munich (TU), Munich, Germany
| | - Fabian Riedel
- Department of Gynecology and Obstetrics, Heidelberg University Hospital, Heidelberg, Germany
| | - Bastian Meyer
- Department of Gynecology and Obstetrics, Klinikum Rechts der Isar, Technical University Munich (TU), Munich, Germany
| |
Collapse
|
63
|
Wu Z, Bian L, Geng H, Zheng Z, Wang H, Zhai B. Application and challenges of ChatGPT in interventional surgery. Int J Surg 2023; 109:3747-3749. [PMID: 37713498 PMCID: PMC10720859 DOI: 10.1097/js9.0000000000000704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 08/12/2023] [Indexed: 09/17/2023]
Affiliation(s)
| | | | - Haigang Geng
- Department of Gastrointestinal Surgery, Renji Hospital, Shanghai Jiao Tong University School of Medicine
| | - Zhigang Zheng
- Department of Liver Surgery and Liver Transplantation, School of Medicine, Renji Hospital, Shanghai Jiao Tong University, Shanghai, People’s Republic of China
| | | | - Bo Zhai
- Department of Interventional Oncology
| |
Collapse
|
64
|
Hermann CE, Patel JM, Boyd L, Growdon WB, Aviki E, Stasenko M. Let's chat about cervical cancer: Assessing the accuracy of ChatGPT responses to cervical cancer questions. Gynecol Oncol 2023; 179:164-168. [PMID: 37988948 DOI: 10.1016/j.ygyno.2023.11.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2023] [Revised: 10/31/2023] [Accepted: 11/08/2023] [Indexed: 11/23/2023]
Abstract
OBJECTIVE To quantify the accuracy of ChatGPT in answering commonly asked questions pertaining to cervical cancer prevention, diagnosis, treatment, and survivorship/quality-of-life (QOL). METHODS ChatGPT was queried with 64 questions adapted from professional society websites and the authors' clinical experiences. The answers were scored by two attending Gynecologic Oncologists according to the following scale: 1) correct and comprehensive, 2) correct but not comprehensive, 3) some correct, some incorrect, and 4) completely incorrect. Scoring discrepancies were resolved by additional reviewers as needed. The proportion of responses earning each score were calculated overall and within each question category. RESULTS ChatGPT provided correct and comprehensive answers to 34 (53.1%) questions, correct but not comprehensive answers to 19 (29.7%) questions, partially incorrect answers to 10 (15.6%) questions, and completely incorrect answers to 1 (1.6%) question. Prevention and survivorship/QOL had the highest proportion of "correct" scores (scores of 1 or 2) at 22/24 (91.7%) and 15/16 (93.8%), respectively. ChatGPT performed less well in the treatment category, with 15/21 (71.4%) correct scores. It performed the worst in the diagnosis category with only 1/3 (33.3%) correct scores. CONCLUSION ChatGPT accurately answers questions about cervical cancer prevention, survivorship, and QOL. It performs less accurately for cervical cancer diagnosis and treatment. Further development of this immensely popular large language model should include physician input before it can be utilized as a tool for Gynecologists or recommended as a patient resource for information on cervical cancer diagnosis and treatment.
Collapse
Affiliation(s)
- Catherine E Hermann
- New York University Langone Health, Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, New York, NY, United States of America.
| | - Jharna M Patel
- New York University Langone Health, Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, New York, NY, United States of America
| | - Leslie Boyd
- New York University Langone Health, Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, New York, NY, United States of America
| | - Whitfield B Growdon
- New York University Langone Health, Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, New York, NY, United States of America
| | - Emeline Aviki
- New York University Langone Health Long Island, Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, Mineola, NY, United States of America
| | - Marina Stasenko
- New York University Langone Health, Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, New York, NY, United States of America
| |
Collapse
|
65
|
Ray PP. Broadening the horizon: a call for extensive exploration of ChatGPT's potential in obstetrics and gynecology. Am J Obstet Gynecol 2023; 229:706. [PMID: 37454961 DOI: 10.1016/j.ajog.2023.07.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2023] [Accepted: 07/12/2023] [Indexed: 07/18/2023]
|
66
|
Grünebaum A, Chervenak FA. The sky is the limit to explore ChatGPT's impact in obstetrics and gynecology: a response. Am J Obstet Gynecol 2023; 229:706-707. [PMID: 37460034 DOI: 10.1016/j.ajog.2023.07.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2023] [Accepted: 07/11/2023] [Indexed: 08/07/2023]
Affiliation(s)
- Amos Grünebaum
- Lenox Hill Hospital, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, 100 E. 77 St., New York, NY 10075.
| | - Frank A Chervenak
- Lenox Hill Hospital, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, 100 E. 77 St., New York, NY 10075
| |
Collapse
|
67
|
Peng C, Yang X, Chen A, Smith KE, PourNejatian N, Costa AB, Martin C, Flores MG, Zhang Y, Magoc T, Lipori G, Mitchell DA, Ospina NS, Ahmed MM, Hogan WR, Shenkman EA, Guo Y, Bian J, Wu Y. A study of generative large language model for medical research and healthcare. NPJ Digit Med 2023; 6:210. [PMID: 37973919 PMCID: PMC10654385 DOI: 10.1038/s41746-023-00958-w] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Accepted: 11/01/2023] [Indexed: 11/19/2023] Open
Abstract
There are enormous enthusiasm and concerns in applying large language models (LLMs) to healthcare. Yet current assumptions are based on general-purpose LLMs such as ChatGPT, which are not developed for medical use. This study develops a generative clinical LLM, GatorTronGPT, using 277 billion words of text including (1) 82 billion words of clinical text from 126 clinical departments and approximately 2 million patients at the University of Florida Health and (2) 195 billion words of diverse general English text. We train GatorTronGPT using a GPT-3 architecture with up to 20 billion parameters and evaluate its utility for biomedical natural language processing (NLP) and healthcare text generation. GatorTronGPT improves biomedical natural language processing. We apply GatorTronGPT to generate 20 billion words of synthetic text. Synthetic NLP models trained using synthetic text generated by GatorTronGPT outperform models trained using real-world clinical text. Physicians' Turing test using 1 (worst) to 9 (best) scale shows that there are no significant differences in linguistic readability (p = 0.22; 6.57 of GatorTronGPT compared with 6.93 of human) and clinical relevance (p = 0.91; 7.0 of GatorTronGPT compared with 6.97 of human) and that physicians cannot differentiate them (p < 0.001). This study provides insights into the opportunities and challenges of LLMs for medical research and healthcare.
Collapse
Affiliation(s)
- Cheng Peng
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA
| | - Xi Yang
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA
- Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, FL, USA
| | - Aokun Chen
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA
- Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, FL, USA
| | | | | | | | | | | | - Ying Zhang
- Research Computing, University of Florida, Gainesville, FL, USA
| | - Tanja Magoc
- Integrated Data Repository Research Services, University of Florida, Gainesville, FL, USA
| | - Gloria Lipori
- Integrated Data Repository Research Services, University of Florida, Gainesville, FL, USA
- Lillian S. Wells Department of Neurosurgery, Clinical and Translational Science Institute, University of Florida, Gainesville, FL, USA
| | - Duane A Mitchell
- Lillian S. Wells Department of Neurosurgery, Clinical and Translational Science Institute, University of Florida, Gainesville, FL, USA
| | - Naykky S Ospina
- Division of Endocrinology, Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, USA
| | - Mustafa M Ahmed
- Division of Cardiovascular Medicine, Department of Medicine, College of Medicine, University of Florida, Gainesville, FL, USA
| | - William R Hogan
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA
| | - Elizabeth A Shenkman
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA
| | - Yi Guo
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA
- Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, FL, USA
| | - Jiang Bian
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA
- Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, FL, USA
| | - Yonghui Wu
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA.
- Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, FL, USA.
| |
Collapse
|
68
|
Tiwari K, Matthews L, May B, Shamovsky V, Orlic-Milacic M, Rothfels K, Ragueneau E, Gong C, Stephan R, Li N, Wu G, Stein L, D'Eustachio P, Hermjakob H. ChatGPT usage in the Reactome curation process. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.11.08.566195. [PMID: 37986970 PMCID: PMC10659344 DOI: 10.1101/2023.11.08.566195] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/22/2023]
Abstract
Appreciating the rapid advancement and ubiquity of generative AI, particularly ChatGPT, a chatbot using large language models like GPT, we endeavour to explore the potential application of ChatGPT in the data collection and annotation stages within the Reactome curation process. This exploration aimed to create an automated or semi-automated framework to mitigate the extensive manual effort traditionally required for gathering and annotating information pertaining to biological pathways, adopting a Reactome "reaction-centric" approach. In this pilot study, we used ChatGPT/GPT4 to address gaps in the pathway annotation and enrichment in parallel with the conventional manual curation process. This approach facilitated a comparative analysis, where we assessed the outputs generated by ChatGPT against manually extracted information. The primary objective of this comparison was to ascertain the efficiency of integrating ChatGPT or other large language models into the Reactome curation workflow and helping plan our annotation pipeline, ultimately improving our protein-to-pathway association in a reliable and automated or semi-automated way. In the process, we identified some promising capabilities and inherent challenges associated with the utilisation of ChatGPT/GPT4 in general and also specifically in the context of Reactome curation processes. We describe approaches and tools for refining the output given by ChatGPT/GPT4 that aid in generating more accurate and detailed output.
Collapse
Affiliation(s)
- Krishna Tiwari
- European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Genome Campus, Hinxton, Cambridgeshire, CB10 1SD, UK
- Open Targets, Wellcome Genome Campus, Hinxton, Cambridgeshire, CB10 1SD, UK
| | - Lisa Matthews
- NYU Grossman School of Medicine, New York, NY 10016, USA
| | - Bruce May
- Ontario Institute for Cancer Research, Toronto, Ontario, M5G 0A3, Canada
| | | | | | - Karen Rothfels
- Ontario Institute for Cancer Research, Toronto, Ontario, M5G 0A3, Canada
| | - Eliot Ragueneau
- European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Genome Campus, Hinxton, Cambridgeshire, CB10 1SD, UK
- Open Targets, Wellcome Genome Campus, Hinxton, Cambridgeshire, CB10 1SD, UK
| | - Chuqiao Gong
- European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Genome Campus, Hinxton, Cambridgeshire, CB10 1SD, UK
- Open Targets, Wellcome Genome Campus, Hinxton, Cambridgeshire, CB10 1SD, UK
| | - Ralf Stephan
- Ontario Institute for Cancer Research, Toronto, Ontario, M5G 0A3, Canada
| | - Nancy Li
- Ontario Institute for Cancer Research, Toronto, Ontario, M5G 0A3, Canada
| | - Guanming Wu
- Oregon Health and Science University, Portland, OR 97239, USA
| | - Lincoln Stein
- Ontario Institute for Cancer Research, Toronto, Ontario, M5G 0A3, Canada
| | | | - Henning Hermjakob
- European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Genome Campus, Hinxton, Cambridgeshire, CB10 1SD, UK
- Open Targets, Wellcome Genome Campus, Hinxton, Cambridgeshire, CB10 1SD, UK
| |
Collapse
|
69
|
Kaarre J, Feldt R, Keeling LE, Dadoo S, Zsidai B, Hughes JD, Samuelsson K, Musahl V. Exploring the potential of ChatGPT as a supplementary tool for providing orthopaedic information. Knee Surg Sports Traumatol Arthrosc 2023; 31:5190-5198. [PMID: 37553552 PMCID: PMC10598178 DOI: 10.1007/s00167-023-07529-2] [Citation(s) in RCA: 24] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Accepted: 07/26/2023] [Indexed: 08/10/2023]
Abstract
PURPOSE To investigate the potential use of large language models (LLMs) in orthopaedics by presenting queries pertinent to anterior cruciate ligament (ACL) surgery to generative pre-trained transformer (ChatGPT, specifically using its GPT-4 model of March 14th 2023). Additionally, this study aimed to evaluate the depth of the LLM's knowledge and investigate its adaptability to different user groups. It was hypothesized that the ChatGPT would be able to adapt to different target groups due to its strong language understanding and processing capabilities. METHODS ChatGPT was presented with 20 questions and response was requested for two distinct target audiences: patients and non-orthopaedic medical doctors. Two board-certified orthopaedic sports medicine surgeons and two expert orthopaedic sports medicine surgeons independently evaluated the responses generated by ChatGPT. Mean correctness, completeness, and adaptability to the target audiences (patients and non-orthopaedic medical doctors) were determined. A three-point response scale facilitated nuanced assessment. RESULTS ChatGPT exhibited fair accuracy, with average correctness scores of 1.69 and 1.66 (on a scale from 0, incorrect, 1, partially correct, to 2, correct) for patients and medical doctors, respectively. Three of the 20 questions (15.0%) were deemed incorrect by any of the four orthopaedic sports medicine surgeon assessors. Moreover, overall completeness was calculated to be 1.51 and 1.64 for patients and medical doctors, respectively, while overall adaptiveness was determined to be 1.75 and 1.73 for patients and doctors, respectively. CONCLUSION Overall, ChatGPT was successful in generating correct responses in approximately 65% of the cases related to ACL surgery. The findings of this study imply that LLMs offer potential as a supplementary tool for acquiring orthopaedic knowledge. However, although ChatGPT can provide guidance and effectively adapt to diverse target audiences, it cannot supplant the expertise of orthopaedic sports medicine surgeons in diagnostic and treatment planning endeavours due to its limited understanding of orthopaedic domains and its potential for erroneous responses. LEVEL OF EVIDENCE V.
Collapse
Affiliation(s)
- Janina Kaarre
- Department of Orthopaedic Surgery, UPMC Freddie Fu Sports Medicine Center, University of Pittsburgh, Pittsburgh, USA
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Göteborgsvägen 31, 431 80 Mölndal, Sweden
| | - Robert Feldt
- Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden
| | - Laura E. Keeling
- Department of Orthopaedic Surgery, UPMC Freddie Fu Sports Medicine Center, University of Pittsburgh, Pittsburgh, USA
| | - Sahil Dadoo
- Department of Orthopaedic Surgery, UPMC Freddie Fu Sports Medicine Center, University of Pittsburgh, Pittsburgh, USA
| | - Bálint Zsidai
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Göteborgsvägen 31, 431 80 Mölndal, Sweden
| | - Jonathan D. Hughes
- Department of Orthopaedic Surgery, UPMC Freddie Fu Sports Medicine Center, University of Pittsburgh, Pittsburgh, USA
| | - Kristian Samuelsson
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Göteborgsvägen 31, 431 80 Mölndal, Sweden
- Department of Orthopaedics, Sahlgrenska University Hospital, Mölndal, Sweden
| | - Volker Musahl
- Department of Orthopaedic Surgery, UPMC Freddie Fu Sports Medicine Center, University of Pittsburgh, Pittsburgh, USA
| |
Collapse
|
70
|
Duey AH, Nietsch KS, Zaidat B, Ren R, Ndjonko LCM, Shrestha N, Rajjoub R, Ahmed W, Hoang T, Saturno MP, Tang JE, Gallate ZS, Kim JS, Cho SK. Thromboembolic prophylaxis in spine surgery: an analysis of ChatGPT recommendations. Spine J 2023; 23:1684-1691. [PMID: 37499880 DOI: 10.1016/j.spinee.2023.07.015] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Revised: 07/10/2023] [Accepted: 07/18/2023] [Indexed: 07/29/2023]
Abstract
BACKGROUND CONTEXT Venous thromboembolism is a negative outcome of elective spine surgery. However, the use of thromboembolic chemoprophylaxis in this patient population is controversial due to the possible increased risk of epidural hematoma. ChatGPT is an artificial intelligence model which may be able to generate recommendations for thromboembolic prophylaxis in spine surgery. PURPOSE To evaluate the accuracy of ChatGPT recommendations for thromboembolic prophylaxis in spine surgery. STUDY DESIGN/SETTING Comparative analysis. PATIENT SAMPLE None. OUTCOME MEASURES Accuracy, over-conclusiveness, supplemental, and incompleteness of ChatGPT responses compared to the North American Spine Society (NASS) clinical guidelines. METHODS ChatGPT was prompted with questions from the 2009 NASS clinical guidelines for antithrombotic therapies and evaluated for concordance with the clinical guidelines. ChatGPT-3.5 responses were obtained on March 5, 2023, and ChatGPT-4.0 responses were obtained on April 7, 2023. A ChatGPT response was classified as accurate if it did not contradict the clinical guideline. Three additional categories were created to further evaluate the ChatGPT responses in comparison to the NASS guidelines: over-conclusiveness, supplementary, and incompleteness. ChatGPT was classified as over-conclusive if it made a recommendation where the NASS guideline did not provide one. ChatGPT was classified as supplementary if it included additional relevant information not specified by the NASS guideline. ChatGPT was classified as incomplete if it failed to provide relevant information included in the NASS guideline. RESULTS Twelve clinical guidelines were evaluated in total. Compared to the NASS clinical guidelines, ChatGPT-3.5 was accurate in 4 (33%) of its responses while ChatGPT-4.0 was accurate in 11 (92%) responses. ChatGPT-3.5 was over-conclusive in 6 (50%) of its responses while ChatGPT-4.0 was over-conclusive in 1 (8%) response. ChatGPT-3.5 provided supplemental information in 8 (67%) of its responses, and ChatGPT-4.0 provided supplemental information in 11 (92%) responses. Four (33%) responses from ChatGPT-3.5 were incomplete, and 4 (33%) responses from ChatGPT-4.0 were incomplete. CONCLUSIONS ChatGPT was able to provide recommendations for thromboembolic prophylaxis with reasonable accuracy. ChatGPT-3.5 tended to cite nonexistent sources and was more likely to give specific recommendations while ChatGPT-4.0 was more conservative in its answers. As ChatGPT is continuously updated, further validation is needed before it can be used as a guideline for clinical practice.
Collapse
Affiliation(s)
- Akiro H Duey
- Department of Orthopedics, Icahn School of Medicine, New York, NY
| | | | - Bashar Zaidat
- Department of Orthopedics, Icahn School of Medicine, New York, NY
| | - Renee Ren
- Department of Orthopedics, Icahn School of Medicine, New York, NY
| | | | - Nancy Shrestha
- Department of Orthopedics, Icahn School of Medicine, New York, NY
| | - Rami Rajjoub
- Department of Orthopedics, Icahn School of Medicine, New York, NY
| | - Wasil Ahmed
- Department of Orthopedics, Icahn School of Medicine, New York, NY
| | - Timothy Hoang
- Department of Orthopedics, Icahn School of Medicine, New York, NY
| | | | - Justin E Tang
- Department of Orthopedics, Icahn School of Medicine, New York, NY
| | | | - Jun S Kim
- Department of Orthopedics, Icahn School of Medicine, New York, NY
| | - Samuel K Cho
- Department of Orthopedics, Icahn School of Medicine, New York, NY.
| |
Collapse
|
71
|
Chakraborty C, Pal S, Bhattacharya M, Dash S, Lee SS. Overview of Chatbots with special emphasis on artificial intelligence-enabled ChatGPT in medical science. Front Artif Intell 2023; 6:1237704. [PMID: 38028668 PMCID: PMC10644239 DOI: 10.3389/frai.2023.1237704] [Citation(s) in RCA: 15] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Accepted: 10/05/2023] [Indexed: 12/01/2023] Open
Abstract
The release of ChatGPT has initiated new thinking about AI-based Chatbot and its application and has drawn huge public attention worldwide. Researchers and doctors have started thinking about the promise and application of AI-related large language models in medicine during the past few months. Here, the comprehensive review highlighted the overview of Chatbot and ChatGPT and their current role in medicine. Firstly, the general idea of Chatbots, their evolution, architecture, and medical use are discussed. Secondly, ChatGPT is discussed with special emphasis of its application in medicine, architecture and training methods, medical diagnosis and treatment, research ethical issues, and a comparison of ChatGPT with other NLP models are illustrated. The article also discussed the limitations and prospects of ChatGPT. In the future, these large language models and ChatGPT will have immense promise in healthcare. However, more research is needed in this direction.
Collapse
Affiliation(s)
- Chiranjib Chakraborty
- Department of Biotechnology, School of Life Science and Biotechnology, Adamas University, Kolkata, West Bengal, India
| | - Soumen Pal
- School of Mechanical Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India
| | | | - Snehasish Dash
- School of Mechanical Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India
| | - Sang-Soo Lee
- Institute for Skeletal Aging and Orthopedic Surgery, Hallym University Chuncheon Sacred Heart Hospital, Chuncheon-si, Gangwon-do, Republic of Korea
| |
Collapse
|
72
|
Cavazzotto TG, Dantas DB, Queiroga MR. ChatGPT and exercise prescription: Human vs. machine or human plus machine? JOURNAL OF SPORT AND HEALTH SCIENCE 2023; 13:S2095-2546(23)00106-0. [PMID: 39492473 PMCID: PMC11282324 DOI: 10.1016/j.jshs.2023.10.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Accepted: 05/08/2023] [Indexed: 11/05/2024]
Affiliation(s)
| | - Diego Bessa Dantas
- Postgraduate Program in Physical Education UEM-UEL, State University of Londrina, Parana 86057-970, Brazil
| | - Marcos Roberto Queiroga
- Department of Physical Education, Midwestern Parana State University, Parana 85040-167, Brazil
| |
Collapse
|
73
|
Zalzal HG, Cheng J, Shah RK. Evaluating the Current Ability of ChatGPT to Assist in Professional Otolaryngology Education. OTO Open 2023; 7:e94. [PMID: 38020045 PMCID: PMC10663981 DOI: 10.1002/oto2.94] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 10/11/2023] [Accepted: 11/04/2023] [Indexed: 12/01/2023] Open
Abstract
Objective To quantify ChatGPT's concordance with expert Otolaryngologists when posed with high-level questions that require blending rote memorization and critical thinking. Study Design Cross-sectional survey. Setting OpenAI's ChatGPT-3.5 Platform. Methods Two board-certified otolaryngologists (HZ, RS) input 2 sets of 30 text-based questions (open-ended and single-answer multiple-choice) into the ChatGPT-3.5 model. Responses were rated on a scale (correct, partially correct, incorrect) by each Otolaryngologist working simultaneously with the AI model. Interrater agreement percentage was based on binomial distribution for calculating the 95% confidence intervals and performing significance tests. Statistical significance was defined as P < .05 for 2-sided tests. Results In testing open-ended questions, the ChatGPT model had 56.7% of initially answering questions with complete accuracy, and 86.7% chance of answer with some accuracy (corrected agreement = 80.1%; P < .001). For repeat questions, ChatGPT improved to 73.3% with complete accuracy and 96.7% with some accuracy (corrected agreement = 88.8%; P < .001). For multiple-choice questions, the ChatGPT model performed substantially worse (43.3% correct). Conclusion ChatGPT currently does not provide reliably accurate responses to sophisticated questions in Otolaryngology. Professional societies must be aware of the potential of this tool and prevent unscrupulous use during test-taking situations and consider guidelines for clinical scenarios. Expert clinical oversight is still necessary for myriad use cases (eg, hallucination).
Collapse
Affiliation(s)
- Habib G. Zalzal
- Division of Otolaryngology–Head and Neck SurgeryChildren's National HospitalWashingtonDistrict of ColumbiaUSA
| | - Jenhao Cheng
- Quality, Safety, AnalyticsChildren's National HospitalWashingtonDistrict of ColumbiaUSA
| | - Rahul K. Shah
- Division of Otolaryngology–Head and Neck SurgeryChildren's National HospitalWashingtonDistrict of ColumbiaUSA
| |
Collapse
|
74
|
Li J, Zhong J, Li Z, Xiao Y, Wang S. Ectopic Pituitary Neuroendocrine Tumor: A Case Report Written With the Help of ChatGPT. Cureus 2023; 15:e46999. [PMID: 37965416 PMCID: PMC10641033 DOI: 10.7759/cureus.46999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/12/2023] [Indexed: 11/16/2023] Open
Abstract
ChatGPT (Chat Generative Pre-trained Transformer) is a chat robot program developed by OpenAI (OpenAI Incorporated, Mission District, San Francisco, USA). It is a natural-language processing tool driven by artificial intelligence technology. It can complete tasks such as writing emails, video scripts, copywriting, translation, coding, and writing papers. At present, several studies have been conducted on the use of this tool for academic writing, so we report a rare case of suprasellar ectopic pituitary neuroendocrine tumor (PitNET) secreting growth hormones, in a female 53-year-old patient with facial changes and hypertrophy of her hands and feet for six years. When asking questions on unusual topics, ChatGPT can provide answers but not specific ones, and similar to most machine learning solutions, verifying the correctness of information is a problem. Question asked from ChatGPT: Summarize ectopic pituitary neuroendocrine tumor. "Suprasellar heterotopic pituitary neuroendocrine tumors refer to tumors formed by the heterotopic growth of pituitary neuroendocrine tumor cells in the sellar region other than the pituitary glands. This tumor is common in middle-aged women, and it often manifests as headache, visual impairment, and endocrine dysfunction. Diagnosis is usually made by combining history, physical examination, blood tests, and imaging. Treatment is primarily performed through surgical resection of the tumor, radiotherapy, and drug therapy, and the prognosis is mostly good." The content in quotation marks is generated by ChatGPT.
Collapse
Affiliation(s)
- Jun Li
- Neurosurgery, Department of Neurosurgery, Fuzhou 900th Hospital, Fuzong Clinical Medical College of Fujian Medical University, Fuzhou, CHN
| | - Jiansheng Zhong
- Neurosurgery, Department of Neurosurgery, Fuzhou 900th Hospital, Fuzong Clinical Medical College of Fujian Medical University, Fuzhou, CHN
| | - Ziqi Li
- Neurosurgery, Department of Neurosurgery, Oriental Hospital Affiliated to Xiamen University, Fuzhou, CHN
| | - Yong Xiao
- Neurosurgery, Central Institute for Mental Health, University of Heidelberg, Heidelberg, DEU
| | - Shousen Wang
- Neurosurgery, Department of Neurosurgery, Oriental Hospital Affiliated to Xiamen University, Fuzhou, CHN
| |
Collapse
|
75
|
Liu G, Ma X, Zhang Y, Su B, Liu P. GPT4: The Indispensable Helper for Neurosurgeons in the New Era. Ann Biomed Eng 2023; 51:2113-2115. [PMID: 37204548 DOI: 10.1007/s10439-023-03241-x] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Accepted: 05/11/2023] [Indexed: 05/20/2023]
Abstract
GPT4 is the newest multimodal language model released by OpenAI. With its powerful capabilities, GPT4 has great potential to revolutionize the healthcare industry. In this study, we proposed various ways GPT4 could display its talents in the field of neurosurgery in future. We believe that GPT4 is prone to become an indispensable assistant for neurosurgeons in the new era.
Collapse
Affiliation(s)
- Gemingtian Liu
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Xin Ma
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Yu Zhang
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Boyan Su
- Department of Neurosurgery, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua University, Beijing, China
| | - Pinan Liu
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
- Department of Neural Reconstruction, Beijing Neurosurgery Institute, Capital Medical University, Beijing, China.
| |
Collapse
|
76
|
Eggmann F, Weiger R, Zitzmann NU, Blatz MB. Implications of large language models such as ChatGPT for dental medicine. J ESTHET RESTOR DENT 2023; 35:1098-1102. [PMID: 37017291 DOI: 10.1111/jerd.13046] [Citation(s) in RCA: 44] [Impact Index Per Article: 44.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 03/25/2023] [Accepted: 03/28/2023] [Indexed: 04/06/2023]
Abstract
OBJECTIVE This article provides an overview of the implications of ChatGPT and other large language models (LLMs) for dental medicine. OVERVIEW ChatGPT, a LLM trained on massive amounts of textual data, is adept at fulfilling various language-related tasks. Despite its impressive capabilities, ChatGPT has serious limitations, such as occasionally giving incorrect answers, producing nonsensical content, and presenting misinformation as fact. Dental practitioners, assistants, and hygienists are not likely to be significantly impacted by LLMs. However, LLMs could affect the work of administrative personnel and the provision of dental telemedicine. LLMs offer potential for clinical decision support, text summarization, efficient writing, and multilingual communication. As more people seek health information from LLMs, it is crucial to safeguard against inaccurate, outdated, and biased responses to health-related queries. LLMs pose challenges for patient data confidentiality and cybersecurity that must be tackled. In dental education, LLMs present fewer challenges than in other academic fields. LLMs can enhance academic writing fluency, but acceptable usage boundaries in science need to be established. CONCLUSIONS While LLMs such as ChatGPT may have various useful applications in dental medicine, they come with risks of malicious use and serious limitations, including the potential for misinformation. CLINICAL SIGNIFICANCE Along with the potential benefits of using LLMs as an additional tool in dental medicine, it is crucial to carefully consider the limitations and potential risks inherent in such artificial intelligence technologies.
Collapse
Affiliation(s)
- Florin Eggmann
- Department of Preventive and Restorative Sciences, Penn Dental Medicine, Robert Schattner Center, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Periodontology, Endodontology, and Cariology, University Center for Dental Medicine Basel UZB, University of Basel, Basel, Switzerland
| | - Roland Weiger
- Department of Periodontology, Endodontology, and Cariology, University Center for Dental Medicine Basel UZB, University of Basel, Basel, Switzerland
| | - Nicola U Zitzmann
- Department of Reconstructive Dentistry, University Center for Dental Medicine Basel UZB, University of Basel, Basel, Switzerland
| | - Markus B Blatz
- Department of Preventive and Restorative Sciences, Penn Dental Medicine, Robert Schattner Center, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
77
|
Nadarzynski T, Lunt A, Knights N, Bayley J, Llewellyn C. "But can chatbots understand sex?" Attitudes towards artificial intelligence chatbots amongst sexual and reproductive health professionals: An exploratory mixed-methods study. Int J STD AIDS 2023; 34:809-816. [PMID: 37269292 PMCID: PMC10561522 DOI: 10.1177/09564624231180777] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Accepted: 05/22/2023] [Indexed: 06/05/2023]
Abstract
BACKGROUND Artificial Intelligence (AI)-enabled chatbots can offer anonymous education about sexual and reproductive health (SRH). Understanding chatbot acceptability and feasibility allows the identification of barriers to the design and implementation. METHODS In 2020, we conducted an online survey and qualitative interviews with SRH professionals recruited online to explore the views on AI, automation and chatbots. Qualitative data were analysed thematically. RESULTS Amongst 150 respondents (48% specialist doctor/consultant), only 22% perceived chatbots as effective and 24% saw them as ineffective for SRH advice [Mean = 2.91, SD = 0.98, range: 1-5]. Overall, there were mixed attitudes towards SRH chatbots [Mean = 4.03, SD = 0.87, range: 1-7]. Chatbots were most acceptable for appointment booking, general sexual health advice and signposting, but not acceptable for safeguarding, virtual diagnosis, and emotional support. Three themes were identified: "Moving towards a 'digital' age'", "AI improving access and service efficacy", and "Hesitancy towards AI". CONCLUSIONS Half of SRH professionals were hesitant about the use of chatbots in SRH services, attributed to concerns about patient safety, and lack of familiarity with this technology. Future studies should explore the role of AI chatbots as supplementary tools for SRH promotion. Chatbot designers need to address the concerns of health professionals to increase acceptability and engagement with AI-enabled services.
Collapse
Affiliation(s)
| | - Alexandria Lunt
- Brighton and Sussex Medical School, University of Sussex, Brighton
| | | | | | - Carrie Llewellyn
- Brighton and Sussex Medical School, University of Sussex, Brighton
| |
Collapse
|
78
|
Luykx JJ, Gerritse F, Habets PC, Vinkers CH. The performance of ChatGPT in generating answers to clinical questions in psychiatry: a two-layer assessment. World Psychiatry 2023; 22:479-480. [PMID: 37713576 PMCID: PMC10503909 DOI: 10.1002/wps.21145] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 05/30/2023] [Indexed: 09/17/2023] Open
Affiliation(s)
- Jurjen J Luykx
- Department of Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, Maastricht University Medical Centre, Maastricht, The Netherlands
- Department of Psychiatry, UMC Utrecht Brain Center, University Medical Center Utrecht, Utrecht, The Netherlands
- Outpatient Second Opinion Clinic, GGNet Mental Health, Warnsveld, The Netherlands
| | - Frank Gerritse
- Department of Psychiatry, Tergooi MC, Hilversum, The Netherlands
| | - Philippe C Habets
- Department of Psychiatry and Anatomy & Neurosciences, Amsterdam University Medical Center, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
- Department of Internal Medicine, Leiden University Medical Center, Leiden, The Netherlands
| | - Christiaan H Vinkers
- Department of Psychiatry and Anatomy & Neurosciences, Amsterdam University Medical Center, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
- Amsterdam Public Health, Mental Health Program and Amsterdam Neuroscience, Mood, Anxiety, Psychosis, Sleep & Stress Program, Amsterdam, The Netherlands
- GGZ inGeest Mental Health Care, Amsterdam, The Netherlands
| |
Collapse
|
79
|
Mészáros B, Kukor Z, Valent S. Recent Advances in the Prevention and Screening of Preeclampsia. J Clin Med 2023; 12:6020. [PMID: 37762960 PMCID: PMC10532380 DOI: 10.3390/jcm12186020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 09/08/2023] [Accepted: 09/13/2023] [Indexed: 09/29/2023] Open
Abstract
Throughout the history of medicine, preeclampsia has remained an enigmatic field of obstetrics. In 2023, despite its prevalence and impact, preeclampsia's exact cause and effective treatment remain elusive; the current options are limited to delivery. The purpose of this review is to summarize the knowledge of the possible novel prophylactic therapies and screening methods for preeclampsia, thereby providing valuable insights for healthcare professionals and researchers. Aspirin and LMWH have already been widely used; meanwhile, calcium, vitamin D, and pravastatin show promise, and endothelin receptor antagonists are being explored. Stress reduction, dietary changes, and lifestyle modifications are also being investigated. Another interesting and fast-growing area is AI- and software-based screening methods. It is also key to find novel biomarkers, which, in some cases, are not only able to predict the development of the disease, but some of them hold promise to be a potential therapeutic target. We conclude that, while a definitive cure for preeclampsia may not be eligible in the near future, it is likely that the assessment and enhancement of preventive methods will lead to the prevention of many cases. However, it is also important to highlight that more additional research is needed in the future to clarify the exact pathophysiology of preeclampsia and to thus identify potential therapeutic targets for more improved treatment methods.
Collapse
Affiliation(s)
- Balázs Mészáros
- Department of Obstetrics and Gynecology, Semmelweis University, 1082 Budapest, Hungary
| | - Zoltán Kukor
- Department of Molecular Biology, Institute of Biochemistry and Molecular Biology, Semmelweis University, 1082 Budapest, Hungary
| | - Sándor Valent
- Department of Obstetrics and Gynecology, Semmelweis University, 1082 Budapest, Hungary
| |
Collapse
|
80
|
Zhang L, Tashiro S, Mukaino M, Yamada S. Use of artificial intelligence large language models as a clinical tool in rehabilitation medicine: a comparative test case. J Rehabil Med 2023; 55:jrm13373. [PMID: 37691497 PMCID: PMC10501385 DOI: 10.2340/jrm.v55.13373] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2023] [Accepted: 07/05/2023] [Indexed: 09/12/2023] Open
Abstract
OBJECTIVE To explore the potential use of artificial intelligence language models in formulating rehabilitation prescriptions and International Classification of Functioning, Disability and Health (ICF) codes. Design: Comparative study based on a single case report compared to standard answers from a textbook. SUBJECTS A stroke case from textbook. Methods: Chat Generative Pre-Trained Transformer-4 (ChatGPT-4)was used to generate comprehensive medical and rehabilitation prescription information and ICF codes pertaining to the stroke case. This information was compared with standard answers from textbook, and 2 licensed Physical Medicine and Rehabilitation (PMR) clinicians reviewed the artificial intelligence recommendations for further discussion. RESULTS ChatGPT-4 effectively formulated rehabilitation prescriptions and ICF codes for a typical stroke case, together with a rationale to support its recommendations. This information was generated in seconds. Compared with standard answers, the large language model generated broader and more general prescriptions in terms of medical problems and management plans, rehabilitation problems and management plans, as well as rehabilitation goals. It also demonstrated the ability to propose specified approaches for each rehabilitation therapy. The language model made an error regarding the ICF category for the stroke case, but no mistakes were identified in the ICF codes assigned. Conclusion: This test case suggests that artificial intelligence language models have potential use in facilitating clinical practice and education in the field of rehabilitation medicine.
Collapse
Affiliation(s)
- Liang Zhang
- Department of Rehabilitation Medicine, Kyorin University School of Medicine, Japan
| | - Syoichi Tashiro
- Department of Rehabilitation Medicine, Kyorin University School of Medicine, Japan; Department of Rehabilitation Medicine, Keio University School of Medicine, Japan
| | - Masahiko Mukaino
- Department of Rehabilitation Medicine, Hokkaido University Hospital, Japan
| | - Shin Yamada
- Department of Rehabilitation Medicine, Kyorin University School of Medicine, Japan.
| |
Collapse
|
81
|
Zhang H, Guan Y, Chen J, Tong W. Commentary: AI-based online chat and the future of oncology care: a promising technology or a solution in search of a problem? Front Oncol 2023; 13:1239932. [PMID: 37746294 PMCID: PMC10517627 DOI: 10.3389/fonc.2023.1239932] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Accepted: 08/21/2023] [Indexed: 09/26/2023] Open
Affiliation(s)
- Hui Zhang
- Department of Rehabilitation and Elderly Care, Gannan Healthcare Vocational College, Ganzhou, China
| | - Yongfu Guan
- Department of Rehabilitation and Elderly Care, Gannan Healthcare Vocational College, Ganzhou, China
| | - Jinping Chen
- Department of Rehabilitation and Elderly Care, Gannan Healthcare Vocational College, Ganzhou, China
| | - Wenting Tong
- Department of Pharmacy, Gannan Healthcare Vocational College, Ganzhou, China
| |
Collapse
|
82
|
Li W, Zhang Y, Chen F. ChatGPT in Colorectal Surgery: A Promising Tool or a Passing Fad? Ann Biomed Eng 2023; 51:1892-1897. [PMID: 37162695 DOI: 10.1007/s10439-023-03232-y] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2023] [Accepted: 05/03/2023] [Indexed: 05/11/2023]
Abstract
Colorectal surgery is a specialized branch of surgery that involves the diagnosis and treatment of conditions affecting the colon, rectum, and anus. In the recent years, the use of artificial intelligence (AI) has gained considerable interest in various medical specialties, including surgery. Chatbot Generative Pre-Trained Transformer (ChatGPT), an AI-based chatbot developed by OpenAI, has shown great potential in improving the quality of healthcare delivery by providing accurate and timely information to both patients and healthcare professionals. In this paper, we investigate the potential application of ChatGPT in colorectal surgery. We also discuss the potential advantages and challenges associated with the implementation of ChatGPT in the surgical setting. Furthermore, we address the socio-ethical implications of utilizing ChatGPT in healthcare. This includes concerns over patient privacy, liability, and the potential impact on the doctor-patient relationship. Our findings suggest that ChatGPT has the potential to revolutionize the field of colorectal surgery by providing personalized and precise medical information, reducing errors and complications, and improving patient outcomes.
Collapse
Affiliation(s)
- Wenbo Li
- Department of Nursing, Jinzhou Medical University, Jinzhou, China
| | - Yinxu Zhang
- Department of Colorectal Surgery, The First Affiliated Hospital, Jinzhou Medical University, Jinzhou, 121001, China
| | - Fengmin Chen
- Department of Colorectal Surgery, The First Affiliated Hospital, Jinzhou Medical University, Jinzhou, 121001, China.
| |
Collapse
|
83
|
Sanchez-Ramos L, Lin L, Romero R. Beware of references when using ChatGPT as a source of information to write scientific articles. Am J Obstet Gynecol 2023; 229:356-357. [PMID: 37031761 PMCID: PMC10524915 DOI: 10.1016/j.ajog.2023.04.004] [Citation(s) in RCA: 17] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Accepted: 04/03/2023] [Indexed: 04/11/2023]
Affiliation(s)
- Luis Sanchez-Ramos
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, University of Florida College of Medicine, 653 West 8th St, Jacksonville, FL 32209.
| | - Lifeng Lin
- Department of Epidemiology and Biostatistics, The University of Arizona, Tucson, AZ
| | - Roberto Romero
- Pregnancy Research Branch, Division of Obstetrics and Maternal-Fetal Medicine, Division of Intramural Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, U.S. Department of Health and Human Services, Bethesda, MD and Detroit, MI; Department of Obstetrics and Gynecology, University of Michigan, Ann Arbor, MI; Department of Epidemiology and Biostatistics, Michigan University, East Lansing, MI
| |
Collapse
|
84
|
Hoch CC, Wollenberg B, Lüers JC, Knoedler S, Knoedler L, Frank K, Cotofana S, Alfertshofer M. ChatGPT's quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questions. Eur Arch Otorhinolaryngol 2023; 280:4271-4278. [PMID: 37285018 PMCID: PMC10382366 DOI: 10.1007/s00405-023-08051-4] [Citation(s) in RCA: 67] [Impact Index Per Article: 67.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 05/26/2023] [Indexed: 06/08/2023]
Abstract
PURPOSE With the increasing adoption of artificial intelligence (AI) in various domains, including healthcare, there is growing acceptance and interest in consulting AI models to provide medical information and advice. This study aimed to evaluate the accuracy of ChatGPT's responses to practice quiz questions designed for otolaryngology board certification and decipher potential performance disparities across different otolaryngology subspecialties. METHODS A dataset covering 15 otolaryngology subspecialties was collected from an online learning platform funded by the German Society of Oto-Rhino-Laryngology, Head and Neck Surgery, designed for board certification examination preparation. These questions were entered into ChatGPT, with its responses being analyzed for accuracy and variance in performance. RESULTS The dataset included 2576 questions (479 multiple-choice and 2097 single-choice), of which 57% (n = 1475) were answered correctly by ChatGPT. An in-depth analysis of question style revealed that single-choice questions were associated with a significantly higher rate (p < 0.001) of correct responses (n = 1313; 63%) compared to multiple-choice questions (n = 162; 34%). Stratified by question categories, ChatGPT yielded the highest rate of correct responses (n = 151; 72%) in the field of allergology, whereas 7 out of 10 questions (n = 65; 71%) on legal otolaryngology aspects were answered incorrectly. CONCLUSION The study reveals ChatGPT's potential as a supplementary tool for otolaryngology board certification preparation. However, its propensity for errors in certain otolaryngology areas calls for further refinement. Future research should address these limitations to improve ChatGPT's educational use. An approach, with expert collaboration, is recommended for the reliable and accurate integration of such AI models.
Collapse
Affiliation(s)
- Cosima C Hoch
- Department of Otolaryngology, Head and Neck Surgery, School of Medicine, Technical University of Munich (TUM), Ismaningerstrasse 22, 81675, Munich, Germany.
| | - Barbara Wollenberg
- Department of Otolaryngology, Head and Neck Surgery, School of Medicine, Technical University of Munich (TUM), Ismaningerstrasse 22, 81675, Munich, Germany
| | - Jan-Christoffer Lüers
- Department of Otorhinolaryngology, Head and Neck Surgery, Medical Faculty, University of Cologne, 50937, Cologne, Germany
| | - Samuel Knoedler
- Division of Plastic Surgery, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, 02152, USA
- Department of Plastic Surgery and Hand Surgery, Klinikum Rechts Der Isar, Technical University of Munich, Munich, Germany
| | - Leonard Knoedler
- Division of Plastic and Reconstructive Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, 02115, USA
| | | | - Sebastian Cotofana
- Department of Dermatology, Erasmus Hospital, Rotterdam, The Netherlands
- Centre for Cutaneous Research, Blizard Institute, Queen Mary University of London, London, UK
| | - Michael Alfertshofer
- Division of Hand, Plastic and Aesthetic Surgery, Ludwig-Maximilians-University Munich, Munich, Germany
| |
Collapse
|
85
|
Shao CY, Li H, Liu XL, Li C, Yang LQ, Zhang YJ, Luo J, Zhao J. Appropriateness and Comprehensiveness of Using ChatGPT for Perioperative Patient Education in Thoracic Surgery in Different Language Contexts: Survey Study. Interact J Med Res 2023; 12:e46900. [PMID: 37578819 PMCID: PMC10463083 DOI: 10.2196/46900] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 07/22/2023] [Accepted: 07/27/2023] [Indexed: 08/15/2023] Open
Abstract
BACKGROUND ChatGPT, a dialogue-based artificial intelligence language model, has shown promise in assisting clinical workflows and patient-clinician communication. However, there is a lack of feasibility assessments regarding its use for perioperative patient education in thoracic surgery. OBJECTIVE This study aimed to assess the appropriateness and comprehensiveness of using ChatGPT for perioperative patient education in thoracic surgery in both English and Chinese contexts. METHODS This pilot study was conducted in February 2023. A total of 37 questions focused on perioperative patient education in thoracic surgery were created based on guidelines and clinical experience. Two sets of inquiries were made to ChatGPT for each question, one in English and the other in Chinese. The responses generated by ChatGPT were evaluated separately by experienced thoracic surgical clinicians for appropriateness and comprehensiveness based on a hypothetical draft response to a patient's question on the electronic information platform. For a response to be qualified, it required at least 80% of reviewers to deem it appropriate and 50% to deem it comprehensive. Statistical analyses were performed using the unpaired chi-square test or Fisher exact test, with a significance level set at P<.05. RESULTS The set of 37 commonly asked questions covered topics such as disease information, diagnostic procedures, perioperative complications, treatment measures, disease prevention, and perioperative care considerations. In both the English and Chinese contexts, 34 (92%) out of 37 responses were qualified in terms of both appropriateness and comprehensiveness. The remaining 3 (8%) responses were unqualified in these 2 contexts. The unqualified responses primarily involved the diagnosis of disease symptoms and surgical-related complications symptoms. The reasons for determining the responses as unqualified were similar in both contexts. There was no statistically significant difference (34/37, 92% vs 34/37, 92%; P=.99) in the qualification rate between the 2 language sets. CONCLUSIONS This pilot study demonstrates the potential feasibility of using ChatGPT for perioperative patient education in thoracic surgery in both English and Chinese contexts. ChatGPT is expected to enhance patient satisfaction, reduce anxiety, and improve compliance during the perioperative period. In the future, there will be remarkable potential application for using artificial intelligence, in conjunction with human review, for patient education and health consultation after patients have provided their informed consent.
Collapse
Affiliation(s)
- Chen-Ye Shao
- Department of Thoracic Surgery, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Hui Li
- Department of Obstetrics and Gynecology, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Xiao-Long Liu
- Department of Cardiothoracic Surgery, Jinling Hospital, Medical School of Nanjing University, Nanjing, China
| | - Chang Li
- Department of Thoracic Surgery, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Li-Qin Yang
- Department of Thoracic Surgery, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Yue-Juan Zhang
- Department of Thoracic Surgery, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Jing Luo
- Department of Thoracic Surgery, The First Affiliated Hospital of Soochow University, Suzhou, China
| | - Jun Zhao
- Department of Thoracic Surgery, The First Affiliated Hospital of Soochow University, Suzhou, China
| |
Collapse
|
86
|
Wang C, Liu S, Yang H, Guo J, Wu Y, Liu J. Ethical Considerations of Using ChatGPT in Health Care. J Med Internet Res 2023; 25:e48009. [PMID: 37566454 PMCID: PMC10457697 DOI: 10.2196/48009] [Citation(s) in RCA: 78] [Impact Index Per Article: 78.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 07/05/2023] [Accepted: 07/25/2023] [Indexed: 08/12/2023] Open
Abstract
ChatGPT has promising applications in health care, but potential ethical issues need to be addressed proactively to prevent harm. ChatGPT presents potential ethical challenges from legal, humanistic, algorithmic, and informational perspectives. Legal ethics concerns arise from the unclear allocation of responsibility when patient harm occurs and from potential breaches of patient privacy due to data collection. Clear rules and legal boundaries are needed to properly allocate liability and protect users. Humanistic ethics concerns arise from the potential disruption of the physician-patient relationship, humanistic care, and issues of integrity. Overreliance on artificial intelligence (AI) can undermine compassion and erode trust. Transparency and disclosure of AI-generated content are critical to maintaining integrity. Algorithmic ethics raise concerns about algorithmic bias, responsibility, transparency and explainability, as well as validation and evaluation. Information ethics include data bias, validity, and effectiveness. Biased training data can lead to biased output, and overreliance on ChatGPT can reduce patient adherence and encourage self-diagnosis. Ensuring the accuracy, reliability, and validity of ChatGPT-generated content requires rigorous validation and ongoing updates based on clinical practice. To navigate the evolving ethical landscape of AI, AI in health care must adhere to the strictest ethical standards. Through comprehensive ethical guidelines, health care professionals can ensure the responsible use of ChatGPT, promote accurate and reliable information exchange, protect patient privacy, and empower patients to make informed decisions about their health care.
Collapse
Affiliation(s)
- Changyu Wang
- Department of Medical Informatics, West China Medical School, Sichuan University, Chengdu, China
- West China College of Stomatology, Sichuan University, Chengdu, China
| | - Siru Liu
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Hao Yang
- Information Center, West China Hospital, Sichuan University, Chengdu, China
| | - Jiulin Guo
- Information Center, West China Hospital, Sichuan University, Chengdu, China
| | - Yuxuan Wu
- Department of Medical Informatics, West China Medical School, Sichuan University, Chengdu, China
| | - Jialin Liu
- Department of Medical Informatics, West China Medical School, Sichuan University, Chengdu, China
- Information Center, West China Hospital, Sichuan University, Chengdu, China
- Department of Otolaryngology-Head and Neck Surgery, West China Hospital, Sichuan University, Chengdu, China
| |
Collapse
|
87
|
Sarink MJ, Bakker IL, Anas AA, Yusuf E. A study on the performance of ChatGPT in infectious diseases clinical consultation. Clin Microbiol Infect 2023; 29:1088-1089. [PMID: 37207982 DOI: 10.1016/j.cmi.2023.05.017] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2023] [Revised: 05/10/2023] [Accepted: 05/11/2023] [Indexed: 05/21/2023]
Affiliation(s)
- Maarten J Sarink
- Department of Medical Microbiology and Infectious Diseases, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Ingrid L Bakker
- Department of Medical Microbiology and Infectious Diseases, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Adam A Anas
- Department of Medical Microbiology and Infectious Diseases, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Erlangga Yusuf
- Department of Medical Microbiology and Infectious Diseases, Erasmus University Medical Center, Rotterdam, The Netherlands.
| |
Collapse
|
88
|
Vintzileos AM, Chavez MR, Romero R. A role for artificial intelligence chatbots in the writing of scientific articles. Am J Obstet Gynecol 2023; 229:89-90. [PMID: 37117103 PMCID: PMC10524709 DOI: 10.1016/j.ajog.2023.03.040] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 03/24/2023] [Indexed: 04/30/2023]
Affiliation(s)
- Anthony M Vintzileos
- Department of Obstetrics and Gynecology, Lenox Hill Hospital Northwell Health, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, New York, NY.
| | - Martin R Chavez
- Department of Obstetrics and Gynecology, NYU Langone Hospital-Long Island, NYU Long Island School of Medicine, Mineola, NY
| | - Roberto Romero
- Pregnancy Research Branch, Division of Obstetrics and Maternal-Fetal Medicine, Division of Intramural Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, US Department of Health and Human Services, Bethesda, MD and Detroit, MI; Department of Obstetrics and Gynecology, University of Michigan, Ann Arbor, MI; Department of Epidemiology and Biostatistics, Michigan State University, East Lansing, MI
| |
Collapse
|
89
|
Suhag A, Kidd J, McGath M, Rajesh R, Gelfinbein J, Cacace N, Monteleone B, Chavez MR. ChatGPT: a pioneering approach to complex prenatal differential diagnosis. Am J Obstet Gynecol MFM 2023; 5:101029. [PMID: 37257586 DOI: 10.1016/j.ajogmf.2023.101029] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Accepted: 05/19/2023] [Indexed: 06/02/2023]
Abstract
This commentary examines how ChatGPT can assist healthcare teams in the prenatal diagnosis of rare and complex cases by creating a differential diagnoses based on deidentified clinical findings, while also acknowledging its limitations.
Collapse
Affiliation(s)
- Anju Suhag
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, NYU Langone Health, NYU Langone Hospital-Long Island, NYU Long Island School of Medicine, Mineola, NY (Drs Suhag and Kidd, Mses McGath and Cacace, and Dr Chavez).
| | - Jennifer Kidd
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, NYU Langone Health, NYU Langone Hospital-Long Island, NYU Long Island School of Medicine, Mineola, NY (Drs Suhag and Kidd, Mses McGath and Cacace, and Dr Chavez)
| | - Meghan McGath
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, NYU Langone Health, NYU Langone Hospital-Long Island, NYU Long Island School of Medicine, Mineola, NY (Drs Suhag and Kidd, Mses McGath and Cacace, and Dr Chavez); Department of Clinical Genetics, NYU Langone Hospital-Long Island, Mineola, NY (Mses McGath and Cacace, and Dr Monteleone)
| | - Raeshmma Rajesh
- Department of Obstetrics and Gynecology, Richmond University Medical Center, Staten Island, NY (Dr Rajesh)
| | | | - Nicole Cacace
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, NYU Langone Health, NYU Langone Hospital-Long Island, NYU Long Island School of Medicine, Mineola, NY (Drs Suhag and Kidd, Mses McGath and Cacace, and Dr Chavez); Department of Clinical Genetics, NYU Langone Hospital-Long Island, Mineola, NY (Mses McGath and Cacace, and Dr Monteleone)
| | - Berrin Monteleone
- Department of Clinical Genetics, NYU Langone Hospital-Long Island, Mineola, NY (Mses McGath and Cacace, and Dr Monteleone)
| | - Martin R Chavez
- Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynecology, NYU Langone Health, NYU Langone Hospital-Long Island, NYU Long Island School of Medicine, Mineola, NY (Drs Suhag and Kidd, Mses McGath and Cacace, and Dr Chavez)
| |
Collapse
|
90
|
Daykan Y, O'Reilly BA. The role of artificial intelligence in the future of urogynecology. Int Urogynecol J 2023; 34:1663-1666. [PMID: 37486359 DOI: 10.1007/s00192-023-05612-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Accepted: 07/08/2023] [Indexed: 07/25/2023]
Abstract
Artificial intelligence (AI) in medicine is a rapidly growing field aimed at using machine learning models to improve health outcomes and patient experiences. Many new platforms have become accessible and therefore it seems inevitable that we consider how to implement them in our day-to-day practice. Currently, the specialty of urogynecology faces new challenges as the population grows, life expectancy increases, and quality of life expectation is much improved. As AI has a lot of potential to promote the discipline of urogynecology, we aim to explore its abilities and possible use in the future. Challenges and risks are associated with using AI, and a responsible use of such resources is required.
Collapse
Affiliation(s)
- Yair Daykan
- Department of Urogynaecology, Cork University Maternity Hospital, Cork, Ireland.
- Department of Obstetrics and Gynecology, Meir Medical Center, Kfar Saba, Israel.
- Sackler School of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Barry A O'Reilly
- Department of Urogynaecology, Cork University Maternity Hospital, Cork, Ireland
| |
Collapse
|
91
|
Bernstein IA, Zhang Y(V, Govil D, Majid I, Chang RT, Sun Y, Shue A, Chou JC, Schehlein E, Christopher KL, Groth SL, Ludwig C, Wang SY. Comparison of Ophthalmologist and Large Language Model Chatbot Responses to Online Patient Eye Care Questions. JAMA Netw Open 2023; 6:e2330320. [PMID: 37606922 PMCID: PMC10445188 DOI: 10.1001/jamanetworkopen.2023.30320] [Citation(s) in RCA: 52] [Impact Index Per Article: 52.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 07/13/2023] [Indexed: 08/23/2023] Open
Abstract
Importance Large language models (LLMs) like ChatGPT appear capable of performing a variety of tasks, including answering patient eye care questions, but have not yet been evaluated in direct comparison with ophthalmologists. It remains unclear whether LLM-generated advice is accurate, appropriate, and safe for eye patients. Objective To evaluate the quality of ophthalmology advice generated by an LLM chatbot in comparison with ophthalmologist-written advice. Design, Setting, and Participants This cross-sectional study used deidentified data from an online medical forum, in which patient questions received responses written by American Academy of Ophthalmology (AAO)-affiliated ophthalmologists. A masked panel of 8 board-certified ophthalmologists were asked to distinguish between answers generated by the ChatGPT chatbot and human answers. Posts were dated between 2007 and 2016; data were accessed January 2023 and analysis was performed between March and May 2023. Main Outcomes and Measures Identification of chatbot and human answers on a 4-point scale (likely or definitely artificial intelligence [AI] vs likely or definitely human) and evaluation of responses for presence of incorrect information, alignment with perceived consensus in the medical community, likelihood to cause harm, and extent of harm. Results A total of 200 pairs of user questions and answers by AAO-affiliated ophthalmologists were evaluated. The mean (SD) accuracy for distinguishing between AI and human responses was 61.3% (9.7%). Of 800 evaluations of chatbot-written answers, 168 answers (21.0%) were marked as human-written, while 517 of 800 human-written answers (64.6%) were marked as AI-written. Compared with human answers, chatbot answers were more frequently rated as probably or definitely written by AI (prevalence ratio [PR], 1.72; 95% CI, 1.52-1.93). The likelihood of chatbot answers containing incorrect or inappropriate material was comparable with human answers (PR, 0.92; 95% CI, 0.77-1.10), and did not differ from human answers in terms of likelihood of harm (PR, 0.84; 95% CI, 0.67-1.07) nor extent of harm (PR, 0.99; 95% CI, 0.80-1.22). Conclusions and Relevance In this cross-sectional study of human-written and AI-generated responses to 200 eye care questions from an online advice forum, a chatbot appeared capable of responding to long user-written eye health posts and largely generated appropriate responses that did not differ significantly from ophthalmologist-written responses in terms of incorrect information, likelihood of harm, extent of harm, or deviation from ophthalmologist community standards. Additional research is needed to assess patient attitudes toward LLM-augmented ophthalmologists vs fully autonomous AI content generation, to evaluate clarity and acceptability of LLM-generated answers from the patient perspective, to test the performance of LLMs in a greater variety of clinical contexts, and to determine an optimal manner of utilizing LLMs that is ethical and minimizes harm.
Collapse
Affiliation(s)
- Isaac A. Bernstein
- Department of Ophthalmology, Byers Eye Institute, Stanford University, Stanford, California
| | - Youchen (Victor) Zhang
- Department of Ophthalmology, Byers Eye Institute, Stanford University, Stanford, California
| | - Devendra Govil
- Department of Ophthalmology, Byers Eye Institute, Stanford University, Stanford, California
| | - Iyad Majid
- Department of Ophthalmology, Byers Eye Institute, Stanford University, Stanford, California
| | - Robert T. Chang
- Department of Ophthalmology, Byers Eye Institute, Stanford University, Stanford, California
| | - Yang Sun
- Department of Ophthalmology, Byers Eye Institute, Stanford University, Stanford, California
| | - Ann Shue
- Department of Ophthalmology, Byers Eye Institute, Stanford University, Stanford, California
| | - Jonathan C. Chou
- Department of Ophthalmology, Kaiser Permanente San Francisco, San Francisco, California
| | | | | | - Sylvia L. Groth
- Department of Ophthalmology and Visual Sciences, Vanderbilt Eye Institute, Nashville, Tennessee
| | - Cassie Ludwig
- Department of Ophthalmology, Byers Eye Institute, Stanford University, Stanford, California
| | - Sophia Y. Wang
- Department of Ophthalmology, Byers Eye Institute, Stanford University, Stanford, California
| |
Collapse
|
92
|
Allahqoli L, Ghiasvand MM, Mazidimoradi A, Salehiniya H, Alkatout I. Diagnostic and Management Performance of ChatGPT in Obstetrics and Gynecology. Gynecol Obstet Invest 2023; 88:310-313. [PMID: 37494894 DOI: 10.1159/000533177] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 07/20/2023] [Indexed: 07/28/2023]
Abstract
OBJECTIVES The use of artificial intelligence (AI) in clinical patient management and medical education has been advancing over time. ChatGPT was developed and trained recently, using a large quantity of textual data from the internet. Medical science is expected to be transformed by its use. The present study was conducted to evaluate the diagnostic and management performance of the ChatGPT AI model in obstetrics and gynecology. DESIGN A cross-sectional study was conducted. PARTICIPANTS/MATERIALS, SETTING, METHODS This study was conducted in Iran in March 2023. Medical histories and examination results of 30 cases were determined in six areas of obstetrics and gynecology. The cases were presented to a gynecologist and ChatGPT for diagnosis and management. Answers from the gynecologist and ChatGPT were compared, and the diagnostic and management performance of ChatGPT were determined. RESULTS Ninety percent (27 of 30) of the cases in obstetrics and gynecology were correctly handled by ChatGPT. Its responses were eloquent, informed, and free of a significant number of errors or misinformation. Even when the answers provided by ChatGPT were incorrect, the responses contained a logical explanation about the case as well as information provided in the question stem. LIMITATIONS The data used in this study were taken from the electronic book and may reflect bias in the diagnosis of ChatGPT. CONCLUSIONS This is the first evaluation of ChatGPT's performance in diagnosis and management in the field of obstetrics and gynecology. It appears that ChatGPT has potential applications in the practice of medicine and is (currently) free and simple to use. However, several ethical considerations and limitations such as bias, validity, copyright infringement, and plagiarism need to be addressed in future studies.
Collapse
Affiliation(s)
- Leila Allahqoli
- Midwifery Department, Ministry of Health and Medical Education, Tehran, Iran,
| | | | - Afrooz Mazidimoradi
- Student Research Committee, Shiraz University of Medical Sciences, Shiraz, Iran
| | - Hamid Salehiniya
- Social Determinants of Health Research Center, Birjand University of Medical Sciences, Birjand, Iran
| | - Ibrahim Alkatout
- Campus Kiel, Kiel School of Gynaecological Endoscopy, University Hospitals Schleswig-Holstein, Kiel, Germany
| |
Collapse
|
93
|
Liu J, Wang C, Liu S. Utility of ChatGPT in Clinical Practice. J Med Internet Res 2023; 25:e48568. [PMID: 37379067 PMCID: PMC10365580 DOI: 10.2196/48568] [Citation(s) in RCA: 117] [Impact Index Per Article: 117.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Revised: 05/29/2023] [Accepted: 06/15/2023] [Indexed: 06/29/2023] Open
Abstract
ChatGPT is receiving increasing attention and has a variety of application scenarios in clinical practice. In clinical decision support, ChatGPT has been used to generate accurate differential diagnosis lists, support clinical decision-making, optimize clinical decision support, and provide insights for cancer screening decisions. In addition, ChatGPT has been used for intelligent question-answering to provide reliable information about diseases and medical queries. In terms of medical documentation, ChatGPT has proven effective in generating patient clinical letters, radiology reports, medical notes, and discharge summaries, improving efficiency and accuracy for health care providers. Future research directions include real-time monitoring and predictive analytics, precision medicine and personalized treatment, the role of ChatGPT in telemedicine and remote health care, and integration with existing health care systems. Overall, ChatGPT is a valuable tool that complements the expertise of health care providers and improves clinical decision-making and patient care. However, ChatGPT is a double-edged sword. We need to carefully consider and study the benefits and potential dangers of ChatGPT. In this viewpoint, we discuss recent advances in ChatGPT research in clinical practice and suggest possible risks and challenges of using ChatGPT in clinical practice. It will help guide and support future artificial intelligence research similar to ChatGPT in health.
Collapse
Affiliation(s)
- Jialin Liu
- Information Center, West China Hospital, Sichuan University, Chengdu, China
- Department of Medical Informatics, West China Medical School, Chengdu, China
- Department of Otolaryngology-Head and Neck Surgery, West China Hospital, Sichuan University, Chengdu, China
| | - Changyu Wang
- Information Center, West China Hospital, Sichuan University, Chengdu, China
- West China College of Stomatology, Sichuan University, Chengdu, China
| | - Siru Liu
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| |
Collapse
|
94
|
Alhaidry HM, Fatani B, Alrayes JO, Almana AM, Alfhaed NK. ChatGPT in Dentistry: A Comprehensive Review. Cureus 2023; 15:e38317. [PMID: 37266053 PMCID: PMC10230850 DOI: 10.7759/cureus.38317] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/29/2023] [Indexed: 06/03/2023] Open
Abstract
Chat generative pre-trained transformer (ChatGPT) is an artificial intelligence chatbot that uses natural language processing that can respond to human input in a conversational manner. ChatGPT has numerous applications in the health care system including dentistry; it is used in diagnoses and for assessing disease risk and scheduling appointments. It also has a role in scientific research. In the dental field, it has provided many benefits such as detecting dental and maxillofacial abnormalities on panoramic radiographs and identifying different dental restorations. Therefore, it helps in decreasing the workload. But even with these benefits, one should take into consideration the risks and limitations of this chatbot. Few articles mentioned the use of ChatGPT in dentistry. This comprehensive review represents data collected from 66 relevant articles using PubMed and Google Scholar as databases. This review aims to discuss all relevant published articles on the use of ChatGPT in dentistry.
Collapse
Affiliation(s)
- Hind M Alhaidry
- Advanced General Dentistry, Prince Sultan Military Medical City, Riyadh, SAU
| | - Bader Fatani
- Dentistry, College of Dentistry, King Saud University, Riyadh, SAU
| | - Jenan O Alrayes
- Dentistry, College of Dentistry, King Saud University, Riyadh, SAU
| | | | - Nawaf K Alfhaed
- Dentistry, College of Dentistry, King Saud University, Riyadh, SAU
| |
Collapse
|