1
|
Tepe M, Emekli E. Decoding medical jargon: The use of AI language models (ChatGPT-4, BARD, microsoft copilot) in radiology reports. PATIENT EDUCATION AND COUNSELING 2024; 126:108307. [PMID: 38743965 DOI: 10.1016/j.pec.2024.108307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 03/20/2024] [Accepted: 04/24/2024] [Indexed: 05/16/2024]
Abstract
OBJECTIVE Evaluate Artificial Intelligence (AI) language models (ChatGPT-4, BARD, Microsoft Copilot) in simplifying radiology reports, assessing readability, understandability, actionability, and urgency classification. METHODS This study evaluated the effectiveness of these AI models in translating radiology reports into patient-friendly language and providing understandable and actionable suggestions and urgency classifications. Thirty radiology reports were processed using AI tools, and their outputs were assessed for readability (Flesch Reading Ease, Flesch-Kincaid Grade Level), understandability (PEMAT), and the accuracy of urgency classification. ANOVA and Chi-Square tests were performed to compare the models' performances. RESULTS All three AI models successfully transformed medical jargon into more accessible language, with BARD showing superior readability scores. In terms of understandability, all models achieved scores above 70%, with ChatGPT-4 and BARD leading (p < 0.001, both). However, the AI models varied in accuracy of urgency recommendations, with no significant statistical difference (p = 0.284). CONCLUSION AI language models have proven effective in simplifying radiology reports, thereby potentially improving patient comprehension and engagement in their health decisions. However, their accuracy in assessing the urgency of medical conditions based on radiology reports suggests a need for further refinement. PRACTICE IMPLICATIONS Incorporating AI in radiology communication can empower patients, but further development is crucial for comprehensive and actionable patient support.
Collapse
Affiliation(s)
- Murat Tepe
- Department of Radiology, King's College Hospital London, Dubai, United Arab Emirates.
| | - Emre Emekli
- Department of Radiology, Eskişehir Osmangazi University, Eskişehir, Turkiye; Department of Medical Education, Gazi University, Ankara, Turkiye
| |
Collapse
|
2
|
Anaya F, Prasad R, Bashour M, Yaghmour R, Alameh A, Blakumaran K. Evaluating ChatGPT platform in delivering heart failure educational material: A comparison with the leading national cardiology institutes. Curr Probl Cardiol 2024:102797. [PMID: 39159709 DOI: 10.1016/j.cpcardiol.2024.102797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2024] [Accepted: 08/16/2024] [Indexed: 08/21/2024]
Abstract
BACKGROUND Patient education plays a crucial role in improving the quality of life for patients with heart failure. As artificial intelligence continues to advance, new chatbots are emerging as valuable tools across various aspects of life. One prominent example is ChatGPT, a widely used chatbot among the public. Our study aims to evaluate the readability of ChatGPT answers for common patients' questions about heart failure. METHODS We performed a comparative analysis between ChatGPT responses and existing heart failure educational materials from top US cardiology institutes. Validated readability calculators were employed to assess and compare the reading difficulty and grade level of the materials. Furthermore, blind assessment using The Patient Education Materials Assessment Tool (PEMAT) was done by four advanced heart failure attendings to evaluate the readability and actionability of each resource. RESULTS Our study revealed that responses generated by ChatGPT were longer and more challenging to read compared to other materials. Additionally, these responses were written at a higher educational level (undergraduate and 9-10th grade), similar to those from the Heart Failure Society of America. Despite achieving a competitive PEMAT readability score (75%), surpassing the American Heart Association score (68%), ChatGPT's actionability score was the lowest (66.7%) among all materials included in our study. CONCLUSION Despite its current limitations, artificial intelligence chatbots has the potential to revolutionize the field of patient education especially given theirs ongoing improvements. However, further research is necessary to ensure the integrity and reliability of these chatbots before endorsing them as reliable resources for patient education.
Collapse
Affiliation(s)
- Firas Anaya
- Department of Medicine, Metrohealth Medical Center, Cleveland, Ohio, USA; Case Western Reserve University, Cleveland, Ohio, USA.
| | - Rahul Prasad
- Cleveland Clinic Akron General Hospital, Akron, Ohio, USA.
| | - Marla Bashour
- Department of Medicine, Metrohealth Medical Center, Cleveland, Ohio, USA; Case Western Reserve University, Cleveland, Ohio, USA.
| | - Raghad Yaghmour
- Department of Medicine, Metrohealth Medical Center, Cleveland, Ohio, USA.
| | - Anas Alameh
- Hear and Vascular Center, Metrohealth Medical Center, Cleveland, Ohio, USA; Case Western Reserve University, Cleveland, Ohio, USA.
| | - Kathir Blakumaran
- Hear and Vascular Center, Metrohealth Medical Center, Cleveland, Ohio, USA; Case Western Reserve University, Cleveland, Ohio, USA.
| |
Collapse
|
3
|
Hancı V, Ergün B, Gül Ş, Uzun Ö, Erdemir İ, Hancı FB. Assessment of readability, reliability, and quality of ChatGPT®, BARD®, Gemini®, Copilot®, Perplexity® responses on palliative care. Medicine (Baltimore) 2024; 103:e39305. [PMID: 39151545 PMCID: PMC11332738 DOI: 10.1097/md.0000000000039305] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/14/2024] [Revised: 07/20/2024] [Accepted: 07/25/2024] [Indexed: 08/19/2024] Open
Abstract
There is no study that comprehensively evaluates data on the readability and quality of "palliative care" information provided by artificial intelligence (AI) chatbots ChatGPT®, Bard®, Gemini®, Copilot®, Perplexity®. Our study is an observational and cross-sectional original research study. In our study, AI chatbots ChatGPT®, Bard®, Gemini®, Copilot®, and Perplexity® were asked to present the answers of the 100 questions most frequently asked by patients about palliative care. Responses from each 5 AI chatbots were analyzed separately. This study did not involve any human participants. Study results revealed significant differences between the readability assessments of responses from all 5 AI chatbots (P < .05). According to the results of our study, when different readability indexes were evaluated holistically, the readability of AI chatbot responses was evaluated as Bard®, Copilot®, Perplexity®, ChatGPT®, Gemini®, from easy to difficult (P < .05). In our study, the median readability indexes of each of the 5 AI chatbots Bard®, Copilot®, Perplexity®, ChatGPT®, Gemini® responses were compared to the "recommended" 6th grade reading level. According to the results of our study answers of all 5 AI chatbots were compared with the 6th grade reading level, statistically significant differences were observed in the all formulas (P < .001). The answers of all 5 artificial intelligence robots were determined to be at an educational level well above the 6th grade level. The modified DISCERN and Journal of American Medical Association scores was found to be the highest in Perplexity® (P < .001). Gemini® responses were found to have the highest Global Quality Scale score (P < .001). It is emphasized that patient education materials should have a readability level of 6th grade level. Of the 5 AI chatbots whose answers about palliative care were evaluated, Bard®, Copilot®, Perplexity®, ChatGPT®, Gemini®, their current answers were found to be well above the recommended levels in terms of readability of text content. Text content quality assessment scores are also low. Both the quality and readability of texts should be brought to appropriate recommended limits.
Collapse
Affiliation(s)
- Volkan Hancı
- Clinic of Anesthesiology and Critical Care, Sincan Education and Research Hospital, Ankara, Turkey
| | - Bişar Ergün
- Clinic of Internal Medicine and Critical Care, Dr. Ismail Fehmi Cumalioğlu City Hospital, Tekirdağ, Turkey
| | - Şanser Gül
- Clinic of Neurosurgery, Ankara Ataturk Sanatory Education and Research Hospital, Ankara, Turkey
| | - Özcan Uzun
- Clinic of Internal Medicine and Nephrology, Yalova City Hospital, Yalova, Turkey
| | - İsmail Erdemir
- Department of Anesthesiology and Critical Care, Faculty of Medicine, Dokuz Eylül University, Izmir, Turkey
| | - Ferid Baran Hancı
- Artificial Intelligence Engineering Department, Faculty of Engineering, Ostim Technical University, Ankara, Turkey
| |
Collapse
|
4
|
Zaki HA, Mai M, Abdel-Megid H, Liew SQR, Kidanemariam S, Omar AS, Tiwari U, Hamze J, Ahn SH, Maxwell AWP. Using ChatGPT to Improve Readability of Interventional Radiology Procedure Descriptions. Cardiovasc Intervent Radiol 2024; 47:1134-1141. [PMID: 38981939 DOI: 10.1007/s00270-024-03803-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/02/2024] [Accepted: 06/26/2024] [Indexed: 07/11/2024]
Abstract
PURPOSE This project examines ChatGPT's potential to enhance the readability of patient educational materials about interventional radiology (IR) procedures. METHODS AND MATERIALS The descriptions of IR procedures from the Cardiovascular and Interventional Radiological Society of Europe (CIRSE) were used as the original text. Readability scores were calculated using three metrics: Flesch Reading Ease (FRE), Gunning Fog (GF), and the Automated Readability Index (ARI) using an online calculator ( https://readabilityformulas.com ). FRE is scored on a scale of 0-100, where 100 indicates easy-to-read texts, and GF and ARI represent the grade level required to comprehend the text. The DISCERN instrument measured credibility and reliability. ChatGPT was prompted to simplify the texts to a fifth-grade reading level, with subsequent recalculation of readability and DISCERN scores for comparison. Statistical significance was determined using a Wilcoxon Signed-Rank Test. Articles were subsequently organized by subgroups and analyzed. RESULTS 73 interventional radiology procedures from CIRSE were analyzed. The original FRE score was 47.2 (Difficult), improved to 78.4 (Fairly Easy) by ChatGPT. GF and ARI scores dropped from 14.4 and 11.2 to 7.8 and 5.8, respectively, after simplification, showing significant improvement (p < 0.001). However, the average DISCERN score decreased from 3.73 to 2.99 (p < 0.001) post-ChatGPT simplification. CONCLUSION This study shows ChatGPT's ability to make interventional radiology descriptions more readable but highlights its struggle to maintain the original's reliability, suggesting the need for human review and prompt engineering to enhance outcomes. LEVEL OF EVIDENCE Level 6.
Collapse
Affiliation(s)
- Hossam A Zaki
- Department of Diagnostic Imaging, The Warren Alpert Medical School of Brown University/Rhode Island Hospital, Providence, RI, USA.
| | - Michelle Mai
- Department of Diagnostic Imaging, The Warren Alpert Medical School of Brown University/Rhode Island Hospital, Providence, RI, USA
| | - Hazem Abdel-Megid
- Department of Diagnostic Imaging, The Warren Alpert Medical School of Brown University/Rhode Island Hospital, Providence, RI, USA
| | - Sabrina Q R Liew
- Department of Diagnostic Imaging, The Warren Alpert Medical School of Brown University/Rhode Island Hospital, Providence, RI, USA
| | - Simon Kidanemariam
- Department of Diagnostic Imaging, The Warren Alpert Medical School of Brown University/Rhode Island Hospital, Providence, RI, USA
| | - Abdifatah S Omar
- Department of Diagnostic Imaging, The Warren Alpert Medical School of Brown University/Rhode Island Hospital, Providence, RI, USA
| | - Urvi Tiwari
- Department of Diagnostic Imaging, The Warren Alpert Medical School of Brown University/Rhode Island Hospital, Providence, RI, USA
| | - Jad Hamze
- Department of Diagnostic Imaging, The Warren Alpert Medical School of Brown University/Rhode Island Hospital, Providence, RI, USA
| | - Sun Ho Ahn
- Department of Diagnostic Imaging, The Warren Alpert Medical School of Brown University/Rhode Island Hospital, Providence, RI, USA
| | - Aaron W P Maxwell
- Department of Diagnostic Imaging, The Warren Alpert Medical School of Brown University/Rhode Island Hospital, Providence, RI, USA
| |
Collapse
|
5
|
Connors C, Gupta K, Khusid JA, Khargi R, Yaghoubian AJ, Levy M, Gallante B, Atallah W, Gupta M. Evaluation of the Current Status of Artificial Intelligence for Endourology Patient Education: A Blind Comparison of ChatGPT and Google Bard Against Traditional Information Resources. J Endourol 2024; 38:843-851. [PMID: 38441078 DOI: 10.1089/end.2023.0696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/06/2024] Open
Abstract
Introduction: Artificial intelligence (AI) platforms such as ChatGPT and Bard are increasingly utilized to answer patient health care questions. We present the first study to blindly evaluate AI-generated responses to common endourology patient questions against official patient education materials. Methods: Thirty-two questions and answers spanning kidney stones, ureteral stents, benign prostatic hyperplasia (BPH), and upper tract urothelial carcinoma were extracted from official Urology Care Foundation (UCF) patient education documents. The same questions were input into ChatGPT 4.0 and Bard, limiting responses to within ±10% of the word count of the corresponding UCF response to ensure fair comparison. Six endourologists blindly evaluated responses from each platform using Likert scales for accuracy, clarity, comprehensiveness, and patient utility. Reviewers identified which response they believed was not AI generated. Finally, Flesch-Kincaid Reading Grade Level formulas assessed the readability of each platform response. Ratings were compared using analysis of variance (ANOVA) and chi-square tests. Results: ChatGPT responses were rated the highest across all categories, including accuracy, comprehensiveness, clarity, and patient utility, while UCF answers were consistently scored the lowest, all p < 0.01. A subanalysis revealed that this trend was consistent across question categories (i.e., kidney stones, BPH, etc.). However, AI-generated responses were more likely to be classified at an advanced reading level, while UCF responses showed improved readability (college or higher reading level: ChatGPT = 100%, Bard = 66%, and UCF = 19%), p < 0.001. When asked to identify which answer was not AI generated, 54.2% of responses indicated ChatGPT, 26.6% indicated Bard, and only 19.3% correctly identified it as the UCF response. Conclusions: In a blind evaluation, AI-generated responses from ChatGPT and Bard surpassed the quality of official patient education materials in endourology, suggesting that current AI platforms are already a reliable resource for basic urologic care information. AI-generated responses do, however, tend to require a higher reading level, which may limit their applicability to a broader audience.
Collapse
Affiliation(s)
- Christopher Connors
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Kavita Gupta
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Johnathan A Khusid
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Raymond Khargi
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Alan J Yaghoubian
- Department of Urology, David Geffen School of Medicine at University of California, Los Angeles, California, USA
| | - Micah Levy
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Blair Gallante
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - William Atallah
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Mantu Gupta
- Department of Urology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| |
Collapse
|
6
|
Warren BE, Bilbily A, Gichoya JW, Conway A, Li B, Fawzy A, Barragán C, Jaberi A, Mafeld S. An Introductory Guide to Artificial Intelligence in Interventional Radiology: Part 1 Foundational Knowledge. Can Assoc Radiol J 2024; 75:558-567. [PMID: 38445497 DOI: 10.1177/08465371241236376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/07/2024] Open
Abstract
Artificial intelligence (AI) is rapidly evolving and has transformative potential for interventional radiology (IR) clinical practice. However, formal training in AI may be limited for many clinicians and therefore presents a challenge for initial implementation and trust in AI. An understanding of the foundational concepts in AI may help familiarize the interventional radiologist with the field of AI, thus facilitating understanding and participation in the development and deployment of AI. A pragmatic classification system of AI based on the complexity of the model may guide clinicians in the assessment of AI. Finally, the current state of AI in IR and the patterns of implementation are explored (pre-procedural, intra-procedural, and post-procedural).
Collapse
Affiliation(s)
- Blair Edward Warren
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Alexander Bilbily
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- 16 Bit Inc., Toronto, ON, Canada
- Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | | | - Aaron Conway
- Prince Charles Hospital, Queensland University of Technology, Brisbane, QLD, Australia
| | - Ben Li
- Division of Vascular Surgery, Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - Aly Fawzy
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
| | - Camilo Barragán
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Arash Jaberi
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| | - Sebastian Mafeld
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
- Joint Department of Medical Imaging, University Health Network, Toronto, ON, Canada
| |
Collapse
|
7
|
Pompili D, Richa Y, Collins P, Richards H, Hennessey DB. Using artificial intelligence to generate medical literature for urology patients: a comparison of three different large language models. World J Urol 2024; 42:455. [PMID: 39073590 PMCID: PMC11286728 DOI: 10.1007/s00345-024-05146-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2024] [Accepted: 06/23/2024] [Indexed: 07/30/2024] Open
Abstract
PURPOSE Large language models (LLMs) are a form of artificial intelligence (AI) that uses deep learning techniques to understand, summarize and generate content. The potential benefits of LLMs in healthcare is predicted to be immense. The objective of this study was to examine the quality of patient information leaflets (PILs) produced by 3 LLMs on urological topics. METHODS Prompts were created to generate PILs from 3 LLMs: ChatGPT-4, PaLM 2 (Google Bard) and Llama 2 (Meta) across four urology topics (circumcision, nephrectomy, overactive bladder syndrome, and transurethral resection of the prostate). PILs were evaluated using a quality assessment checklist. PIL readability was assessed by the Average Reading Level Consensus Calculator. RESULTS PILs generated by PaLM 2 had the highest overall average quality score (3.58), followed by Llama 2 (3.34) and ChatGPT-4 (3.08). PaLM 2 generated PILs were of the highest quality in all topics except TURP and was the only LLM to include images. Medical inaccuracies were present in all generated content including instances of significant error. Readability analysis identified PaLM 2 generated PILs as the simplest (age 14-15 average reading level). Llama 2 PILs were the most difficult (age 16-17 average). CONCLUSION While LLMs can generate PILs that may help reduce healthcare professional workload, generated content requires clinician input for accuracy and inclusion of health literacy aids, such as images. LLM-generated PILs were above the average reading level for adults, necessitating improvement in LLM algorithms and/or prompt design. How satisfied patients are to LLM-generated PILs remains to be evaluated.
Collapse
Affiliation(s)
- David Pompili
- School of Medicine, University College Cork, Cork, Ireland
| | - Yasmina Richa
- School of Medicine, University College Cork, Cork, Ireland
| | - Patrick Collins
- Department of Urology, Mercy University Hospital, Cork, Ireland
| | - Helen Richards
- School of Medicine, University College Cork, Cork, Ireland
- Department of Clinical Psychology, Mercy University Hospital, Cork, Ireland
| | - Derek B Hennessey
- School of Medicine, University College Cork, Cork, Ireland.
- Department of Urology, Mercy University Hospital, Cork, Ireland.
| |
Collapse
|
8
|
Keshavarz P, Bagherieh S, Nabipoorashrafi SA, Chalian H, Rahsepar AA, Kim GHJ, Hassani C, Raman SS, Bedayat A. ChatGPT in radiology: A systematic review of performance, pitfalls, and future perspectives. Diagn Interv Imaging 2024; 105:251-265. [PMID: 38679540 DOI: 10.1016/j.diii.2024.04.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2024] [Revised: 03/11/2024] [Accepted: 04/16/2024] [Indexed: 05/01/2024]
Abstract
PURPOSE The purpose of this study was to systematically review the reported performances of ChatGPT, identify potential limitations, and explore future directions for its integration, optimization, and ethical considerations in radiology applications. MATERIALS AND METHODS After a comprehensive review of PubMed, Web of Science, Embase, and Google Scholar databases, a cohort of published studies was identified up to January 1, 2024, utilizing ChatGPT for clinical radiology applications. RESULTS Out of 861 studies derived, 44 studies evaluated the performance of ChatGPT; among these, 37 (37/44; 84.1%) demonstrated high performance, and seven (7/44; 15.9%) indicated it had a lower performance in providing information on diagnosis and clinical decision support (6/44; 13.6%) and patient communication and educational content (1/44; 2.3%). Twenty-four (24/44; 54.5%) studies reported the proportion of ChatGPT's performance. Among these, 19 (19/24; 79.2%) studies recorded a median accuracy of 70.5%, and in five (5/24; 20.8%) studies, there was a median agreement of 83.6% between ChatGPT outcomes and reference standards [radiologists' decision or guidelines], generally confirming ChatGPT's high accuracy in these studies. Eleven studies compared two recent ChatGPT versions, and in ten (10/11; 90.9%), ChatGPTv4 outperformed v3.5, showing notable enhancements in addressing higher-order thinking questions, better comprehension of radiology terms, and improved accuracy in describing images. Risks and concerns about using ChatGPT included biased responses, limited originality, and the potential for inaccurate information leading to misinformation, hallucinations, improper citations and fake references, cybersecurity vulnerabilities, and patient privacy risks. CONCLUSION Although ChatGPT's effectiveness has been shown in 84.1% of radiology studies, there are still multiple pitfalls and limitations to address. It is too soon to confirm its complete proficiency and accuracy, and more extensive multicenter studies utilizing diverse datasets and pre-training techniques are required to verify ChatGPT's role in radiology.
Collapse
Affiliation(s)
- Pedram Keshavarz
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA 90095, USA; School of Science and Technology, The University of Georgia, Tbilisi 0171, Georgia
| | - Sara Bagherieh
- Independent Clinical Radiology Researcher, Los Angeles, CA 90024, USA
| | | | - Hamid Chalian
- Department of Radiology, Cardiothoracic Imaging, University of Washington, Seattle, WA 98195, USA
| | - Amir Ali Rahsepar
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA 90095, USA
| | - Grace Hyun J Kim
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA 90095, USA; Department of Radiological Sciences, Center for Computer Vision and Imaging Biomarkers, University of California, Los Angeles (UCLA), Los Angeles, CA 90095, USA
| | - Cameron Hassani
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA 90095, USA
| | - Steven S Raman
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA 90095, USA
| | - Arash Bedayat
- Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA 90095, USA.
| |
Collapse
|
9
|
AlShehri Y, McConkey M, Lodhia P. ChatGPT Provides Satisfactory but Occasionally Inaccurate Answers to Common Patient Hip Arthroscopy Questions. Arthroscopy 2024:S0749-8063(24)00452-3. [PMID: 38914299 DOI: 10.1016/j.arthro.2024.06.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/27/2024] [Revised: 05/24/2024] [Accepted: 06/09/2024] [Indexed: 06/26/2024]
Abstract
PURPOSE To assess the ability of ChatGPT to answer common patient questions regarding hip arthroscopy, and to analyze the accuracy and appropriateness of its responses. METHODS Ten questions were selected from well-known patient education websites, and ChatGPT (version 3.5) responses to these questions were graded by 2 fellowship-trained hip preservation surgeons. Responses were analyzed, compared with the current literature, and graded from A to D (A being the highest, and D being the lowest) in a grading scale on the basis of the accuracy and completeness of the response. If the grading differed between the 2 surgeons, a consensus was reached. Inter-rater agreement was calculated. The readability of responses was also assessed using the Flesch-Kincaid Reading Ease Score (FRES) and Flesch-Kincaid Grade Level (FKGL). RESULTS Responses received the following consensus grades: A (50%, n = 5), B (30%, n = 3), C (10%, n = 1), D (10%, n = 1). Inter-rater agreement on the basis of initial individual grading was 30%. The mean FRES was 28.2 (± 9.2 standard deviation), corresponding to a college graduate level, ranging from 11.7 to 42.5. The mean FKGL was 14.4 (±1.8 standard deviation), ranging from 12.1 to 18, indicating a college student reading level. CONCLUSIONS ChatGPT can answer common patient questions regarding hip arthroscopy with satisfactory accuracy graded by 2 high-volume hip arthroscopists; however, incorrect information was identified in more than one instance. Caution must be observed when using ChatGPT for patient education related to hip arthroscopy. CLINICAL RELEVANCE Given the increasing number of hip arthroscopies being performed annually, ChatGPT has the potential to aid physicians in educating their patients about this procedure and addressing any questions they may have.
Collapse
Affiliation(s)
- Yasir AlShehri
- Department of Orthopaedics, Faculty of Medicine, The University of British Columbia, Vancouver, BC, Canada; Department of Orthopedics, College of Medicine, Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia
| | - Mark McConkey
- Department of Orthopaedics, Faculty of Medicine, The University of British Columbia, Vancouver, BC, Canada
| | - Parth Lodhia
- Department of Orthopaedics, Faculty of Medicine, The University of British Columbia, Vancouver, BC, Canada.
| |
Collapse
|
10
|
Güneş YC, Cesur T. Diagnostic Accuracy of Large Language Models in the European Board of Interventional Radiology Examination (EBIR) Sample Questions. Cardiovasc Intervent Radiol 2024; 47:836-837. [PMID: 38388818 DOI: 10.1007/s00270-024-03674-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/22/2024] [Accepted: 01/26/2024] [Indexed: 02/24/2024]
Affiliation(s)
- Yasin Celal Güneş
- Department of Radiology, Kırıkkale Yuksek Ihtisas Hospital, Bağlarbaşı, Ahmet Ay Caddesi, 71300, Merkez/Kırıkkale, Turkey.
| | - Turay Cesur
- Department of Radiology, Mamak State Hospital, Ankara, Turkey
| |
Collapse
|
11
|
Horiuchi D, Tatekawa H, Oura T, Oue S, Walston SL, Takita H, Matsushita S, Mitsuyama Y, Shimono T, Miki Y, Ueda D. Comparing the Diagnostic Performance of GPT-4-based ChatGPT, GPT-4V-based ChatGPT, and Radiologists in Challenging Neuroradiology Cases. Clin Neuroradiol 2024:10.1007/s00062-024-01426-y. [PMID: 38806794 DOI: 10.1007/s00062-024-01426-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2024] [Accepted: 05/06/2024] [Indexed: 05/30/2024]
Abstract
PURPOSE To compare the diagnostic performance among Generative Pre-trained Transformer (GPT)-4-based ChatGPT, GPT‑4 with vision (GPT-4V) based ChatGPT, and radiologists in challenging neuroradiology cases. METHODS We collected 32 consecutive "Freiburg Neuropathology Case Conference" cases from the journal Clinical Neuroradiology between March 2016 and December 2023. We input the medical history and imaging findings into GPT-4-based ChatGPT and the medical history and images into GPT-4V-based ChatGPT, then both generated a diagnosis for each case. Six radiologists (three radiology residents and three board-certified radiologists) independently reviewed all cases and provided diagnoses. ChatGPT and radiologists' diagnostic accuracy rates were evaluated based on the published ground truth. Chi-square tests were performed to compare the diagnostic accuracy of GPT-4-based ChatGPT, GPT-4V-based ChatGPT, and radiologists. RESULTS GPT‑4 and GPT-4V-based ChatGPTs achieved accuracy rates of 22% (7/32) and 16% (5/32), respectively. Radiologists achieved the following accuracy rates: three radiology residents 28% (9/32), 31% (10/32), and 28% (9/32); and three board-certified radiologists 38% (12/32), 47% (15/32), and 44% (14/32). GPT-4-based ChatGPT's diagnostic accuracy was lower than each radiologist, although not significantly (all p > 0.07). GPT-4V-based ChatGPT's diagnostic accuracy was also lower than each radiologist and significantly lower than two board-certified radiologists (p = 0.02 and 0.03) (not significant for radiology residents and one board-certified radiologist [all p > 0.09]). CONCLUSION While GPT-4-based ChatGPT demonstrated relatively higher diagnostic performance than GPT-4V-based ChatGPT, the diagnostic performance of GPT‑4 and GPT-4V-based ChatGPTs did not reach the performance level of either radiology residents or board-certified radiologists in challenging neuroradiology cases.
Collapse
Affiliation(s)
- Daisuke Horiuchi
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Hiroyuki Tatekawa
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Tatsushi Oura
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Satoshi Oue
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Shannon L Walston
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Hirotaka Takita
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Shu Matsushita
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Yasuhito Mitsuyama
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Taro Shimono
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Yukio Miki
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan
| | - Daiju Ueda
- Department of Diagnostic and Interventional Radiology, Graduate School of Medicine, Osaka Metropolitan University, Osaka, Japan.
- Center for Health Science Innovation, Osaka Metropolitan University, Osaka, Japan.
| |
Collapse
|
12
|
Liu L, Qu S, Zhao H, Kong L, Xie Z, Jiang Z, Zou P. Global trends and hotspots of ChatGPT in medical research: a bibliometric and visualized study. Front Med (Lausanne) 2024; 11:1406842. [PMID: 38818395 PMCID: PMC11137200 DOI: 10.3389/fmed.2024.1406842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2024] [Accepted: 05/06/2024] [Indexed: 06/01/2024] Open
Abstract
Objective With the rapid advancement of Chat Generative Pre-Trained Transformer (ChatGPT) in medical research, our study aimed to identify global trends and focal points in this domain. Method All publications on ChatGPT in medical research were retrieved from the Web of Science Core Collection (WoSCC) by Clarivate Analytics from January 1, 2023, to January 31, 2024. The research trends and focal points were visualized and analyzed using VOSviewer and CiteSpace. Results A total of 1,239 publications were collected and analyzed. The USA contributed the largest number of publications (458, 37.145%) with the highest total citation frequencies (2,461) and the largest H-index. Harvard University contributed the highest number of publications (33) among all full-time institutions. The Cureus Journal of Medical Science published the most ChatGPT-related research (127, 10.30%). Additionally, Wiwanitkit V contributed the majority of publications in this field (20). "Artificial Intelligence (AI) and Machine Learning (ML)," "Education and Training," "Healthcare Applications," and "Data Analysis and Technology" emerged as the primary clusters of keywords. These areas are predicted to remain hotspots in future research in this field. Conclusion Overall, this study signifies the interdisciplinary nature of ChatGPT research in medicine, encompassing AI and ML technologies, education and training initiatives, diverse healthcare applications, and data analysis and technology advancements. These areas are expected to remain at the forefront of future research, driving continued innovation and progress in the field of ChatGPT in medical research.
Collapse
Affiliation(s)
- Ling Liu
- Nanxishan Hospital of Guangxi Zhuang Autonomous Region (The Second People’s Hospital of Guangxi Zhuang Autonomous Region), Guilin, China
- School of Integrated Chinese and Western Medicine, Hunan University of Chinese Medicine, Changsha, China
| | - Shenhong Qu
- Department of Otolaryngology-Head and Neck Oncology, The People’s Hospital of Guangxi Zhuang Autonoms Region, Nanning, China
| | - Haiyun Zhao
- Nanxishan Hospital of Guangxi Zhuang Autonomous Region (The Second People’s Hospital of Guangxi Zhuang Autonomous Region), Guilin, China
| | - Lingping Kong
- Nanxishan Hospital of Guangxi Zhuang Autonomous Region (The Second People’s Hospital of Guangxi Zhuang Autonomous Region), Guilin, China
| | - Zhuzhu Xie
- School of Integrated Chinese and Western Medicine, Hunan University of Chinese Medicine, Changsha, China
| | - Zhichao Jiang
- Hunan Provincial Brain Hospital, The Second People’s Hospital of Hunan Province, Changsha, China
| | - Pan Zou
- Nanxishan Hospital of Guangxi Zhuang Autonomous Region (The Second People’s Hospital of Guangxi Zhuang Autonomous Region), Guilin, China
| |
Collapse
|
13
|
Covington EW, Watts Alexander CS, Sewell J, Hutchison AM, Kay J, Tocco L, Hyte M. Unlocking the future of patient Education: ChatGPT vs. LexiComp® as sources of patient education materials. J Am Pharm Assoc (2003) 2024:102119. [PMID: 38729608 DOI: 10.1016/j.japh.2024.102119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 04/23/2024] [Accepted: 05/02/2024] [Indexed: 05/12/2024]
Abstract
BACKGROUND ChatGPT is a conversational artificial intelligence technology that has shown application in various facets of healthcare. With the increased use of AI, it is imperative to assess the accuracy and comprehensibility of AI platforms. OBJECTIVE This pilot project aimed to assess the understandability, readability, and accuracy of ChatGPT as a source of medication-related patient education as compared with an evidence-based medicine tertiary reference resource, LexiComp®. METHODS Patient education materials (PEMs) were obtained from ChatGPT and LexiComp® for 8 common medications (albuterol, apixaban, atorvastatin, hydrocodone/acetaminophen, insulin glargine, levofloxacin, omeprazole, and sacubitril/valsartan). PEMs were extracted, blinded, and assessed by 2 investigators independently. The primary outcome was a comparison of the Patient Education Materials Assessment Tool-printable (PEMAT-P). Secondary outcomes included Flesch reading ease, Flesch Kincaid grade level, percent passive sentences, word count, and accuracy. A 7-item accuracy checklist for each medication was generated by expert consensus among pharmacist investigators, with LexiComp® PEMs serving as the control. PEMAT-P interrater reliability was determined via intraclass correlation coefficient (ICC). Flesch reading ease, Flesch Kincaid grade level, percent passive sentences, and word count were calculated by Microsoft® Word®. Continuous data were assessed using the Student's t-test via SPSS (version 20.0). RESULTS No difference was found in the PEMAT-P understandability score of PEMs produced by ChatGPT versus LexiComp® [77.9% (11.0) vs. 72.5% (2.4), P=0.193]. Reading level was higher with ChatGPT [8.6 (1.2) vs. 5.6 (0.3), P < 0.001). ChatGPT PEMs had a lower percentage of passive sentences and lower word count. The average accuracy score of ChatGPT PEMs was 4.25/7 (61%), with scores ranging from 29% to 86%. CONCLUSION Despite comparable PEMAT-P scores, ChatGPT PEMs did not meet grade level targets. Lower word count and passive text with ChatGPT PEMs could benefit patients, but the variable accuracy scores prevent routine use of ChatGPT to produce medication-related PEMs at this time.
Collapse
|
14
|
Gül Ş, Erdemir İ, Hanci V, Aydoğmuş E, Erkoç YS. How artificial intelligence can provide information about subdural hematoma: Assessment of readability, reliability, and quality of ChatGPT, BARD, and perplexity responses. Medicine (Baltimore) 2024; 103:e38009. [PMID: 38701313 PMCID: PMC11062651 DOI: 10.1097/md.0000000000038009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/21/2024] [Accepted: 04/04/2024] [Indexed: 05/05/2024] Open
Abstract
Subdural hematoma is defined as blood collection in the subdural space between the dura mater and arachnoid. Subdural hematoma is a condition that neurosurgeons frequently encounter and has acute, subacute and chronic forms. The incidence in adults is reported to be 1.72-20.60/100.000 people annually. Our study aimed to evaluate the quality, reliability and readability of the answers to questions asked to ChatGPT, Bard, and perplexity about "Subdural Hematoma." In this observational and cross-sectional study, we asked ChatGPT, Bard, and perplexity to provide the 100 most frequently asked questions about "Subdural Hematoma" separately. Responses from both chatbots were analyzed separately for readability, quality, reliability and adequacy. When the median readability scores of ChatGPT, Bard, and perplexity answers were compared with the sixth-grade reading level, a statistically significant difference was observed in all formulas (P < .001). All 3 chatbot responses were found to be difficult to read. Bard responses were more readable than ChatGPT's (P < .001) and perplexity's (P < .001) responses for all scores evaluated. Although there were differences between the results of the evaluated calculators, perplexity's answers were determined to be more readable than ChatGPT's answers (P < .05). Bard answers were determined to have the best GQS scores (P < .001). Perplexity responses had the best Journal of American Medical Association and modified DISCERN scores (P < .001). ChatGPT, Bard, and perplexity's current capabilities are inadequate in terms of quality and readability of "Subdural Hematoma" related text content. The readability standard for patient education materials as determined by the American Medical Association, National Institutes of Health, and the United States Department of Health and Human Services is at or below grade 6. The readability levels of the responses of artificial intelligence applications such as ChatGPT, Bard, and perplexity are significantly higher than the recommended 6th grade level.
Collapse
Affiliation(s)
- Şanser Gül
- Department of Neurosurgery, Ankara Ataturk Sanatory Education and Research Hospital, Ankara, Turkey
| | - İsmail Erdemir
- Department of Anesthesiology and Critical Care, Faculty of Medicine, Dokuz Eylül University, Izmir, Turkey
| | - Volkan Hanci
- Department of Anesthesiology and Reanimation, Ankara Sincan Education and Research Hospital, Ankara, Turkey
| | - Evren Aydoğmuş
- Department of Neurosurgery, Istanbul Kartal Dr Lütfi Kırdar City Hospital, Istanbul, Turkey
| | - Yavuz Selim Erkoç
- Department of Neurosurgery, Ankara Ataturk Sanatory Education and Research Hospital, Ankara, Turkey
| |
Collapse
|
15
|
Hu M, Qian J, Pan S, Li Y, Qiu RLJ, Yang X. Advancing medical imaging with language models: featuring a spotlight on ChatGPT. Phys Med Biol 2024; 69:10TR01. [PMID: 38537293 DOI: 10.1088/1361-6560/ad387d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Accepted: 03/27/2024] [Indexed: 05/04/2024]
Abstract
This review paper aims to serve as a comprehensive guide and instructional resource for researchers seeking to effectively implement language models in medical imaging research. First, we presented the fundamental principles and evolution of language models, dedicating particular attention to large language models. We then reviewed the current literature on how language models are being used to improve medical imaging, emphasizing a range of applications such as image captioning, report generation, report classification, findings extraction, visual question response systems, interpretable diagnosis and so on. Notably, the capabilities of ChatGPT were spotlighted for researchers to explore its further applications. Furthermore, we covered the advantageous impacts of accurate and efficient language models in medical imaging analysis, such as the enhancement of clinical workflow efficiency, reduction of diagnostic errors, and assistance of clinicians in providing timely and accurate diagnoses. Overall, our goal is to have better integration of language models with medical imaging, thereby inspiring new ideas and innovations. It is our aspiration that this review can serve as a useful resource for researchers in this field, stimulating continued investigative and innovative pursuits of the application of language models in medical imaging.
Collapse
Affiliation(s)
- Mingzhe Hu
- Department of Computer Science and Informatics, Emory University, Atlanta, GA, United States of America
| | - Joshua Qian
- Department of Radiation Oncology, Winship Cancer Institute, School of Medicine, Emory University, Atlanta, GA, United States of America
| | - Shaoyan Pan
- Department of Computer Science and Informatics, Emory University, Atlanta, GA, United States of America
| | - Yuheng Li
- Department of Biomedical Engineering, Emory University, Atlanta, GA, United States of America
| | - Richard L J Qiu
- Department of Radiation Oncology, Winship Cancer Institute, School of Medicine, Emory University, Atlanta, GA, United States of America
| | - Xiaofeng Yang
- Department of Computer Science and Informatics, Emory University, Atlanta, GA, United States of America
- Department of Radiation Oncology, Winship Cancer Institute, School of Medicine, Emory University, Atlanta, GA, United States of America
- Department of Biomedical Engineering, Emory University, Atlanta, GA, United States of America
| |
Collapse
|
16
|
Eleiwa TK, Elhusseiny AM. Re: Kianian et al.: Enhancing the assessment of large language models in medical information generation (Ophthalmol Retina. 2024;8:195-201). Ophthalmol Retina 2024; 8:e15. [PMID: 38363243 DOI: 10.1016/j.oret.2024.01.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2023] [Revised: 01/05/2024] [Accepted: 01/08/2024] [Indexed: 02/17/2024]
Affiliation(s)
- Taher K Eleiwa
- Department of Ophthalmology, Benha Faculty of Medicine, Benha University, Benha, Egypt
| | - Abdelrahman M Elhusseiny
- Department of Ophthalmology, Harvey and Bernice Jones Eye Institute, University of Arkansas for Medical Sciences, Little Rock, Arkansas; Department of Ophthalmology, Boston Children's Hospital, Harvard Medical School, Boston, Massachusetts.
| |
Collapse
|
17
|
Kim MJ, Admane S, Chang YK, Shih KSK, Reddy A, Tang M, Cruz MDL, Taylor TP, Bruera E, Hui D. Chatbot Performance in Defining and Differentiating Palliative Care, Supportive Care, Hospice Care. J Pain Symptom Manage 2024; 67:e381-e391. [PMID: 38219964 DOI: 10.1016/j.jpainsymman.2024.01.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Revised: 12/22/2023] [Accepted: 01/03/2024] [Indexed: 01/16/2024]
Abstract
CONTEXT Artificial intelligence (AI) chatbot platforms are increasingly used by patients as sources of information. However, there is limited data on the performance of these platforms, especially regarding palliative care terms. OBJECTIVES We evaluated the accuracy, comprehensiveness, reliability, and readability of three AI platforms in defining and differentiating "palliative care," "supportive care," and "hospice care." METHODS We asked ChatGPT, Microsoft Bing Chat, Google Bard to define and differentiate "palliative care," "supportive care," and "hospice care" and provide three references. Outputs were randomized and assessed by six blinded palliative care physicians using 0-10 scales (10 = best) for accuracy, comprehensiveness, and reliability. Readability was assessed using Flesch Kincaid Grade Level and Flesch Reading Ease scores. RESULTS The mean (SD) accuracy scores for ChatGPT, Bard, and Bing Chat were 9.1 (1.3), 8.7 (1.5), and 8.2 (1.7), respectively; for comprehensiveness, the scores for the three platforms were 8.7 (1.5), 8.1 (1.9), and 5.6 (2.0), respectively; for reliability, the scores were 6.3 (2.5), 3.2 (3.1), and 7.1 (2.4), respectively. Despite generally high accuracy, we identified some major errors (e.g., Bard stated that supportive care had "the goal of prolonging life or even achieving a cure"). We found several major omissions, particularly with Bing Chat (e.g., no mention of interdisciplinary teams in palliative care or hospice care). References were often unreliable. Readability scores did not meet recommended levels for patient educational materials. CONCLUSION We identified important concerns regarding the accuracy, comprehensiveness, reliability, and readability of outputs from AI platforms. Further research is needed to improve their performance.
Collapse
Affiliation(s)
- Min Ji Kim
- Department of Palliative Care (M.J.K., S.A., Y.K.C., A.R., M.T., E.B., D.H.), Rehabilitation, and Integrative Medicine, University of Texas MD Anderson Cancer Center, Houston, Texas, USA.
| | - Sonal Admane
- Department of Palliative Care (M.J.K., S.A., Y.K.C., A.R., M.T., E.B., D.H.), Rehabilitation, and Integrative Medicine, University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Yuchieh Kathryn Chang
- Department of Palliative Care (M.J.K., S.A., Y.K.C., A.R., M.T., E.B., D.H.), Rehabilitation, and Integrative Medicine, University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | | | - Akhila Reddy
- Department of Palliative Care (M.J.K., S.A., Y.K.C., A.R., M.T., E.B., D.H.), Rehabilitation, and Integrative Medicine, University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Michael Tang
- Department of Palliative Care (M.J.K., S.A., Y.K.C., A.R., M.T., E.B., D.H.), Rehabilitation, and Integrative Medicine, University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Maxine De La Cruz
- Beth Israel Deaconess Medical Center, Harvard Medical School (M.C.), Boston, Massachusetts, USA
| | - Terry Pham Taylor
- Department of Hospital Medicine, University of Texas MD Anderson Cancer Center (T.P.T.), Houston, Texas, USA
| | - Eduardo Bruera
- Department of Palliative Care (M.J.K., S.A., Y.K.C., A.R., M.T., E.B., D.H.), Rehabilitation, and Integrative Medicine, University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - David Hui
- Department of Palliative Care (M.J.K., S.A., Y.K.C., A.R., M.T., E.B., D.H.), Rehabilitation, and Integrative Medicine, University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| |
Collapse
|
18
|
Doğan L, Özçakmakcı GB, Yılmaz ĬE. The Performance of Chatbots and the AAPOS Website as a Tool for Amblyopia Education. J Pediatr Ophthalmol Strabismus 2024:1-7. [PMID: 38661309 DOI: 10.3928/01913913-20240409-01] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 04/26/2024]
Abstract
PURPOSE To evaluate the understandability, actionability, and readability of responses provided by the website of the American Association for Pediatric Ophthalmology and Strabismus (AAPOS), ChatGPT-3.5, Bard, and Bing Chat about amblyopia and the appropriateness of the responses generated by the chatbots. METHOD Twenty-five questions provided by the AAPOS website were directed three times to fresh ChatGPT-3.5, Bard, and Bing Chat interfaces. Two experienced pediatric ophthalmologists categorized the responses of the chatbots in terms of their appropriateness. Flesch Reading Ease (FRE), Flesch Kincaid Grade Level (FKGL), and Coleman-Liau Index (CLI) were used to evaluate the readability of the responses of the AAPOS website and chatbots. Furthermore, the understandability scores were evaluated using the Patient Education Materials Assessment Tool (PEMAT). RESULTS The appropriateness of the chatbots' responses was 84.0% for ChatGPT-3.5 and Bard and 80% for Bing Chat (P > .05). For understandability (mean PEMAT-U score AAPOS website: 81.5%, Bard: 77.6%, ChatGPT-3.5: 76.1%, and Bing Chat: 71.5%, P < .05) and actionability (mean PEMAT-A score AAPOS website: 74.6%, Bard: 69.2%, ChatGPT-3.5: 67.8%, and Bing Chat: 64.8%, P < .05), the AAPOs website scored better than the chat-bots. Three readability analyses showed that Bard had the highest mean score, followed by the AAPOS website, Bing Chat, and ChatGPT-3.5, and these scores were more challenging than the recommended level. CONCLUSIONS Chatbots have the potential to provide detailed and appropriate responses at acceptable levels. The AAPOS website has the advantage of providing information that is more understandable and actionable. The AAPOS website and chatbots, especially Chat-GPT, provided difficult-to-read data for patient education regarding amblyopia. [J Pediatr Ophthalmol Strabismus. 20XX;X(X):XXX-XXX.].
Collapse
|
19
|
Bajaj S, Gandhi D, Nayar D. Potential Applications and Impact of ChatGPT in Radiology. Acad Radiol 2024; 31:1256-1261. [PMID: 37802673 DOI: 10.1016/j.acra.2023.08.039] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 08/15/2023] [Accepted: 08/28/2023] [Indexed: 10/08/2023]
Abstract
Radiology has always gone hand-in-hand with technology and artificial intelligence (AI) is not new to the field. While various AI devices and algorithms have already been integrated in the daily clinical practice of radiology, with applications ranging from scheduling patient appointments to detecting and diagnosing certain clinical conditions on imaging, the use of natural language processing and large language model based software have been in discussion for a long time. Algorithms like ChatGPT can help in improving patient outcomes, increasing the efficiency of radiology interpretation, and aiding in the overall workflow of radiologists and here we discuss some of its potential applications.
Collapse
Affiliation(s)
- Suryansh Bajaj
- Department of Radiology, University of Arkansas for Medical Sciences, Little Rock, Arkansas 72205 (S.B.)
| | - Darshan Gandhi
- Department of Diagnostic Radiology, University of Tennessee Health Science Center, Memphis, Tennessee 38103 (D.G.).
| | - Divya Nayar
- Department of Neurology, University of Arkansas for Medical Sciences, Little Rock, Arkansas 72205 (D.N.)
| |
Collapse
|
20
|
Desai V. The Future of Artificial Intelligence in Sports Medicine and Return to Play. Semin Musculoskelet Radiol 2024; 28:203-212. [PMID: 38484772 DOI: 10.1055/s-0043-1778019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/19/2024]
Abstract
Artificial intelligence (AI) has shown tremendous growth over the last decade, with the more recent development of clinical applications in health care. The ability of AI to synthesize large amounts of complex data automatically allows health care providers to access previously unavailable metrics and thus enhance and personalize patient care. These innovations include AI-assisted diagnostic tools, prediction models for each treatment pathway, and various tools for workflow optimization. The extension of AI into sports medicine is still early, but numerous AI-driven algorithms, devices, and research initiatives have delved into predicting and preventing athlete injury, aiding in injury assessment, optimizing recovery plans, monitoring rehabilitation progress, and predicting return to play.
Collapse
Affiliation(s)
- Vishal Desai
- Department of Radiology, Thomas Jefferson University, Philadelphia, Pennsylvania
| |
Collapse
|
21
|
Bera K, O'Connor G, Jiang S, Tirumani SH, Ramaiya N. Analysis of ChatGPT publications in radiology: Literature so far. Curr Probl Diagn Radiol 2024; 53:215-225. [PMID: 37891083 DOI: 10.1067/j.cpradiol.2023.10.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Accepted: 10/18/2023] [Indexed: 10/29/2023]
Abstract
OBJECTIVE To perform a detailed qualitative and quantitative analysis of the published literature on ChatGPT and radiology in the nine months since its public release, detailing the scope of the work in the short timeframe. METHODS A systematic literature search was carried out of the MEDLINE, EMBASE databases through August 15, 2023 for articles that were focused on ChatGPT and imaging/radiology. Articles were classified into original research and reviews/perspectives. Quantitative analysis was carried out by two experienced radiologists using objective scoring systems for evaluating original and non-original research. RESULTS 51 articles were published involving ChatGPT and radiology/imaging dating from 26 Jan 2023 to the last article published on 14 Aug 2023. 23 articles were original research while the rest included reviews/perspectives or brief communications. For quantitative analysis scored by two readers, we included 23 original research and 17 non-original research articles (after excluding 11 letters as responses to previous articles). Mean score for original research was 3.20 out of 5 (across five questions), while mean score for non-original research was 1.17 out of 2 (across six questions). Mean score grading performance of ChatGPT in original research was 3.20 out of five (across two questions). DISCUSSION While it is early days for ChatGPT and its impact in radiology, there has already been a plethora of articles talking about the multifaceted nature of the tool and how it can impact every aspect of radiology from patient education, pre-authorization, protocol selection, generating differentials, to structuring radiology reports. Most articles show impressive performance of ChatGPT which can only improve with more research and improvements in the tool itself. There have also been several articles which have highlighted the limitations of ChatGPT in its current iteration, which will allow radiologists and researchers to improve these areas.
Collapse
Affiliation(s)
- Kaustav Bera
- Department of Radiology, University Hospitals Cleveland Medical Center, 11000 Euclid Avenue, Cleveland, OH, 44106, USA.
| | - Gregory O'Connor
- Department of Radiology, University Hospitals Cleveland Medical Center, 11000 Euclid Avenue, Cleveland, OH, 44106, USA
| | - Sirui Jiang
- Department of Radiology, University Hospitals Cleveland Medical Center, 11000 Euclid Avenue, Cleveland, OH, 44106, USA
| | - Sree Harsha Tirumani
- Department of Radiology, University Hospitals Cleveland Medical Center, 11000 Euclid Avenue, Cleveland, OH, 44106, USA
| | - Nikhil Ramaiya
- Department of Radiology, University Hospitals Cleveland Medical Center, 11000 Euclid Avenue, Cleveland, OH, 44106, USA
| |
Collapse
|
22
|
Tao BK, Handzic A, Hua NJ, Vosoughi AR, Margolin EA, Micieli JA. Utility of ChatGPT for Automated Creation of Patient Education Handouts: An Application in Neuro-Ophthalmology. J Neuroophthalmol 2024; 44:119-124. [PMID: 38175720 DOI: 10.1097/wno.0000000000002074] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2024]
Abstract
BACKGROUND Patient education in ophthalmology poses a challenge for physicians because of time and resource limitations. ChatGPT (OpenAI, San Francisco) may assist with automating production of patient handouts on common neuro-ophthalmic diseases. METHODS We queried ChatGPT-3.5 to generate 51 patient education handouts across 17 conditions. We devised the "Quality of Generated Language Outputs for Patients" (QGLOP) tool to assess handouts on the domains of accuracy/comprehensiveness, bias, currency, and tone, each scored out of 4 for a total of 16. A fellowship-trained neuro-ophthalmologist scored each passage. Handout readability was assessed using the Simple Measure of Gobbledygook (SMOG), which estimates years of education required to understand a text. RESULTS The QGLOP scores for accuracy, bias, currency, and tone were found to be 2.43, 3, 3.43, and 3.02 respectively. The mean QGLOP score was 11.9 [95% CI 8.98, 14.8] out of 16 points, indicating a performance of 74.4% [95% CI 56.1%, 92.5%]. The mean SMOG across responses as 10.9 [95% CI 9.36, 12.4] years of education. CONCLUSIONS The mean QGLOP score suggests that a fellowship-trained ophthalmologist may have at-least a moderate level of satisfaction with the write-up quality conferred by ChatGPT. This still requires a final review and editing before dissemination. Comparatively, the rarer 5% of responses collectively on either extreme would require very mild or extensive revision. Also, the mean SMOG score exceeded the accepted upper limits of grade 8 reading level for health-related patient handouts. In its current iteration, ChatGPT should be used as an efficiency tool to generate an initial draft for the neuro-ophthalmologist, who may then refine the accuracy and readability for a lay readership.
Collapse
Affiliation(s)
- Brendan K Tao
- Faculty of Medicine (BKT), The University of British Columbia, Vancouver, Canada ; Department of Ophthalmology & Vision Science (AH, EAM, JAM), University of Toronto, Toronto, Canada; Temerty Faculty of Medicine (NJH), University of Toronto, Toronto, Canada; Department of Ophthalmology (ARV), Max Rady College of Medicine, University of Manitoba, Winnipeg, Canada; Mount Sinai Hospital (EAM), Toronto, Canada; Division of Neurology (EAM, JAM), Department of Medicine, University of Toronto, Toronto, Canada; Toronto Western Hospital (EAM, JAM), Toronto, Canada; University Health Network (EAM, JAM), Toronto, Canada; Kensington Vision and Research Center (JAM), Toronto, Canada; and St. Michael's Hospital (JAM), Toronto, Canada
| | | | | | | | | | | |
Collapse
|
23
|
Barabino E, Cittadini G. From Search Engines to Large Language Models: A Big Leap for Patient Education! Cardiovasc Intervent Radiol 2024; 47:251-252. [PMID: 38263526 DOI: 10.1007/s00270-024-03658-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/04/2023] [Accepted: 12/29/2023] [Indexed: 01/25/2024]
Affiliation(s)
- Emanuele Barabino
- Department of Interventional Radiology, Ospedale Santa Corona, 17027, Pietra Ligure, Italy.
| | - Giuseppe Cittadini
- Department of Diagnostic and Interventional Radiology, IRCCS Ospedale Policlinico San Martino, Genoa, Italy
| |
Collapse
|
24
|
Scheschenja M, Viniol S, Bastian MB, Wessendorf J, König AM, Mahnken AH. Feasibility of GPT-3 and GPT-4 for in-Depth Patient Education Prior to Interventional Radiological Procedures: A Comparative Analysis. Cardiovasc Intervent Radiol 2024; 47:245-250. [PMID: 37872295 PMCID: PMC10844465 DOI: 10.1007/s00270-023-03563-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Accepted: 09/09/2023] [Indexed: 10/25/2023]
Abstract
PURPOSE This study explores the utility of the large language models, GPT-3 and GPT-4, for in-depth patient education prior to interventional radiology procedures. Further, differences in answer accuracy between the models were assessed. MATERIALS AND METHODS A total of 133 questions related to three specific interventional radiology procedures (Port implantation, PTA and TACE) covering general information as well as preparation details, risks and complications and post procedural aftercare were compiled. Responses of GPT-3 and GPT-4 were assessed for their accuracy by two board-certified radiologists using a 5-point Likert scale. The performance difference between GPT-3 and GPT-4 was analyzed. RESULTS Both GPT-3 and GPT-4 responded with (5) "completely correct" (4) "very good" answers for the majority of questions ((5) 30.8% + (4) 48.1% for GPT-3 and (5) 35.3% + (4) 47.4% for GPT-4). GPT-3 and GPT-4 provided (3) "acceptable" responses 15.8% and 15.0% of the time, respectively. GPT-3 provided (2) "mostly incorrect" responses in 5.3% of instances, while GPT-4 had a lower rate of such occurrences, at just 2.3%. No response was identified as potentially harmful. GPT-4 was found to give significantly more accurate responses than GPT-3 (p = 0.043). CONCLUSION GPT-3 and GPT-4 emerge as relatively safe and accurate tools for patient education in interventional radiology. GPT-4 showed a slightly better performance. The feasibility and accuracy of these models suggest their promising role in revolutionizing patient care. Still, users need to be aware of possible limitations.
Collapse
Affiliation(s)
- Michael Scheschenja
- Department of Diagnostic and Interventional Radiology, University Hospital Marburg, Philipps-University of Marburg, Baldingerstrasse 1, 35043, Marburg, DE, Germany.
| | - Simon Viniol
- Department of Diagnostic and Interventional Radiology, University Hospital Marburg, Philipps-University of Marburg, Baldingerstrasse 1, 35043, Marburg, DE, Germany
| | - Moritz B Bastian
- Department of Diagnostic and Interventional Radiology, University Hospital Marburg, Philipps-University of Marburg, Baldingerstrasse 1, 35043, Marburg, DE, Germany
| | - Joel Wessendorf
- Department of Diagnostic and Interventional Radiology, University Hospital Marburg, Philipps-University of Marburg, Baldingerstrasse 1, 35043, Marburg, DE, Germany
| | - Alexander M König
- Department of Diagnostic and Interventional Radiology, University Hospital Marburg, Philipps-University of Marburg, Baldingerstrasse 1, 35043, Marburg, DE, Germany
| | - Andreas H Mahnken
- Department of Diagnostic and Interventional Radiology, University Hospital Marburg, Philipps-University of Marburg, Baldingerstrasse 1, 35043, Marburg, DE, Germany
| |
Collapse
|
25
|
Marey A, Saad AM, Killeen BD, Gomez C, Tregubova M, Unberath M, Umair M. Generative Artificial Intelligence: Enhancing Patient Education in Cardiovascular Imaging. BJR Open 2024; 6:tzae018. [PMID: 39086557 PMCID: PMC11290812 DOI: 10.1093/bjro/tzae018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2024] [Revised: 04/18/2024] [Accepted: 07/10/2024] [Indexed: 08/02/2024] Open
Abstract
Cardiovascular disease (CVD) is a major cause of mortality worldwide, especially in resource-limited countries with limited access to healthcare resources. Early detection and accurate imaging are vital for managing CVD, emphasizing the significance of patient education. Generative artificial intelligence (AI), including algorithms to synthesize text, speech, images, and combinations thereof given a specific scenario or prompt, offers promising solutions for enhancing patient education. By combining vision and language models, generative AI enables personalized multimedia content generation through natural language interactions, benefiting patient education in cardiovascular imaging. Simulations, chat-based interactions, and voice-based interfaces can enhance accessibility, especially in resource-limited settings. Despite its potential benefits, implementing generative AI in resource-limited countries faces challenges like data quality, infrastructure limitations, and ethical considerations. Addressing these issues is crucial for successful adoption. Ethical challenges related to data privacy and accuracy must also be overcome to ensure better patient understanding, treatment adherence, and improved healthcare outcomes. Continued research, innovation, and collaboration in generative AI have the potential to revolutionize patient education. This can empower patients to make informed decisions about their cardiovascular health, ultimately improving healthcare outcomes in resource-limited settings.
Collapse
Affiliation(s)
- Ahmed Marey
- Alexandria University Faculty of Medicine, Alexandria, 21521, Egypt
| | | | | | - Catalina Gomez
- Department of Computer Science, Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, 21218, United States
| | - Mariia Tregubova
- Department of Radiology, Amosov National Institute of Cardiovascular Surgery, Kyiv, 02000, Ukraine
| | - Mathias Unberath
- Department of Computer Science, Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, 21218, United States
| | - Muhammad Umair
- Russell H. Morgan Department of Radiology and Radiological Sciences, The Johns Hopkins Hospital, Baltimore, MD, 21205, United States
| |
Collapse
|
26
|
Hillmann HAK, Angelini E, Karfoul N, Feickert S, Mueller-Leisse J, Duncker D. Accuracy and comprehensibility of chat-based artificial intelligence for patient information on atrial fibrillation and cardiac implantable electronic devices. Europace 2023; 26:euad369. [PMID: 38127304 PMCID: PMC10824484 DOI: 10.1093/europace/euad369] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 12/19/2023] [Indexed: 12/23/2023] Open
Abstract
AIMS Natural language processing chatbots (NLPC) can be used to gather information for medical content. However, these tools contain a potential risk of misinformation. This study aims to evaluate different aspects of responses given by different NLPCs on questions about atrial fibrillation (AF) and clinical implantable electronic devices (CIED). METHODS AND RESULTS Questions were entered into three different NLPC interfaces. Responses were evaluated with regard to appropriateness, comprehensibility, appearance of confabulation, absence of relevant content, and recommendations given for clinically relevant decisions. Moreover, readability was assessed by calculating word count and Flesch Reading Ease score. 52, 60, and 84% of responses on AF and 16, 72, and 88% on CIEDs were evaluated to be appropriate for all responses given by Google Bard, (GB) Bing Chat (BC) and ChatGPT Plus (CGP), respectively. Assessment of comprehensibility showed that 96, 88, and 92% of responses on AF and 92 and 88%, and 100% on CIEDs were comprehensible for all responses created by GB, BC, and CGP, respectively. Readability varied between different NLPCs. Relevant aspects were missing in 52% (GB), 60% (BC), and 24% (CGP) for AF, and in 92% (GB), 88% (BC), and 52% (CGP) for CIEDs. CONCLUSION Responses generated by an NLPC are mostly easy to understand with varying readability between the different NLPCs. The appropriateness of responses is limited and varies between different NLPCs. Important aspects are often missed to be mentioned. Thus, chatbots should be used with caution to gather medical information about cardiac arrhythmias and devices.
Collapse
Affiliation(s)
- Henrike A K Hillmann
- Hannover Heart Rhythm Center, Department of Cardiology and Angiology, Hannover Medical School, Carl-Neuberg-Str. 1, 30625 Hannover, Germany
| | - Eleonora Angelini
- Hannover Heart Rhythm Center, Department of Cardiology and Angiology, Hannover Medical School, Carl-Neuberg-Str. 1, 30625 Hannover, Germany
| | - Nizar Karfoul
- Hannover Heart Rhythm Center, Department of Cardiology and Angiology, Hannover Medical School, Carl-Neuberg-Str. 1, 30625 Hannover, Germany
| | - Sebastian Feickert
- Department of Cardiology and Internal Intensive Care Unit, Vivantes Clinic Am Urban, Dieffenbachstraße 1, 10967 Berlin, Germany
- Department of Cardiology, University Medical Center Rostock, Ernst-Heydemann-Straße 6, 18057 Rostock, Germany
| | - Johanna Mueller-Leisse
- Hannover Heart Rhythm Center, Department of Cardiology and Angiology, Hannover Medical School, Carl-Neuberg-Str. 1, 30625 Hannover, Germany
| | - David Duncker
- Hannover Heart Rhythm Center, Department of Cardiology and Angiology, Hannover Medical School, Carl-Neuberg-Str. 1, 30625 Hannover, Germany
| |
Collapse
|
27
|
Gunn AJ. Commentary: The Emerging Role of Artificial Intelligence for Patient Education. J Vasc Interv Radiol 2023; 34:1769-1770. [PMID: 37330212 DOI: 10.1016/j.jvir.2023.06.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Accepted: 06/07/2023] [Indexed: 06/19/2023] Open
Affiliation(s)
- Andrew J Gunn
- Division of Vascular and Interventional Radiology, Department of Radiology, University of Alabama at Birmingham, Birmingham, Alabama.
| |
Collapse
|
28
|
Tippareddy C, Jiang S, Bera K, Ramaiya N. Radiology Reading Room for the Future: Harnessing the Power of Large Language Models Like ChatGPT. Curr Probl Diagn Radiol 2023:S0363-0188(23)00133-0. [PMID: 37758604 DOI: 10.1067/j.cpradiol.2023.08.018] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 08/28/2023] [Accepted: 08/28/2023] [Indexed: 09/29/2023]
Abstract
Radiology has usually been the field of medicine that has been at the forefront of technological advances, often being the first to wholeheartedly embrace them. Whether it's from digitization to cloud side architecture, radiology has led the way for adopting the latest advances. With the advent of large language models (LLMs), especially with the unprecedented explosion of freely available ChatGPT, time is ripe for radiology and radiologists to find novel ways to use the technology to improve their workflow. Towards this, we believe these LLMs have a key role in the radiology reading room not only to expedite processes, simplify mundane and archaic tasks, but also to increase the radiologist's and radiologist trainee's knowledge base at a far faster pace. In this article, we discuss some of the ways we believe ChatGPT, and the likes can be harnessed in the reading room.
Collapse
Affiliation(s)
- Charit Tippareddy
- Department of Radiology, University Hospitals Cleveland Medical Center, Cleveland, OH
| | - Sirui Jiang
- Department of Radiology, University Hospitals Cleveland Medical Center, Cleveland, OH
| | - Kaustav Bera
- Department of Radiology, University Hospitals Cleveland Medical Center, Cleveland, OH.
| | - Nikhil Ramaiya
- Department of Radiology, University Hospitals Cleveland Medical Center, Cleveland, OH
| |
Collapse
|