1
|
Lechien JR, Gorton A, Robertson J, Vaira LA. Is ChatGPT-4 Accurate in Proofread a Manuscript in Otolaryngology-Head and Neck Surgery? Otolaryngol Head Neck Surg 2024; 170:1527-1530. [PMID: 37717252 DOI: 10.1002/ohn.526] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 08/01/2023] [Accepted: 08/18/2023] [Indexed: 09/19/2023]
Abstract
ChatGPT is a new artificial intelligence-powered language model of chatbot able to help otolaryngologists in clinical practice and research. We investigated the ability of ChatGPT-4 in the editing of a manuscript in otolaryngology. Four papers were written by a nonnative English otolaryngologist and edited by a professional editing service. ChatGPT-4 was used to detect and correct errors in manuscripts. From the 171 errors in the manuscripts, ChatGPT-4 detected 86 errors (50.3%) including vocabulary (N = 36), determiner (N = 27), preposition (N = 24), capitalization (N = 20), and number (N = 11). ChatGPT-4 proposed appropriate corrections for 72 (83.7%) errors, while some errors were poorly detected (eg, capitalization [5%] and vocabulary [44.4%] errors. ChatGPT-4 claimed to change something that was already there in 82 cases. ChatGPT demonstrated usefulness in identifying some types of errors but not all. Nonnative English researchers should be aware of the current limits of ChatGPT-4 in the proofreading of manuscripts.
Collapse
Affiliation(s)
- Jerome R Lechien
- Department of Otolaryngology-Head Neck Surgery, Division of Laryngology and Broncho-esophagology, EpiCURA Hospital, UMONS Research Institute for Health Sciences and Technology, University of Mons (UMons), Mons, Belgium
- Department of Otorhinolaryngology and Head and Neck Surgery, Foch Hospital, School of Medicine, UFR Simone Veil, Université Versailles Saint-Quentin-en-Yvelines (Paris Saclay University), Paris, France
- Department of Otorhinolaryngology and Head and Neck Surgery, CHU de Bruxelles, CHU Saint-Pierre, School of Medicine, Université Libre de Bruxelles, Brussels, Belgium
- Polyclinique Elsan de Poitiers, Poitiers, France
| | - Amy Gorton
- Faculty of Translation and Interpretation (FTI-EII), University of Mons, Mons, Belgium
| | - Jean Robertson
- Faculty of Translation and Interpretation (FTI-EII), University of Mons, Mons, Belgium
| | - Luigi A Vaira
- Maxillofacial Surgery Operative Unit, Department of Medicine, Surgery and Pharmacy, University of Sassari, Sassari, Italy
- PhD School of Biomedical Sciences, Department of Biomedical Sciences, University of Sassari, Sassari, Italy
| |
Collapse
|
2
|
González R, Poenaru D, Woo R, Trappey AF, Carter S, Darcy D, Encisco E, Gulack B, Miniati D, Tombash E, Huang EY. ChatGPT: What Every Pediatric Surgeon Should Know About Its Potential Uses and Pitfalls. J Pediatr Surg 2024; 59:941-947. [PMID: 38336588 DOI: 10.1016/j.jpedsurg.2024.01.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 12/30/2023] [Accepted: 01/09/2024] [Indexed: 02/12/2024]
Abstract
ChatGPT - currently the most popular generative artificial intelligence system - has been revolutionizing the world and healthcare since its release in November 2022. ChatGPT is a conversational chatbot that uses machine learning algorithms to enhance its replies based on user interactions and is a part of a broader effort to develop natural language processing that can assist people in their daily lives by understanding and responding to human language in a useful and engaging way. Thus far, many potential applications within healthcare have been described, despite its relatively recent release. This manuscript offers the pediatric surgical community a primer on this new technology and discusses some initial observations about its potential uses and pitfalls. Moreover, it introduces the perspectives of medical journals and surgical societies regarding the use of this artificial intelligence chatbot. As ChatGPT and other large language models continue to evolve, it is the responsibility of the pediatric surgery community to stay abreast of these changes and play an active role in safely incorporating them into our field for the benefit of our patients. LEVEL OF EVIDENCE: V.
Collapse
Affiliation(s)
- Raquel González
- Division of Pediatric Surgery, Johns Hopkins All Children's Hospital, 501 6th Avenue S, Saint Petersburg, FL, 33701, USA.
| | - Dan Poenaru
- McGill University, 5252 Boul. De Maissonneuve O. rm. 3E.05, Montréal, QC, H4a 3S5, Canada
| | - Russell Woo
- Department of Surgery, Division of Pediatric Surgery, University of Hawai'i, John A. Burns School of Medicine, 1319 Punahou Street, Suite 600, Honolulu, HI, 96826, USA
| | - A Francois Trappey
- Pediatric General and Thoracic Surgery, Brooke Army Medical Center, 3551 Roger Brooke Dr, Fort Sam Houston, TX, 78234, USA
| | - Stewart Carter
- Division of Pediatric Surgery, University of Louisville, Norton Children's Hospital, 315 East Broadway, Suite 565, Louisville, KY, 40202, USA
| | - David Darcy
- Golisano Children's Hospital, University of Rochester Medical Center, 601 Elmwood Avenue, Box SURG, Rochester, NY, 14642, USA
| | - Ellen Encisco
- Division of Pediatric General and Thoracic Surgery, Cincinnati Children's Hospital, 3333 Burnet Ave, Cincinnati, OH, 45229, USA
| | - Brian Gulack
- Rush University Medical Center, 1653 W Congress Parkway, Kellogg, Chicago, IL, 60612, USA
| | - Doug Miniati
- Department of Pediatric Surgery, Kaiser Permanente Roseville, 1600 Eureka Road, Building C, Suite C35, Roseville, CA, 95661, USA
| | - Edzhem Tombash
- Division of Pediatric General and Thoracic Surgery, Cincinnati Children's Hospital, 3333 Burnet Ave, Cincinnati, OH, 45229, USA
| | - Eunice Y Huang
- Vanderbilt University Medical Center, Monroe Carell Jr. Children's Hospital, 2200 Children's Way, Suite 7100, Nashville, TN, 37232, USA
| |
Collapse
|
3
|
Mangold S, Ream M. Artificial Intelligence in Graduate Medical Education Applications. J Grad Med Educ 2024; 16:115-118. [PMID: 38993314 PMCID: PMC11234320 DOI: 10.4300/jgme-d-23-00510.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 07/13/2024] Open
Affiliation(s)
- Sarah Mangold
- is a PGY-3 Child Neurology Resident, Division of Child Neurology, Nationwide Children's Hospital, Columbus, Ohio, USA; and
| | - Margie Ream
- is Associate Professor, The Ohio State University College of Medicine, and Program Director, Child Neurology Residency Program, Nationwide Children's Hospital, Columbus, Ohio, USA
| |
Collapse
|
4
|
Gordon M, Daniel M, Ajiboye A, Uraiby H, Xu NY, Bartlett R, Hanson J, Haas M, Spadafore M, Grafton-Clarke C, Gasiea RY, Michie C, Corral J, Kwan B, Dolmans D, Thammasitboon S. A scoping review of artificial intelligence in medical education: BEME Guide No. 84. MEDICAL TEACHER 2024; 46:446-470. [PMID: 38423127 DOI: 10.1080/0142159x.2024.2314198] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Accepted: 01/31/2024] [Indexed: 03/02/2024]
Abstract
BACKGROUND Artificial Intelligence (AI) is rapidly transforming healthcare, and there is a critical need for a nuanced understanding of how AI is reshaping teaching, learning, and educational practice in medical education. This review aimed to map the literature regarding AI applications in medical education, core areas of findings, potential candidates for formal systematic review and gaps for future research. METHODS This rapid scoping review, conducted over 16 weeks, employed Arksey and O'Malley's framework and adhered to STORIES and BEME guidelines. A systematic and comprehensive search across PubMed/MEDLINE, EMBASE, and MedEdPublish was conducted without date or language restrictions. Publications included in the review spanned undergraduate, graduate, and continuing medical education, encompassing both original studies and perspective pieces. Data were charted by multiple author pairs and synthesized into various thematic maps and charts, ensuring a broad and detailed representation of the current landscape. RESULTS The review synthesized 278 publications, with a majority (68%) from North American and European regions. The studies covered diverse AI applications in medical education, such as AI for admissions, teaching, assessment, and clinical reasoning. The review highlighted AI's varied roles, from augmenting traditional educational methods to introducing innovative practices, and underscores the urgent need for ethical guidelines in AI's application in medical education. CONCLUSION The current literature has been charted. The findings underscore the need for ongoing research to explore uncharted areas and address potential risks associated with AI use in medical education. This work serves as a foundational resource for educators, policymakers, and researchers in navigating AI's evolving role in medical education. A framework to support future high utility reporting is proposed, the FACETS framework.
Collapse
Affiliation(s)
- Morris Gordon
- School of Medicine and Dentistry, University of Central Lancashire, Preston, UK
- Blackpool Hospitals NHS Foundation Trust, Blackpool, UK
| | - Michelle Daniel
- School of Medicine, University of California, San Diego, SanDiego, CA, USA
| | - Aderonke Ajiboye
- School of Medicine and Dentistry, University of Central Lancashire, Preston, UK
| | - Hussein Uraiby
- Department of Cellular Pathology, University Hospitals of Leicester NHS Trust, Leicester, UK
| | - Nicole Y Xu
- School of Medicine, University of California, San Diego, SanDiego, CA, USA
| | - Rangana Bartlett
- Department of Cognitive Science, University of California, San Diego, CA, USA
| | - Janice Hanson
- Department of Medicine and Office of Education, School of Medicine, Washington University in Saint Louis, Saint Louis, MO, USA
| | - Mary Haas
- Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, MI, USA
| | - Maxwell Spadafore
- Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, MI, USA
| | | | | | - Colin Michie
- School of Medicine and Dentistry, University of Central Lancashire, Preston, UK
| | - Janet Corral
- Department of Medicine, University of Nevada Reno, School of Medicine, Reno, NV, USA
| | - Brian Kwan
- School of Medicine, University of California, San Diego, SanDiego, CA, USA
| | - Diana Dolmans
- School of Health Professions Education, Faculty of Health, Maastricht University, Maastricht, NL, USA
| | - Satid Thammasitboon
- Center for Research, Innovation and Scholarship in Health Professions Education, Baylor College of Medicine, Houston, TX, USA
| |
Collapse
|
5
|
Magalhães Araujo S, Cruz-Correia R. Incorporating ChatGPT in Medical Informatics Education: Mixed Methods Study on Student Perceptions and Experiential Integration Proposals. JMIR MEDICAL EDUCATION 2024; 10:e51151. [PMID: 38506920 PMCID: PMC10993110 DOI: 10.2196/51151] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Revised: 09/29/2023] [Accepted: 11/10/2023] [Indexed: 03/21/2024]
Abstract
BACKGROUND The integration of artificial intelligence (AI) technologies, such as ChatGPT, in the educational landscape has the potential to enhance the learning experience of medical informatics students and prepare them for using AI in professional settings. The incorporation of AI in classes aims to develop critical thinking by encouraging students to interact with ChatGPT and critically analyze the responses generated by the chatbot. This approach also helps students develop important skills in the field of biomedical and health informatics to enhance their interaction with AI tools. OBJECTIVE The aim of the study is to explore the perceptions of students regarding the use of ChatGPT as a learning tool in their educational context and provide professors with examples of prompts for incorporating ChatGPT into their teaching and learning activities, thereby enhancing the educational experience for students in medical informatics courses. METHODS This study used a mixed methods approach to gain insights from students regarding the use of ChatGPT in education. To accomplish this, a structured questionnaire was applied to evaluate students' familiarity with ChatGPT, gauge their perceptions of its use, and understand their attitudes toward its use in academic and learning tasks. Learning outcomes of 2 courses were analyzed to propose ChatGPT's incorporation in master's programs in medicine and medical informatics. RESULTS The majority of students expressed satisfaction with the use of ChatGPT in education, finding it beneficial for various purposes, including generating academic content, brainstorming ideas, and rewriting text. While some participants raised concerns about potential biases and the need for informed use, the overall perception was positive. Additionally, the study proposed integrating ChatGPT into 2 specific courses in the master's programs in medicine and medical informatics. The incorporation of ChatGPT was envisioned to enhance student learning experiences and assist in project planning, programming code generation, examination preparation, workflow exploration, and technical interview preparation, thus advancing medical informatics education. In medical teaching, it will be used as an assistant for simplifying the explanation of concepts and solving complex problems, as well as for generating clinical narratives and patient simulators. CONCLUSIONS The study's valuable insights into medical faculty students' perspectives and integration proposals for ChatGPT serve as an informative guide for professors aiming to enhance medical informatics education. The research delves into the potential of ChatGPT, emphasizes the necessity of collaboration in academic environments, identifies subject areas with discernible benefits, and underscores its transformative role in fostering innovative and engaging learning experiences. The envisaged proposals hold promise in empowering future health care professionals to work in the rapidly evolving era of digital health care.
Collapse
Affiliation(s)
- Sabrina Magalhães Araujo
- Center for Health Technology and Services Research, Faculty of Medicine, University of Porto, Porto, Portugal
| | - Ricardo Cruz-Correia
- Center for Health Technology and Services Research, Faculty of Medicine, University of Porto, Porto, Portugal
- Department of Community Medicine, Information and Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal
- Working Group Education, European Federation for Medical Informatics, Le Mont-sur-Lausanne, Switzerland
| |
Collapse
|
6
|
Preiksaitis C, Rose C. Opportunities, Challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: Scoping Review. JMIR MEDICAL EDUCATION 2023; 9:e48785. [PMID: 37862079 PMCID: PMC10625095 DOI: 10.2196/48785] [Citation(s) in RCA: 24] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/07/2023] [Revised: 07/28/2023] [Accepted: 09/28/2023] [Indexed: 10/21/2023]
Abstract
BACKGROUND Generative artificial intelligence (AI) technologies are increasingly being utilized across various fields, with considerable interest and concern regarding their potential application in medical education. These technologies, such as Chat GPT and Bard, can generate new content and have a wide range of possible applications. OBJECTIVE This study aimed to synthesize the potential opportunities and limitations of generative AI in medical education. It sought to identify prevalent themes within recent literature regarding potential applications and challenges of generative AI in medical education and use these to guide future areas for exploration. METHODS We conducted a scoping review, following the framework by Arksey and O'Malley, of English language articles published from 2022 onward that discussed generative AI in the context of medical education. A literature search was performed using PubMed, Web of Science, and Google Scholar databases. We screened articles for inclusion, extracted data from relevant studies, and completed a quantitative and qualitative synthesis of the data. RESULTS Thematic analysis revealed diverse potential applications for generative AI in medical education, including self-directed learning, simulation scenarios, and writing assistance. However, the literature also highlighted significant challenges, such as issues with academic integrity, data accuracy, and potential detriments to learning. Based on these themes and the current state of the literature, we propose the following 3 key areas for investigation: developing learners' skills to evaluate AI critically, rethinking assessment methodology, and studying human-AI interactions. CONCLUSIONS The integration of generative AI in medical education presents exciting opportunities, alongside considerable challenges. There is a need to develop new skills and competencies related to AI as well as thoughtful, nuanced approaches to examine the growing use of generative AI in medical education.
Collapse
Affiliation(s)
- Carl Preiksaitis
- Department of Emergency Medicine, Stanford University School of Medicine, Palo Alto, CA, United States
| | - Christian Rose
- Department of Emergency Medicine, Stanford University School of Medicine, Palo Alto, CA, United States
| |
Collapse
|
7
|
A PubMed Scoping Review. ARTIFICIAL INTELLIGENCE APPLICATIONS USING CHATGPT IN EDUCATION 2023:93-105. [DOI: 10.4018/978-1-6684-9300-7.ch009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/02/2024]
Abstract
The current study reviews the challenges of using ChatGPT within medical education to address challenges and proposed solutions given by experts in the medical field. The scoping review uses PubMED using Levac, Colquhoun, and O'Brien's Framework as a reference. A total of 13 studies, out of 102 communication, viewpoints, comments, editorials, brief reports, and letters were included and reviewed. Two reviewers independently conducted the selection process, and the decision was consensually made. The study excluded the review, original, and irrelevant articles. The scoping review did identify concerns about the ethical and rational use of ChatGPT. Several experts have raised questions on how to use ChatGPT so that it does not violate the epistemological methods, which generally involve the critical examination of the sources, methods, and justifications for knowledge claims.
Collapse
|
8
|
Nicolau E, Verduzco-Gutierrez M. Letter to the Editor on "Will ChatGPT Match to Your Program?". Am J Phys Med Rehabil 2023; 102:e115-e116. [PMID: 37026967 DOI: 10.1097/phm.0000000000002259] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/08/2023]
|
9
|
Abstract
ChatGPT is an artificial intelligence (AI) chatbot developed by OpenAI and it first became available to the public in November 2022. ChatGPT can assist in finding academic papers on the web and summarizing them. This chatbot has the potential to be applied in scientific writing, it has the ability to generate automated drafts, summarize articles, and translate content from several languages. This in turn can make academic writing faster and less challenging. However, due to ethical considerations, its use in scientific writing should be regulated and carefully monitored. Few papers have discussed the use of ChatGPT in scientific research writing. This review aims to discuss all the relevant published papers that discuss the use of ChatGPT in medical and dental research.
Collapse
Affiliation(s)
- Bader Fatani
- Dentistry, College of Dentistry, King Saud University, Riyadh, SAU
| |
Collapse
|
10
|
Alhaidry HM, Fatani B, Alrayes JO, Almana AM, Alfhaed NK. ChatGPT in Dentistry: A Comprehensive Review. Cureus 2023; 15:e38317. [PMID: 37266053 PMCID: PMC10230850 DOI: 10.7759/cureus.38317] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/29/2023] [Indexed: 06/03/2023] Open
Abstract
Chat generative pre-trained transformer (ChatGPT) is an artificial intelligence chatbot that uses natural language processing that can respond to human input in a conversational manner. ChatGPT has numerous applications in the health care system including dentistry; it is used in diagnoses and for assessing disease risk and scheduling appointments. It also has a role in scientific research. In the dental field, it has provided many benefits such as detecting dental and maxillofacial abnormalities on panoramic radiographs and identifying different dental restorations. Therefore, it helps in decreasing the workload. But even with these benefits, one should take into consideration the risks and limitations of this chatbot. Few articles mentioned the use of ChatGPT in dentistry. This comprehensive review represents data collected from 66 relevant articles using PubMed and Google Scholar as databases. This review aims to discuss all relevant published articles on the use of ChatGPT in dentistry.
Collapse
Affiliation(s)
- Hind M Alhaidry
- Advanced General Dentistry, Prince Sultan Military Medical City, Riyadh, SAU
| | - Bader Fatani
- Dentistry, College of Dentistry, King Saud University, Riyadh, SAU
| | - Jenan O Alrayes
- Dentistry, College of Dentistry, King Saud University, Riyadh, SAU
| | | | - Nawaf K Alfhaed
- Dentistry, College of Dentistry, King Saud University, Riyadh, SAU
| |
Collapse
|