Can artificial intelligence help for scientific writing?

The Use of Artificial Intelligence in Writing Scientific Review Articles

PURPOSE OF REVIEW

With the recent explosion in the use of artificial intelligence (AI) and specifically ChatGPT, we sought to determine whether ChatGPT could be used to assist in writing credible, peer-reviewed, scientific review articles. We also sought to assess, in a scientific study, the advantages and limitations of using ChatGPT for this purpose. To accomplish this, 3 topics of importance in musculoskeletal research were selected: (1) the intersection of Alzheimer's disease and bone; (2) the neural regulation of fracture healing; and (3) COVID-19 and musculoskeletal health. For each of these topics, 3 approaches to write manuscript drafts were undertaken: (1) human only; (2) ChatGPT only (AI-only); and (3) combination approach of #1 and #2 (AI-assisted). Articles were extensively fact checked and edited to ensure scientific quality, resulting in final manuscripts that were significantly different from the original drafts. Numerous parameters were measured throughout the process to quantitate advantages and disadvantages of approaches.

RECENT FINDINGS

Overall, use of AI decreased the time spent to write the review article, but required more extensive fact checking. With the AI-only approach, up to 70% of the references cited were found to be inaccurate. Interestingly, the AI-assisted approach resulted in the highest similarity indices suggesting a higher likelihood of plagiarism. Finally, although the technology is rapidly changing, at the time of study, ChatGPT 4.0 had a cutoff date of September 2021 rendering identification of recent articles impossible. Therefore, all literature published past the cutoff date was manually provided to ChatGPT, rendering approaches #2 and #3 identical for contemporary citations. As a result, for the COVID-19 and musculoskeletal health topic, approach #2 was abandoned midstream due to the extensive overlap with approach #3. The main objective of this scientific study was to see whether AI could be used in a scientifically appropriate manner to improve the scientific writing process. Indeed, AI reduced the time for writing but had significant inaccuracies. The latter necessitates that AI cannot currently be used alone but could be used with careful oversight by humans to assist in writing scientific review articles.

Ethical Concerns About ChatGPT in Healthcare: A Useful Tool or the Tombstone of Original and Reflective Thinking?

Artificial intelligence (AI), the uprising technology of computer science aiming to create digital systems with human behavior and intelligence, seems to have invaded almost every field of modern life. Launched in November 2022, ChatGPT (Chat Generative Pre-trained Transformer) is a textual AI application capable of creating human-like responses characterized by original language and high coherence. Although AI-based language models have demonstrated impressive capabilities in healthcare, ChatGPT has received controversial annotations from the scientific and academic communities. This chatbot already appears to have a massive impact as an educational tool for healthcare professionals and transformative potential for clinical practice and could lead to dramatic changes in scientific research. Nevertheless, rational concerns were raised regarding whether the pre-trained, AI-generated text would be a menace not only for original thinking and new scientific ideas but also for academic and research integrity, as it gets more and more difficult to distinguish its AI origin due to the coherence and fluency of the produced text. This short review aims to summarize the potential applications and the consequential implications of ChatGPT in the three critical pillars of medicine: education, research, and clinical practice. In addition, this paper discusses whether the current use of this chatbot is in compliance with the ethical principles for the safe use of AI in healthcare, as determined by the World Health Organization. Finally, this review highlights the need for an updated ethical framework and the increased vigilance of healthcare stakeholders to harvest the potential benefits and limit the imminent dangers of this new innovative technology.

Academic publisher guidelines on AI usage: A ChatGPT supported thematic analysis

Background

As Artificial Intelligence (AI) technologies such as Generative AI (GenAI) have become more common in academic settings, it is necessary to examine how these tools interact with issues of authorship, academic integrity, and research methodologies. The current landscape lacks cohesive policies and guidelines for regulating AI's role in academic research which has prompted discussions among publishers, authors, and institutions.

Methods

This study employs inductive thematic analysis to explore publisher policies regarding AI-assisted authorship and academic work. Our methods involved a two-fold analysis using both AI-assisted and traditional unassisted techniques to examine the available policies from leading academic publishers and other publishing or academic entities. The framework was designed to offer multiple perspectives, harnessing the strengths of AI for pattern recognition while leveraging human expertise for nuanced interpretation. The results of these two analyses are combined to form the final themes.

Results

Our findings indicate six overall themes, three of which were independently identified in both the AI-assisted and unassisted, manual analysis using common software tools. A broad consensus appears among publishers that human authorship remains paramount and that the use of GenAI tools is permissible but must be disclosed. However, GenAI tools are increasingly acknowledged for their supportive roles, including text generation and data analysis. The study also discusses the inherent limitations and biases of AI-assisted analysis, necessitating rigorous scrutiny by authors, reviewers, and editors.

Conclusions

There is a growing recognition of AI's role as a valuable auxiliary tool in academic research, but one that comes with caveats pertaining to integrity, accountability, and interpretive limitations. This study used a novel analysis supported by GenAI tools to identify themes emerging in the policy landscape, underscoring the need for an informed, flexible approach to policy formulation that can adapt to the rapidly evolving landscape of AI technologies.

The scholarly footprint of ChatGPT: a bibliometric analysis of the early outbreak phase

This paper presents a comprehensive analysis of the scholarly footprint of ChatGPT, an AI language model, using bibliometric and scientometric methods. The study zooms in on the early outbreak phase from when ChatGPT was launched in November 2022 to early June 2023. It aims to understand the evolution of research output, citation patterns, collaborative networks, application domains, and future research directions related to ChatGPT. By retrieving data from the Scopus database, 533 relevant articles were identified for analysis. The findings reveal the prominent publication venues, influential authors, and countries contributing to ChatGPT research. Collaborative networks among researchers and institutions are visualized, highlighting patterns of co-authorship. The application domains of ChatGPT, such as customer support and content generation, are examined. Moreover, the study identifies emerging keywords and potential research areas for future exploration. The methodology employed includes data extraction, bibliometric analysis using various indicators, and visualization techniques such as Sankey diagrams. The analysis provides valuable insights into ChatGPT's early footprint in academia and offers researchers guidance for further advancements. This study stimulates discussions, collaborations, and innovations to enhance ChatGPT's capabilities and impact across domains.

Artificial intelligence (AI) and ChatGPT involvement in scientific and medical writing, a new concern for researchers. A scoping review

Purpose

The study aims to evaluate PubMed publications on ChatGPT or artificial intelligence (AI) involvement in scientific or medical writing and investigate whether ChatGPT or AI was used to create these articles or listed as authors.

Design/methodology/approach

This scoping review was conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) guidelines. A PubMed database search was performed for articles published between January 1 and November 29, 2023, using appropriate search terms; both authors performed screening and selection independently.

Findings

From the initial search results of 127 articles, 41 were eligible for final analysis. Articles were published in 34 journals. Editorials were the most common article type, with 15 (36.6%) articles. Authors originated from 27 countries, and authors from the USA contributed the most, with 14 (34.1%) articles. The most discussed topic was AI tools and writing capabilities in 19 (46.3%) articles. AI or ChatGPT was involved in manuscript preparation in 31 (75.6%) articles. None of the articles listed AI or ChatGPT as an author, and in 19 (46.3%) articles, the authors acknowledged utilizing AI or ChatGPT.

Practical implications

Researchers worldwide are concerned with AI or ChatGPT involvement in scientific research, specifically the writing process. The authors believe that precise and mature regulations will be developed soon by journals, publishers and editors, which will pave the way for the best usage of these tools.

Originality/value

This scoping review expressed data published on using AI or ChatGPT in various scientific research and writing aspects, besides alluding to the advantages, disadvantages and implications of their usage.

Intelligent Plagiarism as a Misconduct in Academic Integrity

N/a.

Artificial Intelligence-Generated Scientific Literature: A Critical Appraisal

BACKGROUND

Review articles play a critical role in informing medical decisions and identifying avenues for future research. With the introduction of artificial intelligence (AI), there has been a growing interest in the potential of this technology to transform the synthesis of medical literature. Open AI's Generative Pre-trained Transformer (GPT-4) (Open AI Inc, San Francisco, CA) tool provides access to advanced AI that is able to quickly produce medical literature following only simple prompts. The accuracy of the generated articles requires review, especially in subspecialty fields like Allergy/Immunology.

OBJECTIVE

To critically appraise AI-synthesized allergy-focused minireviews.

METHODS

We tasked the GPT-4 Chatbot with generating 2 1,000-word reviews on the topics of hereditary angioedema and eosinophilic esophagitis. Authors critically appraised these articles using the Joanna Briggs Institute (JBI) tool for text and opinion and additionally evaluated domains of interest such as language, reference quality, and accuracy of the content.

RESULTS

The language of the AI-generated minireviews was carefully articulated and logically focused on the topic of interest; however, reviewers of the AI-generated articles indicated that the AI-generated content lacked depth, did not appear to be the result of an analytical process, missed critical information, and contained inaccurate information. Despite being provided instruction to utilize scientific references, the AI chatbot relied mainly on freely available resources, and the AI chatbot fabricated references.

CONCLUSIONS

The AI holds the potential to change the landscape of synthesizing medical literature; however, apparent inaccurate and fabricated information calls for rigorous evaluation and validation of AI tools in generating medical literature, especially on subjects associated with limited resources.

Evaluating ChatGPT responses to frequently asked patient questions regarding periprosthetic joint infection after total hip and knee arthroplasty

Background

Patients access relevant information concerning their orthopaedic surgery resources through multiple information channels before presenting for clinical treatment. Recently, artificial intelligence (AI)-powered chatbots have become another source of information for patients. The currently developed AI chat technology ChatGPT (OpenAI LP) is an application for such purposes and it has been rapidly gaining popularity, including for patient education. This study sought to evaluate whether ChatGPT can correctly answer frequently asked questions (FAQ) regarding periprosthetic joint infection (PJI).

Methods

Twelve FAQs about PJI after hip and knee arthroplasty were identified from the websites of fifteen international clinical expert centres. ChatGPT was confronted with these questions and its responses were analysed for their accuracy using an evidence-based approach by a multidisciplinary team. Responses were categorised in four groups: (1) Excellent response that did not require additional improvement; (2) Satisfactory responses that required a small amount of improvement; (3) Satisfactory responses that required moderate improvement; and (4) Unsatisfactory responses that required a large amount of improvement.

Results

From the analysis of the responses given by the chatbot, no reply received an 'unsatisfactory' rating; one did not require any correction; and the majority of the responses required low (7 out of 12) or moderate (4 out of 12) clarification. Although a few responses required minimal clarification, the chatbot responses were generally unbiased and evidence-based, even when asked controversial questions.

Conclusions

The AI-chatbot ChatGPT was able to effectively answer the FAQs of patients seeking information around PJI diagnosis and treatment. The given information was also written in a manner that can be assumed to be understandable by patients. The chatbot could be a valuable clinical tool for patient education and understanding around PJI treatment in the future. Further studies should evaluate its use and acceptance by patients with PJI.

Can ChatGPT-3.5 Pass a Medical Exam? A Systematic Review of ChatGPT's Performance in Academic Testing

OBJECTIVE

We, therefore, aim to conduct a systematic review to assess the academic potential of ChatGPT-3.5, along with its strengths and limitations when giving medical exams.

METHOD

Following PRISMA guidelines, a systemic search of the literature was performed using electronic databases PUBMED/MEDLINE, Google Scholar, and Cochrane. Articles from their inception till April 4, 2023, were queried. A formal narrative analysis was conducted by systematically arranging similarities and differences between individual findings together.

RESULTS

After rigorous screening, 12 articles underwent this review. All the selected papers assessed the academic performance of ChatGPT-3.5. One study compared the performance of ChatGPT-3.5 with the performance of ChatGPT-4 when giving a medical exam. Overall, ChatGPT performed well in 4 tests, averaged in 4 tests, and performed badly in 4 tests. ChatGPT's performance was directly proportional to the level of the questions' difficulty but was unremarkable on whether the questions were binary, descriptive, or MCQ-based. ChatGPT's explanation, reasoning, memory, and accuracy were remarkably good, whereas it failed to understand image-based questions, and lacked insight and critical thinking.

CONCLUSION

ChatGPT-3.5 performed satisfactorily in the exams it took as an examinee. However, there is a need for future related studies to fully explore the potential of ChatGPT in medical education.

Does Google's Bard Chatbot perform better than ChatGPT on the European hand surgery exam?

PURPOSE

According to a previous research, the chatbot ChatGPT® V3.5 was unable to pass the first part of the European Board of Hand Surgery (EBHS) diploma examination. This study aimed to investigate whether Google's chatbot Bard® would have superior performance compared to ChatGPT on the EBHS diploma examination.

METHODS

Chatbots were asked to answer 18 EBHS multiple choice questions (MCQs) published in the Journal of Hand Surgery (European Volume) in five trials (A1 to A5). After A3, chatbots received correct answers, and after A4, incorrect answers. Consequently, their ability to modify their response was measured and compared.

RESULTS

Bard® scored 3/18 (A1), 1/18 (A2), 4/18 (A3) and 2/18 (A4 and A5). The average percentage of correct answers was 61.1% for A1, 62.2% for A2, 64.4% for A3, 65.6% for A4, 63.3% for A5 and 63.3% for all trials combined. Agreement was moderate from A1 to A5 (kappa = 0.62 (IC95% = [0.51; 0.73])) as well as from A1 to A3 (kappa = 0.60 (IC95% = [0.47; 0.74])). The formulation of Bard® responses was homogeneous, but its learning capacity is still developing.

CONCLUSIONS

The main hypothesis of our study was not proved since Bard did not score significantly higher than ChatGPT when answering the MCQs of the EBHS diploma exam. In conclusion, neither ChatGPT® nor Bard®, in their current versions, can pass the first part of the EBHS diploma exam.

Potential of AI-Driven Chatbots in Urology: Revolutionizing Patient Care Through Artificial Intelligence

PURPOSE OF REVIEW

Artificial intelligence (AI) chatbots have emerged as a potential tool to transform urology by improving patient care and physician efficiency. With an emphasis on their potential advantages and drawbacks, this literature review offers a thorough assessment of the state of AI-driven chatbots in urology today.

RECENT FINDINGS

The capacity of AI-driven chatbots in urology to give patients individualized and timely medical advice is one of its key advantages. Chatbots can help patients prioritize their symptoms and give advice on the best course of treatment. By automating administrative duties and offering clinical decision support, chatbots can also help healthcare providers. Before chatbots are widely used in urology, there are a few issues that need to be resolved. The precision of chatbot diagnoses and recommendations might be impacted by technical constraints like system errors and flaws. Additionally, issues regarding the security and privacy of patient data must be resolved, and chatbots must adhere to all applicable laws. Important issues that must be addressed include accuracy and dependability because any mistakes or inaccuracies could seriously harm patients. The final obstacle is resistance from patients and healthcare professionals who are hesitant to use new technology or who value in-person encounters. AI-driven chatbots have the potential to significantly improve urology care and efficiency. However, it is essential to thoroughly test and ensure the accuracy of chatbots, address privacy and security concerns, and design user-friendly chatbots that can integrate into existing workflows. By exploring various scenarios and examining the current literature, this review provides an analysis of the prospects and limitations of implementing chatbots in urology.

ChatGPT performance in laryngology and head and neck surgery: a clinical case-series

OBJECTIVES

To study the performance of ChatGPT in the management of laryngology and head and neck (LHN) cases.

METHODS

History and clinical examination of patients consulting at the Otolaryngology-Head and Neck Surgery department were presented to ChatGPT, which was interrogated for differential diagnosis, management, and treatment. The ChatGPT performance was assessed by two blinded board-certified otolaryngologists using the following items of a composite score and the Ottawa Clinic Assessment Tool: differential diagnosis; additional examination; and treatment options. The complexity of clinical cases was evaluated with the Amsterdam Clinical Challenge Scale test.

RESULTS

Forty clinical cases were submitted to ChatGPT, accounting for 14 (35%), 12 (30%), and 14 (35%) easy, moderate and difficult cases, respectively. ChatGPT indicated a significant higher number of additional examinations compared to practitioners (p = 0.001). There was a significant agreement between practitioners and ChatGPT for the indication of some common examinations (audiometry, ultrasonography, biopsy, gastrointestinal endoscopy or videofluoroscopy). ChatGPT never indicated some important additional examinations (PET-CT, voice quality assessment, or impedance-pH monitoring). ChatGPT reported highest performance in the proposition of the primary (90%) or the most plausible differential diagnoses (65%), and the therapeutic options (60-68%). The ChatGPT performance in the indication of additional examinations was lowest.

CONCLUSIONS

ChatGPT is a promising adjunctive tool in LHN practice, providing extensive documentation about disease-related additional examinations, differential diagnoses, and treatments. The ChatGPT is more efficient in diagnosis and treatment, rather than in the selection of the most adequate additional examination.

Comparisons of Quality, Correctness, and Similarity Between ChatGPT-Generated and Human-Written Abstracts for Basic Research: Cross-Sectional Study

BACKGROUND

ChatGPT may act as a research assistant to help organize the direction of thinking and summarize research findings. However, few studies have examined the quality, similarity (abstracts being similar to the original one), and accuracy of the abstracts generated by ChatGPT when researchers provide full-text basic research papers.

OBJECTIVE

We aimed to assess the applicability of an artificial intelligence (AI) model in generating abstracts for basic preclinical research.

METHODS

We selected 30 basic research papers from Nature, Genome Biology, and Biological Psychiatry. Excluding abstracts, we inputted the full text into ChatPDF, an application of a language model based on ChatGPT, and we prompted it to generate abstracts with the same style as used in the original papers. A total of 8 experts were invited to evaluate the quality of these abstracts (based on a Likert scale of 0-10) and identify which abstracts were generated by ChatPDF, using a blind approach. These abstracts were also evaluated for their similarity to the original abstracts and the accuracy of the AI content.

RESULTS

The quality of ChatGPT-generated abstracts was lower than that of the actual abstracts (10-point Likert scale: mean 4.72, SD 2.09 vs mean 8.09, SD 1.03; P<.001). The difference in quality was significant in the unstructured format (mean difference -4.33; 95% CI -4.79 to -3.86; P<.001) but minimal in the 4-subheading structured format (mean difference -2.33; 95% CI -2.79 to -1.86). Among the 30 ChatGPT-generated abstracts, 3 showed wrong conclusions, and 10 were identified as AI content. The mean percentage of similarity between the original and the generated abstracts was not high (2.10%-4.40%). The blinded reviewers achieved a 93% (224/240) accuracy rate in guessing which abstracts were written using ChatGPT.

CONCLUSIONS

Using ChatGPT to generate a scientific abstract may not lead to issues of similarity when using real full texts written by humans. However, the quality of the ChatGPT-generated abstracts was suboptimal, and their accuracy was not 100%.

New evidence-based practice: Artificial intelligence as a barrier breaker

The concept of evidence-based practice has persisted over several years and remains a cornerstone in clinical practice, representing the gold standard for optimal patient care. However, despite widespread recognition of its significance, practical application faces various challenges and barriers, including a lack of skills in interpreting studies, limited resources, time constraints, linguistic competencies, and more. Recently, we have witnessed the emergence of a groundbreaking technological revolution known as artificial intelligence. Although artificial intelligence has become increasingly integrated into our daily lives, some reluctance persists among certain segments of the public. This article explores the potential of artificial intelligence as a solution to some of the main barriers encountered in the application of evidence-based practice. It highlights how artificial intelligence can assist in staying updated with the latest evidence, enhancing clinical decision-making, addressing patient misinformation, and mitigating time constraints in clinical practice. The integration of artificial intelligence into evidence-based practice has the potential to revolutionize healthcare, leading to more precise diagnoses, personalized treatment plans, and improved doctor-patient interactions. This proposed synergy between evidence-based practice and artificial intelligence may necessitate adjustments to its core concept, heralding a new era in healthcare.

Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students

Introduction

This mixed-methods study evaluates the impact of AI-assisted language learning on Chinese English as a Foreign Language (EFL) students' writing skills and writing motivation. As artificial intelligence (AI) becomes more prevalent in educational settings, understanding its effects on language learning outcomes is crucial.

Methods

The study employs a comprehensive approach, combining quantitative and qualitative methods. The quantitative phase utilizes a pre-test and post-test design to assess writing skills. Fifty EFL students, matched for proficiency, are randomly assigned to experimental (AI-assisted instruction via ChatGPT) or control (traditional instruction) groups. Writing samples are evaluated using established scoring rubrics. Concurrently, semi-structured interviews are conducted with a subset of participants to explore writing motivation and experiences with AI-assisted learning.

Results

Quantitative analysis reveals significant improvements in both writing skills and motivation among students who received AI-assisted instruction compared to the control group. The experimental group demonstrates enhanced proficiency in various aspects of writing, including organization, coherence, grammar, and vocabulary. Qualitative findings showcase diverse perspectives, ranging from recognition of AI's innovative instructional role and its positive influence on writing skills and motivation to concerns about contextual accuracy and over-reliance. Participants also reflect on the long-term impact and sustainability of AI-assisted instruction, emphasizing the need for ongoing development and adaptation of AI tools.

Discussion

The nuanced findings offer a comprehensive understanding of AI's transformative potential in education. These insights have practical implications for practitioners and researchers, emphasizing the benefits, challenges, and the evolving nature of AI's role in language instruction.

AI language models in human reproduction research: exploring ChatGPT's potential to assist academic writing

Artificial intelligence (AI)-driven language models have the potential to serve as an educational tool, facilitate clinical decision-making, and support research and academic writing. The benefits of their use are yet to be evaluated and concerns have been raised regarding the accuracy, transparency, and ethical implications of using this AI technology in academic publishing. At the moment, Chat Generative Pre-trained Transformer (ChatGPT) is one of the most powerful and widely debated AI language models. Here, we discuss its feasibility to answer scientific questions, identify relevant literature, and assist writing in the field of human reproduction. With consideration of the scarcity of data on this topic, we assessed the feasibility of ChatGPT in academic writing, using data from six meta-analyses published in a leading journal of human reproduction. The text generated by ChatGPT was evaluated and compared to the original text by blinded reviewers. While ChatGPT can produce high-quality text and summarize information efficiently, its current ability to interpret data and answer scientific questions is limited, and it cannot be relied upon for a literature search or accurate source citation due to the potential spread of incomplete or false information. We advocate for open discussions within the reproductive medicine research community to explore the advantages and disadvantages of implementing this AI technology. Researchers and reviewers should be informed about AI language models, and we encourage authors to transparently disclose their use.

Innovating Personalized Nephrology Care: Exploring the Potential Utilization of ChatGPT

The rapid advancement of artificial intelligence (AI) technologies, particularly machine learning, has brought substantial progress to the field of nephrology, enabling significant improvements in the management of kidney diseases. ChatGPT, a revolutionary language model developed by OpenAI, is a versatile AI model designed to engage in meaningful and informative conversations. Its applications in healthcare have been notable, with demonstrated proficiency in various medical knowledge assessments. However, ChatGPT's performance varies across different medical subfields, posing challenges in nephrology-related queries. At present, comprehensive reviews regarding ChatGPT's potential applications in nephrology remain lacking despite the surge of interest in its role in various domains. This article seeks to fill this gap by presenting an overview of the integration of ChatGPT in nephrology. It discusses the potential benefits of ChatGPT in nephrology, encompassing dataset management, diagnostics, treatment planning, and patient communication and education, as well as medical research and education. It also explores ethical and legal concerns regarding the utilization of AI in medical practice. The continuous development of AI models like ChatGPT holds promise for the healthcare realm but also underscores the necessity of thorough evaluation and validation before implementing AI in real-world medical scenarios. This review serves as a valuable resource for nephrologists and healthcare professionals interested in fully utilizing the potential of AI in innovating personalized nephrology care.

ChatGPT: unlocking the potential of Artifical Intelligence in COVID-19 monitoring and prediction

BACKGROUND

The COVID-19 pandemic has had an unprecedent impact of everyday life with deleterious consequences on global health, economics, and society. Thus, accurate and timely information is critical for monitoring its spread and mitigating its impact. ChatGPT is a large language model chatbot with artificial intelligence, developed by OpenAI, that can provide both textual content and R code for predictive models. It may prove to be useful in analyzing and interpreting COVID-19-related data.

METHODS

This paper explores the application of ChatGPT to the monitoring of the COVID-19 pandemic, presenting R code for predictive models and demonstrating the model's capabilities in sentiment analysis, information extraction, and predictive modelling. We used the prediction models suggested by ChatGPT to predict the daily number of COVID-19 deaths in Italy. The prediction accuracy of the models was compared using the following metrics: mean squared error (MSE), mean absolute deviation (MAD) and root mean squared error (RMSE).

RESULTS

ChatGPT suggested three different predictive models, including ARIMA, Random Forest and Prophet. The ARIMA model outperformed the other two models in predicting the daily number of COVID-19 deaths in Italy, with lower MSE, MAD, and RMSE values as compared to the Random Forest and Prophet.

CONCLUSIONS

This paper demonstrates the potential of ChatGPT as a valuable tool in the monitoring of the pandemic. By processing large amounts of data and providing relevant information, ChatGPT has the potential to provide accurate and timely insights, and support decision-making processes to mitigate the spread and impact of pandemics. The paper highlights the importance of exploring the capabilities of artificial intelligence in the management of public emergencies and provides a starting point for future research in this area.

What is the rate of text generated by artificial intelligence over a year of publication in Orthopedics & Traumatology: Surgery & Research? Analysis of 425 articles before versus after the launch of ChatGPT in November 2022

BACKGROUND

The use of artificial intelligence (AI) is soaring, and the launch of ChatGPT in November 2022 has accelerated this trend. This "chatbot" can generate complete scientific articles, with risk of plagiarism by mining existing data or downright fraud by fabricating studies with no real data at all. There are tools that detect AI in publications, but to our knowledge they have not been systematically assessed for publication in scientific journals. We therefore conducted a retrospective study on articles published in Orthopaedics & Traumatology: Surgery & Research (OTSR): firstly, to screen for AI-generated content before and after the publicized launch of ChatGPT; secondly, to assess whether AI was more often used in some countries than others to generate content; thirdly, to determine whether plagiarism rate correlated with AI-generation, and lastly, to determine whether elements other than text generation, and notably the translation procedure, could raise suspicion of AI use.

HYPOTHESIS

The rate of AI use increased after the publicized launch of ChatGPT v3.5 in November 2022.

MATERIAL AND METHODS

In all, 425 articles published between February 2022 and September 2023 (221 before and 204 after November 1, 2022) underwent ZeroGPT assessment of the level of AI generation in the final English-language version (abstract and body of the article). Two scores were obtained: probability of AI generation, in six grades from Human to AI; and percentage AI generation. Plagiarism was assessed on the Ithenticate application at submission. Articles in French were assessed in their English-language version as translated by a human translator, with comparison to automatic translation by Google Translate and DeepL.

RESULTS

AI-generated text was detected mainly in Abstracts, with a 10.1% rate of AI or considerable AI generation, compared to only 1.9% for the body of the article and 5.6% for the total body+abstract. Analysis for before and after November 2022 found an increase in AI generation in body+abstract, from 10.30±15.95% (range, 0-100%) to 15.64±19.8% (range, 0-99.93) (p < 0.04; NS for abstracts alone). AI scores differed between types of article: 14.9% for original articles and 9.8% for reviews (p<0.01). The highest rates of probable AI generation were in articles from Japan, China, South America and English-speaking countries (p<0.0001). Plagiarism rates did not increase between the two study periods, and were unrelated to AI rates. On the other hand, when articles were classified as "suspected" of AI generation (plagiarism rate ≥ 20%) or "non-suspected" (rate<20%), the "similarity" score was higher in suspect articles: 25.7±13.23% (range, 10-69%) versus 16.28±10% (range, 0-79%) (p < 0.001). In the body of the article, use of translation software was associated with higher AI rates than with a human translator: 3.5±5% for human translators, versus 18±10% and 21.9±11% respectively for Google Translate and DeepL (p < 0.001).

DISCUSSION

The present study revealed an increasing rate of AI use in articles published in OTSR. AI grades differed according to type of article and country of origin. Use of translation software increased the AI grade. In the long run, use of ChatGPT incurs a risk of plagiarism and scientific misconduct, and needs to be detected and signaled by a digital tag on any robot-generated text.

LEVEL OF EVIDENCE

III; case-control study.

ChatGPT in academic writing: Maximizing its benefits and minimizing the risks

This review article explores the use of ChatGPT in academic writing and provides insights on how to utilize it judiciously. With the increasing popularity of AI-powered language models, ChatGPT has emerged as a potential tool for assisting writers in the research and writing process. We have provided a list of potential uses of ChatGPT by a novice researcher for getting help during research proposal preparation and manuscript writing. However, there are concerns regarding its reliability and potential risks associated with its use. The review highlights the importance of maintaining human judgment in the writing process and using ChatGPT as a complementary tool rather than a replacement for human effort. The article concludes with recommendations for researchers and writers to ensure responsible and effective use of ChatGPT in academic writing.

Evaluation of the impact of large language learning models on articles submitted to Orthopaedics & Traumatology: Surgery & Research (OTSR): A significant increase in the use of artificial intelligence in 2023

INTRODUCTION

There has been an unprecedented rise is the use of artificial intelligence (AI) amongst medical fields. Recently, a dialogue agent called ChatGPT (Generative Pre-trained Transformer) has grown in popularity through its use of large language models (LLM) to clearly and precisely generate text on demand. However, the impact of AI on the creation of scientific articles is remains unknown. A retrospective study was carried out with the aim of answering the following questions: identify the presence of text generated by LLM before and after the increased usage of ChatGPT in articles submitted in OTSR; determine if the type of article, the year of submission, and the country of origin, influenced the proportion of text generated, at least in part by AI.

MATERIAL AND METHODS

A total of 390 English articles were submitted to OTSR in January, February and March 2022 (n=204) and over the same months of 2023 (n=186) were analyzed. All articles were analyzed using the ZeroGPT tool, which provides an assumed rate of AI use expressed as a percentage. A comparison of the average rate of AI use was carried out between the articles submitted in 2022 and 2023. This comparison was repeated keeping only the articles with the highest percentage of suspected AI use (greater than 10 and 20%). A secondary analysis was carried out to identify risk factors for AI use.

RESULTS

The average percentage of suspected LLM use in the entire cohort was 11%±6, with 160 articles (41.0%) having a suspected AI rate greater than 10% and 61 (15.6%) with an assumed AI rate greater than 20%. A comparison between articles submitted in 2022 and 2023 revealed a significant increase in the use of these tools after the launch of ChatGPT 3.5 (9.4% in 2022 and 12.6% in 2023 [p=0.004]). The number of articles with suspected AI rates of greater than 10 and 20% were significantly higher in 2023: >10%: 71 articles (34.8%) versus 89 articles (47.8%) (p=0.008) and >20%: 21 articles (10.3%) versus 40 articles (21.5%) (p=0.002). A risk factor analysis for LLLM use, demonstrated that authors of Asian geographic origin, and the submission year 2023 were associated with a higher rate of suspected AI use. An AI rate >20% was associated to Asian geographical origin with an odds ratio of 1.79 (95% CI: 1.03-3.11) (p=0.029), while the year of submission being 2023 had an odds ratio of 1.7 (95% CI: 1.1-2.5) (p=0.02).

CONCLUSION

This study highlights a significant increase in the use of LLM in the writing of articles submitted to the OTSR journal after the launch of ChatGPT 3.5. The increasing use of these models raises questions about originality and plagiarism in scientific research. AI offers creative opportunities but also raises ethical and methodological challenges.

LEVEL OF EVIDENCE

III; case control study.

Guiding principles and proposed classification system for the responsible adoption of artificial intelligence in scientific writing in medicine

The integration of large language models (LLMs) and artificial intelligence (AI) into scientific writing, especially in medical literature, presents both unprecedented opportunities and inherent challenges. This manuscript evaluates the transformative potential of LLMs for the synthesis of information, linguistic enhancements, and global knowledge dissemination. At the same time, it raises concerns about unintentional plagiarism, the risk of misinformation, data biases, and an over-reliance on AI. To address these, we propose governing principles for AI adoption that ensure integrity, transparency, validity, and accountability. Additionally, guidelines for reporting AI involvement in manuscript development are delineated, and a classification system to specify the level of AI assistance is introduced. This approach uniquely addresses the challenges of AI in scientific writing, emphasizing transparency in authorship, qualification of AI involvement, and ethical considerations. Concerns regarding access equity, potential biases in AI-generated content, authorship dynamics, and accountability are also explored, emphasizing the human author's continued responsibility. Recommendations are made for fostering collaboration between AI developers, researchers, and journal editors and for emphasizing the importance of AI's responsible use in academic writing. Regular evaluations of AI's impact on the quality and biases of medical manuscripts are also advocated. As we navigate the expanding realm of AI in scientific discourse, it is crucial to maintain the human element of creativity, ethics, and oversight, ensuring that the integrity of scientific literature remains uncompromised.

A SWOT (Strengths, Weaknesses, Opportunities, and Threats) Analysis of ChatGPT in the Medical Literature: Concise Review

BACKGROUND

ChatGPT is a 175-billion-parameter natural language processing model that is already involved in scientific content and publications. Its influence ranges from providing quick access to information on medical topics, assisting in generating medical and scientific articles and papers, performing medical data analyses, and even interpreting complex data sets.

OBJECTIVE

The future role of ChatGPT remains uncertain and a matter of debate already shortly after its release. This review aimed to analyze the role of ChatGPT in the medical literature during the first 3 months after its release.

METHODS

We performed a concise review of literature published in PubMed from December 1, 2022, to March 31, 2023. To find all publications related to ChatGPT or considering ChatGPT, the search term was kept simple ("ChatGPT" in AllFields). All publications available as full text in German or English were included. All accessible publications were evaluated according to specifications by the author team (eg, impact factor, publication modus, article type, publication speed, and type of ChatGPT integration or content). The conclusions of the articles were used for later SWOT (strengths, weaknesses, opportunities, and threats) analysis. All data were analyzed on a descriptive basis.

RESULTS

Of 178 studies in total, 160 met the inclusion criteria and were evaluated. The average impact factor was 4.423 (range 0-96.216), and the average publication speed was 16 (range 0-83) days. Among the articles, there were 77 editorials (48,1%), 43 essays (26.9%), 21 studies (13.1%), 6 reviews (3.8%), 6 case reports (3.8%), 6 news (3.8%), and 1 meta-analysis (0.6%). Of those, 54.4% (n=87) were published as open access, with 5% (n=8) provided on preprint servers. Over 400 quotes with information on strengths, weaknesses, opportunities, and threats were detected. By far, most (n=142, 34.8%) were related to weaknesses. ChatGPT excels in its ability to express ideas clearly and formulate general contexts comprehensibly. It performs so well that even experts in the field have difficulty identifying abstracts generated by ChatGPT. However, the time-limited scope and the need for corrections by experts were mentioned as weaknesses and threats of ChatGPT. Opportunities include assistance in formulating medical issues for nonnative English speakers, as well as the possibility of timely participation in the development of such artificial intelligence tools since it is in its early stages and can therefore still be influenced.

CONCLUSIONS

Artificial intelligence tools such as ChatGPT are already part of the medical publishing landscape. Despite their apparent opportunities, policies and guidelines must be implemented to ensure benefits in education, clinical practice, and research and protect against threats such as scientific misconduct, plagiarism, and inaccuracy.

Potential and limitations of ChatGPT and generative artificial intelligence in medical safety education

The primary objectives of medical safety education are to provide the public with essential knowledge about medications and to foster a scientific approach to drug usage. The era of using artificial intelligence to revolutionize medical safety education has already dawned, and ChatGPT and other generative artificial intelligence models have immense potential in this domain. Notably, they offer a wealth of knowledge, anonymity, continuous availability, and personalized services. However, the practical implementation of generative artificial intelligence models such as ChatGPT in medical safety education still faces several challenges, including concerns about the accuracy of information, legal responsibilities, and ethical obligations. Moving forward, it is crucial to intelligently upgrade ChatGPT by leveraging the strengths of existing medical practices. This task involves further integrating the model with real-life scenarios and proactively addressing ethical and security issues with the ultimate goal of providing the public with comprehensive, convenient, efficient, and personalized medical services.

[Scientific integrity faces plagiarism fabricated with the ChatGPT]

Among the malpractices that undermine research integrity, plagiarism is a major threat given its frequency and evolving presentations. Plagiarism implies the intentional grabbing of texts, ideas, images, or data belonging to others and without crediting them. However, the different and even masked forms of plagiarism often difficult a clear identification. Currently, the many kinds of fraud and plagiarism account for most retractions in traditional and open access journals. Further, the rate of retracted articles is higher in the Latin American databases LILACS and Scielo than in PubMed and Web of Science. This difference has been related to the typical laxity of our culture and the lack of English writing skills of non-Anglophone researchers. These features explain the conflict experienced by Latin American students in USA where they face a stricter culture regarding academic and scientific plagiarism. In the internet era, the ease of accessing scientific literature has increased the temptation to plagiarize but this ethical breach has been countered by antiplagiarism software. Now, the so-called "paraphragiarism" prompted by paraphrasing tools exceeds the infamous "copy-paste". For instance, the innovative ChatGPT can be used for plagiarizing and paraphragiarizing. Moreover, its inclusion as coauthor in scientific papers has been banned by prestigious journals and the International Committee of Medical Journal Editors because such chatbot cannot meet the required public responsibility criterium. To avoid plagiarism, it is enough to always give due credit in the proper way. Lastly, I question the ill-fated and now prevailing conjunction of blind faith in progress and zero skepticism that prevents us from foreseeing the negative consequences of technological advances.

The Role of ChatGPT in the Advancement of Diagnosis, Management, and Prognosis of Cardiovascular and Cerebrovascular Disease

Cardiovascular and cerebrovascular disease incidence has risen mainly due to poor control of preventable risk factors and still constitutes a significant financial and health burden worldwide. ChatGPT is an artificial intelligence language-based model developed by OpenAI. Due to the model's unique cognitive capabilities beyond data processing and the production of high-quality text, there has been a surge of research interest concerning its role in the scientific community and contemporary clinical practice. To fully exploit ChatGPT's potential benefits and reduce its possible misuse, extreme caution must be taken to ensure its implications ethically and equitably. In this narrative review, we explore the language model's possible applications and limitations while emphasizing its potential value for diagnosing, managing, and prognosis of cardiovascular and cerebrovascular disease.

Impact of ChatGPT on Medical Research Article Writing and Publication

NONE

ChatGPT-4 and Human Researchers Are Equal in Writing Scientific Introduction Sections: A Blinded, Randomized, Non-inferiority Controlled Study

Background Natural language processing models are increasingly used in scientific research, and their ability to perform various tasks in the research process is rapidly advancing. This study aims to investigate whether Generative Pre-trained Transformer 4 (GPT-4) is equal to humans in writing introduction sections for scientific articles. Methods This randomized non-inferiority study was reported according to the Consolidated Standards of Reporting Trials for non-inferiority trials and artificial intelligence (AI) guidelines. GPT-4 was instructed to synthesize 18 introduction sections based on the aim of previously published studies, and these sections were compared to the human-written introductions already published in a medical journal. Eight blinded assessors randomly evaluated the introduction sections using 1-10 Likert scales. Results There was no significant difference between GPT-4 and human introductions regarding publishability and content quality. GPT-4 had one point significantly better scores in readability, which was considered a non-relevant difference. The majority of assessors (59%) preferred GPT-4, while 33% preferred human-written introductions. Based on Lix and Flesch-Kincaid scores, GPT-4 introductions were 10 and two points higher, respectively, indicating that the sentences were longer and had longer words. Conclusion GPT-4 was found to be equal to humans in writing introductions regarding publishability, readability, and content quality. The majority of assessors preferred GPT-4 introductions and less than half could determine which were written by GPT-4 or humans. These findings suggest that GPT-4 can be a useful tool for writing introduction sections, and further studies should evaluate its ability to write other parts of scientific articles.

GLP-1 Receptor Agonists and Related Mental Health Issues; Insights from a Range of Social Media Platforms Using a Mixed-Methods Approach

The emergence of glucagon-like peptide-1 receptor agonists (GLP-1 RAs; semaglutide and others) now promises effective, non-invasive treatment of obesity for individuals with and without diabetes. Social media platforms' users started promoting semaglutide/Ozempic as a weight-loss treatment, and the associated increase in demand has contributed to an ongoing worldwide shortage of the drug associated with levels of non-prescribed semaglutide intake. Furthermore, recent reports emphasized some GLP-1 RA-associated risks of triggering depression and suicidal thoughts. Consistent with the above, we aimed to assess the possible impact of GLP-1 RAs on mental health as being perceived and discussed in popular open platforms with the help of a mixed-methods approach. Reddit posts yielded 12,136 comments, YouTube videos 14,515, and TikTok videos 17,059, respectively. Out of these posts/entries, most represented matches related to sleep-related issues, including insomnia (n = 620 matches); anxiety (n = 353); depression (n = 204); and mental health issues in general (n = 165). After the initiation of GLP-1 RAs, losing weight was associated with either a marked improvement or, in some cases, a deterioration, in mood; increase/decrease in anxiety/insomnia; and better control of a range of addictive behaviors. The challenges of accessing these medications were a hot topic as well. To the best of our knowledge, this is the first study documenting if and how GLP-1 RAs are perceived as affecting mood, mental health, and behaviors. Establishing a clear cause-and-effect link between metabolic diseases, depression and medications is difficult because of their possible reciprocal relationship, shared underlying mechanisms and individual differences. Further research is needed to better understand the safety profile of these molecules and their putative impact on behavioral and non-behavioral addictions.

The use of ChatGPT in occupational medicine: opportunities and threats

ChatGPT has the potential to revolutionize occupational medicine by providing a powerful tool for analyzing data, improving communication, and increasing efficiency. It can help identify patterns and trends in workplace health and safety, act as a virtual assistant for workers, employers, and occupational health professionals, and automate certain tasks. However, caution is required due to ethical concerns, the need to maintain confidentiality, and the risk of inconsistent or inaccurate results. ChatGPT cannot replace the crucial role of the occupational health professional in the medical surveillance of workers and the analysis of data on workers' health.

GPT4: The Indispensable Helper for Neurosurgeons in the New Era

GPT4 is the newest multimodal language model released by OpenAI. With its powerful capabilities, GPT4 has great potential to revolutionize the healthcare industry. In this study, we proposed various ways GPT4 could display its talents in the field of neurosurgery in future. We believe that GPT4 is prone to become an indispensable assistant for neurosurgeons in the new era.

Clinical Research With Large Language Models Generated Writing-Clinical Research with AI-assisted Writing (CRAW) Study

IMPORTANCE

The scientific community debates Generative Pre-trained Transformer (GPT)-3.5's article quality, authorship merit, originality, and ethical use in scientific writing.

OBJECTIVES

Assess GPT-3.5's ability to craft the background section of critical care clinical research questions compared to medical researchers with H-indices of 22 and 13.

DESIGN

Observational cross-sectional study.

SETTING

Researchers from 20 countries from six continents evaluated the backgrounds.

PARTICIPANTS

Researchers with a Scopus index greater than 1 were included.

MAIN OUTCOMES AND MEASURES

In this study, we generated a background section of a critical care clinical research question on "acute kidney injury in sepsis" using three different methods: researcher with H-index greater than 20, researcher with H-index greater than 10, and GPT-3.5. The three background sections were presented in a blinded survey to researchers with an H-index range between 1 and 96. First, the researchers evaluated the main components of the background using a 5-point Likert scale. Second, they were asked to identify which background was written by humans only or with large language model-generated tools.

RESULTS

A total of 80 researchers completed the survey. The median H-index was 3 (interquartile range, 1-7.25) and most (36%) researchers were from the Critical Care specialty. When compared with researchers with an H-index of 22 and 13, GPT-3.5 was marked high on the Likert scale ranking on main background components (median 4.5 vs. 3.82 vs. 3.6 vs. 4.5, respectively; p < 0.001). The sensitivity and specificity to detect researchers writing versus GPT-3.5 writing were poor, 22.4% and 57.6%, respectively.

CONCLUSIONS AND RELEVANCE

GPT-3.5 could create background research content indistinguishable from the writing of a medical researcher. It was marked higher compared with medical researchers with an H-index of 22 and 13 in writing the background section of a critical care clinical research question.

Is ChatGPT a "Fire of Prometheus" for Non-Native English-Speaking Researchers in Academic Writing?

Large language models (LLMs) such as ChatGPT have garnered considerable interest for their potential to aid non-native English-speaking researchers. These models can function as personal, round-the-clock English tutors, akin to how Prometheus in Greek mythology bestowed fire upon humans for their advancement. LLMs can be particularly helpful for non-native researchers in writing the Introduction and Discussion sections of manuscripts, where they often encounter challenges. However, using LLMs to generate text for research manuscripts entails concerns such as hallucination, plagiarism, and privacy issues; to mitigate these risks, authors should verify the accuracy of generated content, employ text similarity detectors, and avoid inputting sensitive information into their prompts. Consequently, it may be more prudent to utilize LLMs for editing and refining text rather than generating large portions of text. Journal policies concerning the use of LLMs vary, but transparency in disclosing artificial intelligence tool usage is emphasized. This paper aims to summarize how LLMs can lower the barrier to academic writing in English, enabling researchers to concentrate on domain-specific research, provided they are used responsibly and cautiously.