1
|
Májovský M, Černý M, Kasal M, Komarc M, Netuka D. Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora's Box Has Been Opened. J Med Internet Res 2023; 25:e46924. [PMID: 37256685 PMCID: PMC10267787 DOI: 10.2196/46924] [Citation(s) in RCA: 52] [Impact Index Per Article: 52.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 04/25/2023] [Accepted: 05/03/2023] [Indexed: 06/01/2023] Open
Abstract
BACKGROUND Artificial intelligence (AI) has advanced substantially in recent years, transforming many industries and improving the way people live and work. In scientific research, AI can enhance the quality and efficiency of data analysis and publication. However, AI has also opened up the possibility of generating high-quality fraudulent papers that are difficult to detect, raising important questions about the integrity of scientific research and the trustworthiness of published papers. OBJECTIVE The aim of this study was to investigate the capabilities of current AI language models in generating high-quality fraudulent medical articles. We hypothesized that modern AI models can create highly convincing fraudulent papers that can easily deceive readers and even experienced researchers. METHODS This proof-of-concept study used ChatGPT (Chat Generative Pre-trained Transformer) powered by the GPT-3 (Generative Pre-trained Transformer 3) language model to generate a fraudulent scientific article related to neurosurgery. GPT-3 is a large language model developed by OpenAI that uses deep learning algorithms to generate human-like text in response to prompts given by users. The model was trained on a massive corpus of text from the internet and is capable of generating high-quality text in a variety of languages and on various topics. The authors posed questions and prompts to the model and refined them iteratively as the model generated the responses. The goal was to create a completely fabricated article including the abstract, introduction, material and methods, discussion, references, charts, etc. Once the article was generated, it was reviewed for accuracy and coherence by experts in the fields of neurosurgery, psychiatry, and statistics and compared to existing similar articles. RESULTS The study found that the AI language model can create a highly convincing fraudulent article that resembled a genuine scientific paper in terms of word usage, sentence structure, and overall composition. The AI-generated article included standard sections such as introduction, material and methods, results, and discussion, as well a data sheet. It consisted of 1992 words and 17 citations, and the whole process of article creation took approximately 1 hour without any special training of the human user. However, there were some concerns and specific mistakes identified in the generated article, specifically in the references. CONCLUSIONS The study demonstrates the potential of current AI language models to generate completely fabricated scientific articles. Although the papers look sophisticated and seemingly flawless, expert readers may identify semantic inaccuracies and errors upon closer inspection. We highlight the need for increased vigilance and better detection methods to combat the potential misuse of AI in scientific research. At the same time, it is important to recognize the potential benefits of using AI language models in genuine scientific writing and research, such as manuscript preparation and language editing.
Collapse
Affiliation(s)
- Martin Májovský
- Department of Neurosurgery and Neurooncology, First Faculty of Medicine, Charles University, Prague, Czech Republic
| | - Martin Černý
- Department of Neurosurgery and Neurooncology, First Faculty of Medicine, Charles University, Prague, Czech Republic
| | - Matěj Kasal
- Department of Psychiatry, Faculty of Medicine in Pilsen, Charles University, Pilsen, Czech Republic
| | - Martin Komarc
- Institute of Biophysics and Informatics, First Faculty of Medicine, Charles University, Prague, Czech Republic
- Department of Methodology, Faculty of Physical Education and Sport, Charles University, Prague, Czech Republic
| | - David Netuka
- Department of Neurosurgery and Neurooncology, First Faculty of Medicine, Charles University, Prague, Czech Republic
| |
Collapse
|
2
|
Ott DE. Reference Hygiene and Death on the Internet - Decay, Rot, Half-Life, Deterioration, and Corruption. JSLS 2022; 26:JSLS.2021.00082. [PMID: 35281707 PMCID: PMC8896816 DOI: 10.4293/jsls.2021.00082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Abstract
Background and Objectives: References for medical articles are not always retrievable. This eliminates the ability to check on the validity of statements, methodologies, data collection, and conclusions. Methods: References of review, scientific, and research articles published in the 2019 and 2020 Journal of the Society of Laparoscopic & Robotic Surgeons were evaluated for ability to retrieve the reference cited. Results: Ninety-five articles with 2,424 references were evaluated. There were 1,025 (1,025/2,424 = 42.3%) open access versus 1,399 (1,399/2,424 = 47.7%) paywall articles. There were 357 (14.7%) citations unavailable (misreference) due to bad, broken, or nonexistent links and wrong article, abstract or citation only, and missing citations. Conclusion: Loss of reference existence or retrievability is a scientific hazard. Science is self-correcting but is doomed to not knowing what was said or discovered when references are no longer available.
Collapse
Affiliation(s)
- Douglas E Ott
- School of Engineering, Mercer University, Macon, GA, USA
| |
Collapse
|
3
|
Gudi SK, Tiwari KK, Shaik R. References and Citations: Is It Time to Update Their Format? Int J Endocrinol Metab 2021; 19:e116404. [PMID: 34567141 PMCID: PMC8453648 DOI: 10.5812/ijem.116404] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/21/2021] [Revised: 06/09/2021] [Accepted: 06/21/2021] [Indexed: 11/20/2022] Open
Affiliation(s)
- Sai Krishna Gudi
- College of Pharmacy, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Canada
- Corresponding Author: College of Pharmacy, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Canada.
| | - Komal Krishna Tiwari
- Department of Occupational Therapy, College of Rehabilitation Sciences, University of Manitoba, Winnipeg, Canada
| | - Rahamthulla Shaik
- Department of Pharmacy Practice, Chebrolu Hanumaiah Institute of Pharmaceutical Sciences, Andhra Pradesh, India
| |
Collapse
|
4
|
Pavlovic V, Weissgerber T, Stanisavljevic D, Pekmezovic T, Milicevic O, Lazovic JM, Cirkovic A, Savic M, Rajovic N, Piperac P, Djuric N, Madzarevic P, Dimitrijevic A, Randjelovic S, Nestorovic E, Akinyombo R, Pavlovic A, Ghamrawi R, Garovic V, Milic N. How accurate are citations of frequently cited papers in biomedical literature? Clin Sci (Lond) 2021; 135:671-681. [PMID: 33599711 PMCID: PMC8048031 DOI: 10.1042/cs20201573] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2021] [Revised: 02/16/2021] [Accepted: 02/18/2021] [Indexed: 11/17/2022]
Abstract
Citations are an important, but often overlooked, part of every scientific paper. They allow the reader to trace the flow of evidence, serving as a gateway to relevant literature. Most scientists are aware of citations' errors, but few appreciate the prevalence of these problems. The purpose of the present study was to examine how often frequently cited papers in biomedical scientific literature are cited inaccurately. The study included an active participation of the first authors of included papers; to first-hand verify the citations accuracy. Findings from feasibility study, where we reviewed 1540 articles containing 2526 citations of 14 most cited articles in which the authors were affiliated with the Faculty of Medicine University of Belgrade, were further evaluated for external confirmation in an independent verification set of articles. Verification set included 4912 citations identified in 2995 articles that cited 13 most cited articles published by authors affiliated with the Mayo Clinic Division of Nephrology and Hypertension. A citation was defined as being accurate if the cited article supported or was in accordance with the statement by citing authors. At least one inaccurate citation was found in 11 and 15% of articles in the feasibility study and verification set, respectively, suggesting that inaccurate citations are common in biomedical literature. The most common problem was the citation of nonexistent findings (38.4%), followed by an incorrect interpretation of findings (15.4%). One-fifth of inaccurate citations were due to chains of inaccurate citations. Based on these findings, several actions to reduce citation inaccuracies have been proposed.
Collapse
Affiliation(s)
- Vedrana Pavlovic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Tracey Weissgerber
- Charité - Universitätsmedizin Berlin, Berlin Institute of Health, QUEST Center, Berlin
| | - Dejana Stanisavljevic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Tatjana Pekmezovic
- Institute of Epidemiology, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Ognjen Milicevic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Jelena Milin Lazovic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Andja Cirkovic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Marko Savic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Nina Rajovic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Pavle Piperac
- Department of Humanities, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Nemanja Djuric
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Petar Madzarevic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Ana Dimitrijevic
- Clinical Center of Serbia, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Simona Randjelovic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
- Clinical Center of Serbia, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Emilija Nestorovic
- Clinical Center of Serbia, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Remi Akinyombo
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Andrija Pavlovic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - Ranine Ghamrawi
- Division of Nephrology and Hypertension, Department of Internal Medicine, Mayo Clinic, Rochester, MN U.S.A
| | - Vesna Garovic
- Division of Nephrology and Hypertension, Department of Internal Medicine, Mayo Clinic, Rochester, MN U.S.A
| | - Natasa Milic
- Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
- Division of Nephrology and Hypertension, Department of Internal Medicine, Mayo Clinic, Rochester, MN U.S.A
| |
Collapse
|
5
|
Hui A, Rains LS, Todd A, Boaz A, Johnson S. The accuracy and accessibility of cited evidence: a study examining mental health policy documents. Soc Psychiatry Psychiatr Epidemiol 2020; 55:111-121. [PMID: 31641826 DOI: 10.1007/s00127-019-01786-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/16/2019] [Accepted: 10/09/2019] [Indexed: 11/29/2022]
Abstract
PURPOSE Evidence-based policy making is increasingly being advocated by governments and scholars. To show that policies are informed by evidence, policy-related documents that cite external sources should ideally provide direct access to, and accurately represent, the referenced source and the evidence it provides. Our aim was to find a way to systematically assess the prevalence of referencing accuracy and accessibility issues in referenced statements selected from a sample of mental health-related policy documents. METHOD 236 referenced statements were selected from 10 mental health-related policy documents published between 2013 and 2018. Policy documents were chosen as the focus of this investigation because of their relative accessibility and impact on clinical practice. Statements were rated against their referenced sources in terms of the (i) content accuracy in relation to the information provided by the referenced source and (ii) degree of accessibility of the source and the required evidence from the references provided. RESULTS Of the 236 statements, 141 (59.7%) accurately represented the referenced source, 45 (19.1%) contained major errors and 50 (21.2%) contained minor errors in accuracy. For accessibility, 126 (53.4%) directly referenced primary sources of evidence that supported the claims made, 36 (15.3%) contained indirect references, 18 (7.6%) provided 'dead-end' references, and 11 (4.7%) references were completely inaccessible. CONCLUSIONS With only slightly over half of all statements assessed providing fully accessible references and accurately representing the referenced source, these components of referencing quality deserve further attention if evidence-informed policy goals are to be achieved. The rating framework used in the current study proved to be a simple and straightforward method to assess these components and can provide a baseline against which interventions can be designed to improve referencing quality.
Collapse
Affiliation(s)
- Aika Hui
- Division of Psychiatry, University College London, London, W1T 7NF, UK.,Centre for Health and Social Care Research, Kingston University and St George's University of London, London, KT1 2EE, UK
| | - Luke Sheridan Rains
- Division of Psychiatry, University College London, London, W1T 7NF, UK. .,Centre for Health and Social Care Research, Kingston University and St George's University of London, London, KT1 2EE, UK.
| | - Anita Todd
- Division of Psychiatry, University College London, London, W1T 7NF, UK.,Centre for Health and Social Care Research, Kingston University and St George's University of London, London, KT1 2EE, UK
| | - Annette Boaz
- Division of Psychiatry, University College London, London, W1T 7NF, UK.,Centre for Health and Social Care Research, Kingston University and St George's University of London, London, KT1 2EE, UK
| | - Sonia Johnson
- Division of Psychiatry, University College London, London, W1T 7NF, UK.,Centre for Health and Social Care Research, Kingston University and St George's University of London, London, KT1 2EE, UK
| |
Collapse
|
6
|
Mogull SA. Accuracy of cited "facts" in medical research articles: A review of study methodology and recalculation of quotation error rate. PLoS One 2017; 12:e0184727. [PMID: 28910404 PMCID: PMC5599002 DOI: 10.1371/journal.pone.0184727] [Citation(s) in RCA: 34] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2017] [Accepted: 08/23/2017] [Indexed: 11/19/2022] Open
Abstract
Previous reviews estimated that approximately 20 to 25% of assertions cited from original research articles, or "facts," are inaccurately quoted in the medical literature. These reviews noted that the original studies were dissimilar and only began to compare the methods of the original studies. The aim of this review is to examine the methods of the original studies and provide a more specific rate of incorrectly cited assertions, or quotation errors, in original research articles published in medical journals. Additionally, the estimate of quotation errors calculated here is based on the ratio of quotation errors to quotations examined (a percent) rather than the more prevalent and weighted metric of quotation errors to the references selected. Overall, this resulted in a lower estimate of the quotation error rate in original medical research articles. A total of 15 studies met the criteria for inclusion in the primary quantitative analysis. Quotation errors were divided into two categories: content ("factual") or source (improper indirect citation) errors. Content errors were further subdivided into major and minor errors depending on the degree that the assertion differed from the original source. The rate of quotation errors recalculated here is 14.5% (10.5% to 18.6% at a 95% confidence interval). These content errors are predominantly, 64.8% (56.1% to 73.5% at a 95% confidence interval), major errors or cited assertions in which the referenced source either fails to substantiate, is unrelated to, or contradicts the assertion. Minor errors, which are an oversimplification, overgeneralization, or trivial inaccuracies, are 35.2% (26.5% to 43.9% at a 95% confidence interval). Additionally, improper secondary (or indirect) citations, which are distinguished from calculations of quotation accuracy, occur at a rate of 10.4% (3.4% to 17.5% at a 95% confidence interval).
Collapse
Affiliation(s)
- Scott A. Mogull
- Texas State University, San Marcos, Texas, United States of America
| |
Collapse
|
7
|
Jergas H, Baethge C. Quotation accuracy in medical journal articles-a systematic review and meta-analysis. PeerJ 2015; 3:e1364. [PMID: 26528420 PMCID: PMC4627914 DOI: 10.7717/peerj.1364] [Citation(s) in RCA: 40] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2015] [Accepted: 10/09/2015] [Indexed: 11/20/2022] Open
Abstract
Background. Quotations and references are an indispensable element of scientific communication. They should support what authors claim or provide important background information for readers. Studies indicate, however, that quotations not serving their purpose-quotation errors-may be prevalent. Methods. We carried out a systematic review, meta-analysis and meta-regression of quotation errors, taking account of differences between studies in error ascertainment. Results. Out of 559 studies screened we included 28 in the main analysis, and estimated major, minor and total quotation error rates of 11,9%, 95% CI [8.4, 16.6] 11.5% [8.3, 15.7], and 25.4% [19.5, 32.4]. While heterogeneity was substantial, even the lowest estimate of total quotation errors was considerable (6.7%). Indirect references accounted for less than one sixth of all quotation problems. The findings remained robust in a number of sensitivity and subgroup analyses (including risk of bias analysis) and in meta-regression. There was no indication of publication bias. Conclusions. Readers of medical journal articles should be aware of the fact that quotation errors are common. Measures against quotation errors include spot checks by editors and reviewers, correct placement of citations in the text, and declarations by authors that they have checked cited material. Future research should elucidate if and to what degree quotation errors are detrimental to scientific progress.
Collapse
Affiliation(s)
- Hannah Jergas
- University of Cologne Medical School, Cologne, Germany
- Department of Psychiatry and Psychotherapy, University of Cologne Medical School, Cologne, Germany
| | - Christopher Baethge
- Department of Psychiatry and Psychotherapy, University of Cologne Medical School, Cologne, Germany
- Deutsches Ärzteblatt & Deutsches Ärzteblatt International, Cologne, Germany
| |
Collapse
|
8
|
Sungur MO, Seyhan TÖ. Writing references and using citation management software. Turk J Urol 2015; 39:25-32. [PMID: 26328132 DOI: 10.5152/tud.2013.050] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2013] [Accepted: 04/24/2013] [Indexed: 11/22/2022]
Abstract
The correct citation of references is obligatory to gain scientific credibility, to honor the original ideas of previous authors and to avoid plagiarism. Currently, researchers can easily find, cite and store references using citation management software. In this review, two popular citation management software programs (EndNote and Mendeley) are summarized.
Collapse
Affiliation(s)
- Mukadder Orhan Sungur
- Department of Anaesthesiology, İstanbul Faculty of Medicine, İstanbul University, İstanbul, Turkey
| | - Tülay Özkan Seyhan
- Department of Anaesthesiology, İstanbul Faculty of Medicine, İstanbul University, İstanbul, Turkey
| |
Collapse
|
9
|
Kelleher DK, Piazena H. The interpretation of clinical studies on the photodynamic treatment of actinic keratosis. GERMAN MEDICAL SCIENCE : GMS E-JOURNAL 2012; 10:Doc17. [PMID: 23255880 PMCID: PMC3525885 DOI: 10.3205/000168] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Subscribe] [Scholar Register] [Received: 09/04/2012] [Indexed: 11/30/2022]
Abstract
Actinic keratosis is one of the most commonly treated skin conditions. A number of studies have recently been published on the treatment of this ailment using photodynamic therapy. The authors of this letter are concerned about the interpretation of some of these studies and would like to outline possible misinterpretations which may arise due to an incomplete analysis of the study reports available. Clearly, the "ideal" therapy for actinic keratosis should be a carefully chosen compromise between undesired side-effects and therapeutic efficacy and needs to be based on a consideration of all of the relevant clinical studies.
Collapse
Affiliation(s)
- Debra K Kelleher
- University Medical Center, Johannes Gutenberg University Mainz, Institute of Functional and Clinical Anatomy, Mainz, Germany.
| | | |
Collapse
|