1
|
Sciberras M, Farrugia Y, Gordon H, Furfaro F, Allocca M, Torres J, Arebi N, Fiorino G, Iacucci M, Verstockt B, Magro F, Katsanos K, Busuttil J, De Giovanni K, Fenech VA, Chetcuti Zammit S, Ellul P. Accuracy of Information given by ChatGPT for patients with Inflammatory Bowel Disease in relation to ECCO Guidelines. J Crohns Colitis 2024:jjae040. [PMID: 38520394 DOI: 10.1093/ecco-jcc/jjae040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/09/2023] [Indexed: 03/25/2024]
Abstract
INTRODUCTION As acceptance of AI platforms increases, more patients will consider these tools as sources of information. The ChatGPT architecture utilizes a neural network to process natural language, thus generating responses based on the context of input text. The accuracy and completeness of ChatGPT3.5 in the context of Inflammatory Bowel Disease remains unclear. METHODS In this prospective study, 38 questions worded by IBD patients were inputted into ChatGPT3.5. The following topics were covered: 1) CD, UC and malignancy, 2) maternal medicine 3) infection and vaccination 4) complementary medicine. Responses given by Chat GPT were assessed for accuracy (1 - completely incorrect to 5 - completely correct) and completeness (3-point Likert scale; range 1 - incomplete to 3 - complete) by 14 expert gastroenterologists, in comparison with relevant ECCO guidelines. RESULTS In terms of accuracy, most replies (84.2%) had a median score of ≥4 (IQR:2) and a mean score of 3.87 (SD: +/- 0.6). For completeness, 34.2% of the replies had a median score of 3 and 55.3 % had a median score of between 2 and <3. Overall, the mean rating was 2.24 (SD: +/- 0.4, Median:2 IQR :1). Though group 3 and 4 had a higher mean for both accuracy and completeness, there was no significant scoring variation between the 4 question groups (Kruskal-Wallis test p:>0.05). However, statistical analysis for the different individual questions revealed a significant difference both for accuracy (p<0.001) and completeness (p<0.001). The questions which rated the highest for both accuracy and completeness were related to smoking, while the lowest rating was related to screening for malignancy and vaccinations especially in the context of immunosuppression and family planning. CONCLUSION This is the first study to demonstrate the capability of an AI-based system to provide accurate and comprehensive answers to real-world patient queries in IBD. AI systems may serve as a useful adjunct for patients, in addition to standard of care in clinic and validated patient information resources. However, responses in specialist areas may deviate from evidence-based guidance and the replies need to give more firm advice.
Collapse
Affiliation(s)
- Martina Sciberras
- Department of Medicine, Division of Gastroenterology, Mater Dei Hospital, Malta
| | - Yvette Farrugia
- Department of Medicine, Division of Gastroenterology, Mater Dei Hospital, Malta
| | - Hannah Gordon
- Department of Gastroenterology, Barts Health NHS Trust, London, UK
- Translational Gastroenterology and Liver Unit, John Radcliffe Hospital, University of Oxford
| | - Federica Furfaro
- IRCCS OSPEDALE San Raffaele, Gastroenterology and Endoscopy, IBD Center, Milan, Italy
| | - Mariangela Allocca
- IRCCS OSPEDALE San Raffaele, Gastroenterology and Endoscopy, IBD Center, Milan, Italy
| | - Joana Torres
- Division of Gastroenterology, Hospital da Luz, Lisbon, Portugal
- Division of Gastroenterology, Hospital Beatriz Ângelo, Portugal
- Faculdade de Medicina, Universidade de Lisboa, Portugal
| | - Naila Arebi
- Department of Inflammatory Bowel Disease, St Mark's National Bowel Hospital, London, UK
- Department of Metabolism, Digestion and Reproduction, Imperial College London, UK
| | - Gionata Fiorino
- IRCCS OSPEDALE San Raffaele, Gastroenterology and Endoscopy, IBD Center, Milan, Italy
- IBD Unit, San Camillo-Forlanini Hospital, Rome, Italy
| | - Marietta Iacucci
- APC Microbiome Ireland, College of Medicine and Health, University College of Cork, Cork
| | - Bram Verstockt
- Department of Gastroenterology and Hepatology, University Hospitals Leuven, KU Leuven, Leuven, Belgium
- Department of Chronic Diseases and Metabolism, KU Leuven, Leuven, Belgium
| | - Fernando Magro
- CINTESIS@RISE, Faculty of Medicine of the University of Porto, 4200-319 Porto, Portugal
| | - Kostas Katsanos
- Division of Gastroenterology, Department of Internal Medicine, Faculty of Medicine, University of Ioannina School of Health Sciences,45110 Ioannina, Greece
| | | | | | - Valerie Anne Fenech
- Department of Medicine, Division of Gastroenterology, Mater Dei Hospital, Malta
| | | | - Pierre Ellul
- Department of Medicine, Division of Gastroenterology, Mater Dei Hospital, Malta
| |
Collapse
|
2
|
Lumley S, Atkinson B, Dowall S, Pitman J, Staplehurst S, Busuttil J, Simpson A, Aarons E, Petridou C, Nijjar M, Glover S, Brooks T, Hewson R. Non-fatal case of Crimean-Congo haemorrhagic fever imported into the United Kingdom (ex Bulgaria), June 2014. ACTA ACUST UNITED AC 2014; 19. [PMID: 25108534 DOI: 10.2807/1560-7917.es2014.19.30.20864] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Abstract
Crimean-Congo haemorrhagic fever (CCHF) was diagnosed in a United Kingdom traveller who returned from Bulgaria in June 2014. The patient developed a moderately severe disease including fever, headaches and petechial rash. CCHF was diagnosed following identification of CCHF virus (CCHFV) RNA in a serum sample taken five days after symptom onset. Sequence analysis of the CCHFV genome showed that the virus clusters within the Europe 1 clade, which includes viruses from eastern Europe.
Collapse
Affiliation(s)
- S Lumley
- Research Department, Microbiology Services Division, Public Health England, Porton Down, Salisbury, United Kingdom
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
3
|
Fabret C, Quentin Y, Guiseppi A, Busuttil J, Haiech J, Denizot F. Analysis of errors in finished DNA sequences: the surfactin operon of Bacillus subtilis as an example. Microbiology (Reading) 1995; 141 ( Pt 2):345-50. [PMID: 7704264 DOI: 10.1099/13500872-141-2-345] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
Abstract
Increased productivity in DNA sequencing would not be valid without a straightforward detection and estimation of errors in finished sequences. The sequence of the surfactin operon from Bacillus subtilis was obtained by two different groups and by chance we were also working on the same chromosome region. Taking advantage of this situation we report in this paper, the number and nature of errors found in the overlapping part of the DNA sequences obtained by the three laboratories. The coincidence of some of the errors with compression in sequence ladders and with secondary DNA structures as well as the detection of frameshift errors using computer programs, are demonstrated. Finally we discuss the definition of a new sequencing strategy that might minimize both the error rate and the cost of sequencing.
Collapse
Affiliation(s)
- C Fabret
- Laboratoire de Chimie Bactérienne, Marseille, France
| | | | | | | | | | | |
Collapse
|