Özcan F, Örücü Atar M, Köroğlu Ö, Yılmaz B. Assessment of the reliability and usability of ChatGPT in response to spinal cord injury questions.
J Spinal Cord Med 2024:1-6. [PMID:
38860862 DOI:
10.1080/10790268.2024.2361551]
[Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/12/2024] Open
Abstract
OBJECTIVE
The use of artificial intelligence chatbots to obtain information about patients' diseases is increasing. This study aimed to determine the reliability and usability of ChatGPT for spinal cord injury-related questions.
METHODS
Three raters simultaneously evaluated a total of 47 questions on a 7-point Likert scale for reliability and usability, based on the three most frequently searched keywords in Google Trends ('general information', 'complications' and 'treatment').
RESULTS
Inter-rater Cronbach α scores indicated substantial agreement for both reliability and usability scores (α between 0.558 and 0.839, and α between 0.373 and 0.772, respectively). The highest mean reliability score was for 'complications' (mean 5.38). The lowest average was for the 'general information' section (mean 4.20). The 'treatment' had the highest mean scores for the usability (mean 5.87) and the lowest mean value was recorded in the 'general information' section (mean 4.80).
CONCLUSION
The answers given by ChatGPT to questions related to spinal cord injury were reliable and useful. Nevertheless, it should be kept in mind that ChatGPT may provide incorrect or incomplete information, especially in the 'general information' section, which may mislead patients and their relatives.
Collapse