1
|
Gargari OK, Fatehi F, Mohammadi I, Firouzabadi SR, Shafiee A, Habibi G. Diagnostic accuracy of large language models in psychiatry. Asian J Psychiatr 2024; 100:104168. [PMID: 39111087 DOI: 10.1016/j.ajp.2024.104168] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/27/2024] [Revised: 07/20/2024] [Accepted: 07/22/2024] [Indexed: 09/13/2024]
Abstract
INTRODUCTION Medical decision-making is crucial for effective treatment, especially in psychiatry where diagnosis often relies on subjective patient reports and a lack of high-specificity symptoms. Artificial intelligence (AI), particularly Large Language Models (LLMs) like GPT, has emerged as a promising tool to enhance diagnostic accuracy in psychiatry. This comparative study explores the diagnostic capabilities of several AI models, including Aya, GPT-3.5, GPT-4, GPT-3.5 clinical assistant (CA), Nemotron, and Nemotron CA, using clinical cases from the DSM-5. METHODS We curated 20 clinical cases from the DSM-5 Clinical Cases book, covering a wide range of psychiatric diagnoses. Four advanced AI models (GPT-3.5 Turbo, GPT-4, Aya, Nemotron) were tested using prompts to elicit detailed diagnoses and reasoning. The models' performances were evaluated based on accuracy and quality of reasoning, with additional analysis using the Retrieval Augmented Generation (RAG) methodology for models accessing the DSM-5 text. RESULTS The AI models showed varied diagnostic accuracy, with GPT-3.5 and GPT-4 performing notably better than Aya and Nemotron in terms of both accuracy and reasoning quality. While models struggled with specific disorders such as cyclothymic and disruptive mood dysregulation disorders, others excelled, particularly in diagnosing psychotic and bipolar disorders. Statistical analysis highlighted significant differences in accuracy and reasoning, emphasizing the superiority of the GPT models. DISCUSSION The application of AI in psychiatry offers potential improvements in diagnostic accuracy. The superior performance of the GPT models can be attributed to their advanced natural language processing capabilities and extensive training on diverse text data, enabling more effective interpretation of psychiatric language. However, models like Aya and Nemotron showed limitations in reasoning, indicating a need for further refinement in their training and application. CONCLUSION AI holds significant promise for enhancing psychiatric diagnostics, with certain models demonstrating high potential in interpreting complex clinical descriptions accurately. Future research should focus on expanding the dataset and integrating multimodal data to further enhance the diagnostic capabilities of AI in psychiatry.
Collapse
Affiliation(s)
- Omid Kohandel Gargari
- Farzan Artificial Intelligence Team, Farzan Clinical Research Institute, Tehran, Islamic Republic of Iran
| | - Farhad Fatehi
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Brisbane, Australia; School of Psychological Sciences, Monash University, Melbourne, Australia
| | - Ida Mohammadi
- Farzan Artificial Intelligence Team, Farzan Clinical Research Institute, Tehran, Islamic Republic of Iran
| | - Shahryar Rajai Firouzabadi
- Farzan Artificial Intelligence Team, Farzan Clinical Research Institute, Tehran, Islamic Republic of Iran
| | - Arman Shafiee
- Farzan Artificial Intelligence Team, Farzan Clinical Research Institute, Tehran, Islamic Republic of Iran
| | - Gholamreza Habibi
- Farzan Artificial Intelligence Team, Farzan Clinical Research Institute, Tehran, Islamic Republic of Iran.
| |
Collapse
|
2
|
Salah M, Abdelfattah F, Al Halbusi H. The good, the bad, and the GPT: Reviewing the impact of generative artificial intelligence on psychology. Curr Opin Psychol 2024; 59:101872. [PMID: 39197407 DOI: 10.1016/j.copsyc.2024.101872] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2024] [Revised: 07/10/2024] [Accepted: 08/12/2024] [Indexed: 09/01/2024]
Abstract
This review explores the impact of Generative Artificial Intelligence (GenAI)-a technology capable of autonomously creating new content, ideas, or solutions by learning from extensive data-on psychology. GenAI is changing research methodologies, diagnostics, and treatments by enhancing diagnostic accuracy, personalizing therapeutic interventions, and providing deeper insights into cognitive processes. However, these advancements come with significant ethical concerns, including privacy, bias, and the risk of depersonalization in therapy. By focusing on the current capabilities of GenAI, this study aims to provide a balanced understanding and guide the ethical integration of AI into psychological practices and research. We argue that while GenAI presents profound opportunities, its integration must be approached cautiously using robust ethical frameworks.
Collapse
Affiliation(s)
- Mohammed Salah
- Management Department, College of Business Administration (COBA), A'Sharqiyah University (ASU), Ibra, Oman; Modern College of Business and Science (MCBS), Muscat, Oman.
| | | | | |
Collapse
|
3
|
Krysta K, Cullivan R, Brittlebank A, Dragasek J, Hermans M, Strkalj Ivezics S, van Veelen N, Casanova Dias M. Artificial Intelligence in Healthcare and Psychiatry. ACADEMIC PSYCHIATRY : THE JOURNAL OF THE AMERICAN ASSOCIATION OF DIRECTORS OF PSYCHIATRIC RESIDENCY TRAINING AND THE ASSOCIATION FOR ACADEMIC PSYCHIATRY 2024:10.1007/s40596-024-02036-z. [PMID: 39313674 DOI: 10.1007/s40596-024-02036-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Accepted: 08/18/2024] [Indexed: 09/25/2024]
Affiliation(s)
- Krzysztof Krysta
- Faculty of Medical Sciences in Katowice, Medical University of Silesia in Katowice, Katowice, Poland
| | - Rachael Cullivan
- Cavan/Monaghan Mental Health Services Ireland, Monaghan, Ireland
| | - Andrew Brittlebank
- Cumbria, Northumberland, Tyne and Wear NHS Foundation Trust, Cumbria, UK
| | - Jozef Dragasek
- Faculty of Medicine, University Hospital of Louis Pasteur and Pavol Jozef Safarik University, Trieda, Kosice, Slovak Republic
| | - Marc Hermans
- European Union of Medical Specialists, Brussels, Belgium
| | | | - Nicoletta van Veelen
- Brain Center, Psychiatry, Diagnostic and Early Psychosis, Universitair Medisch Centrum Utrecht, Utrecht, the Netherlands
| | | |
Collapse
|
4
|
Grosshans M, Paul T, Fischer SKM, Lotzmann N, List H, Haag C, Mutschler J. Conversation-based AI for anxiety disorders might lower the threshold for traditional medical assistance: a case report. Front Public Health 2024; 12:1399702. [PMID: 39371214 PMCID: PMC11449728 DOI: 10.3389/fpubh.2024.1399702] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2024] [Accepted: 09/11/2024] [Indexed: 10/08/2024] Open
Abstract
Artificial intelligence (AI) offers a wealth of opportunities for medicine, if we also bear in mind the risks associated with this technology. In recent years the potential future integration of AI with medicine has been the subject of much debate, although practical clinical experience of relevant cases is still largely absent. This case study examines a particular patient's experience with different forms of care. Initially, the patient communicated with the conversation (chat) based AI (CAI) for self-treatment. However, over time she found herself increasingly drawn to a low-threshold internal company support system that is grounded in an existing, more traditional human-based care structure. This pattern of treatment May represent a useful addition to existing care structures, particularly for patients receptive to technology.
Collapse
Affiliation(s)
- Martin Grosshans
- Department of Global Health, Safety and Well-being, SAP SE, Walldorf, Germany
| | - Torsten Paul
- Department of Global Health, Safety and Well-being, SAP SE, Walldorf, Germany
| | - Sebastian Karl Maximilian Fischer
- Psychiatric Services Lucerne, Lucerne, Switzerland
- Institute of General Practice and Family Medicine, University Hospital of the Ludwig-Maximilians University of Munich, Munich, Germany
| | - Natalie Lotzmann
- Department of Global Health, Safety and Well-being, SAP SE, Walldorf, Germany
| | - Hannah List
- Department of Global Health, Safety and Well-being, SAP SE, Walldorf, Germany
| | - Christina Haag
- Institute for Implementation Science in Health Care, University of Zurich, Zurich, Switzerland
- Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich, Switzerland
| | | |
Collapse
|
5
|
Shin D, Kim H, Lee S, Cho Y, Jung W. Using Large Language Models to Detect Depression From User-Generated Diary Text Data as a Novel Approach in Digital Mental Health Screening: Instrument Validation Study. J Med Internet Res 2024; 26:e54617. [PMID: 39292502 PMCID: PMC11447422 DOI: 10.2196/54617] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 05/17/2024] [Accepted: 08/11/2024] [Indexed: 09/19/2024] Open
Abstract
BACKGROUND Depressive disorders have substantial global implications, leading to various social consequences, including decreased occupational productivity and a high disability burden. Early detection and intervention for clinically significant depression have gained attention; however, the existing depression screening tools, such as the Center for Epidemiologic Studies Depression Scale, have limitations in objectivity and accuracy. Therefore, researchers are identifying objective indicators of depression, including image analysis, blood biomarkers, and ecological momentary assessments (EMAs). Among EMAs, user-generated text data, particularly from diary writing, have emerged as a clinically significant and analyzable source for detecting or diagnosing depression, leveraging advancements in large language models such as ChatGPT. OBJECTIVE We aimed to detect depression based on user-generated diary text through an emotional diary writing app using a large language model (LLM). We aimed to validate the value of the semistructured diary text data as an EMA data source. METHODS Participants were assessed for depression using the Patient Health Questionnaire and suicide risk was evaluated using the Beck Scale for Suicide Ideation before starting and after completing the 2-week diary writing period. The text data from the daily diaries were also used in the analysis. The performance of leading LLMs, such as ChatGPT with GPT-3.5 and GPT-4, was assessed with and without GPT-3.5 fine-tuning on the training data set. The model performance comparison involved the use of chain-of-thought and zero-shot prompting to analyze the text structure and content. RESULTS We used 428 diaries from 91 participants; GPT-3.5 fine-tuning demonstrated superior performance in depression detection, achieving an accuracy of 0.902 and a specificity of 0.955. However, the balanced accuracy was the highest (0.844) for GPT-3.5 without fine-tuning and prompt techniques; it displayed a recall of 0.929. CONCLUSIONS Both GPT-3.5 and GPT-4.0 demonstrated relatively reasonable performance in recognizing the risk of depression based on diaries. Our findings highlight the potential clinical usefulness of user-generated text data for detecting depression. In addition to measurable indicators, such as step count and physical activity, future research should increasingly emphasize qualitative digital expression.
Collapse
Affiliation(s)
- Daun Shin
- Department of Psychiatry, Anam Hospital, Korea University, Seoul, Republic of Korea
- Doctorpresso, Seoul, Republic of Korea
| | | | | | - Younhee Cho
- Doctorpresso, Seoul, Republic of Korea
- Department of Design, Seoul National University, Seoul, Republic of Korea
| | | |
Collapse
|
6
|
Pool J, Indulska M, Sadiq S. Large language models and generative AI in telehealth: a responsible use lens. J Am Med Inform Assoc 2024; 31:2125-2136. [PMID: 38441296 PMCID: PMC11339524 DOI: 10.1093/jamia/ocae035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Revised: 02/05/2024] [Accepted: 02/14/2024] [Indexed: 08/23/2024] Open
Abstract
OBJECTIVE This scoping review aims to assess the current research landscape of the application and use of large language models (LLMs) and generative Artificial Intelligence (AI), through tools such as ChatGPT in telehealth. Additionally, the review seeks to identify key areas for future research, with a particular focus on AI ethics considerations for responsible use and ensuring trustworthy AI. MATERIALS AND METHODS Following the scoping review methodological framework, a search strategy was conducted across 6 databases. To structure our review, we employed AI ethics guidelines and principles, constructing a concept matrix for investigating the responsible use of AI in telehealth. Using the concept matrix in our review enabled the identification of gaps in the literature and informed future research directions. RESULTS Twenty studies were included in the review. Among the included studies, 5 were empirical, and 15 were reviews and perspectives focusing on different telehealth applications and healthcare contexts. Benefit and reliability concepts were frequently discussed in these studies. Privacy, security, and accountability were peripheral themes, with transparency, explainability, human agency, and contestability lacking conceptual or empirical exploration. CONCLUSION The findings emphasized the potential of LLMs, especially ChatGPT, in telehealth. They provide insights into understanding the use of LLMs, enhancing telehealth services, and taking ethical considerations into account. By proposing three future research directions with a focus on responsible use, this review further contributes to the advancement of this emerging phenomenon of healthcare AI.
Collapse
Affiliation(s)
- Javad Pool
- ARC Industrial Transformation Training Centre for Information Resilience (CIRES), The University of Queensland, Brisbane 4072, Australia
- School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane 4072, Australia
| | - Marta Indulska
- ARC Industrial Transformation Training Centre for Information Resilience (CIRES), The University of Queensland, Brisbane 4072, Australia
- Business School, The University of Queensland, Brisbane 4072, Australia
| | - Shazia Sadiq
- ARC Industrial Transformation Training Centre for Information Resilience (CIRES), The University of Queensland, Brisbane 4072, Australia
- School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane 4072, Australia
| |
Collapse
|
7
|
Alliende LM, Sands BR, Mittal VA. Chatbots and Stigma in Schizophrenia: The Need for Transparency. Schizophr Bull 2024; 50:957-960. [PMID: 38917476 PMCID: PMC11348995 DOI: 10.1093/schbul/sbae105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/27/2024]
Affiliation(s)
| | - Beckett Ryden Sands
- Weinberg College, Department of Psychology, Northwestern University, Evanston, IL, USA
| | | |
Collapse
|
8
|
Sawamura S, Kohiyama K, Takenaka T, Sera T, Inoue T, Nagai T. Performance of ChatGPT 4.0 on Japan's National Physical Therapist Examination: A Comprehensive Analysis of Text and Visual Question Handling. Cureus 2024; 16:e67347. [PMID: 39310431 PMCID: PMC11413471 DOI: 10.7759/cureus.67347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/20/2024] [Indexed: 09/25/2024] Open
Abstract
INTRODUCTION ChatGPT 4.0, a large-scale language model (LLM) developed by OpenAI, has demonstrated the capability to pass Japan's national medical examination and other medical assessments. However, the impact of imaging-based questions and different question types on its performance has not been thoroughly examined. This study evaluated ChatGPT 4.0's performance on Japan's national examination for physical therapists, particularly its ability to handle complex questions involving images and tables. The study also assessed the model's potential in the field of rehabilitation and its performance with Japanese language inputs. METHODS The evaluation utilized 1,000 questions from the 54th to 58th national exams for physical therapists in Japan, comprising 160 general questions and 40 practical questions per exam. All questions were input in Japanese and included additional information such as images or tables. The answers generated by ChatGPT were then compared with the official correct answers. ANALYSIS ChatGPT's performance was evaluated based on accuracy rates using various criteria: general and practical questions were analyzed with Fisher's exact test, A-type (single correct answer) and X2-type (two correct answers) questions, text-only questions versus questions with images and tables, and different question lengths using Student's t-test. RESULTS ChatGPT 4.0 met the passing criteria with an overall accuracy of 73.4%. The accuracy rates for general and practical questions were 80.1% and 46.6%, respectively. No significant difference was found between the accuracy rates for A-type (74.3%) and X2-type (67.4%) questions. However, a significant difference was observed between the accuracy rates for text-only questions (80.5%) and questions with images and tables (35.4%). DISCUSSION The results indicate that ChatGPT 4.0 satisfies the passing criteria for the national exam and demonstrates adequate knowledge and application skills. However, its performance on practical questions and those with images and tables is lower, indicating areas for improvement. The effective handling of Japanese inputs suggests its potential use in non-English-speaking regions. CONCLUSION ChatGPT 4.0 can pass the national examination for physical therapists, particularly with text-based questions. However, improvements are needed for specialized practical questions and those involving images and tables. The model shows promise for supporting clinical rehabilitation and medical education in Japanese-speaking contexts, though further enhancements are required for a comprehensive application.
Collapse
Affiliation(s)
- Shogo Sawamura
- Department of Rehabilitation, Heisei College of Health Sciences, Gifu, JPN
| | - Kengo Kohiyama
- Department of Rehabilitation, Heisei College of Health Sciences, Gifu, JPN
| | - Takahiro Takenaka
- Department of Rehabilitation, Heisei College of Health Sciences, Gifu, JPN
| | - Tatsuya Sera
- Department of Rehabilitation, Heisei College of Health Sciences, Gifu, JPN
| | - Tadatoshi Inoue
- Department of Rehabilitation, Heisei College of Health Sciences, Gifu, JPN
| | - Takashi Nagai
- Department of Rehabilitation, Heisei College of Health Sciences, Gifu, JPN
| |
Collapse
|
9
|
Su Z, Tang G, Huang R, Qiao Y, Zhang Z, Dai X. Based on Medicine, The Now and Future of Large Language Models. Cell Mol Bioeng 2024; 17:263-277. [PMID: 39372551 PMCID: PMC11450117 DOI: 10.1007/s12195-024-00820-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2024] [Accepted: 09/08/2024] [Indexed: 10/08/2024] Open
Abstract
Objectives This review explores the potential applications of large language models (LLMs) such as ChatGPT, GPT-3.5, and GPT-4 in the medical field, aiming to encourage their prudent use, provide professional support, and develop accessible medical AI tools that adhere to healthcare standards. Methods This paper examines the impact of technologies such as OpenAI's Generative Pre-trained Transformers (GPT) series, including GPT-3.5 and GPT-4, and other large language models (LLMs) in medical education, scientific research, clinical practice, and nursing. Specifically, it includes supporting curriculum design, acting as personalized learning assistants, creating standardized simulated patient scenarios in education; assisting with writing papers, data analysis, and optimizing experimental designs in scientific research; aiding in medical imaging analysis, decision-making, patient education, and communication in clinical practice; and reducing repetitive tasks, promoting personalized care and self-care, providing psychological support, and enhancing management efficiency in nursing. Results LLMs, including ChatGPT, have demonstrated significant potential and effectiveness in the aforementioned areas, yet their deployment in healthcare settings is fraught with ethical complexities, potential lack of empathy, and risks of biased responses. Conclusion Despite these challenges, significant medical advancements can be expected through the proper use of LLMs and appropriate policy guidance. Future research should focus on overcoming these barriers to ensure the effective and ethical application of LLMs in the medical field.
Collapse
Affiliation(s)
- Ziqing Su
- Department of Neurosurgery, The First Affiliated Hospital of Anhui Medical University, 218 Jixi Road, Hefei, 230022 P.R. China
- Department of Clinical Medicine, The First Clinical College of Anhui Medical University, Hefei, 230022 P.R. China
| | - Guozhang Tang
- Department of Neurosurgery, The First Affiliated Hospital of Anhui Medical University, 218 Jixi Road, Hefei, 230022 P.R. China
- Department of Clinical Medicine, The Second Clinical College of Anhui Medical University, Hefei, 230032 Anhui P.R. China
| | - Rui Huang
- Department of Neurosurgery, The First Affiliated Hospital of Anhui Medical University, 218 Jixi Road, Hefei, 230022 P.R. China
- Department of Clinical Medicine, The First Clinical College of Anhui Medical University, Hefei, 230022 P.R. China
| | - Yang Qiao
- Department of Neurosurgery, The First Affiliated Hospital of Anhui Medical University, 218 Jixi Road, Hefei, 230022 P.R. China
| | - Zheng Zhang
- Department of Neurosurgery, The First Affiliated Hospital of Anhui Medical University, 218 Jixi Road, Hefei, 230022 P.R. China
- Department of Clinical Medicine, The First Clinical College of Anhui Medical University, Hefei, 230022 P.R. China
| | - Xingliang Dai
- Department of Neurosurgery, The First Affiliated Hospital of Anhui Medical University, 218 Jixi Road, Hefei, 230022 P.R. China
- Department of Research & Development, East China Institute of Digital Medical Engineering, Shangrao, 334000 P.R. China
| |
Collapse
|
10
|
Yang W. Beyond algorithms: The human touch machine-generated titles for enhancing click-through rates on social media. PLoS One 2024; 19:e0306639. [PMID: 38995930 PMCID: PMC11244827 DOI: 10.1371/journal.pone.0306639] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Accepted: 06/20/2024] [Indexed: 07/14/2024] Open
Abstract
Artificial intelligence (AI) has the potential to revolutionize various domains by automating language-driven tasks. This study evaluates the effectiveness of an AI-assisted methodology, called the "POP Title AI Five-Step Optimization Method," in optimizing content titles on the RED social media platform. By leveraging advancements in natural language generation, this methodology aims to enhance the impact of titles by incorporating emotional sophistication and cultural proficiency, addressing existing gaps in AI capabilities. The methodology entails training generative models using human-authored examples that align with the aspirations of the target audience. By incorporating popular keywords derived from user searches, the relevance and discoverability of titles are enhanced. Audience-centric filtering is subsequently employed to further refine the generated outputs. Furthermore, human oversight is introduced to provide essential intuition that AI systems alone may lack. A total of one thousand titles, generated by AI, underwent linguistic and engagement analyses. Qualitatively, 65% of the titles exhibited intrigue and conveyed meaning comparable to those generated by humans. However, attaining full emotional sophistication remained a challenge. Quantitatively, titles emphasizing curiosity and contrast demonstrated positive correlations with user interactions, thus validating the efficacy of these techniques. Consequently, the machine-generated titles achieved coherence on par with 65% of human-generated titles, signifying significant progress and potential for further refinement. Nevertheless, achieving socio-cultural awareness is vital to match human understanding across diverse contexts, thus presenting a critical avenue for future improvement in the methodology. Continuous advancements in AI can enhance adaptability and reduce subjectivity by promoting flexibility instead of relying solely on manual reviews. As AI gains a deeper understanding of humanity, opportunities for its application across various industries through experiential reasoning abilities emerge. This case study exemplifies the nurturing of AI's potential by refining its skills through an evolutionary process.
Collapse
Affiliation(s)
- Wenyu Yang
- Foki Media Co., Ltd. Hangzhou, Hangzhou, Zhejiang Province, China
| |
Collapse
|
11
|
Ferrario A, Sedlakova J, Trachsel M. The Role of Humanization and Robustness of Large Language Models in Conversational Artificial Intelligence for Individuals With Depression: A Critical Analysis. JMIR Ment Health 2024; 11:e56569. [PMID: 38958218 PMCID: PMC11231450 DOI: 10.2196/56569] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Revised: 04/27/2024] [Accepted: 04/27/2024] [Indexed: 07/04/2024] Open
Abstract
Unlabelled Large language model (LLM)-powered services are gaining popularity in various applications due to their exceptional performance in many tasks, such as sentiment analysis and answering questions. Recently, research has been exploring their potential use in digital health contexts, particularly in the mental health domain. However, implementing LLM-enhanced conversational artificial intelligence (CAI) presents significant ethical, technical, and clinical challenges. In this viewpoint paper, we discuss 2 challenges that affect the use of LLM-enhanced CAI for individuals with mental health issues, focusing on the use case of patients with depression: the tendency to humanize LLM-enhanced CAI and their lack of contextualized robustness. Our approach is interdisciplinary, relying on considerations from philosophy, psychology, and computer science. We argue that the humanization of LLM-enhanced CAI hinges on the reflection of what it means to simulate "human-like" features with LLMs and what role these systems should play in interactions with humans. Further, ensuring the contextualization of the robustness of LLMs requires considering the specificities of language production in individuals with depression, as well as its evolution over time. Finally, we provide a series of recommendations to foster the responsible design and deployment of LLM-enhanced CAI for the therapeutic support of individuals with depression.
Collapse
Affiliation(s)
- Andrea Ferrario
- Institute Biomedical Ethics and History of Medicine, University of Zurich, Zurich, Switzerland
- Mobiliar Lab for Analytics at ETH, ETH Zurich, Zurich, Switzerland
| | - Jana Sedlakova
- Institute Biomedical Ethics and History of Medicine, University of Zurich, Zurich, Switzerland
- Digital Society Initiative, University of Zurich, Zurich, Switzerland
- Institute for Implementation Science in Health Care, University of Zurich, Zurich, Switzerland
| | - Manuel Trachsel
- University of Basel, Basel, Switzerland
- University Hospital Basel, Basel, Switzerland
- University Psychiatric Clinics Basel, Basel, Switzerland
| |
Collapse
|
12
|
Omar M, Soffer S, Charney AW, Landi I, Nadkarni GN, Klang E. Applications of large language models in psychiatry: a systematic review. Front Psychiatry 2024; 15:1422807. [PMID: 38979501 PMCID: PMC11228775 DOI: 10.3389/fpsyt.2024.1422807] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/24/2024] [Accepted: 06/05/2024] [Indexed: 07/10/2024] Open
Abstract
Background With their unmatched ability to interpret and engage with human language and context, large language models (LLMs) hint at the potential to bridge AI and human cognitive processes. This review explores the current application of LLMs, such as ChatGPT, in the field of psychiatry. Methods We followed PRISMA guidelines and searched through PubMed, Embase, Web of Science, and Scopus, up until March 2024. Results From 771 retrieved articles, we included 16 that directly examine LLMs' use in psychiatry. LLMs, particularly ChatGPT and GPT-4, showed diverse applications in clinical reasoning, social media, and education within psychiatry. They can assist in diagnosing mental health issues, managing depression, evaluating suicide risk, and supporting education in the field. However, our review also points out their limitations, such as difficulties with complex cases and potential underestimation of suicide risks. Conclusion Early research in psychiatry reveals LLMs' versatile applications, from diagnostic support to educational roles. Given the rapid pace of advancement, future investigations are poised to explore the extent to which these models might redefine traditional roles in mental health care.
Collapse
Affiliation(s)
- Mahmud Omar
- Faculty of Medicine, Tel-Aviv University, Tel-Aviv, Israel
| | - Shelly Soffer
- Internal Medicine B, Assuta Medical Center, Ashdod, Israel
- Ben-Gurion University of the Negev, Be'er Sheva, Israel
| | | | - Isotta Landi
- Icahn School of Medicine at Mount Sinai, New York, NY, United States
| | - Girish N Nadkarni
- Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| | - Eyal Klang
- Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| |
Collapse
|
13
|
Maggio MG, Tartarisco G, Cardile D, Bonanno M, Bruschetta R, Pignolo L, Pioggia G, Calabrò RS, Cerasa A. Exploring ChatGPT's potential in the clinical stream of neurorehabilitation. Front Artif Intell 2024; 7:1407905. [PMID: 38903157 PMCID: PMC11187276 DOI: 10.3389/frai.2024.1407905] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2024] [Accepted: 05/13/2024] [Indexed: 06/22/2024] Open
Abstract
In several medical fields, generative AI tools such as ChatGPT have achieved optimal performance in identifying correct diagnoses only by evaluating narrative clinical descriptions of cases. The most active fields of application include oncology and COVID-19-related symptoms, with preliminary relevant results also in psychiatric and neurological domains. This scoping review aims to introduce the arrival of ChatGPT applications in neurorehabilitation practice, where such AI-driven solutions have the potential to revolutionize patient care and assistance. First, a comprehensive overview of ChatGPT, including its design, and potential applications in medicine is provided. Second, the remarkable natural language processing skills and limitations of these models are examined with a focus on their use in neurorehabilitation. In this context, we present two case scenarios to evaluate ChatGPT ability to resolve higher-order clinical reasoning. Overall, we provide support to the first evidence that generative AI can meaningfully integrate as a facilitator into neurorehabilitation practice, aiding physicians in defining increasingly efficacious diagnostic and personalized prognostic plans.
Collapse
Affiliation(s)
| | - Gennaro Tartarisco
- Institute for Biomedical Research and Innovation (IRIB), National Research Council of Italy (CNR), Messina, Italy
| | | | | | - Roberta Bruschetta
- Institute for Biomedical Research and Innovation (IRIB), National Research Council of Italy (CNR), Messina, Italy
| | | | - Giovanni Pioggia
- Institute for Biomedical Research and Innovation (IRIB), National Research Council of Italy (CNR), Messina, Italy
| | | | - Antonio Cerasa
- Institute for Biomedical Research and Innovation (IRIB), National Research Council of Italy (CNR), Messina, Italy
- S’Anna Institute, Crotone, Italy
- Pharmacotechnology Documentation and Transfer Unit, Preclinical and Translational Pharmacology, Department of Pharmacy, Health and Nutritional Sciences, University of Calabria, Rende, Italy
| |
Collapse
|
14
|
Monosov IE, Zimmermann J, Frank MJ, Mathis MW, Baker JT. Ethological computational psychiatry: Challenges and opportunities. Curr Opin Neurobiol 2024; 86:102881. [PMID: 38696972 PMCID: PMC11162904 DOI: 10.1016/j.conb.2024.102881] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 04/02/2024] [Accepted: 04/03/2024] [Indexed: 05/04/2024]
Abstract
Studying the intricacies of individual subjects' moods and cognitive processing over extended periods of time presents a formidable challenge in medicine. While much of systems neuroscience appropriately focuses on the link between neural circuit functions and well-constrained behaviors over short timescales (e.g., trials, hours), many mental health conditions involve complex interactions of mood and cognition that are non-stationary across behavioral contexts and evolve over extended timescales. Here, we discuss opportunities, challenges, and possible future directions in computational psychiatry to quantify non-stationary continuously monitored behaviors. We suggest that this exploratory effort may contribute to a more precision-based approach to treating mental disorders and facilitate a more robust reverse translation across animal species. We conclude with ethical considerations for any field that aims to bridge artificial intelligence and patient monitoring.
Collapse
Affiliation(s)
- Ilya E. Monosov
- Departments of Neuroscience, Biomedical Engineering, Electrical Engineering, and Neurosurgery, Washington University School of Medicine, St. Louis, MO, USA
| | - Jan Zimmermann
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - Michael J. Frank
- Carney Center for Computational Brain Science, Brown University, Providence, RI, USA
| | | | | |
Collapse
|
15
|
Li DJ, Kao YC, Tsai SJ, Bai YM, Yeh TC, Chu CS, Hsu CW, Cheng SW, Hsu TW, Liang CS, Su KP. Comparing the performance of ChatGPT GPT-4, Bard, and Llama-2 in the Taiwan Psychiatric Licensing Examination and in differential diagnosis with multi-center psychiatrists. Psychiatry Clin Neurosci 2024; 78:347-352. [PMID: 38404249 DOI: 10.1111/pcn.13656] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 12/08/2023] [Accepted: 02/05/2024] [Indexed: 02/27/2024]
Abstract
AIM Large language models (LLMs) have been suggested to play a role in medical education and medical practice. However, the potential of their application in the psychiatric domain has not been well-studied. METHOD In the first step, we compared the performance of ChatGPT GPT-4, Bard, and Llama-2 in the 2022 Taiwan Psychiatric Licensing Examination conducted in traditional Mandarin. In the second step, we compared the scores of these three LLMs with those of 24 experienced psychiatrists in 10 advanced clinical scenario questions designed for psychiatric differential diagnosis. RESULT Only GPT-4 passed the 2022 Taiwan Psychiatric Licensing Examination (scoring 69 and ≥ 60 being considered a passing grade), while Bard scored 36 and Llama-2 scored 25. GPT-4 outperformed Bard and Llama-2, especially in the areas of 'Pathophysiology & Epidemiology' (χ2 = 22.4, P < 0.001) and 'Psychopharmacology & Other therapies' (χ2 = 15.8, P < 0.001). In the differential diagnosis, the mean score of the 24 experienced psychiatrists (mean 6.1, standard deviation 1.9) was higher than that of GPT-4 (5), Bard (3), and Llama-2 (1). CONCLUSION Compared to Bard and Llama-2, GPT-4 demonstrated superior abilities in identifying psychiatric symptoms and making clinical judgments. Besides, GPT-4's ability for differential diagnosis closely approached that of the experienced psychiatrists. GPT-4 revealed a promising potential as a valuable tool in psychiatric practice among the three LLMs.
Collapse
Affiliation(s)
- Dian-Jeng Li
- Department of Addiction Science, Kaohsiung Municipal Kai-Syuan Psychiatric Hospital, Kaohsiung, Taiwan
- Department of Nursing, Meiho University, Pingtung, Taiwan
| | - Yu-Chen Kao
- Department of Psychiatry, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan
- Department of Psychiatry, Tri-Service General Hospital, Beitou branch, Taipei, Taiwan
| | - Shih-Jen Tsai
- Department of Psychiatry, Taipei Veterans General Hospital, Taipei, Taiwan
- Department of Psychiatry, College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan
| | - Ya-Mei Bai
- Department of Psychiatry, Taipei Veterans General Hospital, Taipei, Taiwan
- Department of Psychiatry, College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan
- Institute of Brain Science, National Yang Ming Chiao Tung University, Taipei, Taiwan
| | - Ta-Chuan Yeh
- Department of Psychiatry, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan
| | - Che-Sheng Chu
- Center for Geriatric and Gerontology, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan
- Non-invasive Neuromodulation Consortium for Mental Disorders, Society of Psychophysiology, Taipei, Taiwan
- Graduate Institute of Medicine, College of Medicine, Kaohsiung Medical University, Kaohsiung, Taiwan
- Department of Psychiatry, Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan
| | - Chih-Wei Hsu
- Department of Psychiatry, Kaohsiung Chang Gung Memorial Hospital, Kaohsiung, Taiwan
| | - Szu-Wei Cheng
- Department of General Medicine, Chi Mei Medical Center, Tainan, Taiwan
- Mind-Body Interface Laboratory (MBI-Lab) and Department of Psychiatry, China Medical University Hospital, Taichung, Taiwan
| | - Tien-Wei Hsu
- Department of Psychiatry, E-DA Dachang Hospital, I-Shou University, Kaohsiung, Taiwan
- Department of Psychiatry, E-DA Hospital, I-Shou University, Kaohsiung, Taiwan
| | - Chih-Sung Liang
- Department of Psychiatry, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan
- Department of Psychiatry, Tri-Service General Hospital, Beitou branch, Taipei, Taiwan
| | - Kuan-Pin Su
- Mind-Body Interface Laboratory (MBI-Lab) and Department of Psychiatry, China Medical University Hospital, Taichung, Taiwan
- College of Medicine, China Medical University, Taichung, Taiwan
- An-Nan Hospital, China Medical University, Tainan, Taiwan
| |
Collapse
|
16
|
Treder MS, Lee S, Tsvetanov KA. Introduction to Large Language Models (LLMs) for dementia care and research. FRONTIERS IN DEMENTIA 2024; 3:1385303. [PMID: 39081594 PMCID: PMC11285660 DOI: 10.3389/frdem.2024.1385303] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/12/2024] [Accepted: 04/23/2024] [Indexed: 08/02/2024]
Abstract
Introduction Dementia is a progressive neurodegenerative disorder that affects cognitive abilities including memory, reasoning, and communication skills, leading to gradual decline in daily activities and social engagement. In light of the recent advent of Large Language Models (LLMs) such as ChatGPT, this paper aims to thoroughly analyse their potential applications and usefulness in dementia care and research. Method To this end, we offer an introduction into LLMs, outlining the key features, capabilities, limitations, potential risks, and practical considerations for deployment as easy-to-use software (e.g., smartphone apps). We then explore various domains related to dementia, identifying opportunities for LLMs to enhance understanding, diagnostics, and treatment, with a broader emphasis on improving patient care. For each domain, the specific contributions of LLMs are examined, such as their ability to engage users in meaningful conversations, deliver personalized support, and offer cognitive enrichment. Potential benefits encompass improved social interaction, enhanced cognitive functioning, increased emotional well-being, and reduced caregiver burden. The deployment of LLMs in caregiving frameworks also raises a number of concerns and considerations. These include privacy and safety concerns, the need for empirical validation, user-centered design, adaptation to the user's unique needs, and the integration of multimodal inputs to create more immersive and personalized experiences. Additionally, ethical guidelines and privacy protocols must be established to ensure responsible and ethical deployment of LLMs. Results We report the results on a questionnaire filled in by people with dementia (PwD) and their supporters wherein we surveyed the usefulness of different application scenarios of LLMs as well as the features that LLM-powered apps should have. Both PwD and supporters were largely positive regarding the prospect of LLMs in care, although concerns were raised regarding bias, data privacy and transparency. Discussion Overall, this review corroborates the promising utilization of LLMs to positively impact dementia care by boosting cognitive abilities, enriching social interaction, and supporting caregivers. The findings underscore the importance of further research and development in this field to fully harness the benefits of LLMs and maximize their potential for improving the lives of individuals living with dementia.
Collapse
Affiliation(s)
- Matthias S. Treder
- School of Computer Science & Informatics, Cardiff University, Cardiff, United Kingdom
| | - Sojin Lee
- Olive AI Limited, London, United Kingdom
| | - Kamen A. Tsvetanov
- Department of Clinical Neurosciences, University of Cambridge, Cambridge, United Kingdom
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
| |
Collapse
|
17
|
Bartal A, Jagodnik KM, Chan SJ, Dekel S. AI and narrative embeddings detect PTSD following childbirth via birth stories. Sci Rep 2024; 14:8336. [PMID: 38605073 PMCID: PMC11009279 DOI: 10.1038/s41598-024-54242-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Accepted: 02/10/2024] [Indexed: 04/13/2024] Open
Abstract
Free-text analysis using machine learning (ML)-based natural language processing (NLP) shows promise for diagnosing psychiatric conditions. Chat Generative Pre-trained Transformer (ChatGPT) has demonstrated preliminary initial feasibility for this purpose; however, whether it can accurately assess mental illness remains to be determined. This study evaluates the effectiveness of ChatGPT and the text-embedding-ada-002 (ADA) model in detecting post-traumatic stress disorder following childbirth (CB-PTSD), a maternal postpartum mental illness affecting millions of women annually, with no standard screening protocol. Using a sample of 1295 women who gave birth in the last six months and were 18+ years old, recruited through hospital announcements, social media, and professional organizations, we explore ChatGPT's and ADA's potential to screen for CB-PTSD by analyzing maternal childbirth narratives. The PTSD Checklist for DSM-5 (PCL-5; cutoff 31) was used to assess CB-PTSD. By developing an ML model that utilizes numerical vector representation of the ADA model, we identify CB-PTSD via narrative classification. Our model outperformed (F1 score: 0.81) ChatGPT and six previously published large text-embedding models trained on mental health or clinical domains data, suggesting that the ADA model can be harnessed to identify CB-PTSD. Our modeling approach could be generalized to assess other mental health disorders.
Collapse
Affiliation(s)
- Alon Bartal
- The School of Business Administration, Bar-Ilan University, Ramat Gan, 5290002, Israel
| | - Kathleen M Jagodnik
- The School of Business Administration, Bar-Ilan University, Ramat Gan, 5290002, Israel
- Department of Psychiatry, Massachusetts General Hospital, Boston, MA, 02114, USA
- Department of Psychiatry, Harvard Medical School, Boston, MA, 02115, USA
| | - Sabrina J Chan
- Department of Psychiatry, Massachusetts General Hospital, Boston, MA, 02114, USA
| | - Sharon Dekel
- Department of Psychiatry, Massachusetts General Hospital, Boston, MA, 02114, USA.
- Department of Psychiatry, Harvard Medical School, Boston, MA, 02115, USA.
| |
Collapse
|
18
|
Sathyam PKR, Surapaneni KM. Assessing the performance of ChatGPT in psychiatry: A study using clinical cases from foreign medical graduate examination (FMGE). Indian J Psychiatry 2024; 66:408-410. [PMID: 38778847 PMCID: PMC11107915 DOI: 10.4103/indianjpsychiatry.indianjpsychiatry_919_23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Revised: 01/01/2024] [Accepted: 02/08/2024] [Indexed: 05/25/2024] Open
Affiliation(s)
- Praveen Kumar Ratavarapu Sathyam
- Department of Psychiatry, Panimalar Medical College Hospital & Research Institute, Varadharajapuram, Poonamallee, Chennai, Tamil Nadu, India
| | - Krishna Mohan Surapaneni
- Department of Biochemistry, Panimalar Medical College Hospital & Research Institute, Varadharajapuram, Poonamallee, Chennai, Tamil Nadu, India E-mail:
- Department of Medical Education, Panimalar Medical College Hospital & Research Institute, Varadharajapuram, Poonamallee, Chennai, Tamil Nadu, India
| |
Collapse
|
19
|
Cheng J. Applications of Large Language Models in Pathology. Bioengineering (Basel) 2024; 11:342. [PMID: 38671764 PMCID: PMC11047860 DOI: 10.3390/bioengineering11040342] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2024] [Revised: 03/27/2024] [Accepted: 03/29/2024] [Indexed: 04/28/2024] Open
Abstract
Large language models (LLMs) are transformer-based neural networks that can provide human-like responses to questions and instructions. LLMs can generate educational material, summarize text, extract structured data from free text, create reports, write programs, and potentially assist in case sign-out. LLMs combined with vision models can assist in interpreting histopathology images. LLMs have immense potential in transforming pathology practice and education, but these models are not infallible, so any artificial intelligence generated content must be verified with reputable sources. Caution must be exercised on how these models are integrated into clinical practice, as these models can produce hallucinations and incorrect results, and an over-reliance on artificial intelligence may lead to de-skilling and automation bias. This review paper provides a brief history of LLMs and highlights several use cases for LLMs in the field of pathology.
Collapse
Affiliation(s)
- Jerome Cheng
- Department of Pathology, University of Michigan, Ann Arbor, MI 48105, USA
| |
Collapse
|
20
|
Liu XQ, Zhang ZR. Potential use of large language models for mitigating students' problematic social media use: ChatGPT as an example. World J Psychiatry 2024; 14:334-341. [PMID: 38617990 PMCID: PMC11008388 DOI: 10.5498/wjp.v14.i3.334] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Revised: 01/15/2024] [Accepted: 02/05/2024] [Indexed: 03/19/2024] Open
Abstract
The problematic use of social media has numerous negative impacts on individuals' daily lives, interpersonal relationships, physical and mental health, and more. Currently, there are few methods and tools to alleviate problematic social media, and their potential is yet to be fully realized. Emerging large language models (LLMs) are becoming increasingly popular for providing information and assistance to people and are being applied in many aspects of life. In mitigating problematic social media use, LLMs such as ChatGPT can play a positive role by serving as conversational partners and outlets for users, providing personalized information and resources, monitoring and intervening in problematic social media use, and more. In this process, we should recognize both the enormous potential and endless possibilities of LLMs such as ChatGPT, leveraging their advantages to better address problematic social media use, while also acknowledging the limitations and potential pitfalls of ChatGPT technology, such as errors, limitations in issue resolution, privacy and security concerns, and potential overreliance. When we leverage the advantages of LLMs to address issues in social media usage, we must adopt a cautious and ethical approach, being vigilant of the potential adverse effects that LLMs may have in addressing problematic social media use to better harness technology to serve individuals and society.
Collapse
Affiliation(s)
- Xin-Qiao Liu
- School of Education, Tianjin University, Tianjin 300350, China
| | - Zi-Ru Zhang
- School of Education, Tianjin University, Tianjin 300350, China
| |
Collapse
|
21
|
Dimitriadis F, Alkagiet S, Tsigkriki L, Kleitsioti P, Sidiropoulos G, Efstratiou D, Askalidi T, Tsaousidis A, Siarkos M, Giannakopoulou P, Mavrogianni AD, Zarifis J, Koulaouzidis G. ChatGPT and Patients With Heart Failure. Angiology 2024:33197241238403. [PMID: 38451243 DOI: 10.1177/00033197241238403] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/08/2024]
Abstract
ChatGPT (Generative Pre-trained Transformer) is a large-scale language processing model, with possibilities for professional patient support in a patient-friendly way. The aim of the study was to examine the accuracy and reproducibility of ChatGPT in answering questions about knowledge and management of heart failure (HF). First, we recorded 47 most frequently asked questions by patients about HF. The answers of ChatGPT to these questions were independently assessed by two researchers. ChatGPT was able to render the definition of the disease in a very simple and explanatory way. It listed a number of the most important causes of HF and the most important risk factors for its occurrence. It provided correct answers about the most important diagnostic tests and why they are recommended. In addition, it answered health and dietary questions, such as the daily fluid and the alcohol intake. ChatGPT listed the most important classes of drugs in HF and their mechanism of action. It also answered with arguments to questions about patient's sex life, whether they could work, drive, or travel by plane. The performance of ChatGPT was described as very good as it was able to adequately answer all questions posed to it.
Collapse
Affiliation(s)
- Fotis Dimitriadis
- Cardiology Department, General Hospital G. Papanikolaou, Thessaloniki, Greece
| | - Stelina Alkagiet
- Cardiology Department, General Hospital G. Papanikolaou, Thessaloniki, Greece
| | - Lamprini Tsigkriki
- Cardiology Department, General Hospital G. Papanikolaou, Thessaloniki, Greece
| | | | - George Sidiropoulos
- Cardiology Department, General Hospital G. Papanikolaou, Thessaloniki, Greece
| | - Dimitris Efstratiou
- Cardiology Department, General Hospital G. Papanikolaou, Thessaloniki, Greece
| | - Taisa Askalidi
- Cardiology Department, General Hospital G. Papanikolaou, Thessaloniki, Greece
| | - Adam Tsaousidis
- Cardiology Department, General Hospital G. Papanikolaou, Thessaloniki, Greece
| | - Michail Siarkos
- Cardiology Department, General Hospital G. Papanikolaou, Thessaloniki, Greece
| | | | | | - John Zarifis
- Cardiology Department, General Hospital G. Papanikolaou, Thessaloniki, Greece
| | - George Koulaouzidis
- Department of Biochemical Sciences, Pomeranian Medical University, Szczecin, Poland
| |
Collapse
|
22
|
Bartal A, Jagodnik KM, Chan SJ, Dekel S. OpenAI's Narrative Embeddings Can Be Used for Detecting Post-Traumatic Stress Following Childbirth Via Birth Stories. RESEARCH SQUARE 2024:rs.3.rs-3428787. [PMID: 37886525 PMCID: PMC10602164 DOI: 10.21203/rs.3.rs-3428787/v2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 04/30/2024]
Abstract
Free-text analysis using Machine Learning (ML)-based Natural Language Processing (NLP) shows promise for diagnosing psychiatric conditions. Chat Generative Pre-trained Transformer (ChatGPT) has demonstrated preliminary initial feasibility for this purpose; however, whether it can accurately assess mental illness remains to be determined. This study evaluates the effectiveness of ChatGPT and the text-embedding-ada-002 (ADA) model in detecting post-traumatic stress disorder following childbirth (CB-PTSD), a maternal postpartum mental illness affecting millions of women annually, with no standard screening protocol. Using a sample of 1,295 women who gave birth in the last six months and were 18+ years old, recruited through hospital announcements, social media, and professional organizations, we explore ChatGPT's and ADA's potential to screen for CB-PTSD by analyzing maternal childbirth narratives. The PTSD Checklist for DSM-5 (PCL-5; cutoff 31) was used to assess CB-PTSD. By developing an ML model that utilizes numerical vector representation of the ADA model, we identify CB-PTSD via narrative classification. Our model outperformed (F1 score: 0.82) ChatGPT and six previously published large language models (LLMs) trained on mental health or clinical domains data, suggesting that the ADA model can be harnessed to identify CB-PTSD. Our modeling approach could be generalized to assess other mental health disorders.
Collapse
Affiliation(s)
- Alon Bartal
- The School of Business Administration, Bar-Ilan University, Max and Anna Web, Ramat Gan, 5290002, Israel
| | - Kathleen M. Jagodnik
- The School of Business Administration, Bar-Ilan University, Max and Anna Web, Ramat Gan, 5290002, Israel
- Department of Psychiatry, Massachusetts General Hospital, 55 Fruit St., Boston, 02114, Massachusetts, USA
- Department of Psychiatry, Harvard Medical School, 25 Shattuck St., Boston, 02115, Massachusetts, USA
| | - Sabrina J. Chan
- Department of Psychiatry, Massachusetts General Hospital, 55 Fruit St., Boston, 02114, Massachusetts, USA
| | - Sharon Dekel
- Department of Psychiatry, Massachusetts General Hospital, 55 Fruit St., Boston, 02114, Massachusetts, USA
- Department of Psychiatry, Harvard Medical School, 25 Shattuck St., Boston, 02115, Massachusetts, USA
| |
Collapse
|
23
|
Kalam KT, Rahman JM, Islam MR, Dewan SMR. ChatGPT and mental health: Friends or foes? Health Sci Rep 2024; 7:e1912. [PMID: 38361805 PMCID: PMC10867692 DOI: 10.1002/hsr2.1912] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Revised: 11/30/2023] [Accepted: 01/31/2024] [Indexed: 02/17/2024] Open
Abstract
Background ChatGPT is an artificial intelligence (AI) language model that has gained popularity as a virtual assistant because of its exceptional capacity to solve problems and make decisions. However, there are some ways in which technological misuse and incorrect interpretations can have potentially hazardous consequences for a user's mental health. Discussion Because it lacks real-time fact-checking capabilities, ChatGPT may create misleading or erroneous information. Considering AI technology has the potential to influence a person's thinking, we anticipate ChatGPT's future repercussions on mental health by considering instances in which inappropriate usage may lead to mental disorders. While several studies have demonstrated how the AI model may transform mental health care and therapy, certain drawbacks, including bias and privacy violations, have also been identified. Conclusion Educating people and organizing workshops on AI technology usage, strengthening privacy measures, and updating ethical standards are crucial initiatives to prevent misuse and resultant dire impacts on mental health. Longitudinal research on the potential of these platforms to impact a variety of mental health problems is recommended in the future.
Collapse
Affiliation(s)
| | - Jannatul Mabia Rahman
- Department of Electrical and Electronic EngineeringUniversity of Asia PacificDhakaBangladesh
| | | | | |
Collapse
|
24
|
Liu I, Liu F, Xiao Y, Huang Y, Wu S, Ni S. Investigating the Key Success Factors of Chatbot-Based Positive Psychology Intervention with Retrieval- and Generative Pre-Trained Transformer (GPT)-Based Chatbots. INTERNATIONAL JOURNAL OF HUMAN–COMPUTER INTERACTION 2024:1-12. [DOI: 10.1080/10447318.2023.2300015] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/30/2023] [Accepted: 12/21/2023] [Indexed: 10/06/2024]
Affiliation(s)
- Ivan Liu
- Department of Psychology, Faculty of Arts and Science, Beijing Normal University at Zhuhai, China
- Faculty of Psychology, Beijing Normal University, China
| | - Fangyuan Liu
- Department of Psychology, Faculty of Arts and Science, Beijing Normal University at Zhuhai, China
| | - Yuting Xiao
- Faculty of Psychology, Beijing Normal University, China
| | - Yajia Huang
- Faculty of Psychology, Beijing Normal University, China
| | - Shuming Wu
- Faculty of Psychology, Beijing Normal University, China
| | - Shiguang Ni
- Shenzhen International Graduate School, Tsinghua University, China
| |
Collapse
|
25
|
Bekbolatova M, Mayer J, Ong CW, Toma M. Transformative Potential of AI in Healthcare: Definitions, Applications, and Navigating the Ethical Landscape and Public Perspectives. Healthcare (Basel) 2024; 12:125. [PMID: 38255014 PMCID: PMC10815906 DOI: 10.3390/healthcare12020125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Revised: 12/27/2023] [Accepted: 01/02/2024] [Indexed: 01/24/2024] Open
Abstract
Artificial intelligence (AI) has emerged as a crucial tool in healthcare with the primary aim of improving patient outcomes and optimizing healthcare delivery. By harnessing machine learning algorithms, natural language processing, and computer vision, AI enables the analysis of complex medical data. The integration of AI into healthcare systems aims to support clinicians, personalize patient care, and enhance population health, all while addressing the challenges posed by rising costs and limited resources. As a subdivision of computer science, AI focuses on the development of advanced algorithms capable of performing complex tasks that were once reliant on human intelligence. The ultimate goal is to achieve human-level performance with improved efficiency and accuracy in problem-solving and task execution, thereby reducing the need for human intervention. Various industries, including engineering, media/entertainment, finance, and education, have already reaped significant benefits by incorporating AI systems into their operations. Notably, the healthcare sector has witnessed rapid growth in the utilization of AI technology. Nevertheless, there remains untapped potential for AI to truly revolutionize the industry. It is important to note that despite concerns about job displacement, AI in healthcare should not be viewed as a threat to human workers. Instead, AI systems are designed to augment and support healthcare professionals, freeing up their time to focus on more complex and critical tasks. By automating routine and repetitive tasks, AI can alleviate the burden on healthcare professionals, allowing them to dedicate more attention to patient care and meaningful interactions. However, legal and ethical challenges must be addressed when embracing AI technology in medicine, alongside comprehensive public education to ensure widespread acceptance.
Collapse
Affiliation(s)
- Molly Bekbolatova
- Department of Osteopathic Manipulative Medicine, College of Osteopathic Medicine, New York Institute of Technology, Old Westbury, NY 11568, USA; (M.B.); (J.M.)
| | - Jonathan Mayer
- Department of Osteopathic Manipulative Medicine, College of Osteopathic Medicine, New York Institute of Technology, Old Westbury, NY 11568, USA; (M.B.); (J.M.)
| | - Chi Wei Ong
- School of Chemistry, Chemical Engineering, and Biotechnology, Nanyang Technological University, 62 Nanyang Drive, Singapore 637459, Singapore
| | - Milan Toma
- Department of Osteopathic Manipulative Medicine, College of Osteopathic Medicine, New York Institute of Technology, Old Westbury, NY 11568, USA; (M.B.); (J.M.)
| |
Collapse
|
26
|
Cho CH, Lee HJ, Kim YK. The New Emerging Treatment Choice for Major Depressive Disorders: Digital Therapeutics. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2024; 1456:307-331. [PMID: 39261436 DOI: 10.1007/978-981-97-4402-2_16] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/13/2024]
Abstract
The chapter provides an in-depth analysis of digital therapeutics (DTx) as a revolutionary approach to managing major depressive disorder (MDD). It discusses the evolution and definition of DTx, their application across various medical fields, regulatory considerations, and their benefits and limitations. This chapter extensively covers DTx for MDD, including smartphone applications, virtual reality interventions, cognitive-behavioral therapy (CBT) platforms, artificial intelligence (AI) and chatbot therapies, biofeedback, wearable technologies, and serious games. It evaluates the effectiveness of these digital interventions, comparing them with traditional treatments and examining patient perspectives, compliance, and engagement. The integration of DTx into clinical practice is also explored, along with the challenges and barriers to their adoption, such as technological limitations, data privacy concerns, ethical considerations, reimbursement issues, and the need for improved digital literacy. This chapter concludes by looking at the future direction of DTx in mental healthcare, emphasizing the need for personalized treatment plans, integration with emerging modalities, and the expansion of access to these innovative solutions globally.
Collapse
Affiliation(s)
- Chul-Hyun Cho
- Department of Psychiatry, Korea University College of Medicine, Seoul, Republic of Korea
| | - Heon-Jeong Lee
- Department of Psychiatry, Korea University College of Medicine, Seoul, Republic of Korea
| | - Yong-Ku Kim
- Department of Psychiatry, Korea University College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
27
|
Di H, Wen Y. Generalist medical artificial intelligence: Embracing the future with flexible interactions. Psychiatry Clin Neurosci 2023; 77:625-626. [PMID: 37671749 DOI: 10.1111/pcn.13595] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/26/2023] [Revised: 08/31/2023] [Accepted: 09/01/2023] [Indexed: 09/07/2023]
Affiliation(s)
- Huajie Di
- Department of Pediatrics, Xuzhou Medical University, Xuzhou, China
- Evidence-Based Medicine Research Center, Xuzhou Medical University, Xuzhou, China
- Department of Pediatric Urology, The Affiliated Xuzhou Children's Hospital of Xuzhou Medical University, Xuzhou, China
| | - Yi Wen
- Department of Pediatrics, Xuzhou Medical University, Xuzhou, China
- Department of Pediatric Urology, The Affiliated Xuzhou Children's Hospital of Xuzhou Medical University, Xuzhou, China
| |
Collapse
|
28
|
Randhawa J, Khan A. A Conversation With ChatGPT About the Usage of Lithium in Pregnancy for Bipolar Disorder. Cureus 2023; 15:e46548. [PMID: 37933339 PMCID: PMC10625495 DOI: 10.7759/cureus.46548] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/05/2023] [Indexed: 11/08/2023] Open
Abstract
This conversation with ChatGPT explores the use of lithium in pregnancy for bipolar disorder, a topic of significant importance in psychiatry. Bipolar disorder is characterized by extreme mood swings, and its prevalence varies globally. ChatGPT provides valuable information on bipolar disorder, its prevalence, age of onset, and gender differences. It also discusses the use of lithium during pregnancy, emphasizing the need for individualized decisions, close monitoring, and potential risks and benefits. However, it is essential to note that ChatGPT's responses lack specific references, raising concerns about the reliability of the information provided. Further research is needed to quantify the correctness and dependability of ChatGPT-generated answers in the healthcare context.
Collapse
Affiliation(s)
- Jaismeen Randhawa
- Psychiatry, Sri Guru Ram Das Institute of Medical Sciences and Research, Amritsar, IND
| | - Aadil Khan
- Trauma Surgery, OSF Saint Francis Medical Center, University of Illinois Chicago, Peoria, USA
- Cardiology, University of Illinois Chicago, Illinois, USA
- Internal Medicine, Lala Lajpat Rai Hospital, Kanpur, IND
| |
Collapse
|