1
|
Moylan K, Doherty K. Expert and Interdisciplinary Analysis of AI-Driven Chatbots for Mental Health Support: Mixed Methods Study. J Med Internet Res 2025; 27:e67114. [PMID: 40279575 PMCID: PMC12064976 DOI: 10.2196/67114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2024] [Revised: 12/20/2024] [Accepted: 02/25/2025] [Indexed: 04/27/2025] Open
Abstract
BACKGROUND Recent years have seen an immense surge in the creation and use of chatbots as social and mental health companions. Aiming to provide empathic responses in support of the delivery of personalized support, these tools are often presented as offering immense potential. However, it is also essential that we understand the risks of their deployment, including their potential adverse impacts on the mental health of users, including those most at risk. OBJECTIVE The study aims to assess the ethical and pragmatic clinical implications of using chatbots that claim to aid mental health. While several studies within human-computer interaction and related fields have examined users' perceptions of such systems, few studies have engaged mental health professionals in critical analysis of their conduct as mental health support tools. This paper comprises, in turn, an effort to assess the ethical and pragmatic clinical implications of using chatbots that claim to aid mental health. METHODS This study included 8 interdisciplinary mental health professional participants (from psychology and psychotherapy to social care and crisis volunteer workers) in a mixed methods and hands-on analysis of 2 popular mental health-related chatbots' data handling, interface design, and responses. This analysis was carried out through profession-specific tasks with each chatbot, eliciting participants' perceptions through both the Trust in Automation scale and semistructured interviews. Through thematic analysis and a 2-tailed, paired t test, these chatbots' implications for mental health support were thus evaluated. RESULTS Qualitative analysis revealed emphatic initial impressions among mental health professionals of chatbot responses likely to produce harm, exhibiting a generic mode of care, and risking user dependence and manipulation given the central role of trust in the therapeutic relationship. Trust scores from the Trust in Automation scale, while exhibiting no statistically significant differences between the chatbots (t6=-0.76; P=.48), indicated medium to low trust scores for each chatbot. The findings of this work highlight that the design and development of artificial intelligence (AI)-driven mental health-related solutions must be undertaken with utmost caution. The mental health professionals in this study collectively resist these chatbots and make clear that AI-driven chatbots used for mental health by at-risk users invite several potential and specific harms. CONCLUSIONS Through this work, we contributed insights into the mental health professional perspective on the design of chatbots used for mental health and underscore the necessity of ongoing critical assessment and iterative refinement to maximize the benefits and minimize the risks associated with integrating AI into mental health support.
Collapse
Affiliation(s)
- Kayley Moylan
- School of Information and Communication Studies, University College Dublin, Dublin, Ireland
| | - Kevin Doherty
- School of Information and Communication Studies, University College Dublin, Dublin, Ireland
| |
Collapse
|
2
|
Chen TH, Chu G, Pan RH, Ma WF. Effectiveness of mental health chatbots in depression and anxiety for adolescents and young adults: a meta-analysis of randomized controlled trials. Expert Rev Med Devices 2025; 22:233-241. [PMID: 39935147 DOI: 10.1080/17434440.2025.2466742] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2024] [Accepted: 01/25/2025] [Indexed: 02/13/2025]
Abstract
BACKGROUND The mental health chatbot is dedicated to providing assistance to individuals grappling with the complexities of depression and anxiety. OBJECTIVE The study aimed to evaluate the effectiveness of the mental health chatbot in alleviating symptoms of depression and anxiety among adolescents and young adults. METHODS A systematic review framework was employed with a protocol pre-registered on Prospero (CRD42023418877). Databases were systematically searched, including PubMed, ACM Digital Library, Embase, Cochrane and IEEE. Data synthesis was conducted narratively, and meta-analysis was performed by pooling data from the original studies. RESULTS Ten randomized controlled trials focused on an acute population, mainly females and university students. Chatbots designed for daily conversations and mood monitoring, using cognitive behavioral therapy techniques, showed efficacy in treating depression (95% CI = -1.09 to -0.23; p = .003). However, it is essential to highlight that these interventions utilizing chatbots for mental health were not found to be efficacious in managing symptoms of anxiety (95% CI = -0.56 to 0.4; p = .74). CONCLUSIONS Evidence supports the effectiveness of mental health chatbots in treating depression, but further exploration and refinement are needed to optimize their efficacy in managing anxiety.
Collapse
Affiliation(s)
- Tzu Han Chen
- PhD Program for Health Science and Industry, China Medical University, Taichung, Taiwan
| | - Ginger Chu
- School of Nursing and Midwifery, College of Health, Medicine and Wellbeing, The University of Newcastle, New South Wales, Australia
- College of Health, Medicine and Wellbeing, The University of Newcastle, New South Wales, Australia
| | - Ren-Hao Pan
- Founder, La Vida Tec. Co. Ltd., Taichung, Taichung, Taiwan (R.O.C.)
- Institute of Hospital and Health Care Administration, National Yang Ming Chiao Tung University, Taipei, Taiwan (R.O.C.)
- Department of Information Management, Tunghai University, Taichung, Taiwan (R.O.C.)
| | - Wei-Fen Ma
- School of Nursing, China Medical University, Taichung, Taiwan
- Department of Nursing, China Medical University Hospital, Taichung, Taiwan
| |
Collapse
|
3
|
Hu M, Chua XCW, Diong SF, Kasturiratna KTAS, Majeed NM, Hartanto A. AI as your ally: The effects of AI-assisted venting on negative affect and perceived social support. Appl Psychol Health Well Being 2025; 17:e12621. [PMID: 39496509 DOI: 10.1111/aphw.12621] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2024] [Accepted: 10/20/2024] [Indexed: 11/06/2024]
Abstract
In recent years, artificial intelligence (AI) chatbots have made significant strides in generating human-like conversations. With AI's expanding capabilities in mimicking human interactions, its affordability and accessibility underscore the potential of AI chatbots to facilitate negative emotional disclosure or venting. The study's primary objective is to highlight the potential benefits of AI-assisted venting by comparing its effectiveness to venting through a traditional journaling platform in reducing negative affect and increasing perceived social support. We conducted a pre-registered within-subject experiment involving 150 participants who completed both traditional venting and AI-assisted venting conditions with counterbalancing and a wash-out period of 1-week between the conditions. Results from the frequentist and Bayesian dependent samples t-test revealed that AI-assisted venting effectively reduced high and medium arousal negative affect such as anger, frustration and fear. However, participants in the AI-assisted venting condition did not experience a significant increase in perceived social support and perceived loneliness, suggesting that participants did not perceive the effective assistance from AI as social support. This study demonstrates the promising role of AI in improving individuals' emotional well-being, serving as a catalyst for a broader discussion on the evolving role of AI and its potential psychological implications.
Collapse
Affiliation(s)
- Meilan Hu
- School of Social Sciences, Singapore Management University, Singapore
| | | | - Shu Fen Diong
- School of Social Sciences, Singapore Management University, Singapore
| | | | - Nadyanna M Majeed
- Department of Psychology, National University of Singapore, Singapore
| | - Andree Hartanto
- School of Social Sciences, Singapore Management University, Singapore
| |
Collapse
|
4
|
Cruz-Gonzalez P, He AWJ, Lam EP, Ng IMC, Li MW, Hou R, Chan JNM, Sahni Y, Vinas Guasch N, Miller T, Lau BWM, Sánchez Vidaña DI. Artificial intelligence in mental health care: a systematic review of diagnosis, monitoring, and intervention applications. Psychol Med 2025; 55:e18. [PMID: 39911020 PMCID: PMC12017374 DOI: 10.1017/s0033291724003295] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Revised: 10/26/2024] [Accepted: 11/26/2024] [Indexed: 02/07/2025]
Abstract
Artificial intelligence (AI) has been recently applied to different mental health illnesses and healthcare domains. This systematic review presents the application of AI in mental health in the domains of diagnosis, monitoring, and intervention. A database search (CCTR, CINAHL, PsycINFO, PubMed, and Scopus) was conducted from inception to February 2024, and a total of 85 relevant studies were included according to preestablished inclusion criteria. The AI methods most frequently used were support vector machine and random forest for diagnosis, machine learning for monitoring, and AI chatbot for intervention. AI tools appeared to be accurate in detecting, classifying, and predicting the risk of mental health conditions as well as predicting treatment response and monitoring the ongoing prognosis of mental health disorders. Future directions should focus on developing more diverse and robust datasets and on enhancing the transparency and interpretability of AI models to improve clinical practice.
Collapse
Affiliation(s)
- Pablo Cruz-Gonzalez
- Rehabilitation Research Institute of Singapore, Nanyang Technological University, Singapore, Singapore
| | - Aaron Wan-Jia He
- School of Public Health, LKS Faculty of Medicine, The University of Hong Kong, Hong Kong, Hong Kong
| | - Elly PoPo Lam
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Ingrid Man Ching Ng
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Mandy Wingman Li
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Rangchun Hou
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Jackie Ngai-Man Chan
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Yuvraj Sahni
- Department of Building Environment and Energy Engineering, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Nestor Vinas Guasch
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Tiev Miller
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Benson Wui-Man Lau
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
- Mental Health Research Center, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Dalinda Isabel Sánchez Vidaña
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
- Mental Health Research Center, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| |
Collapse
|
5
|
Luo X, Zhang A, Li Y, Zhang Z, Ying F, Lin R, Yang Q, Wang J, Huang G. Emergence of Artificial Intelligence Art Therapies (AIATs) in Mental Health Care: A Systematic Review. Int J Ment Health Nurs 2024; 33:1743-1760. [PMID: 39020473 DOI: 10.1111/inm.13384] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/19/2024] [Revised: 05/26/2024] [Accepted: 06/11/2024] [Indexed: 07/19/2024]
Abstract
The application of artificial intelligence art therapies (AIATs) in mental health care represents an innovative merger between digital technology and the therapeutic potential of creative arts. This systematic review aimed to assess the effectiveness and ethical considerations of AIATs, incorporating robots, AI painting and AI Chatbots to augment traditional art therapies. Aligning with the Preferred Reporting Items for systematic reviews (PRISMA) guidelines, we meticulously searched PubMed, Cochrane Library, Web of Science and CNKI, resulting in 15 selected articles for detailed analysis. To ensure methodological quality, we applied the Joanna Briggs Institute (JBI) criteria for quality assessment and extracted data using the PICO(S) format, specifically targeting randomised controlled trials (RCTs). Our findings suggest that AIATs can profoundly enhance the therapeutic experience by providing new creative outlets and reinforcing existing methods, despite possible drawbacks and ethical challenges. This examination underscores AIATs' potential to enrich mental health therapies, emphasising the critical importance of ethical considerations and the responsible application of AI as the field evolves. With a focus on expanding treatment efficacy and patient expressiveness, the promise of AIATs in mental health care necessitates a careful balance between innovation and ethical responsibility. Trial Registration: PROSPERO: CRD42024504472.
Collapse
Affiliation(s)
- Xuexing Luo
- Faculty of Humanities and Arts, Macau University of Science and Technology, Taipa, Macau
| | - Aijia Zhang
- Faculty of Humanities and Arts, Macau University of Science and Technology, Taipa, Macau
| | - Yu Li
- State Key Laboratory of Quality Research in Chinese Medicines, Macau University of Science and Technology, Taipa, Macau
- Faculty of Chinese Medicine, Macau University of Science and Technology, Taipa, Macau
| | - Zheyu Zhang
- Faculty of Humanities and Arts, Macau University of Science and Technology, Taipa, Macau
| | - Fangtian Ying
- Faculty of Humanities and Arts, Macau University of Science and Technology, Taipa, Macau
- Zhejiang University, Hangzhou, China
| | - Runqing Lin
- Faculty of Humanities and Arts, Macau University of Science and Technology, Taipa, Macau
| | - Qianxu Yang
- Department of Social and Preventive Medicine, Faculty of Medicine, Centre for Epidemiology and Evidence-Based Practice, University of Malaya, Kuala Lumpur, Malaysia
| | - Jue Wang
- State Key Laboratory of Quality Research in Chinese Medicines, Macau University of Science and Technology, Taipa, Macau
- Faculty of Chinese Medicine, Macau University of Science and Technology, Taipa, Macau
- Guangdong-Hong Kong-Macao Joint Laboratory for Contaminants Exposure and Health, Guangzhou, China
| | - Guanghui Huang
- Faculty of Humanities and Arts, Macau University of Science and Technology, Taipa, Macau
- Zhuhai M.U.S.T. Science and Technology Research Institute, Macau University of Science and Technology, Taipa, Macau
| |
Collapse
|
6
|
Kolding S, Lundin RM, Hansen L, Østergaard SD. Use of generative artificial intelligence (AI) in psychiatry and mental health care: a systematic review. Acta Neuropsychiatr 2024; 37:e37. [PMID: 39523628 DOI: 10.1017/neu.2024.50] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2024]
Abstract
OBJECTIVES Tools based on generative artificial intelligence (AI) such as ChatGPT have the potential to transform modern society, including the field of medicine. Due to the prominent role of language in psychiatry, e.g., for diagnostic assessment and psychotherapy, these tools may be particularly useful within this medical field. Therefore, the aim of this study was to systematically review the literature on generative AI applications in psychiatry and mental health. METHODS We conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The search was conducted across three databases, and the resulting articles were screened independently by two researchers. The content, themes, and findings of the articles were qualitatively assessed. RESULTS The search and screening process resulted in the inclusion of 40 studies. The median year of publication was 2023. The themes covered in the articles were mainly mental health and well-being in general - with less emphasis on specific mental disorders (substance use disorder being the most prevalent). The majority of studies were conducted as prompt experiments, with the remaining studies comprising surveys, pilot studies, and case reports. Most studies focused on models that generate language, ChatGPT in particular. CONCLUSIONS Generative AI in psychiatry and mental health is a nascent but quickly expanding field. The literature mainly focuses on applications of ChatGPT, and finds that generative AI performs well, but notes that it is limited by significant safety and ethical concerns. Future research should strive to enhance transparency of methods, use experimental designs, ensure clinical relevance, and involve users/patients in the design phase.
Collapse
Affiliation(s)
- Sara Kolding
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
- Department of Affective Disorders, Aarhus University Hospital - Psychiatry, Aarhus, Denmark
- Center for Humanities Computing, Aarhus University, Aarhus, Denmark
| | - Robert M Lundin
- Deakin University, Institute for Mental and Physical Health and Clinical Translation (IMPACT), Geelong, VIC, Australia
- Mildura Base Public Hospital, Mental Health Services, Alcohol and Other Drugs Integrated Treatment Team, Mildura, VIC, Australia
- Barwon Health, Change to Improve Mental Health (CHIME), Mental Health Drugs and Alcohol Services, Geelong, VIC, Australia
| | - Lasse Hansen
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
- Department of Affective Disorders, Aarhus University Hospital - Psychiatry, Aarhus, Denmark
- Center for Humanities Computing, Aarhus University, Aarhus, Denmark
| | - Søren Dinesen Østergaard
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
- Department of Affective Disorders, Aarhus University Hospital - Psychiatry, Aarhus, Denmark
| |
Collapse
|
7
|
Chiu YH, Lee YF, Lin HL, Cheng LC. Exploring the Role of Mobile Apps for Insomnia in Depression: Systematic Review. J Med Internet Res 2024; 26:e51110. [PMID: 39423009 PMCID: PMC11530740 DOI: 10.2196/51110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 01/01/2024] [Accepted: 09/22/2024] [Indexed: 10/19/2024] Open
Abstract
BACKGROUND The COVID-19 pandemic has profoundly affected mental health, leading to an increased prevalence of depression and insomnia. Currently, artificial intelligence (AI) and deep learning have thoroughly transformed health care-related mobile apps, offered more effective mental health support, and alleviated the psychological stress that may have emerged during the pandemic. Early reviews outlined the use of mobile apps for dealing with depression and insomnia separately. However, there is now an urgent need for a systematic evaluation of mobile apps that address both depression and insomnia to reveal new applications and research gaps. OBJECTIVE This study aims to systematically review and evaluate mobile apps targeting depression and insomnia, highlighting their features, effectiveness, and gaps in the current research. METHODS We systematically searched PubMed, Scopus, and Web of Science for peer-reviewed journal articles published between 2017 and 2023. The inclusion criteria were studies that (1) focused on mobile apps addressing both depression and insomnia, (2) involved young people or adult participants, and (3) provided data on treatment efficacy. Data extraction was independently conducted by 2 reviewers. Title and abstract screening, as well as full-text screening, were completed in duplicate. Data were extracted by a single reviewer and verified by a second reviewer, and risk of bias assessments were completed accordingly. RESULTS Of the initial 383 studies we found, 365 were excluded after title, abstract screening, and removal of duplicates. Eventually, 18 full-text articles met our criteria and underwent full-text screening. The analysis revealed that mobile apps related to depression and insomnia were primarily utilized for early detection, assessment, and screening (n=5 studies); counseling and psychological support (n=3 studies); and cognitive behavioral therapy (CBT; n=10 studies). Among the 10 studies related to depression, our findings showed that chatbots demonstrated significant advantages in improving depression symptoms, a promising development in the field. Additionally, 2 studies evaluated the effectiveness of mobile apps as alternative interventions for depression and sleep, further expanding the potential applications of this technology. CONCLUSIONS The integration of AI and deep learning into mobile apps, particularly chatbots, is a promising avenue for personalized mental health support. Through innovative features, such as early detection, assessment, counseling, and CBT, these apps significantly contribute toward improving sleep quality and addressing depression. The reviewed chatbots leveraged advanced technologies, including natural language processing, machine learning, and generative dialog, to provide intelligent and autonomous interactions. Compared with traditional face-to-face therapies, their feasibility, acceptability, and potential efficacy highlight their user-friendly, cost-effective, and accessible nature with the aim of enhancing sleep and mental health outcomes.
Collapse
Affiliation(s)
- Yi-Hang Chiu
- Department of Psychiatry, Wan Fang Hospital, Taipei Medical University, Taipei City, Taiwan
- Psychiatric Research Center, Wan Fang Hospital, Taipei Medical University, Taipei City, Taiwan
| | - Yen-Fen Lee
- Department of Information and Finance Management, National Taipei University of Technology, Taipei City, Taiwan
| | - Huang-Li Lin
- Department of Psychiatry, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan
- College of Medicine, Chang Gung University, Taoyuan, Taiwan
| | - Li-Chen Cheng
- Department of Information and Finance Management, National Taipei University of Technology, Taipei City, Taiwan
| |
Collapse
|
8
|
Das KP, Gavade P. A review on the efficacy of artificial intelligence for managing anxiety disorders. Front Artif Intell 2024; 7:1435895. [PMID: 39479229 PMCID: PMC11523650 DOI: 10.3389/frai.2024.1435895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2024] [Accepted: 09/16/2024] [Indexed: 11/02/2024] Open
Abstract
Anxiety disorders are psychiatric conditions characterized by prolonged and generalized anxiety experienced by individuals in response to various events or situations. At present, anxiety disorders are regarded as the most widespread psychiatric disorders globally. Medication and different types of psychotherapies are employed as the primary therapeutic modalities in clinical practice for the treatment of anxiety disorders. However, combining these two approaches is known to yield more significant benefits than medication alone. Nevertheless, there is a lack of resources and a limited availability of psychotherapy options in underdeveloped areas. Psychotherapy methods encompass relaxation techniques, controlled breathing exercises, visualization exercises, controlled exposure exercises, and cognitive interventions such as challenging negative thoughts. These methods are vital in the treatment of anxiety disorders, but executing them proficiently can be demanding. Moreover, individuals with distinct anxiety disorders are prescribed medications that may cause withdrawal symptoms in some instances. Additionally, there is inadequate availability of face-to-face psychotherapy and a restricted capacity to predict and monitor the health, behavioral, and environmental aspects of individuals with anxiety disorders during the initial phases. In recent years, there has been notable progress in developing and utilizing artificial intelligence (AI) based applications and environments to improve the precision and sensitivity of diagnosing and treating various categories of anxiety disorders. As a result, this study aims to establish the efficacy of AI-enabled environments in addressing the existing challenges in managing anxiety disorders, reducing reliance on medication, and investigating the potential advantages, issues, and opportunities of integrating AI-assisted healthcare for anxiety disorders and enabling personalized therapy.
Collapse
Affiliation(s)
- K. P. Das
- Department of Computer Science, Christ University, Bengaluru, India
| | - P. Gavade
- Independent Practitioner, San Francisco, CA, United States
| |
Collapse
|
9
|
Gallegos C, Kausler R, Alderden J, Davis M, Wang L. Can Artificial Intelligence Chatbots Improve Mental Health?: A Scoping Review. Comput Inform Nurs 2024; 42:731-736. [PMID: 38934788 DOI: 10.1097/cin.0000000000001155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/28/2024]
Abstract
BACKGROUND AND OBJECTIVES Mental health disorders, including anxiety and depression, are the leading causes of global health-related burden and have increased dramatically since the 1990s. Delivering mental healthcare using artificial intelligence chatbots may be one option for closing the gaps in mental healthcare access. The overall aim of this scoping review was to describe the use, efficacy, and advantages/disadvantages of using an artificial intelligence chatbot for mental healthcare (stress, anxiety, depression). METHODS PubMed, PsycINFO, CINAHL, and Web of Science databases were searched. When possible, Medical Subject Headings terms were searched in combination with keywords. Two independent reviewers reviewed a total of 5768 abstracts. RESULTS Fifty-four articles were chosen for further review, with 10 articles included in the final analysis. Regarding quality assessment, the overall quality of the evidence was lower than expected. Overall, most studies showed positive trends in improving anxiety, stress, and depression. DISCUSSION Overall, using an artificial intelligence chatbot for mental health has some promising effects. However, many studies were done using rudimentary versions of artificial intelligence chatbots. In addition, lack of guardrails and privacy issues were identified. More research is needed to determine the effectiveness of artificial intelligence chatbots and to describe undesirable effects.
Collapse
Affiliation(s)
- Cara Gallegos
- Author Affiliations: School of Nursing (Drs Gallegos, Kausler, and Alderden; and Ms. Wang) and Albertson's Library (Mrs. Davis), Boise State University, ID
| | | | | | | | | |
Collapse
|
10
|
Villarreal-Zegarra D, Reategui-Rivera CM, García-Serna J, Quispe-Callo G, Lázaro-Cruz G, Centeno-Terrazas G, Galvez-Arevalo R, Escobar-Agreda S, Dominguez-Rodriguez A, Finkelstein J. Self-Administered Interventions Based on Natural Language Processing Models for Reducing Depressive and Anxiety Symptoms: Systematic Review and Meta-Analysis. JMIR Ment Health 2024; 11:e59560. [PMID: 39167795 PMCID: PMC11375382 DOI: 10.2196/59560] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Revised: 06/12/2024] [Accepted: 07/02/2024] [Indexed: 08/23/2024] Open
Abstract
BACKGROUND The introduction of natural language processing (NLP) technologies has significantly enhanced the potential of self-administered interventions for treating anxiety and depression by improving human-computer interactions. Although these advances, particularly in complex models such as generative artificial intelligence (AI), are highly promising, robust evidence validating the effectiveness of the interventions remains sparse. OBJECTIVE The aim of this study was to determine whether self-administered interventions based on NLP models can reduce depressive and anxiety symptoms. METHODS We conducted a systematic review and meta-analysis. We searched Web of Science, Scopus, MEDLINE, PsycINFO, IEEE Xplore, Embase, and Cochrane Library from inception to November 3, 2023. We included studies with participants of any age diagnosed with depression or anxiety through professional consultation or validated psychometric instruments. Interventions had to be self-administered and based on NLP models, with passive or active comparators. Outcomes measured included depressive and anxiety symptom scores. We included randomized controlled trials and quasi-experimental studies but excluded narrative, systematic, and scoping reviews. Data extraction was performed independently by pairs of authors using a predefined form. Meta-analysis was conducted using standardized mean differences (SMDs) and random effects models to account for heterogeneity. RESULTS In all, 21 articles were selected for review, of which 76% (16/21) were included in the meta-analysis for each outcome. Most of the studies (16/21, 76%) were recent (2020-2023), with interventions being mostly AI-based NLP models (11/21, 52%); most (19/21, 90%) delivered some form of therapy (primarily cognitive behavioral therapy: 16/19, 84%). The overall meta-analysis showed that self-administered interventions based on NLP models were significantly more effective in reducing both depressive (SMD 0.819, 95% CI 0.389-1.250; P<.001) and anxiety (SMD 0.272, 95% CI 0.116-0.428; P=.001) symptoms compared to various control conditions. Subgroup analysis indicated that AI-based NLP models were effective in reducing depressive symptoms (SMD 0.821, 95% CI 0.207-1.436; P<.001) compared to pooled control conditions. Rule-based NLP models showed effectiveness in reducing both depressive (SMD 0.854, 95% CI 0.172-1.537; P=.01) and anxiety (SMD 0.347, 95% CI 0.116-0.578; P=.003) symptoms. The meta-regression showed no significant association between participants' mean age and treatment outcomes (all P>.05). Although the findings were positive, the overall certainty of evidence was very low, mainly due to a high risk of bias, heterogeneity, and potential publication bias. CONCLUSIONS Our findings support the effectiveness of self-administered NLP-based interventions in alleviating depressive and anxiety symptoms, highlighting their potential to increase accessibility to, and reduce costs in, mental health care. Although the results were encouraging, the certainty of evidence was low, underscoring the need for further high-quality randomized controlled trials and studies examining implementation and usability. These interventions could become valuable components of public health strategies to address mental health issues. TRIAL REGISTRATION PROSPERO International Prospective Register of Systematic Reviews CRD42023472120; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42023472120.
Collapse
Affiliation(s)
- David Villarreal-Zegarra
- Instituto Peruano de Orientación Psicológica, Lima, Peru
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT, United States
| | - C Mahony Reategui-Rivera
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT, United States
| | | | | | | | | | | | | | | | - Joseph Finkelstein
- Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, UT, United States
| |
Collapse
|
11
|
Laymouna M, Ma Y, Lessard D, Schuster T, Engler K, Lebouché B. Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review. J Med Internet Res 2024; 26:e56930. [PMID: 39042446 PMCID: PMC11303905 DOI: 10.2196/56930] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 04/07/2024] [Accepted: 04/12/2024] [Indexed: 07/24/2024] Open
Abstract
BACKGROUND Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements in artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various health care needs. However, no comprehensive synthesis of health care chatbots' roles, users, benefits, and limitations is available to inform future research and application in the field. OBJECTIVE This review aims to describe health care chatbots' characteristics, focusing on their diverse roles in the health care pathway, user groups, benefits, and limitations. METHODS A rapid review of published literature from 2017 to 2023 was performed with a search strategy developed in collaboration with a health sciences librarian and implemented in the MEDLINE and Embase databases. Primary research studies reporting on chatbot roles or benefits in health care were included. Two reviewers dual-screened the search results. Extracted data on chatbot roles, users, benefits, and limitations were subjected to content analysis. RESULTS The review categorized chatbot roles into 2 themes: delivery of remote health services, including patient support, care management, education, skills building, and health behavior promotion, and provision of administrative assistance to health care providers. User groups spanned across patients with chronic conditions as well as patients with cancer; individuals focused on lifestyle improvements; and various demographic groups such as women, families, and older adults. Professionals and students in health care also emerged as significant users, alongside groups seeking mental health support, behavioral change, and educational enhancement. The benefits of health care chatbots were also classified into 2 themes: improvement of health care quality and efficiency and cost-effectiveness in health care delivery. The identified limitations encompassed ethical challenges, medicolegal and safety concerns, technical difficulties, user experience issues, and societal and economic impacts. CONCLUSIONS Health care chatbots offer a wide spectrum of applications, potentially impacting various aspects of health care. While they are promising tools for improving health care efficiency and quality, their integration into the health care system must be approached with consideration of their limitations to ensure optimal, safe, and equitable use.
Collapse
Affiliation(s)
- Moustafa Laymouna
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
| | - Yuanchao Ma
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
- Department of Biomedical Engineering, Polytechnique Montréal, Montreal, QC, Canada
| | - David Lessard
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
| | - Tibor Schuster
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
| | - Kim Engler
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
| | - Bertrand Lebouché
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
| |
Collapse
|
12
|
Zhong W, Luo J, Zhang H. The therapeutic effectiveness of artificial intelligence-based chatbots in alleviation of depressive and anxiety symptoms in short-course treatments: A systematic review and meta-analysis. J Affect Disord 2024; 356:459-469. [PMID: 38631422 DOI: 10.1016/j.jad.2024.04.057] [Citation(s) in RCA: 19] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/07/2023] [Revised: 04/10/2024] [Accepted: 04/14/2024] [Indexed: 04/19/2024]
Abstract
BACKGROUND The emergence of artificial intelligence-based chatbot has revolutionized the field of clinical psychology and psychotherapy, granting individuals unprecedented access to professional assistance, overcoming time constraints and geographical limitations with cost-effective convenience. However, despite its potential, there has been a noticeable gap in the literature regarding their effectiveness in addressing common mental health issues like depression and anxiety. This meta-analysis aims to evaluate the efficacy of AI-based chatbots in treating these conditions. METHODS A systematic search was executed across multiple databases, including PubMed, Cochrane Library, Web of Science, PsycINFO, and Embase on April 4th, 2024. The effect size of treatment efficacy was calculated using the standardized mean difference (Hedge's g). Quality assessment measures were implemented to ensure trial's quality. RESULTS In our analysis of 18 randomized controlled trials involving 3477 participants, we observed noteworthy improvements in depression (g = -0.26, 95 % CI = -0.34, -0.17) and anxiety (g = -0.19, 95 % CI = -0.29, -0.09) symptoms. The most significant benefits were evident after 8 weeks of treatment. However, at the three-month follow-up, no substantial effects were detected for either condition. LIMITATIONS Several limitations should be considered. These include the lack of diversity in the study populations, variations in chatbot design, and the use of different psychotherapeutic approaches. These factors may limit the generalizability of our findings. CONCLUSION This meta-analysis highlights the promising role of AI-based chatbot interventions in alleviating depressive and anxiety symptoms among adults. Our results indicate that these interventions can yield substantial improvements over a relatively brief treatment period.
Collapse
Affiliation(s)
- Wenjun Zhong
- Center for Studies of Education and Psychology of Ethnic Minorities in Southwest China, Southwest University, Chongqing, China
| | - Jianghua Luo
- Center for Studies of Education and Psychology of Ethnic Minorities in Southwest China, Southwest University, Chongqing, China.
| | - Hong Zhang
- Center for Psychological Health Education, Xinjiang University of Finance & Economics, Urumqi, Xinjiang, China
| |
Collapse
|
13
|
Li H, Zhang R, Lee YC, Kraut RE, Mohr DC. Systematic review and meta-analysis of AI-based conversational agents for promoting mental health and well-being. NPJ Digit Med 2023; 6:236. [PMID: 38114588 PMCID: PMC10730549 DOI: 10.1038/s41746-023-00979-5] [Citation(s) in RCA: 57] [Impact Index Per Article: 28.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2023] [Accepted: 11/29/2023] [Indexed: 12/21/2023] Open
Abstract
Conversational artificial intelligence (AI), particularly AI-based conversational agents (CAs), is gaining traction in mental health care. Despite their growing usage, there is a scarcity of comprehensive evaluations of their impact on mental health and well-being. This systematic review and meta-analysis aims to fill this gap by synthesizing evidence on the effectiveness of AI-based CAs in improving mental health and factors influencing their effectiveness and user experience. Twelve databases were searched for experimental studies of AI-based CAs' effects on mental illnesses and psychological well-being published before May 26, 2023. Out of 7834 records, 35 eligible studies were identified for systematic review, out of which 15 randomized controlled trials were included for meta-analysis. The meta-analysis revealed that AI-based CAs significantly reduce symptoms of depression (Hedge's g 0.64 [95% CI 0.17-1.12]) and distress (Hedge's g 0.7 [95% CI 0.18-1.22]). These effects were more pronounced in CAs that are multimodal, generative AI-based, integrated with mobile/instant messaging apps, and targeting clinical/subclinical and elderly populations. However, CA-based interventions showed no significant improvement in overall psychological well-being (Hedge's g 0.32 [95% CI -0.13 to 0.78]). User experience with AI-based CAs was largely shaped by the quality of human-AI therapeutic relationships, content engagement, and effective communication. These findings underscore the potential of AI-based CAs in addressing mental health issues. Future research should investigate the underlying mechanisms of their effectiveness, assess long-term effects across various mental health outcomes, and evaluate the safe integration of large language models (LLMs) in mental health care.
Collapse
Affiliation(s)
- Han Li
- Department of Communications and New Media, National University of Singapore, Singapore, 117416, Singapore
| | - Renwen Zhang
- Department of Communications and New Media, National University of Singapore, Singapore, 117416, Singapore.
| | - Yi-Chieh Lee
- Department of Computer Science, National University of Singapore, Singapore, 117416, Singapore
| | - Robert E Kraut
- Human-Computer Interaction Institute Carnegie Mellon University, Pittsburgh, PA, 15213, USA
| | - David C Mohr
- Center for Behavioral Intervention Technologies, Department of Preventive Medicine, Northwestern University, Chicago, IL, 60611, USA
| |
Collapse
|
14
|
Chakraborty C, Pal S, Bhattacharya M, Dash S, Lee SS. Overview of Chatbots with special emphasis on artificial intelligence-enabled ChatGPT in medical science. Front Artif Intell 2023; 6:1237704. [PMID: 38028668 PMCID: PMC10644239 DOI: 10.3389/frai.2023.1237704] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Accepted: 10/05/2023] [Indexed: 12/01/2023] Open
Abstract
The release of ChatGPT has initiated new thinking about AI-based Chatbot and its application and has drawn huge public attention worldwide. Researchers and doctors have started thinking about the promise and application of AI-related large language models in medicine during the past few months. Here, the comprehensive review highlighted the overview of Chatbot and ChatGPT and their current role in medicine. Firstly, the general idea of Chatbots, their evolution, architecture, and medical use are discussed. Secondly, ChatGPT is discussed with special emphasis of its application in medicine, architecture and training methods, medical diagnosis and treatment, research ethical issues, and a comparison of ChatGPT with other NLP models are illustrated. The article also discussed the limitations and prospects of ChatGPT. In the future, these large language models and ChatGPT will have immense promise in healthcare. However, more research is needed in this direction.
Collapse
Affiliation(s)
- Chiranjib Chakraborty
- Department of Biotechnology, School of Life Science and Biotechnology, Adamas University, Kolkata, West Bengal, India
| | - Soumen Pal
- School of Mechanical Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India
| | | | - Snehasish Dash
- School of Mechanical Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India
| | - Sang-Soo Lee
- Institute for Skeletal Aging and Orthopedic Surgery, Hallym University Chuncheon Sacred Heart Hospital, Chuncheon-si, Gangwon-do, Republic of Korea
| |
Collapse
|
15
|
Xu Y, Yang H, Jin Z, Xiang J, Xu H, Pokay YH, Mao H, Cai X, Wu Y, Wang DB. Application of a Digital Mental Health Clinic in Secondary Schools: Functionality and Effectiveness Evaluation. JMIR Form Res 2023; 7:e46494. [PMID: 37883144 PMCID: PMC10636614 DOI: 10.2196/46494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 09/25/2023] [Accepted: 09/27/2023] [Indexed: 10/27/2023] Open
Abstract
BACKGROUND Adolescents experience relatively more stress than other populations as they are facing rapid physical changes and adapting to complex social environments. However, access for this population to professional service providers is limited. Therefore, there is an increasing need for access to mental health services and new mental health care resources tailored to adolescents. OBJECTIVE The aim of this study was to evaluate the functionality and effectiveness of a school digital mental health clinic (DMHC) created by a Chinese psychiatric hospital and provided to secondary school students for a trial. METHODS The trial period of the DMHC was from January to July 2021 at three secondary schools in Taizhou City, China. Under a collaborative agreement between the local educational bureau and provider, use of the DMHC was free to all students, teachers, and staff of the schools. The functionality of the DMHC was compared with existing digital health interventions introduced in the literature and its effectiveness was quantitatively analyzed in terms of the volume of received counseling calls, number of calls per 100 students, length and time of calls, and reasons for the calls. The mini course video views were analyzed by topics and viewing time. RESULTS The design functions of the DMHC are well aligned with required factors defined in the literature. The first advantage of this DMHC is its high accessibility to students in the three schools. All functions of the DMHC are free to use by students, thereby eliminating the economic barriers to seeking and receiving care. Students can receive virtual counseling during or after regular working hours. Acceptability of the DHMC was further ensured by the full support from a national top-tier mental health facility. Any audio or video call from a student user would connect them to a live, qualified professional (ie, a psychiatrist or psychologist). Options are provided to view and listen to resources for stress relief or tips to help address mental health needs. The major reasons for the counseling calls included difficulties in learning, interpersonal relationships, and emotional distress. The three topics with the highest level of interest for the mini course videos were emotional assistance, personal growth, and family member relationships. The DMHC served as an effective tool for crisis prevention and intervention during nonworking hours as most of the live calls and mini video viewing occurred after school or over the weekend. Furthermore, the DMHC helped three students at high risk for suicide and self-injury through live-call intervention. CONCLUSIONS The DMHC is an effective complementary solution to improve access to professional mental health care facilities, especially during nonworking hours, thereby helping adolescents meet their mental health needs. Extension of the DMHC into more schools and other settings is recommended.
Collapse
Affiliation(s)
- Yi Xu
- Zhejiang Jerinte Health Technology Co, Ltd., Wenzhou, China
- Zhejiang Provincial Clinical Research Center for Mental Disorders, The Affiliated Kangning Hospital, Wenzhou Medical University, Wenzhou, China
| | - Hongshen Yang
- Zhejiang Provincial Clinical Research Center for Mental Disorders, The Affiliated Kangning Hospital, Wenzhou Medical University, Wenzhou, China
| | - Zhou Jin
- Key Laboratory of Alzheimer's Disease of Zhejiang Province, School of Mental Health and The Affiliated Kangning Hospital, Wenzhou Medical University, Wenzhou, China
| | - Jian Xiang
- Zhejiang Jerinte Health Technology Co, Ltd., Wenzhou, China
- Zhejiang Provincial Clinical Research Center for Mental Disorders, The Affiliated Kangning Hospital, Wenzhou Medical University, Wenzhou, China
| | - Haiyun Xu
- Zhejiang Jerinte Health Technology Co, Ltd., Wenzhou, China
- Zhejiang Provincial Clinical Research Center for Mental Disorders, The Affiliated Kangning Hospital, Wenzhou Medical University, Wenzhou, China
| | | | - Haibo Mao
- Zhejiang Jerinte Health Technology Co, Ltd., Wenzhou, China
- Zhejiang Provincial Clinical Research Center for Mental Disorders, The Affiliated Kangning Hospital, Wenzhou Medical University, Wenzhou, China
| | - Xugong Cai
- Zhejiang Jerinte Health Technology Co, Ltd., Wenzhou, China
- Zhejiang Provincial Clinical Research Center for Mental Disorders, The Affiliated Kangning Hospital, Wenzhou Medical University, Wenzhou, China
| | - Yili Wu
- Zhejiang Provincial Clinical Research Center for Mental Disorders, The Affiliated Kangning Hospital, Wenzhou Medical University, Wenzhou, China
- Key Laboratory of Alzheimer's Disease of Zhejiang Province, School of Mental Health and The Affiliated Kangning Hospital, Wenzhou Medical University, Wenzhou, China
| | - Deborah Baofeng Wang
- Zhejiang Provincial Clinical Research Center for Mental Disorders, The Affiliated Kangning Hospital, Wenzhou Medical University, Wenzhou, China
| |
Collapse
|
16
|
Chou YH, Lin C, Lee SH, Chang Chien YW, Cheng LC. Potential Mobile Health Applications for Improving the Mental Health of the Elderly: A Systematic Review. Clin Interv Aging 2023; 18:1523-1534. [PMID: 37727447 PMCID: PMC10506600 DOI: 10.2147/cia.s410396] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Accepted: 09/05/2023] [Indexed: 09/21/2023] Open
Abstract
The rapid aging of the global population presents challenges in providing mental health care resources for older adults aged 65 and above. The COVID-19 pandemic has further exacerbated the global population's psychological distress due to social isolation and distancing. Thus, there is an urgent need to update scholarly knowledge on the effectiveness of mHealth applications to improve older people's mental health. This systematic review summarizes recent literature on chatbots aimed at enhancing mental health and well-being. Sixteen papers describing six apps or prototypes were reviewed, indicating the practicality, feasibility, and acceptance of chatbots for promoting mental health in older adults. Engaging with chatbots led to improvements in well-being and stress reduction, as well as a decrement in depressive symptoms. Mobile health applications addressing these studies are categorized for reference.
Collapse
Affiliation(s)
- Ya-Hsin Chou
- Department of Psychiatry, Taoyuan Chang Gung Memorial Hospital, Taoyuan County, Taiwan
| | - Chemin Lin
- College of Medicine, Chang Gung University, Taoyuan County, Taiwan
- Department of Psychiatry, Keelung Chang Gung Memorial Hospital, Keelung City, Taiwan
- Community Medicine Research Center, Chang Gung Memorial Hospital, Keelung, Taiwan
| | - Shwu-Hua Lee
- College of Medicine, Chang Gung University, Taoyuan County, Taiwan
- Department of Psychiatry, Linkou Chang Gung Memorial Hospital, Taoyuan County, Taiwan
| | - Ya-Wen Chang Chien
- Department of Photography and Virtual Reality Design, Huafan University, New Taipei, Taiwan
| | - Li-Chen Cheng
- Department of Information and Finance Management, National Taipei University of Technology, Taipei, Taiwan
| |
Collapse
|