1
|
Malgaroli M, Tseng E, Hull TD, Jennings E, Choudhury TK, Simon NM. Association of Health Care Work With Anxiety and Depression During the COVID-19 Pandemic: Structural Topic Modeling Study. JMIR AI 2023; 2:e47223. [PMID: 38875560 PMCID: PMC11041488 DOI: 10.2196/47223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Revised: 06/28/2023] [Accepted: 09/07/2023] [Indexed: 06/16/2024]
Abstract
BACKGROUND Stressors for health care workers (HCWs) during the COVID-19 pandemic have been manifold, with high levels of depression and anxiety alongside gaps in care. Identifying the factors most tied to HCWs' psychological challenges is crucial to addressing HCWs' mental health needs effectively, now and for future large-scale events. OBJECTIVE In this study, we used natural language processing methods to examine deidentified psychotherapy transcripts from telemedicine treatment during the initial wave of COVID-19 in the United States. Psychotherapy was delivered by licensed therapists while HCWs were managing increased clinical demands and elevated hospitalization rates, in addition to population-level social distancing measures and infection risks. Our goal was to identify specific concerns emerging in treatment for HCWs and to compare differences with matched non-HCW patients from the general population. METHODS We conducted a case-control study with a sample of 820 HCWs and 820 non-HCW matched controls who received digitally delivered psychotherapy in 49 US states in the spring of 2020 during the first US wave of the COVID-19 pandemic. Depression was measured during the initial assessment using the Patient Health Questionnaire-9, and anxiety was measured using the General Anxiety Disorder-7 questionnaire. Structural topic models (STMs) were used to determine treatment topics from deidentified transcripts from the first 3 weeks of treatment. STM effect estimators were also used to examine topic prevalence in patients with moderate to severe anxiety and depression. RESULTS The median treatment enrollment date was April 15, 2020 (IQR March 31 to April 27, 2020) for HCWs and April 19, 2020 (IQR April 5 to April 27, 2020) for matched controls. STM analysis of deidentified transcripts identified 4 treatment topics centered on health care and 5 on mental health for HCWs. For controls, 3 STM topics on pandemic-related disruptions and 5 on mental health were identified. Several STM treatment topics were significantly associated with moderate to severe anxiety and depression, including working on the hospital unit (topic prevalence 0.035, 95% CI 0.022-0.048; P<.001), mood disturbances (prevalence 0.014, 95% CI 0.002-0.026; P=.03), and sleep disturbances (prevalence 0.016, 95% CI 0.002-0.030; P=.02). No significant associations emerged between pandemic-related topics and moderate to severe anxiety and depression for non-HCW controls. CONCLUSIONS The study provides large-scale quantitative evidence that during the initial wave of the COVID-19 pandemic, HCWs faced unique work-related challenges and stressors associated with anxiety and depression, which required dedicated treatment efforts. The study further demonstrates how natural language processing methods have the potential to surface clinically relevant markers of distress while preserving patient privacy.
Collapse
Affiliation(s)
- Matteo Malgaroli
- Department of Psychiatry, Grossman School of Medicine, New York University, New York, NY, United States
| | - Emily Tseng
- Ann S Bowers College of Computing and Information Science, Cornell University, Ithaca, NY, United States
| | - Thomas D Hull
- Research and Development, Talkspace, New York, NY, United States
| | - Emma Jennings
- Department of Psychiatry, Grossman School of Medicine, New York University, New York, NY, United States
| | - Tanzeem K Choudhury
- Ann S Bowers College of Computing and Information Science, Cornell University, Ithaca, NY, United States
| | - Naomi M Simon
- Department of Psychiatry, Grossman School of Medicine, New York University, New York, NY, United States
| |
Collapse
|
2
|
Malgaroli M, Hull TD, Zech JM, Althoff T. Natural language processing for mental health interventions: a systematic review and research framework. Transl Psychiatry 2023; 13:309. [PMID: 37798296 PMCID: PMC10556019 DOI: 10.1038/s41398-023-02592-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Revised: 08/31/2023] [Accepted: 09/04/2023] [Indexed: 10/07/2023] Open
Abstract
Neuropsychiatric disorders pose a high societal cost, but their treatment is hindered by lack of objective outcomes and fidelity metrics. AI technologies and specifically Natural Language Processing (NLP) have emerged as tools to study mental health interventions (MHI) at the level of their constituent conversations. However, NLP's potential to address clinical and research challenges remains unclear. We therefore conducted a pre-registered systematic review of NLP-MHI studies using PRISMA guidelines (osf.io/s52jh) to evaluate their models, clinical applications, and to identify biases and gaps. Candidate studies (n = 19,756), including peer-reviewed AI conference manuscripts, were collected up to January 2023 through PubMed, PsycINFO, Scopus, Google Scholar, and ArXiv. A total of 102 articles were included to investigate their computational characteristics (NLP algorithms, audio features, machine learning pipelines, outcome metrics), clinical characteristics (clinical ground truths, study samples, clinical focus), and limitations. Results indicate a rapid growth of NLP MHI studies since 2019, characterized by increased sample sizes and use of large language models. Digital health platforms were the largest providers of MHI data. Ground truth for supervised learning models was based on clinician ratings (n = 31), patient self-report (n = 29) and annotations by raters (n = 26). Text-based features contributed more to model accuracy than audio markers. Patients' clinical presentation (n = 34), response to intervention (n = 11), intervention monitoring (n = 20), providers' characteristics (n = 12), relational dynamics (n = 14), and data preparation (n = 4) were commonly investigated clinical categories. Limitations of reviewed studies included lack of linguistic diversity, limited reproducibility, and population bias. A research framework is developed and validated (NLPxMHI) to assist computational and clinical researchers in addressing the remaining gaps in applying NLP to MHI, with the goal of improving clinical utility, data access, and fairness.
Collapse
Affiliation(s)
- Matteo Malgaroli
- Department of Psychiatry, New York University, Grossman School of Medicine, New York, NY, 10016, USA.
| | | | - James M Zech
- Talkspace, New York, NY, 10025, USA
- Department of Psychology, Florida State University, Tallahassee, FL, 32306, USA
| | - Tim Althoff
- Department of Computer Science, University of Washington, Seattle, WA, 98195, USA
| |
Collapse
|
3
|
Darnell D, Pullmann MD, Hull TD, Chen S, Areán P. Predictors of Disengagement and Symptom Improvement Among Adults With Depression Enrolled in Talkspace, a Technology-Mediated Psychotherapy Platform: Naturalistic Observational Study. JMIR Form Res 2022; 6:e36521. [PMID: 35731563 PMCID: PMC9260528 DOI: 10.2196/36521] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2022] [Revised: 05/06/2022] [Accepted: 05/22/2022] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Depression is a common psychiatric condition with an estimated lifetime prevalence for major depression of 16.6% in the US adult population and is effectively treated through psychotherapy. The widespread availability of the internet and personal devices such as smartphones are changing the landscape of delivery of psychotherapy; however, little is known about whether and for whom this type of therapy is beneficial, and whether having synchronous video-based sessions provides additional benefits to clients above and beyond messaging-based therapy. OBJECTIVE This study examined the outcomes associated with the use of a digital platform (Talkspace) for technology-mediated psychotherapy. We examined the duration of client engagement in therapy and client depression score trajectories over 16 weeks. We explored the association of client characteristics, therapist characteristics, and service plan type with time-to-disengagement and trajectories of change in depression scores. METHODS This naturalistic observational study assessed data collected routinely by the platform between January 2016 and January 2018 and examined psychotherapy outcomes among a large representative sample of adult clients with clinically significant depression. Treatment disengagement was defined as a lack of client-initiated communication for more than 4 weeks. Clients completed the Patient Health Questionnaire-8 item (PHQ-8) at intake and every 3 weeks via an in-app survey. Cox regression analysis was used to examine the time until and predictors of disengagement. Changes in depression scores and predictors of change over time were examined using mixed-effects regression. RESULTS The study included 5890 clients and 1271 therapists. Client scores on the PHQ-8 declined over time, with the average client improving from a score of 15 to below the clinical cutoff of 10 by week 6. At the same time point, 37% of clients had disengaged from the therapy. When combined into a final Cox regression model, those who were more likely to disengage were clients aged 18 to 25 years versus those aged ≥50 years (odds ratio [OR] 0.82, 95% CI 0.74-0.9; P<.001), had higher education (OR 1.14, 95% CI 1.06-1.22; P<.001), had been in therapy before (OR 1.09, 95% CI 1.02-1.17; P=.01), and were living with a partner but unmarried versus single (OR 1.14, 95% CI 1.02-1.27; P=.02). Having a therapist with >10 years of experience was related to lower odds of disengagement (OR 0.87, 95% CI 0.8-0.94; P=.01). When combined into a final regression model predicting improvement in depression scores over time, clients showing more improvement were those with an associate's degree or higher (linear estimate=-0.07, P=.002) and higher intake PHQ-8 scores (estimate=3.73, P<.001). There were no differences based on the plan type. CONCLUSIONS Our findings add to the growing literature showing the benefits of technology-mediated psychotherapy over a relatively brief period (16 weeks).
Collapse
Affiliation(s)
- Doyanne Darnell
- Department of Psychiatry & Behavioral Sciences, University of Washington, Seattle, WA, United States
| | - Michael D Pullmann
- Department of Psychiatry & Behavioral Sciences, University of Washington, Seattle, WA, United States
| | | | - Shiyu Chen
- Department of Psychiatry & Behavioral Sciences, University of Washington, Seattle, WA, United States
| | - Patricia Areán
- Department of Psychiatry & Behavioral Sciences, University of Washington, Seattle, WA, United States
| |
Collapse
|
4
|
Abstract
Human-computer interaction (HCI) has contributed to the design and development of some efficient, user-friendly, cost-effective, and adaptable digital mental health solutions. But HCI has not been well-combined into technological developments resulting in quality and safety concerns. Digital platforms and artificial intelligence (AI) have a good potential to improve prediction, identification, coordination, and treatment by mental health care and suicide prevention services. AI is driving web-based and smartphone apps; mostly it is used for self-help and guided cognitive behavioral therapy (CBT) for anxiety and depression. Interactive AI may help real-time screening and treatment in outdated, strained or lacking mental healthcare systems. The barriers for using AI in mental healthcare include accessibility, efficacy, reliability, usability, safety, security, ethics, suitable education and training, and socio-cultural adaptability. Apps, real-time machine learning algorithms, immersive technologies, and digital phenotyping are notable prospects. Generally, there is a need for faster and better human factors in combination with machine interaction and automation, higher levels of effectiveness evaluation and the application of blended, hybrid or stepped care in an adjunct approach. HCI modeling may assist in the design and development of usable applications, and to effectively recognize, acknowledge, and address the inequities of mental health care and suicide prevention and assist in the digital therapeutic alliance.
Collapse
|
5
|
Eis S, Solà-Morales O, Duarte-Díaz A, Vidal-Alaball J, Perestelo-Pérez L, Robles N, Carrion C. Mobile Applications in Mood Disorders and Mental Health: Systematic Search in Apple App Store and Google Play Store and Review of the Literature. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19042186. [PMID: 35206373 PMCID: PMC8871536 DOI: 10.3390/ijerph19042186] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 02/03/2022] [Accepted: 02/10/2022] [Indexed: 02/04/2023]
Abstract
OBJECTIVES The main objective of this work was to explore and characterize the current landscape of mobile applications available to treat mood disorders such as depression, bipolar disorder, and dysthymia. METHODS We developed a tool that makes both the Apple App Store and the Google Play Store searchable using keywords and that facilitates the extraction of basic app information of the search results. All app results were filtered using various inclusion and exclusion criteria. We characterized all resultant applications according to their technical details. Furthermore, we searched for scientific publications on each app's website and PubMed, to understand whether any of the apps were supported by any type of scientific evidence on their acceptability, validation, use, effectiveness, etc. Results: Thirty apps were identified that fit the inclusion and exclusion criteria. The literature search yielded 27 publications related to the apps. However, these did not exclusively concern mood disorders. 6 were randomized studies and the rest included a protocol, pilot-, feasibility, case-, or qualitative studies, among others. The majority of studies were conducted on relatively small scales and 9 of the 27 studies did not explicitly study the effects of mobile application use on mental wellbeing. CONCLUSION While there exists a wealth of mobile applications aimed at the treatment of mental health disorders, including mood disorders, this study showed that only a handful of these are backed by robust scientific evidence. This result uncovers a need for further clinically oriented and systematic validation and testing of such apps.
Collapse
Affiliation(s)
- Sophie Eis
- Fundació HiTT (Health Innovation Technology Transfer), 08015 Barcelona, Spain;
| | - Oriol Solà-Morales
- Fundació HiTT (Health Innovation Technology Transfer), 08015 Barcelona, Spain;
- Correspondence:
| | - Andrea Duarte-Díaz
- Canary Islands Health Research Institute Foundation (FIISC), 38109 Tenerife, Spain;
| | - Josep Vidal-Alaball
- Health Promotion in Rural Areas Research Group, Gerència Territorial de la Catalunya Central, Institut Català de la Salut, 08272 Barcelona, Spain;
- Unitat de Suport a la Recerca de la Catalunya Central, Fundació Institut Universitari per a la Recerca a l’Atenció Primària de Salut Jordi Gol i Gurina, 08007 Barcelona, Spain
- Faculty of Medicine, University of Vic-Central University of Catalonia (UVIC-UCC), 08500 Vic, Spain
| | | | - Noemí Robles
- eHealth Lab Research Group, School of Health Sciences and eHealth Centre, Universitat Oberta de Catalunya (UOC), 08035 Barcelona, Spain; (N.R.); (C.C.)
| | - Carme Carrion
- eHealth Lab Research Group, School of Health Sciences and eHealth Centre, Universitat Oberta de Catalunya (UOC), 08035 Barcelona, Spain; (N.R.); (C.C.)
| |
Collapse
|
6
|
COVID 19 and Implementing E-Mental Health using the VRIO Strategic Planning Framework. INTERNATIONAL JOURNAL OF E-COLLABORATION 2022. [DOI: 10.4018/ijec.299005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Coronavirus has disrupted normalcy for communities and businesses across the globe. With massive shutdowns across the world, organizations were forced to assess their business operations and determine how to sustain in such turbulent times. Mental health clinics faced equivalent barriers as the structure of engaging with clients in a face to face setting was similarly disrupted. The implementation of e-mental health structures were clear targets of change as clinics needed to continue to serve the mentally fragile. Strategic planning calls for thoughtful collaboration, collection, and consideration of the necessary adaptations to meet the mission and vision of the organization. Strategic frameworks like the Value, Rarity, Imitability, and Organization are useful tools for mental health clinics to plan on shifting their operations from an in-person to a virtual environment.
Collapse
|