Improving Moderator Responsiveness in Online Peer Support Through Automated Triage

The Role of Moderators in Facilitating and Encouraging Peer-to-Peer Support in an Online Mental Health Community: A Qualitative Exploratory Study

Online peer support platforms have gained popularity as a potential way for people struggling with mental health problems to share information and provide support to each other. While these platforms can offer an open space to discuss emotionally difficult issues, unsafe or unmoderated communities can allow potential harm to users by spreading triggering content, misinformation or hostile interactions. The purpose of this study was to explore the role of moderators in these online communities, and how moderators can facilitate peer-to-peer support, while minimizing harms to users and amplifying potential benefits. Moderators of the Togetherall peer support platform were recruited to participate in qualitative interviews. The moderators, referred to as 'Wall Guides', were asked about their day-to-day responsibilities, positive and negative experiences they have witnessed on the platform and the strategies they employ when encountering problems such as lack of engagement or posting of inappropriate content. The data were then analyzed qualitatively using thematic content analysis and consensus codes were deduced and reviewed to reach final results and representative themes. In total, 20 moderators participated in this study, and described their experiences and efforts to follow a consistent and shared protocol for responding to common scenarios in the online community. Many reported the deep connections formed by the online community, the helpful and thoughtful responses that members give each other and the satisfaction of seeing progress in members' recovery. They also reported occasional aggressive, sensitive or inconsiderate comments and posts on the platform. They respond by removing or revising the hurtful post or reaching out to the affected member to maintain the 'house rules'. Lastly, many discussed strategies they elicit to promote engagement from members within the community and ensure each member is supported through their use of the platform. This study sheds light on the critical role of moderators of online peer support communities, and their ability to contribute to the potential benefits of digital peer support while minimizing risks to users. The findings reported here accentuate the importance of having well-trained moderators on online peer support platforms and can guide future efforts to effectively train and supervise prospective peer support moderators. Moderators can become an active 'shaping force' and bring a cohesive culture of expressed empathy, sensitivity and care. The delivery of a healthy and safe community contrasts starkly with non-moderated online forums, which can become unhealthy and unsafe as a result.

Multi-task learning to detect suicide ideation and mental disorders among social media users

Mental disorders and suicide are considered global health problems faced by many countries worldwide. Even though advancements have been made to improve mental wellbeing through research, there is room for improvement. Using Artificial Intelligence to early detect individuals susceptible to mental illness and suicide ideation based on their social media postings is one way to start. This research investigates the effectiveness of using a shared representation to automatically extract features between the two different yet related tasks of mental illness and suicide ideation detection using data in parallel from social media platforms with different distributions. In addition to discovering the shared features between users with suicidal thoughts and users who self-declared a single mental disorder, we further investigate the impact of comorbidity on suicide ideation and use two datasets during inference to test the generalizability of the trained models and provide satisfactory evidence to validate the increased predictive accurateness of suicide risk when using data from users diagnosed with multiple mental disorders compared to a single mental disorder for the mental illness detection task. Our results also demonstrate different mental disorders' impact on suicidal risk and discover a noticeable impact when using data from users diagnosed with Post-Traumatic Stress Disorder. We use multi-task learning (MTL) with soft and hard parameter sharing to produce state-of-the-art results for detecting users with suicide ideation who require urgent attention. We further improve the predictability of the proposed model by demonstrating the effectiveness of cross-platform knowledge sharing and predefined auxiliary inputs.

Resilience in Web-Based Mental Health Communities: Building a Resilience Dictionary With Semiautomatic Text Analysis

Background

Resilience is an accepted strengths-based concept that responds to change, adversity, and crises. This concept underpins both personal and community-based preventive approaches to mental health issues and shapes digital interventions. Online mental health peer-support forums have played a prominent role in enhancing resilience by providing accessible places for sharing lived experiences of mental issues and finding support. However, little research has been conducted on whether and how resilience is realized, hindering service providers’ ability to optimize resilience outcomes.

Objective

This study aimed to create a resilience dictionary that reflects the characteristics and realization of resilience within online mental health peer-support forums. The findings can be used to guide further analysis and improve resilience outcomes in mental health forums through targeted moderation and management.

Methods

A semiautomatic approach to creating a resilience dictionary was proposed using topic modeling and qualitative content analysis. We present a systematic 4-phase analysis pipeline that preprocesses raw forum posts, discovers core themes, conceptualizes resilience indicators, and generates a resilience dictionary. Our approach was applied to a mental health forum run by SANE (Schizophrenia: A National Emergency) Australia, with 70,179 forum posts between 2018 and 2020 by 2357 users being analyzed.

Results

The resilience dictionary and taxonomy developed in this study, reveal how resilience indicators (ie, “social capital,” “belonging,” “learning,” “adaptive capacity,” and “self-efficacy”) are characterized by themes commonly discussed in the forums; each theme’s top 10 most relevant descriptive terms and their synonyms; and the relatedness of resilience, reflecting a taxonomy of indicators that are more comprehensive (or compound) and more likely to facilitate the realization of others. The study showed that the resilience indicators “learning,” “belonging,” and “social capital” were more commonly realized, and “belonging” and “learning” served as foundations for “social capital” and “adaptive capacity” across the 2-year study period.

Conclusions

This study presents a resilience dictionary that improves our understanding of how aspects of resilience are realized in web-based mental health forums. The dictionary provides novel guidance on how to improve training to support and enhance automated systems for moderating mental health forum discussions.

"I Want to Be Stepping in More" - Professional Online Forum Moderators' Experiences of Supporting Individuals in a Suicide Crisis

INTRODUCTION

Individuals experiencing suicidal crises increasingly turn to online mental health forums for support. Support can come from peers but also from online moderators, many of whom are trained health professionals. Much is known about users' forum experiences; however, the experiences of professional moderators who work to keep users safe has been overlooked. The beneficial nature of online forums cannot be fully realized until there is a clearer understanding of both parties' participation. This study explored the experiences of professional online forum moderators engaged in suicide prevention.

MATERIALS AND METHODS

A purposive sample of professionally qualified moderators was recruited from three online mental health organizations. In-depth semi-structured, video-recorded interviews were conducted with 15 moderators (3 male, 12 female), to explore their experiences and perceptions of working in online suicide prevention spaces. Data was analyzed using inductive thematic analysis.

RESULTS

Five themes were identified related to the experiences and challenges for moderators. These were the sense of the unknown, the scope of the role, limitations of the written word, volume of tasks, and balancing individual vs. community needs.

DISCUSSION

Findings indicate that the professionally qualified moderator role is complex and multifaceted, with organizations failing to recognize these aspects. Organizations restrict moderators from using their full therapeutic skill set, limiting them to only identifying and re-directing at-risk users to crisis services. The benefits of moderated online forums could be enhanced by allowing moderators to use more of their skills. To facilitate this, in-situ research is needed that examines how moderators use their skills to identify at-risk users.

Suicide Risk and Protective Factors in Online Support Forum Posts: Annotation Scheme Development and Validation Study

BACKGROUND

Online communities provide support for individuals looking for help with suicidal ideation and crisis. As community data are increasingly used to devise machine learning models to infer who might be at risk, there have been limited efforts to identify both risk and protective factors in web-based posts. These annotations can enrich and augment computational assessment approaches to identify appropriate intervention points, which are useful to public health professionals and suicide prevention researchers.

OBJECTIVE

This qualitative study aims to develop a valid and reliable annotation scheme for evaluating risk and protective factors for suicidal ideation in posts in suicide crisis forums.

METHODS

We designed a valid, reliable, and clinically grounded process for identifying risk and protective markers in social media data. This scheme draws on prior work on construct validity and the social sciences of measurement. We then applied the scheme to annotate 200 posts from r/SuicideWatch-a Reddit community focused on suicide crisis.

RESULTS

We documented our results on producing an annotation scheme that is consistent with leading public health information coding schemes for suicide and advances attention to protective factors. Our study showed high internal validity, and we have presented results that indicate that our approach is consistent with findings from prior work.

CONCLUSIONS

Our work formalizes a framework that incorporates construct validity into the development of annotation schemes for suicide risk on social media. This study furthers the understanding of risk and protective factors expressed in social media data. This may help public health programming to prevent suicide and computational social science research and investigations that rely on the quality of labels for downstream machine learning tasks.

Improving Mental Health Services: A 50-Year Journey from Randomized Experiments to Artificial Intelligence and Precision Mental Health

This conceptual paper describes the current state of mental health services, identifies critical problems, and suggests how to solve them. I focus on the potential contributions of artificial intelligence and precision mental health to improving mental health services. Toward that end, I draw upon my own research, which has changed over the last half century, to highlight the need to transform the way we conduct mental health services research. I identify exemplars from the emerging literature on artificial intelligence and precision approaches to treatment in which there is an attempt to personalize or fit the treatment to the client in order to produce more effective interventions.

Ethics and Law in Research on Algorithmic and Data-Driven Technology in Mental Health Care: Scoping Review

BACKGROUND

Uncertainty surrounds the ethical and legal implications of algorithmic and data-driven technologies in the mental health context, including technologies characterized as artificial intelligence, machine learning, deep learning, and other forms of automation.

OBJECTIVE

This study aims to survey empirical scholarly literature on the application of algorithmic and data-driven technologies in mental health initiatives to identify the legal and ethical issues that have been raised.

METHODS

We searched for peer-reviewed empirical studies on the application of algorithmic technologies in mental health care in the Scopus, Embase, and Association for Computing Machinery databases. A total of 1078 relevant peer-reviewed applied studies were identified, which were narrowed to 132 empirical research papers for review based on selection criteria. Conventional content analysis was undertaken to address our aims, and this was supplemented by a keyword-in-context analysis.

RESULTS

We grouped the findings into the following five categories of technology: social media (53/132, 40.1%), smartphones (37/132, 28%), sensing technology (20/132, 15.1%), chatbots (5/132, 3.8%), and miscellaneous (17/132, 12.9%). Most initiatives were directed toward detection and diagnosis. Most papers discussed privacy, mainly in terms of respecting the privacy of research participants. There was relatively little discussion of privacy in this context. A small number of studies discussed ethics directly (10/132, 7.6%) and indirectly (10/132, 7.6%). Legal issues were not substantively discussed in any studies, although some legal issues were discussed in passing (7/132, 5.3%), such as the rights of user subjects and privacy law compliance.

CONCLUSIONS

Ethical and legal issues tend to not be explicitly addressed in empirical studies on algorithmic and data-driven technologies in mental health initiatives. Scholars may have considered ethical or legal matters at the ethics committee or institutional review board stage. If so, this consideration seldom appears in published materials in applied research in any detail. The form itself of peer-reviewed papers that detail applied research in this field may well preclude a substantial focus on ethics and law. Regardless, we identified several concerns, including the near-complete lack of involvement of mental health service users, the scant consideration of algorithmic accountability, and the potential for overmedicalization and techno-solutionism. Most papers were published in the computer science field at the pilot or exploratory stages. Thus, these technologies could be appropriated into practice in rarely acknowledged ways, with serious legal and ethical implications.

Naturally occurring language as a source of evidence in suicide prevention

We discuss computational language analysis as it pertains to suicide prevention research, with an emphasis on providing non-technologists with an understanding of key issues and, equally important, considering its relation to the broader enterprise of suicide prevention. Our emphasis here is on naturally occurring language in social media, motivated by its non-intrusive ability to yield high-value information that in the past has been largely unavailable to clinicians.

Evaluation of ReachOut.com, an Unstructured Digital Youth Mental Health Intervention: Prospective Cohort Study

BACKGROUND

Young people experience a disproportionate burden associated with mental illness that Australia's mental health care system is ill-equipped to handle. Despite improvements in the provision of mental health services, the rates of service utilization among young people remain suboptimal, and there are still considerable barriers to seeking help. Digital mental health services can overcome a number of barriers and connect young people requiring support; however, the evidence base of digital interventions is limited.

OBJECTIVE

The aim of this study is to examine the effectiveness of a brief, self-directed, unstructured digital intervention, ReachOut.com (hereafter ReachOut), in reducing depression, anxiety, stress, and risk of suicide.

METHODS

A cohort of 1982 ReachOut users participated in a 12-week longitudinal study, with a retention rate of 81.18% (1609/1982) across the duration of the study. Participants completed web-based surveys, with outcome measures of mental health status and suicide risk assessed at 3 time points across the study period.

RESULTS

The results demonstrated that over the 12-week study period, young people using ReachOut experienced modest yet significant reductions in symptoms of depression, anxiety, and stress. Significant, albeit modest, reductions in the proportion of participants at high risk of suicide were also observed.

CONCLUSIONS

The findings of this research provide preliminary evidence of the promise of an unstructured digital mental health intervention, ReachOut, in alleviating symptoms of mental ill-health and promoting well-being in young people. These findings are particularly important given that digital services are not only acceptable and accessible but also have the potential to cater to the diverse mental health needs of young people at scale, in a way that other services cannot.

Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study

Background

Mental illness affects a significant portion of the worldwide population. Online mental health forums can provide a supportive environment for those afflicted and also generate a large amount of data that can be mined to predict mental health states using machine learning methods.

Objective

This study aimed to benchmark multiple methods of text feature representation for social media posts and compare their downstream use with automated machine learning (AutoML) tools. We tested on datasets that contain posts labeled for perceived suicide risk or moderator attention in the context of self-harm. Specifically, we assessed the ability of the methods to prioritize posts that a moderator would identify for immediate response.

Methods

We used 1588 labeled posts from the Computational Linguistics and Clinical Psychology (CLPsych) 2017 shared task collected from the Reachout.com forum. Posts were represented using lexicon-based tools, including Valence Aware Dictionary and sEntiment Reasoner, Empath, and Linguistic Inquiry and Word Count, and also using pretrained artificial neural network models, including DeepMoji, Universal Sentence Encoder, and Generative Pretrained Transformer-1 (GPT-1). We used Tree-based Optimization Tool and Auto-Sklearn as AutoML tools to generate classifiers to triage the posts.

Results

The top-performing system used features derived from the GPT-1 model, which was fine-tuned on over 150,000 unlabeled posts from Reachout.com. Our top system had a macroaveraged F1 score of 0.572, providing a new state-of-the-art result on the CLPsych 2017 task. This was achieved without additional information from metadata or preceding posts. Error analyses revealed that this top system often misses expressions of hopelessness. In addition, we have presented visualizations that aid in the understanding of the learned classifiers.

Conclusions

In this study, we found that transfer learning is an effective strategy for predicting risk with relatively little labeled data and noted that fine-tuning of pretrained language models provides further gains when large amounts of unlabeled text are available.