Designing a Clinician-Facing Tool for Using Insights From Patients' Social Media Activity: Iterative Co-Design Approach

Integrating Patient-Generated Digital Data Into Mental Health Therapy: Mixed Methods Analysis of User Experience

Background

Therapists and their patients increasingly discuss digital data from social media, smartphone sensors, and other online engagements within the context of psychotherapy.

Objective

We examined patients' and mental health therapists' experiences and perceptions following a randomized controlled trial in which they both received regular summaries of patients' digital data (eg, dashboard) to review and discuss in session. The dashboard included data that patients consented to share from their social media posts, phone usage, and online searches.

Methods

Following the randomized controlled trial, patient (n=56) and therapist (n=44) participants completed a debriefing survey after their study completion (from December 2021 to January 2022). Participants were asked about their experience receiving a digital data dashboard in psychotherapy via closed- and open-ended questions. We calculated descriptive statistics for closed-ended questions and conducted qualitative coding via NVivo (version 10; Lumivero) and natural language processing using the machine learning tool latent Dirichlet allocation to analyze open-ended questions.

Results

Of 100 participants, nearly half (n=48, 49%) described their experience with the dashboard as "positive," while the other half noted a "neutral" experience. Responses to the open-ended questions resulted in three thematic areas (nine subcategories): (1) dashboard experience (positive, neutral or negative, and comfortable); (2) perception of the dashboard's impact on enhancing therapy (accountability, increased awareness over time, and objectivity); and (3) dashboard refinements (additional sources, tailored content, and privacy).

Conclusions

Patients reported that receiving their digital data helped them stay "accountable," while therapists indicated that the dashboard helped "tailor treatment plans." Patient and therapist surveys provided important feedback on their experience regularly discussing dashboards in psychotherapy.

Facilitating Access to Mental Health Services: A Stakeholder-Driven Improvement of the Children and Young People (CYP) as One Referral Platform

(1) Background: Pre-pandemic, child and adolescent mental health service (CAMHS) referrals were paper based in Liverpool and Sefton (England, United Kingdom), causing delays in waiting times. The "CYP as One" online mental health referral platform was co-created to overcome these challenges. (2) Methods: This study aims to improve "CYP as One" accessibility and usability and, subsequently, support CAMHS to improve waiting times. The current study utilised the Living Lab approach. We conducted content analysis on completed online referrals extracted from the "CYP as One" platform. These findings were supplemented by seven online focus groups, with 16-19-year-old young people, parents of children under 16, and health service providers. Thematic analysis was conducted on all data. (3) Results: The thematic analysis returned seven themes, namely (i) "CYP as One" vs. Traditional Referrals, (ii) Gender and Language Dynamics, (iii) Digital Empathy in Action, (iv) the Influence of the Provider Perspective, (v) Age and Social Sensitivity, (vi) Enhancing Access to Information, and (vii) Boosting Admin and Clinical Efficiency. (4) Conclusions: Digital content that seeks to replace in-person referrals can provide adequate support to children and young people who have faced difficulties accessing mental health services.

Methodologies for Monitoring Mental Health on Twitter: Systematic Review

BACKGROUND

The use of social media data to predict mental health outcomes has the potential to allow for the continuous monitoring of mental health and well-being and provide timely information that can supplement traditional clinical assessments. However, it is crucial that the methodologies used to create models for this purpose are of high quality from both a mental health and machine learning perspective. Twitter has been a popular choice of social media because of the accessibility of its data, but access to big data sets is not a guarantee of robust results.

OBJECTIVE

This study aims to review the current methodologies used in the literature for predicting mental health outcomes from Twitter data, with a focus on the quality of the underlying mental health data and the machine learning methods used.

METHODS

A systematic search was performed across 6 databases, using keywords related to mental health disorders, algorithms, and social media. In total, 2759 records were screened, of which 164 (5.94%) papers were analyzed. Information about methodologies for data acquisition, preprocessing, model creation, and validation was collected, as well as information about replicability and ethical considerations.

RESULTS

The 164 studies reviewed used 119 primary data sets. There were an additional 8 data sets identified that were not described in enough detail to include, and 6.1% (10/164) of the papers did not describe their data sets at all. Of these 119 data sets, only 16 (13.4%) had access to ground truth data (ie, known characteristics) about the mental health disorders of social media users. The other 86.6% (103/119) of data sets collected data by searching keywords or phrases, which may not be representative of patterns of Twitter use for those with mental health disorders. The annotation of mental health disorders for classification labels was variable, and 57.1% (68/119) of the data sets had no ground truth or clinical input on this annotation. Despite being a common mental health disorder, anxiety received little attention.

CONCLUSIONS

The sharing of high-quality ground truth data sets is crucial for the development of trustworthy algorithms that have clinical and research utility. Further collaboration across disciplines and contexts is encouraged to better understand what types of predictions will be useful in supporting the management and identification of mental health disorders. A series of recommendations for researchers in this field and for the wider research community are made, with the aim of enhancing the quality and utility of future outputs.

Cross-Platform Detection of Psychiatric Hospitalization via Social Media Data: Comparison Study

BACKGROUND

Previous research has shown the feasibility of using machine learning models trained on social media data from a single platform (eg, Facebook or Twitter) to distinguish individuals either with a diagnosis of mental illness or experiencing an adverse outcome from healthy controls. However, the performance of such models on data from novel social media platforms unseen in the training data (eg, Instagram and TikTok) has not been investigated in previous literature.

OBJECTIVE

Our study examined the feasibility of building machine learning classifiers that can effectively predict an upcoming psychiatric hospitalization given social media data from platforms unseen in the classifiers' training data despite the preliminary evidence on identity fragmentation on the investigated social media platforms.

METHODS

Windowed timeline data of patients with a diagnosis of schizophrenia spectrum disorder before a known hospitalization event and healthy controls were gathered from 3 platforms: Facebook (254/268, 94.8% of participants), Twitter (51/268, 19% of participants), and Instagram (134/268, 50% of participants). We then used a 3 × 3 combinatorial binary classification design to train machine learning classifiers and evaluate their performance on testing data from all available platforms. We further compared results from models in intraplatform experiments (ie, training and testing data belonging to the same platform) to those from models in interplatform experiments (ie, training and testing data belonging to different platforms). Finally, we used Shapley Additive Explanation values to extract the top predictive features to explain and compare the underlying constructs that predict hospitalization on each platform.

RESULTS

We found that models in intraplatform experiments on average achieved an F₁-score of 0.72 (SD 0.07) in predicting a psychiatric hospitalization because of schizophrenia spectrum disorder, which is 68% higher than the average of models in interplatform experiments at an F₁-score of 0.428 (SD 0.11). When investigating the key drivers for divergence in construct validities between models, an analysis of top features for the intraplatform models showed both low predictive feature overlap between the platforms and low pairwise rank correlation (<0.1) between the platforms' top feature rankings. Furthermore, low average cosine similarity of data between platforms within participants in comparison with the same measurement on data within platforms between participants points to evidence of identity fragmentation of participants between platforms.

CONCLUSIONS

We demonstrated that models built on one platform's data to predict critical mental health treatment outcomes such as hospitalization do not generalize to another platform. In our case, this is because different social media platforms consistently reflect different segments of participants' identities. With the changing ecosystem of social media use among different demographic groups and as web-based identities continue to become fragmented across platforms, further research on holistic approaches to harnessing these diverse data sources is required.

Human-Centered Design Approaches in Digital Mental Health Interventions: Exploratory Mapping Review

BACKGROUND

Digital mental health interventions have a great potential to alleviate mental illness and increase access to care. However, these technologies face significant challenges, especially in terms of user engagement and adoption. It has been suggested that this issue stems from a lack of user perspective in the development process; accordingly, several human-centered design approaches have been developed over the years to consider this important aspect. Yet, few human-centered design approaches to digital solutions exist in the field of mental health, and rarely are end users involved in their development.

OBJECTIVE

The main objective of this literature review is to understand how human-centered design is considered in e-mental health intervention research.

METHODS

An exploratory mapping review was conducted of mental health journals with the explicit scope of covering e-mental health technology. The human-centered design approaches reported and the core elements of design activity (ie, object, context, design process, and actors involved) were examined among the eligible studies.

RESULTS

A total of 30 studies met the inclusion criteria, of which 22 mentioned using human-centered design approaches or specific design methods in the development of an e-mental health solution. Reported approaches were classified as participatory design (11/27, 41%), codesign (6/27, 22%), user-centered design (5/27, 19%), or a specific design method (5/27, 19%). Just over half (15/27, 56%) of the approaches mentioned were supported by references. End users were involved in each study to some extent but not necessarily in designing. About 27% (8/30) of all the included studies explicitly mentioned the presence of designers on their team.

CONCLUSIONS

Our results show that some attempts have indeed been made to integrate human-centered design approaches into digital mental health technology development. However, these attempts rely very little on designers and design research. Researchers from other domains and technology developers would be wise to learn the underpinnings of human-centered design methods before selecting one over another. Inviting designers for assistance when implementing a particular approach would also be beneficial. To further motivate interest in and use of human-centered design principles in the world of e-mental health, we make nine suggestions for better reporting of human-centered design approaches in future research.

Social Media Discussions Predict Mental Health Consultations on College Campuses

The mental health of college students is a growing concern, and gauging the mental health needs of college students is difficult to assess in real-time and in scale. To address this gap, researchers and practitioners have encouraged the use of passive technologies. Social media is one such "passive sensor" that has shown potential as a viable "passive sensor" of mental health. However, the construct validity and in-practice reliability of computational assessments of mental health constructs with social media data remain largely unexplored. Towards this goal, we study how assessing the mental health of college students using social media data correspond with ground-truth data of on-campus mental health consultations. For a large U.S. public university, we obtained ground-truth data of on-campus mental health consultations between 2011–2016, and collected 66,000 posts from the university’s Reddit community. We adopted machine learning and natural language methodologies to measure symptomatic mental health expressions of depression, anxiety, stress, suicidal ideation, and psychosis on the social media data. Seasonal auto-regressive integrated moving average (SARIMA) models of forecasting on-campus mental health consultations showed that incorporating social media data led to predictions with r = 0.86 and SMAPE = 13.30, outperforming models without social media data by 41%. Our language analyses revealed that social media discussions during high mental health consultations months consisted of discussions on academics and career, whereas months of low mental health consultations saliently show expressions of positive affect, collective identity, and socialization. This study reveals that social media data can improve our understanding of college students’ mental health, particularly their mental health treatment needs.

Detecting Suicide Ideation in the Era of Social Media: The Population Neuroscience Perspective

Social media platforms are increasingly used across many population groups not only to communicate and consume information, but also to express symptoms of psychological distress and suicidal thoughts. The detection of suicidal ideation (SI) can contribute to suicide prevention. Twitter data suggesting SI have been associated with negative emotions (e.g., shame, sadness) and a number of geographical and ecological variables (e.g., geographic location, environmental stress). Other important research contributions on SI come from studies in neuroscience. To date, very few research studies have been conducted that combine different disciplines (epidemiology, health geography, neurosciences, psychology, and social media big data science), to build innovative research directions on this topic. This article aims to offer a new interdisciplinary perspective, that is, a Population Neuroscience perspective on SI in order to highlight new ways in which multiple scientific fields interact to successfully investigate emotions and stress in social media to detect SI in the population. We argue that a Population Neuroscience perspective may help to better understand the mechanisms underpinning SI and to promote more effective strategies to prevent suicide timely and at scale.

Clinician Perspectives on Using Computational Mental Health Insights From Patients' Social Media Activities: Design and Qualitative Evaluation of a Prototype

BACKGROUND

Previous studies have suggested that social media data, along with machine learning algorithms, can be used to generate computational mental health insights. These computational insights have the potential to support clinician-patient communication during psychotherapy consultations. However, how clinicians perceive and envision using computational insights during consultations has been underexplored.

OBJECTIVE

The aim of this study is to understand clinician perspectives regarding computational mental health insights from patients' social media activities. We focus on the opportunities and challenges of using these insights during psychotherapy consultations.

METHODS

We developed a prototype that can analyze consented patients' Facebook data and visually represent these computational insights. We incorporated the insights into existing clinician-facing assessment tools, the Hamilton Depression Rating Scale and Global Functioning: Social Scale. The design intent is that a clinician will verbally interview a patient (eg, How was your mood in the past week?) while they reviewed relevant insights from the patient's social media activities (eg, number of depression-indicative posts). Using the prototype, we conducted interviews (n=15) and 3 focus groups (n=13) with mental health clinicians: psychiatrists, clinical psychologists, and licensed clinical social workers. The transcribed qualitative data were analyzed using thematic analysis.

RESULTS

Clinicians reported that the prototype can support clinician-patient collaboration in agenda-setting, communicating symptoms, and navigating patients' verbal reports. They suggested potential use scenarios, such as reviewing the prototype before consultations and using the prototype when patients missed their consultations. They also speculated potential negative consequences: patients may feel like they are being monitored, which may yield negative effects, and the use of the prototype may increase the workload of clinicians, which is already difficult to manage. Finally, our participants expressed concerns regarding the prototype: they were unsure whether patients' social media accounts represented their actual behaviors; they wanted to learn how and when the machine learning algorithm can fail to meet their expectations of trust; and they were worried about situations where they could not properly respond to the insights, especially emergency situations outside of clinical settings.

CONCLUSIONS

Our findings support the touted potential of computational mental health insights from patients' social media account data, especially in the context of psychotherapy consultations. However, sociotechnical issues, such as transparent algorithmic information and institutional support, should be addressed in future endeavors to design implementable and sustainable technology.

Development, Implementation, and User Evaluation of COVID-19 Dashboard in a Third-Level Hospital in Iran

INTRODUCTION

The implementation of a dashboard enables managers to make informed and evidence-based decisions through data visualization and graphical presentation of information. This study aimed to design and implement a COVID-19 management dashboard in a third-level hospital in Mashhad, Iran.

MATERIALS AND METHODS

This descriptive developmental applied study was conducted in the second half of 2020 in three stages, using user-centered design methodology in four phases: (1) specification of the application context, (2) specification of requirements, (3) creation of design solutions, and (4) evaluation of designs. Data collection in each phase was performed through holding group discussions with the main users, nominal group techniques, interviews, and questioners. The dashboard prototype for the data display was designed using the Power BI Desktop software. Subsequently, users' comments were obtained using the focus group method and included in the dashboard.

RESULTS

In total, 25 indicators related to input, process, and output areas were identified based on the findings of the first stage. Moreover, eight items were introduced by participants as dashboard requirements. The dashboard was developed based on users' feedback and suggestions, such as the use of colors, reception of periodic and specific reports based on key performance indicators, and rearrangement of the components visible on the page. The result of the user satisfaction survey indicated their satisfaction with the developed dashboard.

CONCLUSION

The selection of proper criteria for the implementation of an effective dashboard is critical for the health care organization since they are designed with a high-tech and content-based environment. The dashboard in the present study was a successful combination of clinical and managerial indicators. Future studies should focus on the design and development of dashboards, as well as benchmarking by using data from several hospitals.

Characterizing Social Media and Digital Data Use in Mental Health Therapy from Patient and Therapist Perspectives (Preprint)

Background

Incorporating insights from social media into the patient-provider encounter is increasingly being explored in health care settings. Less is known about the utility of these data in mental health therapy.

Objective

This study aims to prospectively investigate and characterize how social media and digital data are used in mental health therapy from both the patient and mental health therapist perspective.

Methods

Patients enrolled in mental health therapy and mental health therapists were interviewed using a semistructured interview guide. All interviews were transcribed and coded using a deductive framework analysis. Themes and subthemes were identified. Participants completed a sociodemographic survey, while mental health therapists also completed a behavioral norms and elicitation survey.

Results

Seventeen participants, that is, 8 (48%) mental health therapists and 9 (52%) patients were interviewed. Overall, participants identified 4 themes and 9 subthemes. Themes were current data collection practices, social media and digital data in therapy, advantages of social media and digital data in therapy, and disadvantages of social media and digital data in therapy. Most subthemes were related to the advantages and disadvantages of incorporating digital data in mental health therapy. Advantage subthemes included convenience, objective, builds rapport, and user-friendliness while disadvantage subthemes were nonreflective, ethically ambiguous, and nongeneralizable. The mental health therapists' behavioral norms and elicitation survey found that injunctive and descriptive normative beliefs mapped onto 2 advantage subthemes: convenience and objectivity.

Conclusions

This qualitative pilot study established the advantages and disadvantages of social media and digital data use in mental health therapy. Patients and therapists highlighted similar concerns and uses. This study indicated that overall, both patients and therapists are interested in and are comfortable to use and discuss social media and digital data in mental health therapy.

Unique Internet Search Strategies of Individuals With Self-Stated Autism: Quantitative Analysis of Search Engine Users' Investigative Behaviors

Background

Although autism is often characterized in literature by the presence of repetitive behavior, in structured decision tasks, individuals with autism spectrum disorder (ASD) have been found to examine more options in a given time period than controls.

Objective

We aimed to examine whether this investigative tendency emerges in information searches conducted via the internet.

Methods

In total, 1746 search engine users stated that they had ASD in 2019. This group’s naturally occurring responses following 1491 unique general queries and 78 image queries were compared to those of all other users of the search engine. The main dependent measure was scrolled distance, which denoted the extent to which additional results were scanned beyond the initial results presented on-screen. Additionally, we examined the number of clicks on search results as an indicator of the degree of search outcome exploitation and assessed whether there was a trade-off between increased search range and the time invested in viewing initial search results.

Results

After issuing general queries, individuals with self-stated ASD scanned more results than controls. The scrolled distance in the results page of general queries was 45% larger for the group of individuals with ASD (P<.001; d=0.45). The group of individuals with ASD also made the first scroll faster than the controls (P<.001; d=0.51). The differences in scrolled distance were larger for popular queries. No group differences in scrolled distance emerged for image queries, suggesting that visual load impeded the investigative behavior of individuals with ASD. No differences emerged in the number of clicks on search results.

Conclusions

Individuals who self-stated that they had ASD scrutinized more general search results and fewer image search results than the controls. Thus, our results at least partially support the notion that individuals with ASD exhibit investigative behaviors and suggest that textual searches are an important context for expressing such tendencies.

Blinded Clinical Ratings of Social Media Data are Correlated with In-Person Clinical Ratings in Participants Diagnosed with Either Depression, Schizophrenia, or Healthy Controls

This study investigates clinically valid signals about psychiatric symptoms in social media data, by rating severity of psychiatric symptoms in donated, de-identified Facebook posts and comparing to in-person clinical assessments. Participants with schizophrenia (N=8), depression (N=7), or who were healthy controls (N=8) also consented to the collection of their Facebook activity from three months before the in-person assessments to six weeks after this evaluation. Depressive symptoms were assessed in- person using the Montgomery-Åsberg Depression Rating Scale (MADRS), psychotic symptoms were assessed using the Brief Psychiatric Rating Scale (BPRS), and global functioning was assessed using the Community Assessment of Psychotic Experiences (CAPE-42). Independent raters (psychiatrists, non-psychiatrist mental health clinicians, and two staff members) rated depression, psychosis, and global functioning symptoms from the social media activity of deidentified participants. The correlations between in-person clinical ratings and blinded ratings based on social media data were evaluated. Significant correlations (and trends for significance in the mixed model controlling for multiple raters) were found for psychotic symptoms, global symptom ratings and depressive symptoms. Results like these, indicating the presence of clinically valid signal in social media, are an important step toward developing computational tools that could assist clinicians by providing additional data outside the context of clinical encounters.

Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media

BACKGROUND

The COVID-19 pandemic has caused several disruptions in personal and collective lives worldwide. The uncertainties surrounding the pandemic have also led to multifaceted mental health concerns, which can be exacerbated with precautionary measures such as social distancing and self-quarantining, as well as societal impacts such as economic downturn and job loss. Despite noting this as a "mental health tsunami", the psychological effects of the COVID-19 crisis remain unexplored at scale. Consequently, public health stakeholders are currently limited in identifying ways to provide timely and tailored support during these circumstances.

OBJECTIVE

Our study aims to provide insights regarding people's psychosocial concerns during the COVID-19 pandemic by leveraging social media data. We aim to study the temporal and linguistic changes in symptomatic mental health and support expressions in the pandemic context.

METHODS

We obtained about 60 million Twitter streaming posts originating from the United States from March 24 to May 24, 2020, and compared these with about 40 million posts from a comparable period in 2019 to attribute the effect of COVID-19 on people's social media self-disclosure. Using these data sets, we studied people's self-disclosure on social media in terms of symptomatic mental health concerns and expressions of support. We employed transfer learning classifiers that identified the social media language indicative of mental health outcomes (anxiety, depression, stress, and suicidal ideation) and support (emotional and informational support). We then examined the changes in psychosocial expressions over time and language, comparing the 2020 and 2019 data sets.

RESULTS

We found that all of the examined psychosocial expressions have significantly increased during the COVID-19 crisis-mental health symptomatic expressions have increased by about 14%, and support expressions have increased by about 5%, both thematically related to COVID-19. We also observed a steady decline and eventual plateauing in these expressions during the COVID-19 pandemic, which may have been due to habituation or due to supportive policy measures enacted during this period. Our language analyses highlighted that people express concerns that are specific to and contextually related to the COVID-19 crisis.

CONCLUSIONS

We studied the psychosocial effects of the COVID-19 crisis by using social media data from 2020, finding that people's mental health symptomatic and support expressions significantly increased during the COVID-19 period as compared to similar data from 2019. However, this effect gradually lessened over time, suggesting that people adapted to the circumstances and their "new normal." Our linguistic analyses revealed that people expressed mental health concerns regarding personal and professional challenges, health care and precautionary measures, and pandemic-related awareness. This study shows the potential to provide insights to mental health care and stakeholders and policy makers in planning and implementing measures to mitigate mental health risks amid the health crisis.

Role of Technology in Self-Assessment and Feedback Among Hospitalist Physicians: Semistructured Interviews and Thematic Analysis

Background

Lifelong learning is embedded in the culture of medicine, but there are limited tools currently available for many clinicians, including hospitalists, to help improve their own practice. Although there are requirements for continuing medical education, resources for learning new clinical guidelines, and developing fields aimed at facilitating peer-to-peer feedback, there is a gap in the availability of tools that enable clinicians to learn based on their own patients and clinical decisions.

Objective

The aim of this study was to explore the technologies or modifications to existing systems that could be used to benefit hospitalist physicians in pursuing self-assessment and improvement by understanding physicians’ current practices and their reactions to proposed possibilities.

Methods

Semistructured interviews were conducted in two separate stages with analysis performed after each stage. In the first stage, interviews (N=12) were conducted to understand the ways in which hospitalist physicians are currently gathering feedback and assessing their practice. A thematic analysis of these interviews informed the prototype used to elicit responses in the second stage.

Results

Clinicians actively look for feedback that they can apply to their practice, with the majority of the feedback obtained through self-assessment. The following three themes surrounding this aspect were identified in the first round of semistructured interviews: collaboration, self-reliance, and uncertainty, each with three related subthemes. Using a wireframe, the second round of interviews led to identifying the features that are currently challenging to use or could be made available with technology.

Conclusions

Based on each theme and subtheme, we provide targeted recommendations for use by relevant stakeholders such as institutions, clinicians, and technologists. Most hospitalist self-assessments occur on a rolling basis, specifically using data in electronic medical records as their primary source. Specific objective data points or subjective patient relationships lead clinicians to review their patient cases and to assess their own performance. However, current systems are not built for these analyses or for clinicians to perform self-assessment, making this a burdensome and incomplete process. Building a platform that focuses on providing and curating the information used for self-assessment could help physicians make more accurately informed changes to their own clinical practice and decision-making.