Provision of Care by "Real World" Telemental Health Providers

Machine Learning Model to Predict Assignment of Therapy Homework in Behavioral Treatments: Algorithm Development and Validation

BACKGROUND

Therapeutic homework is a core element of cognitive and behavioral interventions, and greater homework compliance predicts improved treatment outcomes. To date, research in this area has relied mostly on therapists' and clients' self-reports or studies carried out in academic settings, and there is little knowledge on how homework is used as a treatment intervention in routine clinical care.

OBJECTIVE

This study tested whether a machine learning (ML) model using natural language processing could identify homework assignments in behavioral health sessions. By leveraging this technology, we sought to develop a more objective and accurate method for detecting the presence of homework in therapy sessions.

METHODS

We analyzed 34,497 audio-recorded treatment sessions provided in 8 behavioral health care programs via an artificial intelligence (AI) platform designed for therapy provided by Eleos Health. Therapist and client utterances were captured and analyzed via the AI platform. Experts reviewed the homework assigned in 100 sessions to create classifications. Next, we sampled 4000 sessions and labeled therapist-client microdialogues that suggested homework to train an unsupervised sentence embedding model. This model was trained on 2.83 million therapist-client microdialogues.

RESULTS

An analysis of 100 random sessions found that homework was assigned in 61% (n=61) of sessions, and in 34% (n=21) of these cases, more than one homework assignment was provided. Homework addressed practicing skills (n=34, 37%), taking action (n=26, 28.5%), journaling (n=17, 19%), and learning new skills (n=14, 15%). Our classifier reached a 72% F₁-score, outperforming state-of-the-art ML models. The therapists reviewing the microdialogues agreed in 90% (n=90) of cases on whether or not homework was assigned.

CONCLUSIONS

The findings of this study demonstrate the potential of ML and natural language processing to improve the detection of therapeutic homework assignments in behavioral health sessions. Our findings highlight the importance of accurately capturing homework in real-world settings and the potential for AI to support therapists in providing evidence-based care and increasing fidelity with science-backed interventions. By identifying areas where AI can facilitate homework assignments and tracking, such as reminding therapists to prescribe homework and reducing the charting associated with homework, we can ultimately improve the overall quality of behavioral health care. Additionally, our approach can be extended to investigate the impact of homework assignments on therapeutic outcomes, providing insights into the effectiveness of specific types of homework.

Provider Perspectives on Telemental Health Usage After COVID-19: A Retrospective Analysis (Preprint)

Background

Mental health care pivoted to telemedicine during the COVID-19 pandemic, and there is uncertainty around the sustainability of this rapid shift.

Objective

This study examined how intentions to continue using telemedicine after the COVID-19 pandemic are influenced by provider perceptions of usefulness, ease of use, and professional social influence, facilitating organizational conditions.

Methods

We conducted a web-based, cross-sectional survey of 369 telemental health providers between February and March 2021. A hierarchical linear regression analysis was conducted to predict intentions to continue using telemedicine after the COVID-19 pandemic.

Results

Most providers began using telemedicine in March 2020 or later (257/369, 69.6%) and attended to ≥50% of their clients via telemedicine (299/369, 81.0%). Intention to continue using telemedicine after the COVID-19 pandemic was predicted by the telemedicine caseload (β=.10; P=.005), perceived usefulness in general (β=.10; P=.008), ease of use (β=.08; P=.04), social influence (β=.68; P<.001), and facilitating conditions (β=.08; P=.047).

Conclusions

Exploration of the predictors of telemedicine usage beyond the COVID-19 pandemic aids in surveillance of telemedicine usage, integration with future clinic workflows, and the shaping of public policy. It is important to consider telemedicine services as not only a response to a crisis but also an effective and useful solution for everyday life. Our results suggest widespread, sustainable telemedicine adoption.

Perceptions of Telemental Health Care Delivery During COVID-19: A Cross-Sectional Study With Providers, February-March 2021

The COVID-19 pandemic accelerated adoption of telemental health (TMH). Providers with limited TMH experience faced challenges during the rapid switch to remote patient care. We investigated TMH providers' perceptions about remote care one year into the pandemic according to when providers adopted telemedicine (i.e., before vs. after March 2020) and how much of their caseloads were served remotely (i.e., < 50% vs. ≥ 50%). Between February-March 2021, 472 TMH providers completed a cross-sectional, web-based survey that measured perceived benefits and satisfaction with telemedicine, therapeutic alliance, patient-centered communication, eHealth literacy, multicultural counseling self-efficacy, and facilitating factors of using telemedicine. Providers who began using telemedicine before the pandemic reported having better training, task-related therapeutic alliance with patients, and ability to conduct multicultural interventions, assessments, and session management. Providers who served ≥ 50% of their caseload remotely reported greater satisfaction with their practice, stronger beliefs about the benefits of telemedicine, and greater perceived effects of telemedicine on alleviating the impact of COVID-19. There were no differences in reports of patient-centered communication nor eHealth literacy. In conclusion, providers who adopted TMH more recently may require additional training and support to successfully establish a working alliance with their patients, especially with multicultural aspects of care.