Professional Online Support Group Facilitators: Guarantors of Maximal Group Utility

Artificial Intelligence-Based Co-Facilitator (AICF) for Detecting and Monitoring Group Cohesion Outcomes in Web-Based Cancer Support Groups: Single-Arm Trial Study

BACKGROUND

Commonly offered as supportive care, therapist-led online support groups (OSGs) are a cost-effective way to provide support to individuals affected by cancer. One important indicator of a successful OSG session is group cohesion; however, monitoring group cohesion can be challenging due to the lack of nonverbal cues and in-person interactions in text-based OSGs. The Artificial Intelligence-based Co-Facilitator (AICF) was designed to contextually identify therapeutic outcomes from conversations and produce real-time analytics.

OBJECTIVE

The aim of this study was to develop a method to train and evaluate AICF's capacity to monitor group cohesion.

METHODS

AICF used a text classification approach to extract the mentions of group cohesion within conversations. A sample of data was annotated by human scorers, which was used as the training data to build the classification model. The annotations were further supported by finding contextually similar group cohesion expressions using word embedding models as well. AICF performance was also compared against the natural language processing software Linguistic Inquiry Word Count (LIWC).

RESULTS

AICF was trained on 80,000 messages obtained from Cancer Chat Canada. We tested AICF on 34,048 messages. Human experts scored 6797 (20%) of the messages to evaluate the ability of AICF to classify group cohesion. Results showed that machine learning algorithms combined with human input could detect group cohesion, a clinically meaningful indicator of effective OSGs. After retraining with human input, AICF reached an F1-score of 0.82. AICF performed slightly better at identifying group cohesion compared to LIWC.

CONCLUSIONS

AICF has the potential to assist therapists by detecting discord in the group amenable to real-time intervention. Overall, AICF presents a unique opportunity to strengthen patient-centered care in web-based settings by attending to individual needs.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/21453.

Therapist Feedback and Implications on Adoption of an Artificial Intelligence-Based Co-Facilitator for Online Cancer Support Groups: Mixed Methods Single-Arm Usability Study

BACKGROUND

The recent onset of the COVID-19 pandemic and the social distancing requirement have created an increased demand for virtual support programs. Advances in artificial intelligence (AI) may offer novel solutions to management challenges such as the lack of emotional connections within virtual group interventions. Using typed text from online support groups, AI can help identify the potential risk of mental health concerns, alert group facilitator(s), and automatically recommend tailored resources while monitoring patient outcomes.

OBJECTIVE

The aim of this mixed methods, single-arm study was to evaluate the feasibility, acceptability, validity, and reliability of an AI-based co-facilitator (AICF) among CancerChatCanada therapists and participants to monitor online support group participants' distress through a real-time analysis of texts posted during the support group sessions. Specifically, AICF (1) generated participant profiles with discussion topic summaries and emotion trajectories for each session, (2) identified participant(s) at risk for increased emotional distress and alerted the therapist for follow-up, and (3) automatically suggested tailored recommendations based on participant needs. Online support group participants consisted of patients with various types of cancer, and the therapists were clinically trained social workers.

METHODS

Our study reports on the mixed methods evaluation of AICF, including therapists' opinions as well as quantitative measures. AICF's ability to detect distress was evaluated by the patient's real-time emoji check-in, the Linguistic Inquiry and Word Count software, and the Impact of Event Scale-Revised.

RESULTS

Although quantitative results showed only some validity of AICF's ability in detecting distress, the qualitative results showed that AICF was able to detect real-time issues that are amenable to treatment, thus allowing therapists to be more proactive in supporting every group member on an individual basis. However, therapists are concerned about the ethical liability of AICF's distress detection function.

CONCLUSIONS

Future works will look into wearable sensors and facial cues by using videoconferencing to overcome the barriers associated with text-based online support groups.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/21453.

Providing Care Beyond Therapy Sessions With a Natural Language Processing-Based Recommender System That Identifies Cancer Patients Who Experience Psychosocial Challenges and Provides Self-care Support: Pilot Study

BACKGROUND

The negative psychosocial impacts of cancer diagnoses and treatments are well documented. Virtual care has become an essential mode of care delivery during the COVID-19 pandemic, and online support groups (OSGs) have been shown to improve accessibility to psychosocial and supportive care. de Souza Institute offers CancerChatCanada, a therapist-led OSG service where sessions are monitored by an artificial intelligence-based co-facilitator (AICF). The AICF is equipped with a recommender system that uses natural language processing to tailor online resources to patients according to their psychosocial needs.

OBJECTIVE

We aimed to outline the development protocol and evaluate the AICF on its precision and recall in recommending resources to cancer OSG members.

METHODS

Human input informed the design and evaluation of the AICF on its ability to (1) appropriately identify keywords indicating a psychosocial concern and (2) recommend the most appropriate online resource to the OSG member expressing each concern. Three rounds of human evaluation and algorithm improvement were performed iteratively.

RESULTS

We evaluated 7190 outputs and achieved a precision of 0.797, a recall of 0.981, and an F1 score of 0.880 by the third round of evaluation. Resources were recommended to 48 patients, and 25 (52%) accessed at least one resource. Of those who accessed the resources, 19 (75%) found them useful.

CONCLUSIONS

The preliminary findings suggest that the AICF can help provide tailored support for cancer OSG members with high precision, recall, and satisfaction. The AICF has undergone rigorous human evaluation, and the results provide much-needed evidence, while outlining potential strengths and weaknesses for future applications in supportive care.

Implementation of a Statewide Web-Based Caregiver Resource Information System (CareNav): Mixed Methods Study

BACKGROUND

With the aging population, family caregivers provide increasingly complex and intense care for older adults and persons with disabilities. There is growing interest in developing community-based services to support family caregivers. Caregiving occurs around the clock, and caregivers face challenges in accessing community-based services at convenient times owing to the demands of care. Web-based resources hold promise for accessible real-time support. CareNav (TM), a caregiver resource information system, is a web-based platform designed to support real-time universal caregiver assessment, a record of client encounters, development of a care plan, tailored information and resource content, access to web-based caregiver resources, the capacity to track service authorization and contracts, and secure communications. The assessment includes needs and health conditions of both the care recipient and caregiver; current resources; and priorities for support, information, and referral. In 2019, the California Department of Health Care Services funded the 11 nonprofit California Caregiver Resource Centers (CRCs) to expand and improve family caregiver services and enhance CRC information technology services. Deployment of a statewide information system offered a unique opportunity to examine structures and processes facilitating implementation, providing feedback to the sites as well as lessons learned for similar projects in the future.

OBJECTIVE

The aim of this paper was to describe the statewide implementation of the comprehensive CareNav system using the Consolidated Framework for Implementation Research as an organizing structure for synthesizing the evaluation.

METHODS

This mixed methods study used two major approaches to evaluate the implementation process: a survey of all staff who completed training (n=82) and in-depth qualitative interviews with 11 CRC teams and 3 key informants (n=35). We initially analyzed interview transcripts using qualitative descriptive methods and then identified subthemes and relationships among ideas, mapping the findings to the Consolidated Framework for Implementation Research.

RESULTS

We present findings on the outer setting, inner setting, characteristics of the intervention, characteristics of the staff, and the implementation process. The critical elements for success were leadership, communication, harmonization of processes across sites, and motivation to serve clients in more accessible and convenient ways.

CONCLUSIONS

These findings have implications for technology deployment in diverse community-based agencies that aspire to enhance web-based services.

'Weighing' Losses and Gains: Evaluation of the Healthy Lifestyle Modification After Breast Cancer Pilot Program

Objectives

This pilot study sought to develop and evaluate a novel online group-based intervention (Healthy Lifestyle Modification after Breast Cancer; HLM-ABC) to help breast cancer survivors (BCSs) make healthy lifestyle changes intended to yield not only beneficial physical outcomes (i.e., weight loss, reduced body mass index) but also greater behavioral (e.g., increased physical activity, healthier eating), and psychosocial well-being (e.g., self-efficacy, motivation, body image).

Methods

An exploratory single-arm, mixed-method triangulation design was employed to evaluate the feasibility and preliminary effectiveness of the HLM-ABC intervention for overweight BCSs. Fourteen women participated in the 10-week intervention and completed quantitative measures of the above-mentioned outcomes at baseline, post-treatment, 6-month, and 12-month follow-up time points. Qualitative data were obtained post-treatment via semi-structured interviews and a treatment satisfaction questionnaire.

Results

Participants lost an average of 2.83% of their baseline weight (M = 196.65; SD = 38.59) by 1-year follow-up (M = 191.29; SD = 33.91), equal to a small effect size (d = -0.37). Despite achieving only modest weight loss, participants achieved meaningful gains in the form of increased physical activity (d = 0.2), discovery of gratifying movement, more intuitive eating habits (d = 1.12), greater bodily and emotional awareness, and positive shifts in beliefs about being able to make healthy choices regarding food (d = 0.63) and physical activity (d = 0.38). Furthermore, they demonstrated a slight improvement in body image (d = 0.36) and described feeling more self-compassionate, empowered, and acknowledging of variables beyond control (i.e., hormonal therapy, unsatisfactory surgery) that can present barriers to change.

Conclusion

After completing a 10-week online program, participants achieved meaningful and lasting changes on a number of healthful indicators, even when this did not correspond with a significant reduction in weight. Findings highlight the complex, multifaceted nature of "health" and lend support for promotion of healthier lifestyle following cancer treatment that encompasses not only physical weight, but also behavior, psychosocial well-being, and (often unmodifiable) circumstances such as life-preserving hormonal treatments.

Natural Language Processing-Based Virtual Cofacilitator for Online Cancer Support Groups: Protocol for an Algorithm Development and Validation Study

BACKGROUND

Cancer and its treatment can significantly impact the short- and long-term psychological well-being of patients and families. Emotional distress and depressive symptomatology are often associated with poor treatment adherence, reduced quality of life, and higher mortality. Cancer support groups, especially those led by health care professionals, provide a safe place for participants to discuss fear, normalize stress reactions, share solidarity, and learn about effective strategies to build resilience and enhance coping. However, in-person support groups may not always be accessible to individuals; geographic distance is one of the barriers for access, and compromised physical condition (eg, fatigue, pain) is another. Emerging evidence supports the effectiveness of online support groups in reducing access barriers. Text-based and professional-led online support groups have been offered by Cancer Chat Canada. Participants join the group discussion using text in real time. However, therapist leaders report some challenges leading text-based online support groups in the absence of visual cues, particularly in tracking participant distress. With multiple participants typing at the same time, the nuances of the text messages or red flags for distress can sometimes be missed. Recent advances in artificial intelligence such as deep learning-based natural language processing offer potential solutions. This technology can be used to analyze online support group text data to track participants' expressed emotional distress, including fear, sadness, and hopelessness. Artificial intelligence allows session activities to be monitored in real time and alerts the therapist to participant disengagement.

OBJECTIVE

We aim to develop and evaluate an artificial intelligence-based cofacilitator prototype to track and monitor online support group participants' distress through real-time analysis of text-based messages posted during synchronous sessions.

METHODS

An artificial intelligence-based cofacilitator will be developed to identify participants who are at-risk for increased emotional distress and track participant engagement and in-session group cohesion levels, providing real-time alerts for therapist to follow-up; generate postsession participant profiles that contain discussion content keywords and emotion profiles for each session; and automatically suggest tailored resources to participants according to their needs. The study is designed to be conducted in 4 phases consisting of (1) development based on a subset of data and an existing natural language processing framework, (2) performance evaluation using human scoring, (3) beta testing, and (4) user experience evaluation.

RESULTS

This study received ethics approval in August 2019. Phase 1, development of an artificial intelligence-based cofacilitator, was completed in January 2020. As of December 2020, phase 2 is underway. The study is expected to be completed by September 2021.

CONCLUSIONS

An artificial intelligence-based cofacilitator offers a promising new mode of delivery of person-centered online support groups tailored to individual needs.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/21453.