Predicting couple therapy outcomes based on speech acoustic features

Prediction of Chinese clients' satisfaction with psychotherapy by machine learning

Background

Effective psychotherapy should satisfy the client, but that satisfaction depends on many factors. We do not fully understand the factors that affect client satisfaction with psychotherapy and how these factors synergistically affect a client's psychotherapy experience.

Aims

This study aims to use machine learning to predict Chinese clients' satisfaction with psychotherapy and analyze potential outcome contributors.

Methods

In this cross-sectional investigation, a self-compiled online questionnaire was delivered through the WeChat app. The information of 791 participants who had received psychotherapy was used in the study. A series of features, for example, the participants' demographic features and psychotherapy-related features, were chosen to distinguish between participants satisfied and dissatisfied with the psychotherapy they received. With our dataset, we trained seven supervised machine-learning-based algorithms to implement prediction models.

Results

Among the 791 participants, 619 (78.3%) reported being satisfied with the psychotherapy sessions that they received. The occupation of the clients, the location of psychotherapy, and the form of access to psychotherapy are the three most recognizable features that determined whether clients are satisfied with psychotherapy. The machine-learning model based on the CatBoost achieved the highest prediction performance in classifying satisfied and psychotherapy clients with an F1 score of 0.758.

Conclusion

This study clarified the factors related to clients' satisfaction with psychotherapy, and the machine-learning-based classifier accurately distinguished clients who were satisfied or unsatisfied with psychotherapy. These results will help provide better psychotherapy strategies for specific clients, so they may achieve better therapeutic outcomes.

A Machine Learning Approach for Predicting Non-Suicidal Self-Injury in Young Adults

Artificial intelligence techniques were explored to assess the ability to anticipate self-harming behaviour in the mental health context using a database collected by an app previously designed to record the emotional states and activities of a group of subjects exhibiting self-harm. Specifically, the Leave-One-Subject-Out technique was used to train classification trees with a maximum of five splits. The results show an accuracy of 84.78%, a sensitivity of 64.64% and a specificity of 85.53%. In addition, positive and negative predictive values were also obtained, with results of 14.48% and 98.47%, respectively. These results are in line with those reported in previous work using a multilevel mixed-effect regression analysis. The combination of apps and AI techniques is a powerful way to improve the tools to accompany and support the care and treatment of patients with this type of behaviour. These studies also guide the improvement of apps on the user side, simplifying and collecting more meaningful data, and on the therapist side, progressing in pathology treatments. Traditional therapy involves observing and reconstructing what had happened before episodes once they have occurred. This new generation of tools will make it possible to monitor the pathology more closely and to act preventively.

Vocal modulation in human mating and competition

The human voice is dynamic, and people modulate their voices across different social interactions. This article presents a review of the literature examining natural vocal modulation in social contexts relevant to human mating and intrasexual competition. Altering acoustic parameters during speech, particularly pitch, in response to mating and competitive contexts can influence social perception and indicate certain qualities of the speaker. For instance, a lowered voice pitch is often used to exert dominance, display status and compete with rivals. Changes in voice can also serve as a salient medium for signalling a person's attraction to another, and there is evidence to support the notion that attraction and/or romantic interest can be distinguished through vocal tones alone. Individuals can purposely change their vocal behaviour in attempt to sound more attractive and to facilitate courtship success. Several findings also point to the effectiveness of vocal change as a mechanism for communicating relationship status. As future studies continue to explore vocal modulation in the arena of human mating, we will gain a better understanding of how and why vocal modulation varies across social contexts and its impact on receiver psychology. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part I)'.

Ensuring patient and public involvement in the transition to AI-assisted mental health care: A systematic scoping review and agenda for design justice

Background

Machine‐learning algorithms and big data analytics, popularly known as ‘artificial intelligence’ (AI), are being developed and taken up globally. Patient and public involvement (PPI) in the transition to AI‐assisted health care is essential for design justice based on diverse patient needs.

Objective

To inform the future development of PPI in AI‐assisted health care by exploring public engagement in the conceptualization, design, development, testing, implementation, use and evaluation of AI technologies for mental health.

Methods

Systematic scoping review drawing on design justice principles, and (i) structured searches of Web of Science (all databases) and Ovid (MEDLINE, PsycINFO, Global Health and Embase); (ii) handsearching (reference and citation tracking); (iii) grey literature; and (iv) inductive thematic analysis, tested at a workshop with health researchers.

Results

The review identified 144 articles that met inclusion criteria. Three main themes reflect the challenges and opportunities associated with PPI in AI‐assisted mental health care: (a) applications of AI technologies in mental health care; (b) ethics of public engagement in AI‐assisted care; and (c) public engagement in the planning, development, implementation, evaluation and diffusion of AI technologies.

Conclusion

The new data‐rich health landscape creates multiple ethical issues and opportunities for the development of PPI in relation to AI technologies. Further research is needed to understand effective modes of public engagement in the context of AI technologies, to examine pressing ethical and safety issues and to develop new methods of PPI at every stage, from concept design to the final review of technology in practice. Principles of design justice can guide this agenda.

Observational and Accelerometer Analysis of Head Movement Patterns in Psychotherapeutic Dialogue

Psychotherapists, who use their communicative skills to assist people, review their dialogue practices and improve their skills from their experiences. However, technology has not been fully exploited for this purpose. In this study, we analyze the use of head movements during actual psychotherapeutic dialogues between two participants-therapist and client-using video recordings and head-mounted accelerometers. Accelerometers have been utilized in the mental health domain but not for analyzing mental health related communications. We examined the relationship between the state of the interaction and temporally varying head nod and movement patterns in psychological counseling sessions. Head nods were manually annotated and the head movements were measured using accelerometers. Head nod counts were analyzed based on annotations taken from video data. We conducted cross-correlation analysis of the head movements of the two participants using the accelerometer data. The results of two case studies suggest that upward and downward head nod count patterns may reflect stage transitions in counseling dialogues and that peaks of head movement synchrony may be related to emphasis in the interaction.

The influence of algorithms on political and dating decisions

Artificial intelligence algorithms are ubiquitous in daily life, and this is motivating the development of some institutional initiatives to ensure trustworthiness in Artificial Intelligence (AI). However, there is not enough research on how these algorithms can influence people's decisions and attitudes. The present research examines whether algorithms can persuade people, explicitly or covertly, on whom to vote and date, or whether, by contrast, people would reject their influence in an attempt to confirm their personal freedom and independence. In four experiments, we found that persuasion was possible and that different styles of persuasion (e.g., explicit, covert) were more effective depending on the decision context (e.g., political and dating). We conclude that it is important to educate people against trusting and following the advice of algorithms blindly. A discussion on who owns and can use the data that makes these algorithms work efficiently is also necessary.

A scoping review of machine learning in psychotherapy research

Machine learning (ML) offers robust statistical and probabilistic techniques that can help to make sense of large amounts of data. This scoping review paper aims to broadly explore the nature of research activity using ML in the context of psychological talk therapies, highlighting the scope of current methods and considerations for clinical practice and directions for future research. Using a systematic search methodology, fifty-one studies were identified. A narrative synthesis indicates two types of studies, those who developed and tested an ML model (k=44), and those who reported on the feasibility of a particular treatment tool that uses an ML algorithm (k=7). Most model development studies used supervised learning techniques to classify or predict labeled treatment process or outcome data, whereas others used unsupervised techniques to identify clusters in the unlabeled patient or treatment data. Overall, the current applications of ML in psychotherapy research demonstrated a range of possible benefits for indications of treatment process, adherence, therapist skills and treatment response prediction, as well as ways to accelerate research through automated behavioral or linguistic process coding. Given the novelty and potential of this research field, these proof-of-concept studies are encouraging, however, do not necessarily translate to improved clinical practice (yet).

Supervised Speaker Diarization Using Random Forests: A Tool for Psychotherapy Process Research

Speaker diarization is the practice of determining who speaks when in audio recordings. Psychotherapy research often relies on labor intensive manual diarization. Unsupervised methods are available but yield higher error rates. We present a method for supervised speaker diarization based on random forests. It can be considered a compromise between commonly used labor-intensive manual coding and fully automated procedures. The method is validated using the EMRAI synthetic speech corpus and is made publicly available. It yields low diarization error rates (M: 5.61%, STD: 2.19). Supervised speaker diarization is a promising method for psychotherapy research and similar fields.

Spoken words as biomarkers: using machine learning to gain insight into communication as a predictor of anxiety

OBJECTIVE

The goal of this study was to explore whether features of recorded and transcribed audio communication data extracted by machine learning algorithms can be used to train a classifier for anxiety.

MATERIALS AND METHODS

We used a secondary data set generated by a clinical trial examining problem-solving therapy for hospice caregivers consisting of 140 transcripts of multiple, sequential conversations between an interviewer and a family caregiver along with standardized assessments of anxiety prior to each session; 98 of these transcripts (70%) served as the training set, holding the remaining 30% of the data for evaluation.

RESULTS

A classifier for anxiety was developed relying on language-based features. An 86% precision, 78% recall, 81% accuracy, and 84% specificity were achieved with the use of the trained classifiers. High anxiety inflections were found among recently bereaved caregivers and were usually connected to issues related to transitioning out of the caregiving role. This analysis highlighted the impact of lowering anxiety by increasing reciprocity between interviewers and caregivers.

CONCLUSION

Verbal communication can provide a platform for machine learning tools to highlight and predict behavioral health indicators and trends.

The Trust Game for Couples (TGC): A new standardized paradigm to assess trust in romantic relationships

Trust between couples is a prerequisite for stable and satisfactory romantic relationships. However, there has been no valid research tool to assess partner-specific trust behavior including costly investments in the trustworthiness of the romantic partner. We here present a comprehensive validation of the newly developed Trust Game for Couples (TGC) by means of various self-report and implicit relationship-related measures. The TGC operationalizes trust by measuring an individual's willingness to invest his or her own financial resources in pro-relationship attitudes of their romantic partner (collected by dichotomous responses to relationship-relevant items, e.g., answering yes to "I am absolutely sure that I love my partner"). Thirty-five healthy couples between 20 and 34 years completed the TGC in an interactive (both partners present), but anonymous setting (no information on the partner's responses revealed). Trust, as measured by the TGC, correlates positively with self-reported trust, satisfaction, and felt closeness in the relationship, but not with general interpersonal trust, confirming both its convergent and discriminant validity. In addition to explicit criteria for construct validity, implicit measures of partner valence and confidence explained variance in the TGC, demonstrating that it constitutes an economical measure of implicit and explicit ingredients of trust between couples. In sum, the TGC provides a novel, specific behavioral tool for a sensitive assessment of trust in dyadic relationships with potential for numerous research fields.

Linking emotions to behaviors through deep transfer learning

Human behavior refers to the way humans act and interact. Understanding human behavior is a cornerstone of observational practice, especially in psychotherapy. An important cue of behavior analysis is the dynamical changes of emotions during the conversation. Domain experts integrate emotional information in a highly nonlinear manner; thus, it is challenging to explicitly quantify the relationship between emotions and behaviors. In this work, we employ deep transfer learning to analyze their inferential capacity and contextual importance. We first train a network to quantify emotions from acoustic signals and then use information from the emotion recognition network as features for behavior recognition. We treat this emotion-related information as behavioral primitives and further train higher level layers towards behavior quantification. Through our analysis, we find that emotion-related information is an important cue for behavior recognition. Further, we investigate the importance of emotional-context in the expression of behavior by constraining (or not) the neural networks' contextual view of the data. This demonstrates that the sequence of emotions is critical in behavior expression. To achieve these frameworks we employ hybrid architectures of convolutional networks and recurrent networks to extract emotion-related behavior primitives and facilitate automatic behavior recognition from speech.

Machine learning meets partner matching: Predicting the future relationship quality based on personality traits

To what extent is it possible to use machine learning to predict the outcome of a relationship, based on the personality of both partners? In the present study, relationship satisfaction, conflicts, and separation (intents) of 192 partners four years after the completion of questionnaires concerning their personality traits was predicted. A 10x10-fold cross-validation was used to ensure that the results of the linear regression models are reproducible. The findings indicate that machine learning techniques can improve the prediction of relationship quality (37% of variance explained), and that the perceived relationship quality of a partner is mostly dependent on his or her own individual personality traits. Additionally, the influences of different sets of variables on predictions are shown: partner and similarity effects did not incrementally predict relationship quality beyond actor effects and general personality traits predicted relationship quality less strongly than relationship-related personality.

Syncing Up for a Good Conversation: A Clinically Meaningful Methodology for Capturing Conversational Entrainment in the Speech Domain

Purpose Conversational entrainment, the phenomenon whereby communication partners synchronize their behavior, is considered essential for productive and fulfilling conversation. Lack of entrainment could, therefore, negatively impact conversational success. Although studied in many disciplines, entrainment has received limited attention in the field of speech-language pathology, where its implications may have direct clinical relevance. Method A novel computational methodology, informed by expert clinical assessment of conversation, was developed to investigate conversational entrainment across multiple speech dimensions in a corpus of experimentally elicited conversations involving healthy participants. The predictive relationship between the methodology output and an objective measure of conversational success, communicative efficiency, was then examined. Results Using a real versus sham validation procedure, we find evidence of sustained entrainment in rhythmic, articulatory, and phonatory dimensions of speech. We further validate the methodology, showing that models built on speech signal entrainment measures consistently outperform models built on nonentrained speech signal measures in predicting communicative efficiency of the conversations. Conclusions A multidimensional, clinically meaningful methodology for capturing conversational entrainment, validated in healthy populations, has implications for disciplines such as speech-language pathology where conversational entrainment represents a critical knowledge gap in the field, as well as a potential target for remediation.

Everyday couples' communication research: Overcoming methodological barriers with technology

Relationship behaviors contribute to compromised health or resilience. Everyday communication between intimate partners represents the vast majority of their interactions. When intimate partners take on new roles as patients and caregivers, everyday communication takes on a new and important role in managing both the transition and the adaptation to the change in health status. However, everyday communication and its relation to health has been little studied, likely due to barriers in collecting and processing this kind of data. The goal of this paper is to describe deterrents to capturing naturalistic, day-in-the-life communication data and share how technological advances have helped surmount them. We provide examples from a current study and describe how we anticipate technology will further change research capabilities.