Automated Detection of Conversational Pauses from Audio Recordings of Serious Illness Conversations in Natural Hospital Settings

An Acoustical and Lexical Machine-Learning Pipeline to Identify Connectional Silences

Context: Developing scalable methods for conversation analytics is essential for health care communication science and quality improvement. Purpose: To assess the feasibility of automating the identification of a conversational feature, Connectional Silence, which is associated with important patient outcomes. Methods: Using audio recordings from the Palliative Care Communication Research Initiative cohort study, we develop and test an automated measurement pipeline comprising three machine-learning (ML) tools-a random forest algorithm and a custom convolutional neural network that operate in parallel on audio recordings, and subsequently a natural language processing algorithm that uses brief excerpts of automated speech-to-text transcripts. Results: Our ML pipeline identified Connectional Silence with an overall sensitivity of 84% and specificity of 92%. For Emotional and Invitational subtypes, we observed sensitivities of 68% and 67%, and specificities of 95% and 97%, respectively. Conclusion: These findings support the capacity for coordinated and complementary ML methods to fully automate the identification of Connectional Silence in natural hospital-based clinical conversations.

Applications of Machine Learning in Palliative Care: A Systematic Review

Objective: To summarize the available literature on using machine learning (ML) for palliative care practice as well as research and to assess the adherence of the published studies to the most important ML best practices. Methods: The MEDLINE database was searched for the use of ML in palliative care practice or research, and the records were screened according to PRISMA guidelines. Results: In total, 22 publications using machine learning for mortality prediction (n = 15), data annotation (n = 5), predicting morbidity under palliative therapy (n = 1), and predicting response to palliative therapy (n = 1) were included. Publications used a variety of supervised or unsupervised models, but mostly tree-based classifiers and neural networks. Two publications had code uploaded to a public repository, and one publication uploaded the dataset. Conclusions: Machine learning in palliative care is mainly used to predict mortality. Similarly to other applications of ML, external test sets and prospective validations are the exception.

The use of natural language processing in palliative care research: A scoping review

BACKGROUND

Natural language processing has been increasingly used in palliative care research over the last 5 years for its versatility and accuracy.

AIM

To evaluate and characterize natural language processing use in palliative care research, including the most commonly used natural language processing software and computational methods, data sources, trends in natural language processing use over time, and palliative care topics addressed.

DESIGN

A scoping review using the framework by Arksey and O'Malley and the updated recommendations proposed by Levac et al. was conducted.

SOURCES

PubMed, Web of Science, Embase, Scopus, and IEEE Xplore databases were searched for palliative care studies that utilized natural language processing tools. Data on study characteristics and natural language processing instruments used were collected and relevant palliative care topics were identified.

RESULTS

197 relevant references were identified. Of these, 82 were included after full-text review. Studies were published in 48 different journals from 2007 to 2022. The average sample size was 21,541 (median 435). Thirty-two different natural language processing software and 33 machine-learning methods were identified. Nine main sources for data processing and 15 main palliative care topics across the included studies were identified. The most frequent topic was mortality and prognosis prediction. We also identified a trend where natural language processing was frequently used in analyzing clinical serious illness conversations extracted from audio recordings.

CONCLUSIONS

We found 82 papers on palliative care using natural language processing methods for a wide-range of topics and sources of data that could expand the use of this methodology. We encourage researchers to consider incorporating this cutting-edge research methodology in future studies to improve published palliative care data.

Epidemiology of Connectional Silence in specialist serious illness conversations

CONTEXT

Human connection can reduce suffering and facilitate meaningful decision-making amid the often terrifying experience of hospitalization for advanced cancer. Some conversational pauses indicate human connection, but we know little about their prevalence, distribution or association with outcomes.

PURPOSE

To describe the epidemiology of Connectional Silence during serious illness conversations in advanced cancer.

METHODS

We audio-recorded 226 inpatient palliative care consultations at two academic centers. We identified pauses lasting 2+ seconds and distinguished Connectional Silences from other pauses, sub-categorized as either Invitational (ICS) or Emotional (ECS). We identified treatment decisional status pre-consultation from medical records and post-consultation via clinicians. Patients self-reported quality-of-life before and one day after consultation.

RESULTS

Among all 6769 two-second silences, we observed 328 (4.8%) ECS and 240 (3.5%) ICS. ECS prevalence was associated with decisions favoring fewer disease-focused treatments (OR_adj: 2.12; 95% CI: 1.12, 4.06). Earlier conversational ECS was associated with improved quality-of-life (p = 0.01). ICS prevalence was associated with clinicians' prognosis expectations.

CONCLUSIONS

Connectional Silences during specialist serious illness conversations are associated with decision-making and improved patient quality-of-life. Further work is necessary to evaluate potential causal relationships.

PRACTICE IMPLICATIONS

Pauses offer important opportunities to advance the science of human connection in serious illness decision-making.

Conversational stories & self organizing maps: Innovations for the scalable study of uncertainty in healthcare communication

BACKGROUND

Understanding uncertainty in participatory decision-making requires scientific attention to interaction between what actually happens when patients, families and clinicians engage one another in conversation and the multi-level contexts in which these occur. Achieving this understanding will require conceptually grounded and scalable methods for use in large samples of people representing diversity in cultures, speaking and decision-making norms, and clinical situations.

DISCUSSION

Here, we focus on serious illness and describe Conversational Stories as a scalable and conceptually grounded framework for characterizing uncertainty expression in these clinical contexts. Using actual conversations from a large direct-observation cohort study, we demonstrate how natural language processing and unsupervised machine learning methods can reveal underlying types of uncertainty stories in serious illness conversations.

CONCLUSIONS

Conversational Storytelling offers a meaningful analytic framework for scalable computational methods to study uncertainty in healthcare conversations.

Using natural language processing to explore heterogeneity in moral terminology in palliative care consultations

Background

High quality serious illness communication requires good understanding of patients’ values and beliefs for their treatment at end of life. Natural Language Processing (NLP) offers a reliable and scalable method for measuring and analyzing value- and belief-related features of conversations in the natural clinical setting. We use a validated NLP corpus and a series of statistical analyses to capture and explain conversation features that characterize the complex domain of moral values and beliefs. The objective of this study was to examine the frequency, distribution and clustering of morality lexicon expressed by patients during palliative care consultation using the Moral Foundations NLP Dictionary.

Methods

We used text data from 231 audio-recorded and transcribed inpatient PC consultations and data from baseline and follow-up patient questionnaires at two large academic medical centers in the United States. With these data, we identified different moral expressions in patients using text mining techniques. We used latent class analysis to explore if there were qualitatively different underlying patterns in the PC patient population. We used Poisson regressions to analyze if individual patient characteristics, EOL preferences, religion and spiritual beliefs were associated with use of moral terminology.

Results

We found two latent classes: a class in which patients did not use many expressions of morality in their PC consultations and one in which patients did. Age, race (white), education, spiritual needs, and whether a patient was affiliated with Christianity or another religion were all associated with membership of the first class. Gender, financial security and preference for longevity-focused over comfort focused treatment near EOL did not affect class membership.

Conclusions

This study is among the first to use text data from a real-world situation to extract information regarding individual foundations of morality. It is the first to test empirically if individual moral expressions are associated with individual characteristics, attitudes and emotions.

Identifying Connectional Silence in Palliative Care Consultations: A Tandem Machine-Learning and Human Coding Method

Background: Systematic measurement of conversational features in the natural clinical setting is essential to better understand, disseminate, and incentivize high quality serious illness communication. Advances in machine-learning (ML) classification of human speech offer exceptional opportunity to complement human coding (HC) methods for measurement in large scale studies. Objectives: To test the reliability, efficiency, and sensitivity of a tandem ML-HC method for identifying one feature of clinical importance in serious illness conversations: Connectional Silence. Design: This was a cross-sectional analysis of 354 audio-recorded inpatient palliative care consultations from the Palliative Care Communication Research Initiative multisite cohort study. Setting/Subjects: Hospitalized people with advanced cancer. Measurements: We created 1000 brief audio "clips" of randomly selected moments predicted by a screening ML algorithm to be two-second or longer pauses in conversation. Each clip included 10 seconds of speaking before and 5 seconds after each pause. Two HCs independently evaluated each clip for Connectional Silence as operationalized from conceptual taxonomies of silence in serious illness conversations. HCs also evaluated 100 minutes from 10 additional conversations having unique speakers to identify how frequently the ML screening algorithm missed episodes of Connectional Silence. Results: Connectional Silences were rare (5.5%) among all two-second or longer pauses in palliative care conversations. Tandem ML-HC demonstrated strong reliability (kappa 0.62; 95% confidence interval: 0.47-0.76). HC alone required 61% more time than the Tandem ML-HC method. No Connectional Silences were missed by the ML screening algorithm. Conclusions: Tandem ML-HC methods are reliable, efficient, and sensitive for identifying Connectional Silence in serious illness conversations.