Rhythmic synchrony and mediated interaction: towards a framework of rhythm in embodied interaction

Phonetic entrainment in L2 human-robot interaction: an investigation of children with and without autism spectrum disorder

Phonetic entrainment is a phenomenon in which people adjust their phonetic features to approach those of their conversation partner. Individuals with Autism Spectrum Disorder (ASD) have been reported to show some deficits in entrainment during their interactions with human interlocutors, though deficits in terms of significant differences from typically developing (TD) controls were not always registered. One reason related to the inconsistencies of whether deficits are detected or not in autistic individuals is that the conversation partner's speech could hardly be controlled, and both the participants and the partners might be adjusting their phonetic features. The variabilities in the speech of conversation partners and various social traits exhibited might make the phonetic entrainment (if any) of the participants less detectable. In this study, we attempted to reduce the variability of the interlocutors by employing a social robot and having it do a goal-directed conversation task with children with and without ASD. Fourteen autistic children and 12 TD children participated the current study in their second language English. Results showed that autistic children showed comparable vowel formants and mean fundamental frequency (f0) entrainment as their TD peers, but they did not entrain their f0 range as the TD group did. These findings suggest that autistic children were capable of exhibiting phonetic entrainment behaviors similar to TD children in vowel formants and f0, particularly in a less complex situation where the speech features and social traits of the interlocutor were controlled. Furthermore, the utilization of a social robot may have increased the interest of these children in phonetic entrainment. On the other hand, entrainment of f0 range was more challenging for these autistic children even in a more controlled situation. This study demonstrates the viability and potential of using human-robot interactions as a novel method to evaluate abilities and deficits in phonetic entrainment in autistic children.

Dyadic nonverbal synchrony during pre and post music therapy interventions and its relationship to self-reported therapy readiness

Nonverbal interpersonal synchronization has been established as an important factor in therapeutic relationships, and the differentiation of who leads the interaction appears to provide further important information. We investigated nonverbal synchrony - quantified as the coordination of body movement between patient and therapist. This was observed in music therapy dyads, while engaged in verbal interaction before and after a music intervention in the session. We further examined associations with patients' self-reported therapy readiness at the beginning of the session. Eleven neurological in-patients participated in this study. Our results showed an increase in both nonverbal synchrony and patient leading after the music intervention. A significant negative correlation was found between self-reported therapy readiness and nonverbal synchrony after the music intervention. These findings point to the empathic ability of the music therapist to sense patients' therapy readiness. Higher patient leading in nonverbal synchrony after the music intervention may thus indicate that the music intervention may have allowed dyadic entrainment to take place, potentially increasing self-regulation and thus empowering patients.

Resonance as a Design Strategy for AI and Social Robots

Resonance, a powerful and pervasive phenomenon, appears to play a major role in human interactions. This article investigates the relationship between the physical mechanism of resonance and the human experience of resonance, and considers possibilities for enhancing the experience of resonance within human-robot interactions. We first introduce resonance as a widespread cultural and scientific metaphor. Then, we review the nature of "sympathetic resonance" as a physical mechanism. Following this introduction, the remainder of the article is organized in two parts. In part one, we review the role of resonance (including synchronization and rhythmic entrainment) in human cognition and social interactions. Then, in part two, we review resonance-related phenomena in robotics and artificial intelligence (AI). These two reviews serve as ground for the introduction of a design strategy and combinatorial design space for shaping resonant interactions with robots and AI. We conclude by posing hypotheses and research questions for future empirical studies and discuss a range of ethical and aesthetic issues associated with resonance in human-robot interactions.

Aesthetic and technical strategies for networked music performance

Networked music is no longer a future genre: the global quarantine event of 2020 launched the concept of performing together over the Internet into the mainstream. While the demand for performing at a distance may be a new imperative, musicians find themselves faced with technological and performative processes that do not appear to be suitable for performing music together online, due particularly to network latency which disrupts the ability for musicians to synchronize. The research presented in this paper investigates and challenges the reasons why networked music is not readily embraced by musicians and describes how that might change, by way of interviews with practitioners and an in-depth review of the technical constraints. Limitations that might cause frustration are in fact shown to have creative strategies that give rise to aesthetic approaches, distinct to the platform. By exploiting the constraints, in tandem with developing technology designed specifically for remote performance, aesthetic implications arise that not only accommodate the inconveniences of latency and acoustic feedback but can help us adapt and transform how we engage in real-time online, towards a future where we can imagine performing together over even more dramatic distances such as high-latency, low-bandwidth locations outside of urban areas-or even over galactic distances.

Rhythmic Synchrony with Artificial Agents and Its Effects on Frequency of Visual Illusions Seen in White Noise

Rhythmic synchrony among different individuals has often been observed in various religious rituals and it has been known to bring various psychological effects in human minds. This study investigated the effects of induced rhythmic synchrony with artificial agents in drumming on participants’ visual illusions. The participants completed a task with three cartoon agents on a computer screen beating drums taking turns. We then investigated whether participants were tended to find more meaningful shapes in displayed random dots (pareidolia) when rhythms of intervals between each agents’ drumbeats were in-sync rather than out-of-sync. We simultaneously compared an active condition, in which participants took the role as one of three agents to beat a drum, with a passive condition, in which they only observed three agents beating the drums. The results showed that pareidolia appeared strongly in participants where the drum rhythm was in sync, regardless of active and passive conditions.

Using Music Technology Creatively to Enrich Later-Life: A Literature Review

Background: A growing body of evidence has demonstrated significant social and emotional benefits of music-making amongst senior citizens. However, several as-yet unresolved age-related barriers to "musicking" have been identified. Positioned within the emergent field of gerontechnology, concerned with the interface between aging and technology research, this review of literature thus explores the potential for music technologies to function as a vehicle for creative musical opportunities in later-life. Methods: ERIC, PsychInfo, and Web of Science databases were searched, focusing on the intersection between music, technology, and aging. The criteria for inclusion were that the paper should: (1) be in English; (2) report empirical research involving the use of music technologies intended to support receptive (listening, interpreting, reflecting) or active (playing, creating, performing) engagement with music amongst older persons, defined as being aged 60 years or above (United Nations, 2017); (3) be published as a peer reviewed journal article. Results: Of 144 papers screened, 18 papers were retained. 10 studies focused on using technology to support musicking in the form of listening, reflecting, and interpreting. Just five studies explored the utility of technology in promoting singing or playing instruments, while a further three were focused on music and movement. Conclusions: Overall, the literature reviewed suggests that older people, even those with complex needs, are capable of, and interested in using music technologies to access and create personally meaningful music. The limited research that does exist points to multiple and significant benefits that may be derived from receptive or active musicking supported by a range of music technologies.

Facing Sorrow as a Group Unites. Facing Sorrow in a Group Divides

Collective gatherings foster group cohesion through providing occasion for emotional sharing among participants. However, prior studies have failed to disentangle two processes that are involved in emotional sharing: 1) focusing shared attention on the same emotion-eliciting event and 2) actively sharing one’s experiences and disclosing one’s feelings to others. To date, it has remained untested if shared attention influences group cohesion independent of active emotional sharing. Our experiment investigated the effect of shared versus individual attention on cohesion in groups of strangers. We predicted that differences in group cohesion as called forth by shared vs. individual attention are most pronounced when experiencing highly arousing negative affect, in that the act of experiencing intensely negative affect with others buffers negative affect’s otherwise detrimental effect on group cohesion. Two-hundred sixteen participants were assembled in groups of 3 to 4 people to either watch an emotion-eliciting film simultaneously on a common screen or to watch the same emotion-eliciting film clip on a laptop in front of each group member using earphones. The film clips were chosen to elicit either highly arousing negative affect or one of three other affective states representing the other poles in Russel’s Circumplex model of affect. We examined self-reported affective and cognitive group cohesion and a behavioral measure of group cohesion. Results support our buffer-hypothesis, in that experiencing intense negative affect in unison leads to higher levels of group cohesion than experiencing this affect individually despite the group setting. The present study demonstrates that shared attention to intense negative emotional stimuli affects group cohesion independently of active emotional sharing.

Disordered speech disrupts conversational entrainment: a study of acoustic-prosodic entrainment and communicative success in populations with communication challenges

Conversational entrainment, a pervasive communication phenomenon in which dialogue partners adapt their behaviors to align more closely with one another, is considered essential for successful spoken interaction. While well-established in other disciplines, this phenomenon has received limited attention in the field of speech pathology and the study of communication breakdowns in clinical populations. The current study examined acoustic-prosodic entrainment, as well as a measure of communicative success, in three distinctly different dialogue groups: (i) healthy native vs. healthy native speakers (Control), (ii) healthy native vs. foreign-accented speakers (Accented), and (iii) healthy native vs. dysarthric speakers (Disordered). Dialogue group comparisons revealed significant differences in how the groups entrain on particular acoustic–prosodic features, including pitch, intensity, and jitter. Most notably, the Disordered dialogues were characterized by significantly less acoustic-prosodic entrainment than the Control dialogues. Further, a positive relationship between entrainment indices and communicative success was identified. These results suggest that the study of conversational entrainment in speech pathology will have essential implications for both scientific theory and clinical application in this domain.

Embodiment of intersubjective time: relational dynamics as attractors in the temporal coordination of interpersonal behaviors and experiences

This paper addresses the issue of “being together,” and more specifically the issue of “being together in time.” We provide with an integrative framework that is inspired by phenomenology, the enactive approach and dynamical systems theories. To do so, we first define embodiment as a living and lived phenomenon that emerges from agent-world coupling. We then show that embodiment is essentially dynamical and therefore we describe experiential, behavioral and brain dynamics. Both lived temporality and the temporality of the living appear to be complex, multiscale phenomena. Next we discuss embodied dynamics in the context of interpersonal interactions, and briefly review the empirical literature on between-persons temporal coordination. Overall, we propose that being together in time emerges from the relational dynamics of embodied interactions and their flexible co-regulation.

Rhythm as a coordinating device: entrainment with disordered speech

PURPOSE

The rhythmic entrainment (coordination) of behavior during human interaction is a powerful phenomenon, considered essential for successful communication, supporting social and emotional connection, and facilitating sense-making and information exchange. Disruption in entrainment likely occurs in conversations involving those with speech and language impairment, but its contribution to communication disorders has not been defined. As a first step to exploring this phenomenon in clinical populations, the present investigation examined the influence of disordered speech on the speech production properties of healthy interactants.

METHOD

Twenty-nine neurologically healthy interactants participated in a quasi-conversational paradigm, in which they read sentences (response) in response to hearing prerecorded sentences (exposure) from speakers with dysarthria (n = 4) and healthy controls (n = 4). Recordings of read sentences prior to the task were also collected (habitual).

RESULTS

Findings revealed that interactants modified their speaking rate and pitch variation to align more closely with the disordered speech. Production shifts in these rhythmic properties, however, remained significantly different from corresponding properties in dysarthric speech.

CONCLUSION

Entrainment offers a new avenue for exploring speech and language impairment, addressing a communication process not currently explained by existing frameworks. This article offers direction for advancing this line of inquiry.

Sensorimotor synchronization: a review of recent research (2006-2012)

Sensorimotor synchronization (SMS) is the coordination of rhythmic movement with an external rhythm, ranging from finger tapping in time with a metronome to musical ensemble performance. An earlier review (Repp, 2005) covered tapping studies; two additional reviews (Repp, 2006a, b) focused on music performance and on rate limits of SMS, respectively. The present article supplements and extends these earlier reviews by surveying more recent research in what appears to be a burgeoning field. The article comprises four parts, dealing with (1) conventional tapping studies, (2) other forms of moving in synchrony with external rhythms (including dance and nonhuman animals' synchronization abilities), (3) interpersonal synchronization (including musical ensemble performance), and (4) the neuroscience of SMS. It is evident that much new knowledge about SMS has been acquired in the last 7 years.