Prospects for Augmenting Team Interactions with Real-Time Coordination-Based Measures in Human-Autonomy Teams

Complex work in teams requires coordination across team members and their technology as well as the ability to change and adapt over time to achieve effective performance. To support such complex interactions, recent efforts have worked toward the design of adaptive human-autonomy teaming systems that can provide feedback in or near real time to achieve the desired individual or team results. However, while significant advancements have been made to better model and understand the dynamics of team interaction and its relationship with task performance, appropriate measures of team coordination and computational methods to detect changes in coordination have not yet been widely investigated. Having the capacity to measure coordination in real time is quite promising as it provides the opportunity to provide adaptive feedback that may influence and regulate teams' coordination patterns and, ultimately, drive effective team performance. A critical requirement to reach this potential is having the theoretical and empirical foundation from which to do so. Therefore, the first goal of the paper is to review approaches to coordination dynamics, identify current research gaps, and draw insights from other areas, such as social interaction, relationship science, and psychotherapy. The second goal is to collate extant work on feedback and advance ideas for adaptive feedback systems that have potential to influence coordination in a way that can enhance the effectiveness of team interactions. In addressing these two goals, this work lays the foundation as well as plans for the future of human-autonomy teams that augment team interactions using coordination-based measures.

Team Situation Awareness, Cohesion, and Autonomic Synchrony

OBJECTIVE

This study evaluated the causal relationships among situation awareness (SA), cohesion, and autonomic synchrony (SE) within teams. SA is often a team effort and should be more accurate in better-functioning teams.

BACKGROUND

Cohesive teams perform better overall, although the relationship appears reciprocal; the relationship to SA has not been considered previously. SE is a collective neurocognitive activity that has been connected to team coordination, communication, and performance in some circumstances.

METHOD

In this experiment, 71 undergraduates, organized into 16 teams, played two matches of a first-person shooter computer game and completed self-report measures of cohesion and SA. SE was determined through time series analysis of electrodermal responses using the driver-empath framework.

RESULTS

Empaths and those who came from more synchronized teams reported less cohesion in the team. Granger causality regression showed reciprocal relations among SA, SE, and cohesion that were both positive and negative after controlling for match difficulty.

CONCLUSION

The cohesion-SA relationship is similar to the reciprocal cohesion-performance relationship. SE plays an important and independent role in both the social and cognitive aspects of team behavior. It is possible, furthermore, that individuals who are more attuned to their co-workers reported a more accurate, and less obliging, social situation.

APPLICATION

Results are applicable to situations requiring teamwork in a dynamic environment.

Using interbrain synchrony to study teamwork: A systematic review and meta-analysis

It has been proposed that interbrain synchrony (IBS) may help to elucidate the neural mechanisms underpinning teamwork. As hyperscanning studies have provided abundant findings on IBS in team environments, the current review aims to synthesize the findings of hyperscanning studies in a way that is relevant to the teamwork research. A systematic review was conducted. Included studies were classified according to the IPO (i.e. input, process, output) model of teamwork. Three multi-level meta-analyses were performed to quantify the associations between IBS and the three IPO variables. The methodology followed PRISMA guidelines and the protocol was pre-registered (https://osf.io/7h8sa/). Of the 229 studies, 41 were included, representing 1326 teams. The three meta-analyses found statistically significant positive effects, indicating a positive association between IBS and the three IPO teamwork variables. This study provides evidence that IBS is a relevant measure of the teamwork process and argues for the continued use of IBS to study teamwork.

Validating computer-generated measures of linguistic style matching and accommodation in patient-clinician communication

OBJECTIVE

To explore the validity of computer-analyzed linguistic style matching (LSM) in patient-clinician communication.

METHODS

Using 330 transcribed HIV patient encounters, we quantified word use with Linguistic Inquiry and Word Count (LIWC), a dictionary-based text analysis software. We measured LSM by calculating the degree to which clinicians matched patients in the use of LIWC "function words" (e.g., articles, pronouns). We tested associations of different LSM metrics with patients' perceptions that their clinicians spoke similiarly to them.

RESULTS

We developed 3 measures of LSM: 1) at the whole-visit level; (2) at the turn-by-turn level; and (3) using a "rolling-window" approach, measuring matching between clusters of 8 turns per conversant. None of these measures was associated with patient-rated speech similarity. However, we found that increasing trajectories of LSM, from beginning to end of the visit, were associated with higher patient-rated speech similarity (β 0.35, CI 0.06, 0.64), compared to unchanging trajectories.

CONCLUSIONS

Our findings point to the potential value of clinicians' adapting their communication style to match their patients, over the course of the visit.

PRACTICE IMPLICATIONS

With further validation, computer-based linguistic analyses may prove an efficient tool for generating data on communication patterns and providing feedback to clinicians in real time.

Social and nonlinear dynamics unite: musical group synchrony

Synchronization, the human tendency to align behaviors in time with others, is necessary for many survival skills. The ability to synchronize actions with rhythmic (predictable) sound patterns is especially well developed in music making. Recent models of synchrony in musical ensembles rely on pairwise comparisons between group members. This pairwise approach to synchrony has hampered theory development, given current findings from social dynamics indicating shifts in members' influence within larger groups. We draw on social theory and nonlinear dynamics to argue that emergent properties and novel roles arise in musical group synchrony that differ from individual or pairwise behaviors. This transformational shift in defining synchrony sheds light on successful outcomes as well as on disruptions that cause negative behavioral outcomes.

Exploring objective measures for assessing team performance in healthcare: an interview study

Introduction

Effective teamwork plays a critical role in achieving high-performance outcomes in healthcare. Consequently, conducting a comprehensive assessment of team performance is essential for providing meaningful feedback during team trainings and enabling comparisons in scientific studies. However, traditional methods like self-reports or behavior observations have limitations such as susceptibility to bias or being resource consuming. To overcome these limitations and gain a more comprehensive understanding of team processes and performance, the assessment of objective measures, such as physiological parameters, can be valuable. These objective measures can complement traditional methods and provide a more holistic view of team performance. The aim of this study was to explore the potential of the use of objective measures for evaluating team performance for research and training purposes. For this, experts in the field of research and medical simulation training were interviewed to gather their opinions, ideas, and concerns regarding this novel approach.

Methods

A total of 34 medical and research experts participated in this exploratory qualitative study, engaging in semi-structured interviews. During the interview, experts were asked for (a) their opinion on measuring team performance with objective measures, (b) their ideas concerning potential objective measures suitable for measuring team performance of healthcare teams, and (c) their concerns regarding the use of objective measures for evaluating team performance. During data analysis responses were categorized per question.

Results

The findings from the 34 interviews revealed a predominantly positive reception of the idea of utilizing objective measures for evaluating team performance. However, the experts reported limited experience in actively incorporating objective measures into their training and research. Nevertheless, they identified various potential objective measures, including acoustical, visual, physiological, and endocrinological measures and a time layer. Concerns were raised regarding feasibility, complexity, cost, and privacy issues associated with the use of objective measures.

Discussion

The study highlights the opportunities and challenges associated with employing objective measures to assess healthcare team performance. It particularly emphasizes the concerns expressed by medical simulation experts and team researchers, providing valuable insights for developers, trainers, researchers, and healthcare professionals involved in the design, planning or utilization of objective measures in team training or research.

Syncing to perform? A naturalistic uncontrolled prospective case study of emotional and physiological synchrony in a team of male volleyball athletes

Group processes are important for promoting relational and performance-related outcomes in sport; however, research exploring emotional and physiological synchrony and performance outcomes is rare. The objective of this study was to examine perceived emotional synchrony, physiological synchrony, and performance among male volleyball players using a naturalistic uncontrolled prospective case study approach over nine practices. Athletes participated in a coach-led pre-practice group visualization routine, while their heart rate and heart rate variability were continuously monitored. Athletes completed post-practice measures of emotional synchrony, and athletes and coaches completed ratings of individual and team performance. Emotional and physiological synchrony were not significantly correlated, but they were differentially related to performance outcomes, and there were significant interaction effects between physiological synchrony and time. Athletes' ratings of their own and their team's performance were associated with their perceived emotional synchrony. Coach performance ratings were associated with pre-practice team physiological synchrony but were unrelated to athletes' perceived emotional synchrony. Heart rate synchrony was associated with athlete and coach performance ratings and may be important for team dynamics and performance outcomes. The findings demonstrate evidence for emotional and physiological synchrony among athletes, providing a platform for future research examining processes and impacts of synchrony in sport.

multiSyncPy: A Python package for assessing multivariate coordination dynamics

In order to support the burgeoning field of research into intra- and interpersonal synchrony, we present an open-source software package: multiSyncPy. Multivariate synchrony goes beyond the bivariate case and can be useful for quantifying how groups, teams, and families coordinate their behaviors, or estimating the degree to which multiple modalities from an individual become synchronized. Our package includes state-of-the-art multivariate methods including symbolic entropy, multidimensional recurrence quantification analysis, coherence (with an additional sum-normalized modification), the cluster-phase 'Rho' metric, and a statistical test based on the Kuramoto order parameter. We also include functions for two surrogation techniques to compare the observed coordination dynamics with chance levels and a windowing function to examine time-varying coordination for most of the measures. Taken together, our collation and presentation of these methods make the study of interpersonal synchronization and coordination dynamics applicable to larger, more complex and often more ecologically valid study designs. In this work, we summarize the relevant theoretical background and present illustrative practical examples, lessons learned, as well as guidance for the usage of our package - using synthetic as well as empirical data. Furthermore, we provide a discussion of our work and software and outline interesting further directions and perspectives. multiSyncPy is freely available under the LGPL license at: https://github.com/cslab-hub/multiSyncPy , and also available at the Python package index.

Westerink JHD, Rispens S. A Review of Using Wearable Technology to Assess Team Functioning and Performance

Wearable technology enables collecting continuous in situ data from multiple people in various modalities, which can enhance team research and support, as the dynamic coupling of signals between interacting individuals (i.e., team coordination dynamics) is believed to reflect underlying processes and states of team functioning and performance. We conducted a systematic review on existing literature to evaluate the prospective use of wearable technology in research and practice. Using the IMOI framework as an organizing tool, our review revealed considerable support linking team coordination dynamics in different modalities to team functioning and performance, but also explicated the field’s nascent status.

Catastrophe Theory Applied to Neuropsychological Data: Nonlinear Effects of Depression on Financial Capacity in Amnestic Mild Cognitive Impairment and Dementia

Financial incapacity is one of the cognitive deficits observed in amnestic mild cognitive impairment and dementia, while the combined interference of depression remains unexplored. The objective of this research is to investigate and propose a nonlinear model that explains empirical data better than ordinary linear ones and elucidates the role of depression. Four hundred eighteen (418) participants with a diagnosis of amnestic MCI with varying levels of depression were examined with the Geriatric Depression Scale (GDS-15), the Functional Rating Scale for Symptoms of Dementia (FRSSD), and the Legal Capacity for Property Law Transactions Assessment Scale (LCPLTAS). Cusp catastrophe analysis was applied to the data, which suggested that the nonlinear model was superior to the linear and logistic alternatives, demonstrating depression contributes to a bifurcation effect. Depressive symptomatology induces nonlinear effects, that is, beyond a threshold value sudden decline in financial capacity is observed. Implications for theory and practice are discussed.

Beyond Dyadic Coupling: The Method of Multivariate Surrogate Synchrony (mv-SUSY)

Measuring interpersonal synchrony is a promising approach to assess the complexity of social interaction, which however has been mostly limited to dyads. In this study, we introduce multivariate Surrogate Synchrony (mv-SUSY) to extend the current set of computational methods. Methods: mv-SUSY was applied to eight datasets consisting of 10 time series each, all with n = 9600 observations. Datasets 1 to 5 consist of simulated time series with the following characteristics: white noise (dataset 1), non-stationarity with linear time trends (dataset 2), autocorrelation (dataset 3), oscillation (dataset 4), and multivariate correlation (dataset 5). Datasets 6 to 8 comprise empirical multivariate movement data of two individuals (datasets 6 and 7) and between members of a group discussion (dataset 8.) Results: As hypothesized, findings of mv-SUSY revealed absence of synchrony in datasets 1 to 4 and presence of synchrony in dataset 5. In the empirical datasets, mv-SUSY indicated significant movement synchrony. These results were predominantly replicated by two well-established dyadic synchrony approaches, Surrogate Synchrony (SUSY) and Surrogate Concordance (SUCO). Conclusions: The study applied and evaluated a novel synchrony approach, mv-SUSY. We demonstrated the feasibility and validity of estimating multivariate nonverbal synchrony within and between individuals by mv-SUSY.

Windowed multiscale synchrony: modeling time-varying and scale-localized interpersonal coordination dynamics

Social interactions are pervasive in human life with varying forms of interpersonal coordination emerging and spanning different modalities (e.g. behaviors, speech/language, and neurophysiology). However, during social interactions, as in any dynamical system, patterns of coordination form and dissipate at different scales. Historically, researchers have used aggregate measures to capture coordination over time. While those measures (e.g. mean relative phase, cross-correlation, coherence) have provided a wealth of information about coordination in social settings, some evidence suggests that multiscale coordination may change over the time course of a typical empirical observation. To address this gap, we demonstrate an underutilized method, windowed multiscale synchrony, that moves beyond quantifying aggregate measures of coordination by focusing on how the relative strength of coordination changes over time and the scales that comprise social interaction. This method involves using a wavelet transform to decompose time series into component frequencies (i.e. scales), preserving temporal information and then quantifying phase synchronization at each of these scales. We apply this method to both simulated and empirical interpersonal physiological and neuromechanical data. We anticipate that demonstrating this method will stimulate new insights on the mechanisms and functions of synchrony in interpersonal contexts using neurophysiological and behavioral measures.

Group-level physiological synchrony and individual-level anxiety predict positive affective behaviors during a group decision-making task

Joint performance can lead to the synchronization of physiological processes among group members during a shared task. Recently, it has been shown that synchronization is indicative of subjective ratings of group processes and task performance. However, different methods have been used to quantify synchronization, and little is known about the effects of the choice of method and level of analysis (individuals, dyads, or triads) on the results. In this study, participants performed a decision‐making task in groups of three while physiological signals (heart rate and electrodermal activity), positive affective behavior, and personality traits were measured. First, we investigated the effects of different levels of analysis of physiological synchrony on affective behavior. We computed synchrony measures as (a) individual contributions to group synchrony, (b) the average dyadic synchrony within a group, and (c) group‐level synchrony. Second, we assessed the association between physiological synchrony and positive affective behavior. Third, we investigated the moderating effects of trait anxiety and social phobia on behavior. We discovered that the effects of physiological synchrony on positive affective behavior were particularly strong at the group level but nonsignificant at the individual and dyadic levels. Moreover, we found that heart rate and electrodermal synchronization showed opposite effects on group members' display of affective behavior. Finally, trait anxiety moderated the relationship between physiological synchrony and affective behavior, perhaps due to social uncertainty, while social phobia did not have a moderating effect. We discuss these results regarding the role of different physiological signals and task demands during joint action.

Impact Statement

Despite the inherent multilevel structure of groups, little is known regarding how physiological coupling between group members relates to their behaviors during joint group tasks at multiple levels (individual, dyadic, and group). We showed that the relationship between physiological synchrony and smiling/laughing behaviors made by group members were particularly strong at the group level but nonsignificant at the individual and dyadic levels. By using innovative quantification methods—Multidimensional Recurrence Quantification Analysis—we highlight the importance of modeling data in a way that allows for multilevel considerations within groups.

Team Physiological Dynamics: A Critical Review

OBJECTIVE

Review the use of physiological measurement in team settings and propose recommendations to improve the state of the science.

BACKGROUND

New sensor and analytical capabilities enable exploration of relationships between team members' physiological dynamics. We conducted a review of physiological measures used in research on teams to understand (1) how these measures are theoretically and operationally related to team constructs and (2) what types of validity evidence exist for physiological measurement in team settings.

METHOD

We identified 32 articles that investigated task-performing teams using physiological data. Articles were coded on several dimensions, including team characteristics. Study findings were categorized by relationships tested between team physiological dynamics (TPD) and team inputs, mediators/processes, outputs, or psychometric properties.

RESULTS

TPD researchers overwhelmingly measure single physiological systems. Although there is research linking TPD to inputs and outputs, the research on processes is underdeveloped.

CONCLUSION

We recommend several theoretical, methodological, and statistical themes to expand the growth of the TPD field.

APPLICATION

Physiological measures, once established as reliable indicators of team functioning, might be used to diagnose suboptimal team states and cue interventions to ameliorate these states.

Beyond Dyadic Coordination: Multimodal Behavioral Irregularity in Triads Predicts Facets of Collaborative Problem Solving

We hypothesize that effective collaboration is facilitated when individuals and environmental components form a synergy where they work together and regulate one another to produce stable patterns of behavior, or regularity, as well as adaptively reorganize to form new behaviors, or irregularity. We tested this hypothesis in a study with 32 triads who collaboratively solved a challenging visual computer programming task for 20 min following an introductory warm-up phase. Multidimensional recurrence quantification analysis was used to examine fine-grained (i.e., every 10 s) collective patterns of regularity across team members' speech rate, body movement, and team interaction with the shared user interface. We found that teams exhibited significant patterns of regularity as compared to shuffled baselines, but there were no systematic trends in regularity across time. We also found that periods of regularity were associated with a reduction in overall behavior. Notably, the production of irregular behavior predicted expert-coded metrics of collaborative activity, such as teams' ability to construct shared knowledge and effectively negotiate and coordinate execution of solutions, net of overall behavioral production and behavioral self-similarity. Our findings support the theory that groups can interact to form interpersonal synergies and indicate that information about system-level dynamics is a viable way to understand and predict effective collaborative processes.

Interpersonal Coordination Dynamics in Psychotherapy: A Systematic Review

Background

The aim of this study was to gain a better understanding of interpersonal coordination (i.e., covariation over time) in different modalities (e.g., physiology, language) during psychotherapy and their importance for understanding the dynamics of psychotherapeutic interaction and efficacy.

Methods

We conducted a systematic review of all studies examining some form of interpersonal coordination in a psychotherapeutic context.

Results

We first summarize details of the included studies such as gender composition, therapy types, and methods used. The collation of these studies provided evidence that, during psychotherapeutic contexts, interpersonal coordination occurs in physiology, movements, interpersonal displays, and language/vocalizations. Further, it also showed that movement coordination was most frequently associated with psychotherapy outcomes, physiological coordination was most frequently associated with empathy, and coordination in a variety of modalities including language/vocalizations were most frequently associated with therapeutic alliance.

Conclusions

We discuss these results, shortcomings with the current literature, and highlight three crucial questions for future research. Research on interpersonal coordination in psychotherapy has potential to advance the both the research and practice of psychotherapy.

Multiscale movement coordination dynamics in collaborative team problem solving

During collaborative problem solving (CPS), coordination occurs at different spatial and temporal scales. This multiscale coordination should play a functional role in facilitating effective collaboration. To evaluate this, we conducted a study of computer-based CPS with 42 dyadic teams. We used cross-wavelet coherence to examine movement coordination, extracted from videos, at several scales, and tested whether the observed coordination was greater than expected due to chance and due to task demands. We found that coordination at scales less than 2s was greater than chance and at most scales (except 16s, 1m, and 2m), was greater than expected due to task demands. Lastly, we observed that coherence at .25s and 1s scales was predictive of performance. However, when including relative phase, our results suggest that higher in-phase movement coordination at the 1s scale was the strongest predictor of CPS performance. Further, we used growth curve modeling to examine how movement coordination changes across the duration of the task and whether this is moderated by CPS performance. We found that coordination over the duration of the CPS task is quadratic (a U shape) and that better performing teams have higher coordination with a shallower curve. We discuss these findings and their relevance to understanding how low-level movement coordination facilitates CPS.

Advancing Our Understandings of Healthcare Team Dynamics From the Simulation Room to the Operating Room: A Neurodynamic Perspective

The initial models of team and team member dynamics using biometric data in healthcare will likely come from simulations. But how confident are we that the simulation-derived high-resolution dynamics will reflect those of teams working with live patients? We have developed neurodynamic models of a neurosurgery team while they performed a peroneal nerve decompression surgery on a patient to approach this question. The models were constructed from EEG-derived measures that provided second-by-second estimates of the neurodynamic responses of the team and team members to task uncertainty. The anesthesiologist and two neurosurgeons developed peaks, often coordinated, of elevated neurodynamic organization during the patient preparation and surgery which were similar to those seen during simulation training, and which occurred near important episodes of the patient preparation and surgery. As the analyses moved down the neurodynamic hierarchy, and the simulation and live patient neurodynamics occurring during the intubation procedure were compared at progressively smaller time scales, differences emerged across scalp locations and EEG frequencies. The most significant was the pronounced suppression of gamma rhythms detected by the frontal scalp sensors during the live patient intubation which was absent in simulation trials of the intubation procedure. These results indicate that while profiles of the second-by-second neurodynamics of teams were similar in both the simulation and live patient environments, a deeper analysis revealed differences in the EEG frequencies and scalp locations of the signals responsible for those team dynamics. As measures of individual and team performance become more micro-scale and dynamic, and simulations become extended into virtual environments, these results argue for the need for parallel studies in live environments to validate the dynamics of cognition being observed.

Introducing rLSM: An integrated metric assessing temporal reciprocity in language style matching

The way that individuals use function words in a conversation-reflecting how they say things, rather than what they say-is called their individual language style. The dyadic coordination of language styles, called language style matching (LSM), is central to the development of social relationships in conversations. Despite a growing body of research on LSM, conceptual and methodological approaches are inconsistent between scholars. After giving a conceptual overview of LSM, we derive the properties desirable for analyses of LSM in interaction (e.g., reciprocity, consistency, and frequency sensitivity). Building on these properties, the existing three methodological approaches to LSM are reviewed. Since none of the existing metrics fulfills all the desired properties, we introduce a new metric to assess LSM in dyadic interaction, capturing reciprocal adaption throughout the dynamic process of a conversation. Hence, the new metric is called reciprocal LSM (rLSM). To empirically establish the conceptual underpinnings of rLSM, the metric is compared to the LSM metric most commonly used in psychological research. Both metrics are applied to a set of N = 77 transcribed real-life dyadic conversations, analyzed with the Linguistic Inquiry and Word Count software. The results indicate that rLSM is a better estimate of LSM than is the old metric and that there is high conceptual similarity between the two metrics. Implications for existing research and directions for future research are discussed. To facilitate the standardization and comparability of research, guidelines are provided for authors on the use of the new and existing metrics.