"They must have seen it, you know." Body talk, extension talk, and action talk: A qualitative study on how palliative care patients and their significant others express experiencing these nonverbal cues

Communication about life-threatening disease and palliative care is essential but often experienced as difficult by those concerned and has mainly been studied in terms of its verbal components. Despite the fundamentality of nonverbal communication, its dimensions in care, especially in the communication by patients and their significant others, has not been as extensively examined. Drawing on a secondary qualitative content analysis of data from 23 interviews-15 with patients in specialized palliative home care in Sweden and 8 with their significant others-this study aims at understanding and characterizing how patients verbally express experiences of conveying nonverbal cues about life-threatening disease and its consequences and how their significant others express perceiving these cues. Patients expressed experiences of nonverbal communication in the form of cues conveying meaning about their disease and its consequences, often beyond their control. Whether and how the patients reinforced these cues verbally, depended on individual needs, care for others, and evaluations of relationships. Significant others acknowledged the presence of nonverbal cues and tried to interpret their meaning. Both patients and significant others emphasized the importance of nonverbal cues and actively related to how cues in the form of bodily appearance, aids, objects and acts, serve communicative functions about disease and its consequences. These dimensions of nonverbal communication are characterized as: body talk, extension talk and action talk. This study contributes to an international knowledge base on the complexities of nonverbal communicative aspects in these dimensions and how it affects patients and significant others. Professionals should be aware that dimensions of care, such as prescribed aids, from the patients' perspective can be perceived as nonverbal cues that might "speak of" disease progression.

Impaired recognition of interactive intentions in adults with autism spectrum disorder not attributable to differences in visual attention or coordination via eye contact and joint attention

Altered nonverbal communication patterns especially with regard to gaze interactions are commonly reported for persons with autism spectrum disorder (ASD). In this study we investigate and differentiate for the first time the interplay of attention allocation, the establishment of shared focus (eye contact and joint attention) and the recognition of intentions in gaze interactions in adults with ASD compared to control persons. Participants interacted via gaze with a virtual character (VC), who they believed was controlled by another person. Participants were instructed to ascertain whether their partner was trying to interact with them. In fact, the VC was fully algorithm-controlled and showed either interactive or non-interactive gaze behavior. Participants with ASD were specifically impaired in ascertaining whether their partner was trying to interact with them or not as compared to participants without ASD whereas neither the allocation of attention nor the ability to establish a shared focus were affected. Thus, perception and production of gaze cues seem preserved while the evaluation of gaze cues appeared to be impaired. An additional exploratory analysis suggests that especially the interpretation of contingencies between the interactants' actions are altered in ASD and should be investigated more closely.

Influence of autistic traits and communication role on eye contact behavior during face-to-face interaction

Eye contact is a central component in face-to-face interactions. It is important in structuring communicative exchanges and offers critical insights into others' interests and intentions. To better understand eye contact in face-to-face interactions, we applied a novel, non-intrusive deep-learning-based dual-camera system and investigated associations between eye contact and autistic traits as well as self-reported eye contact discomfort during a referential communication task, where participants and the experimenter had to guess, in turn, a word known by the other individual. Corroborating previous research, we found that participants' eye gaze and mutual eye contact were inversely related to autistic traits. In addition, our findings revealed different behaviors depending on the role in the dyad: listening and guessing were associated with increased eye contact compared with describing words. In the listening and guessing condition, only a subgroup who reported eye contact discomfort had a lower amount of eye gaze and eye contact. When describing words, higher autistic traits were associated with reduced eye gaze and eye contact. Our data indicate that eye contact is inversely associated with autistic traits when describing words, and that eye gaze is modulated by the communicative role in a conversation.

The Gaze Dialogue Model: Nonverbal Communication in HHI and HRI

When humans interact with each other, eye gaze movements have to support motor control as well as communication. On the one hand, we need to fixate the task goal to retrieve visual information required for safe and precise action-execution. On the other hand, gaze movements fulfil the purpose of communication, both for reading the intention of our interaction partners, as well as to signal our action intentions to others. We study this Gaze Dialogue between two participants working on a collaborative task involving two types of actions: 1) individual action and 2) action-in-interaction. We recorded the eye-gaze data of both participants during the interaction sessions in order to build a computational model, the Gaze Dialogue, encoding the interplay of the eye movements during the dyadic interaction. The model also captures the correlation between the different gaze fixation points and the nature of the action. This knowledge is used to infer the type of action performed by an individual. We validated the model against the recorded eye-gaze behavior of one subject, taking the eye-gaze behavior of the other subject as the input. Finally, we used the model to design a humanoid robot controller that provides interpersonal gaze coordination in human-robot interaction scenarios. During the interaction, the robot is able to: 1) adequately infer the human action from gaze cues; 2) adjust its gaze fixation according to the human eye-gaze behavior; and 3) signal nonverbal cues that correlate with the robot's own action intentions.

Patterns of Verbal and Nonverbal Communication During Sex

Sexual communication functions as an important relational process expediating satisfying sexual experiences (Sprecher & Hendrick, 2004). Much of the existing literature on sexual communication concerning sexual pleasure biases verbal communication (Babin, 2012). This study adds to the existing research regarding patterns of communication surrounding sex and during sex through qualitative analysis. Further, this inquiry focused on participants' full histories, rather than their tendencies within any current relationship. We analyzed 78 qualitative interview transcripts from participants between the ages of 18 and 69. Participants reported a reluctance to communicate anything but pleasure, discomfort, or dislike during sex to avoid discouraging their partners. Though participants reported a perception that communicating pleasure served as encouragement and affirmation to their partners, most preferred to communicate pleasure nonverbally. Some participants reported a tendency to communicate pain or dislike verbally. Some preferred communication about sexual topics only before or after sexual activities. Participants shared that a high level of comfort with their partner increased sexual communication. How sexual partners communicate sex not only affects pleasure but can only affect intimacy between partners and health. This adds to the scant literature on nonverbal communication during sex and some people's preference for that style (Blunt-Vinti et al., 2019).

Pragmatic features in patients with schizophrenia

Schizophrenia is a heterogeneous disorder. Despite the fact that communication and language skills may be impaired in schizophrenia, only a few studies have examined specific aspects of pragmatic competence in these patients. Therefore, this study aimed to compare the pragmatic skills of schizophrenic patients and a control group. Forty-three schizophrenic patients with a Mean ± SD age of 40 ± 1.21 years old and a control group were assessed using the Adult Pragmatics Profile (APP), which is a validated tool consisting of three scales: verbal, nonverbal and paralinguistic. The participants in the schizophrenia group demonstrated significant impairments in pragmatic abilities compared to the participants in those the control group. The most remarkable difference was in the nonverbal scale (5.00 ± 1.09), while the lowest difference was seen in the verbal scale (18.30 ± 3.91). Within the nonverbal subscales, eye contact was the most impaired. Schizophrenic patients presented with impairments in their pragmatic skills (verbal, nonverbal and paralinguistic). As pragmatic skills play a paramount role in social communication, it is of great significance to address these impairments to enhance patients' quality of life.

Support vector machine prediction of individual Autism Diagnostic Observation Schedule (ADOS) scores based on neural responses during live eye-to-eye contact

Social difficulties during interactions with others are central to autism spectrum disorder (ASD). Understanding the links between these social difficulties and their underlying neural processes is a primary aim focused on improved diagnosis and treatment. In keeping with this goal, we have developed a multivariate classification method based on neural data acquired by functional near infrared spectroscopy, fNIRS, during live eye-to-eye contact with adults who were either typically developed (TD) or individuals with ASD. The ASD diagnosis was based on the gold-standard Autism Diagnostic Observation Schedule (ADOS) which also provides an index of symptom severity. Using a nested cross-validation method, a support vector machine (SVM) was trained to discriminate between ASD and TD groups based on the neural responses during eye-to-eye contact. ADOS scores were not applied in the classification training. To test the hypothesis that SVM identifies neural activity patterns related to one of the neural mechanisms underlying the behavioral symptoms of ASD, we determined the correlation coefficient between the SVM scores and the individual ADOS scores. Consistent with the hypothesis, the correlation between observed and predicted ADOS scores was 0.72 (p < 0.002). Findings suggest that multivariate classification methods combined with the live interaction paradigm of eye-to-eye contact provide a promising approach to link neural processes and social difficulties in individuals with ASD.

The Role of Coherent Robot Behavior and Embodiment in Emotion Perception and Recognition During Human-Robot Interaction: Experimental Study

BACKGROUND

Social robots are becoming increasingly important as companions in our daily lives. Consequently, humans expect to interact with them using the same mental models applied to human-human interactions, including the use of cospeech gestures. Research efforts have been devoted to understanding users' needs and developing robot's behavioral models that can perceive the user state and properly plan a reaction. Despite the efforts made, some challenges regarding the effect of robot embodiment and behavior in the perception of emotions remain open.

OBJECTIVE

The aim of this study is dual. First, it aims to assess the role of the robot's cospeech gestures and embodiment in the user's perceived emotions in terms of valence (stimulus pleasantness), arousal (intensity of evoked emotion), and dominance (degree of control exerted by the stimulus). Second, it aims to evaluate the robot's accuracy in identifying positive, negative, and neutral emotions displayed by interacting humans using 3 supervised machine learning algorithms: support vector machine, random forest, and K-nearest neighbor.

METHODS

Pepper robot was used to elicit the 3 emotions in humans using a set of 60 images retrieved from a standardized database. In particular, 2 experimental conditions for emotion elicitation were performed with Pepper robot: with a static behavior or with a robot that expresses coherent (COH) cospeech behavior. Furthermore, to evaluate the role of the robot embodiment, the third elicitation was performed by asking the participant to interact with a PC, where a graphical interface showed the same images. Each participant was requested to undergo only 1 of the 3 experimental conditions.

RESULTS

A total of 60 participants were recruited for this study, 20 for each experimental condition for a total of 3600 interactions. The results showed significant differences (P<.05) in valence, arousal, and dominance when stimulated with the Pepper robot behaving COH with respect to the PC condition, thus underlying the importance of the robot's nonverbal communication and embodiment. A higher valence score was obtained for the elicitation of the robot (COH and robot with static behavior) with respect to the PC. For emotion recognition, the K-nearest neighbor classifiers achieved the best accuracy results. In particular, the COH modality achieved the highest level of accuracy (0.97) when compared with the static behavior and PC elicitations (0.88 and 0.94, respectively).

CONCLUSIONS

The results suggest that the use of multimodal communication channels, such as cospeech and visual channels, as in the COH modality, may improve the recognition accuracy of the user's emotional state and can reinforce the perceived emotion. Future studies should investigate the effect of age, culture, and cognitive profile on the emotion perception and recognition going beyond the limitation of this work.

Improving counseling effectiveness with virtual counselors through nonverbal compassion involving eye contact, facial mimicry, and head-nodding

An effective way to reduce emotional distress is by sharing negative emotions with others. This is why counseling with a virtual counselor is an emerging methodology, where the sharer can consult freely anytime and anywhere without having to fear being judged. To improve counseling effectiveness, most studies so far have focused on designing verbal compassion for virtual counselors. However, recent studies showed that virtual counselors' nonverbal compassion through eye contact, facial mimicry, and head-nodding also have significant impact on the overall counseling experience. To verify this, we designed the virtual counselor's nonverbal compassion and examined its effects on counseling effectiveness (i.e., reduce the intensity of anger and improve general affect). A total of 40 participants were recruited from the university community. Participants were then randomly assigned to one of two virtual counselor conditions: a neutral virtual counselor condition without nonverbal compassion and a compassionate virtual counselor condition with nonverbal compassion (i.e., eye contact, facial mimicry, and head-nodding). Participants shared their anger-inducing episodes with the virtual counselor for an average of 16.30 min. Note that the virtual counselor was operated by the Wizard-of-Oz method without actually being technically implemented. Results showed that counseling with a compassionate virtual counselor reduced the intensity of anger significantly more than counseling with a neutral virtual counselor (F(1, 37) = 30.822, p < 0.001, ηp2 = 0.454). In addition, participants who counseled with a compassionate virtual counselor responded that they experienced higher empathy than those who counseled with a neutral virtual counselor (p < 0.001). These findings suggest that nonverbal compassion through eye contact, facial mimicry, and head-nodding of the virtual counselor makes the participants feel more empathy, which contributes to improving the counseling effectiveness by reducing the intensity of anger.

Item-Based Analysis of Some ADOS-2 Items with Typically Developing Participants Might help Improve Cross-Cultural Validity of ADOS-2

Most internationally recognized instruments for the screening and diagnosis of autism spectrum disorder have been developed in the USA, which calls into question the degree of their cultural adaptation to diverse populations. The aim of this study is to examine the characteristics of social communication in typically developing Croatian-speaking participants (N = 220) using ADOS-2-defined item-level normative values. Croatian subjects showed the expected ("typical") results in the domain of verbal communication, slightly atpical results in nonverbal communication (primarily gesture use), and more significant deviations in pragmatics (offering and asking for information), relative to the expectations of the ADOS-2. As ADOS-2 has become an important component of thorough ASD diagnostic evaluations worldwide, identifying methods for increasing the cross-cultural validity is essential.

Evaluating the efficacy of a digital App to enhance patient-centred nursing handover: A simulation study

AIM

The study aim was to evaluate the feasibility and efficacy of a digital App developed to enhance patient communication with nurses during bedside nursing handover at shift change.

METHODS

Six nurses and 11 patient actors/volunteers participated in 12 simulated nursing handovers across six simulation workshops. Over half the patients were aged 70+ years (55%); majority were female (82%). Handover video recordings were analysed using a structured observation tool and a revised Four Habits Coding Scheme to assess nurses' handover communication skills. Patient and nurse feedback was also sought. The STROBE checklist (Data S1) guided preparation of the study.

RESULTS

For all simulated handovers (n = 12): Nurses greeted the patient at commencement; nurses made eye contact with the patient; patients were given opportunity to ask questions; and all patient questions were answered. Nurses explained the handover process for less than half the handovers (42%). Familiarity with the patient's history was evident in every handover. Communication behaviours identified in most handovers included: good nonverbal behaviour; allowing time for the patient to absorb information; giving clear explanations; involving the patient in decisions; and exploring acceptability of the care plan. Patient and nurse feedback on the App included: The App was easy to navigate, features were well-liked, with some improvements suggested.

CONCLUSION

Patients and nurses provided positive feedback for the App during hospital stay and at handover. The App has the potential to enhance existing handover processes and increase safety of hospital care by using technology to educate and empower patients/carers to be active partners in communication with nurses during change-of-shift handover.

RELEVANCE TO CLINICAL PRACTICE

The App empowers and enables patients/carers to actively participate in nursing handover and allows patients to communicate concerns and provide information to their nursing team, facilitating a new approach.

PATIENT OR PUBLIC CONTRIBUTION

Patients and carers were involved in the research from the original co-design workshops that guided the development of the handover App. The research aims and outcome measures were informed by the experiences and preferences of patients/carers. Two patient representatives were involved in writing and submission of the grant application for the study to evaluate the efficacy of the App and were listed as co-authors on this paper. Patient volunteers were involved in the current study to pilot test the handover App. Patient volunteers were recruited through a consumer representative and volunteer registry at the health service. They participated in simulated nursing handovers with two nurses to assess the feasibility and acceptability of the handover App and then provided feedback and suggestions for improvement.

Object use in communication of semi-wild chimpanzees

Object interactions play an important role in human communication but the extent to which nonhuman primates incorporate objects in their social interactions remains unknown. To better understand the evolution of object use, this study explored how objects are used in social interactions in semi-wild chimpanzees (Pan troglodytes). We used an observational approach focusing on naturally occurring object actions where we examined their use and tested whether the production of object actions was influenced by the recipients' visual attention as well as by colony membership. The results show that chimpanzees adjusted both the type of object used, and the modality of object actions to match the visual attention of the recipient, as well as colony differences in the use of targeted object actions. These results provide empirical evidence highlighting that chimpanzees use objects in diverse ways to communicate with conspecifics and that their use may be shaped by social factors, contributing to our understanding of the evolution of human nonverbal communication, language, and tool use.

The role of hand gestures in emotion communication: Do type and size of gestures matter?

We communicate emotions in a multimodal way, yet non-verbal emotion communication is a relatively understudied area of research. In three experiments, we investigated the role of gesture characteristics (e.g., type, size in space) on individuals' processing of emotional content. In Experiment 1, participants were asked to rate the emotional intensity of emotional narratives from the videoclips either with iconic or beat gestures. Participants in the iconic gesture condition rated the emotional intensity higher than participants in the beat gesture condition. In Experiment 2, the size of gestures and its interaction with gesture type were investigated in a within-subjects design. Participants again rated the emotional intensity of emotional narratives from the videoclips. Although individuals overall rated narrow gestures more emotionally intense than wider gestures, no effects of gesture type, or gesture size and type interaction were found. Experiment 3 was conducted to check whether findings of Experiment 2 were due to viewing gestures in all videoclips. We compared the gesture and no gesture (i.e., speech only) conditions and showed that there was not a difference between them on emotional ratings. However, we could not replicate the findings related to gesture size of Experiment 2. Overall, these findings indicate the importance of examining gesture's role in emotional contexts and that different gesture characteristics such as size of gestures can be considered in nonverbal communication.

Impaired emotion perception in schizophrenia shows sex differences with channel- and category-specific effects: A pilot study

Individuals with schizophrenia reportedly demonstrate deficits in emotion perception. Relevant studies on the effects of decoder's sex, communication channels and emotion categories have produced mixed findings and seldom explored the interactions among these three key factors. The present pilot study examined how male and female individuals with schizophrenia and healthy controls perceived emotional (e.g., angry, happy, and sad) and neutral expressions from verbal semantic and nonverbal prosodic and facial channels. Twenty-eight (11 females) individuals with schizophrenia and 30 healthy controls (13 females) were asked to recognize emotional facial expressions, emotional prosody, and emotional semantics. Both accuracy and response time showed subpar performance for all communication channels and emotional categories in the schizophrenia group. More severe emotion perception deficits were found with the nonverbal (not the verbal) materials. There was also a reduced level of impairment with anger perception, especially in the female individuals with schizophrenia while biased perception towards emotional semantics was more pronounced in male individuals with schizophrenia. These findings, although preliminary, indicate the channel- and category-specific nature of emotion perception with potential sex differences among people with schizophrenia, which has important theoretical and practical implications.

The promise and peril of interactive embodied agents for studying non-verbal communication: a machine learning perspective

In face-to-face interactions, parties rapidly react and adapt to each other's words, movements and expressions. Any science of face-to-face interaction must develop approaches to hypothesize and rigorously test mechanisms that explain such interdependent behaviour. Yet conventional experimental designs often sacrifice interactivity to establish experimental control. Interactive virtual and robotic agents have been offered as a way to study true interactivity while enforcing a measure of experimental control by allowing participants to interact with realistic but carefully controlled partners. But as researchers increasingly turn to machine learning to add realism to such agents, they may unintentionally distort the very interactivity they seek to illuminate, particularly when investigating the role of non-verbal signals such as emotion or active-listening behaviours. Here I discuss some of the methodological challenges that may arise when machine learning is used to model the behaviour of interaction partners. By articulating and explicitly considering these commitments, researchers can transform 'unintentional distortions' into valuable methodological tools that yield new insights and better contextualize existing experimental findings that rely on learning technology. This article is part of a discussion meeting issue 'Face2face: advancing the science of social interaction'.

Linking Nonverbal Rapport to Health Outcome: Testing an Organizational Pathway Model

A growing body of research on medical communication indicates that nonverbal rapport (e.g., smiling, eye contact, closer proximity) is central to productive health care delivery. However, mechanisms integral to the process by which nonverbal rapport influences health improvement remain under-researched. This study breaks new grounds in proposing and testing mediation pathways that take into account organizational factors. We conducted a cross-sectional survey in a private hospital in Singapore among 417 patients to examine their communication with physicians and nurses. Results indicated that nonverbal rapport did not have a significant direct relationship with perceived health outcome in both the patient-physician dyad and the patient-nurse dyad. Instead, communication satisfaction and organizational identity completely mediated this relationship. In addition, respect positively moderated the relationship between nonverbal rapport and communication satisfaction in both dyads, while health literacy was not a significant moderator. The findings suggest that the organizational context should be considered in pathways research.

Predictability modulates neural response to eye contact in ASD

BACKGROUND

Deficits in establishing and maintaining eye-contact are early and persistent vulnerabilities of autism spectrum disorder (ASD), and the neural bases of these deficits remain elusive. A promising hypothesis is that social features of autism may reflect difficulties in making predictions about the social world under conditions of uncertainty. However, no research in ASD has examined how predictability impacts the neural processing of eye-contact in naturalistic interpersonal interactions.

METHOD

We used eye tracking to facilitate an interactive social simulation wherein onscreen faces would establish eye-contact when the participant looked at them. In Experiment One, receipt of eye-contact was unpredictable; in Experiment Two, receipt of eye-contact was predictable. Neural response to eye-contact was measured via the N170 and P300 event-related potentials (ERPs). Experiment One included 23 ASD and 46 typically developing (TD) adult participants. Experiment Two included 25 ASD and 43 TD adult participants.

RESULTS

When receipt of eye-contact was unpredictable, individuals with ASD showed increased N170 and increased, but non-specific, P300 responses. The magnitude of the N170 responses correlated with measures of sensory and anxiety symptomology, such that increased response to eye-contact was associated with increased symptomology. However, when receipt of eye-contact was predictable, individuals with ASD, relative to controls, exhibited slower N170s and no differences in the amplitude of N170 or P300.

LIMITATIONS

Our ASD sample was composed of adults with IQ > 70 and included only four autistic women. Thus, further research is needed to evaluate how these results generalize across the spectrum of age, sex, and cognitive ability. Additionally, as analyses were exploratory, some findings failed to survive false-discovery rate adjustment.

CONCLUSIONS

Neural response to eye-contact in ASD ranged from attenuated to hypersensitive depending on the predictability of the social context. These findings suggest that the vulnerabilities in eye-contact during social interactions in ASD may arise from differences in anticipation and expectation of eye-contact in addition to the perception of gaze alone.

Can We Do Better than Gesturing? Requirements for a Digital Communication Aid to Support Non-Verbal Communication in Paediatric Emergency Care

Providing urgent and emergency care to migrant children is often hampered or delayed. Reasons for this are language barriers when children, and their care givers, don't speak any of the languages commonly spoken in Switzerland, which include German, French, Italian, and English. By a participatory design process, we want to develop a novel image-based digital communication aid tailored to the needs of migrant patients and nurses within Swiss paediatric clinics.

Physician Communication Skills in Telemedicine: The Role of Eye Contact

Gaze is an important non-verbal behavior in patient-physician communication. We examine the effect of the physician's gaze direction in video consultations on their communication and interpersonal skills ratings. 51 subjects watched videos of a physician providing the same teleconsultations while (a) looking directly at the camera and (b) looking at the computer screen. After each video, the participants rated the physician's skills. The results showed that looking at the camera is perceived as making eye contact and is associated with higher ratings on two communication skill items: (1) using empathy to communicate appreciation of the patient's feelings, and (2) providing support by expressing concern, understanding, and willingness to help. The effect of eye contact depended on the content of the consultation and on the general attitude of the physician. These results highlight the role of eye contact in video consultations and its dependency on other verbal and non-verbal behaviors.

Emojis predict dropouts of remote workers: An empirical study of emoji usage on GitHub

Emotions at work have long been identified as critical signals of work motivations, status, and attitudes, and as predictors of various work-related outcomes. When more and more employees work remotely, these emotional signals of workers become harder to observe through daily, face-to-face communications. The use of online platforms to communicate and collaborate at work provides an alternative channel to monitor the emotions of workers. This paper studies how emojis, as non-verbal cues in online communications, can be used for such purposes and how the emotional signals in emoji usage can be used to predict future behavior of workers. In particular, we present how the developers on GitHub use emojis in their work-related activities. We show that developers have diverse patterns of emoji usage, which can be related to their working status including activity levels, types of work, types of communications, time management, and other behavioral patterns. Developers who use emojis in their posts are significantly less likely to dropout from the online work platform. Surprisingly, solely using emoji usage as features, standard machine learning models can predict future dropouts of developers at a satisfactory accuracy. Features related to the general use and the emotions of emojis appear to be important factors, while they do not rule out paths through other purposes of emoji use.

Is a downwards head tilt a cross-cultural signal of dominance? Evidence for a universal visual illusion

The present pre-registered research provides the first evidence that a downwards head tilt is sufficient to communicate dominance from a neutral facial expression among the Mayangna, members of an unindustrialized, small-scale traditional society in Nicaragua who have had minimal exposure to North American culture. Consistent with the Action Unit imposter effect observed in North American populations (Witkower and Tracy in Psychol Sci 30:893-906, 2019), changes to the appearance of the upper face caused by a downwards head tilt were sufficient to elicit perceptions of dominance among this population. Given that the Mayangna are unlikely to associate a downwards head tilt or related apparent facial changes with dominance as a result of cross-cultural learning, the present results suggest that perceptions of dominance formed from a downwards head tilt, and the visual illusion shaping these perceptions, are a widely generalizable, and possibly universal, feature of human psychology.

Verbalizing spiritual needs in palliative care: a qualitative interview study on verbal and non-verbal communication in two Danish hospices

Denmark is considered one of the World's most secular societies, and spiritual matters are rarely verbalized in public. Patients report that their spiritual needs are not cared for sufficiently. For studying spiritual care and communication, twelve patients admitted to two Danish hospices were interviewed. Verbal and non-verbal communication between patients and healthcare professionals were identified and analysed. Methodically, the Interpretative Phenomenological Analysis was used, and the findings were discussed through the lenses of existential psychology as well as philosophy and theory of caring sciences. Three themes were identified: 1. When death becomes present, 2. Direction of the initiative, and 3. Bodily presence and non-verbal communication. The encounter between patient and healthcare professional is greatly influenced by sensing, decoding, and interpretation. A perceived connection between the patient and the healthcare professional is of great importance as to how the patient experiences the relationship with the healthcare professional.The patient's perception and the patient's bodily experience of the healthcare professional are crucial to whether the patient opens up to the healthcare professional about thoughts and needs of a spiritual nature and initiates a conversation hereabout. In this way we found three dynamically connected movements toward spiritual care: 1. From secular to spiritual aspects of care 2. From bodily, sensory to verbal aspects of spiritual care and 3. From biomedical to spiritual communication and care. Thus, the non-verbal dimension becomes a prerequisite for the verbal dimension of spiritual communication to develop and unfold. The behaviour of the healthcare professionals, characterised by the way they move physically and the way they touch the patient, was found to be just as important as verbal conversation when it comes to spiritual care. The healthcare professional can create a connection to the patient through bodily and relational presence. Furthermore, the healthcare professionals should let their sensing and impressions guide them when meeting the patient in dialog about matters of a spiritual nature. Their perception of the patient and non-verbal communication are a prerequisite for being able to meet patient's spiritual needs with care and verbal communication.

Voice modulation: from origin and mechanism to social impact

Research on within-individual modulation of vocal cues is surprisingly scarce outside of human speech. Yet, voice modulation serves diverse functions in human and nonhuman nonverbal communication, from dynamically signalling motivation and emotion, to exaggerating physical traits such as body size and masculinity, to enabling song and musicality. The diversity of anatomical, neural, cognitive and behavioural adaptations necessary for the production and perception of voice modulation make it a critical target for research on the origins and functions of acoustic communication. This diversity also implicates voice modulation in numerous disciplines and technological applications. In this two-part theme issue comprising 21 articles from leading and emerging international researchers, we highlight the multidisciplinary nature of the voice sciences. Every article addresses at least two, if not several, critical topics: (i) development and mechanisms driving vocal control and modulation; (ii) cultural and other environmental factors affecting voice modulation; (iii) evolutionary origins and adaptive functions of vocal control including cross-species comparisons; (iv) social functions and real-world consequences of voice modulation; and (v) state-of-the-art in multidisciplinary methodologies and technologies in voice modulation research. With this collection of works, we aim to facilitate cross-talk across disciplines to further stimulate the burgeoning field of voice modulation. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part I)'.

Eye Synchrony: A Method to Capture Mutual and Joint Attention in Social Eye Movements

Gaze behavior represents a complex phenomenon in social inter-action. We focus here on dyadic face-to-face interaction during naturally occurring verbal exchanges, where shared attention can be operationalized by joint gazes and eye contact. A multi-step methodology for the analysis of eye synchrony is presented, exemplified by a single case. The dynamics of face-to-face interaction allows estimating the degree of interlocutors' synchrony. While there is growing evidence for interpersonal synchrony of various behavioral and physiological signals, eye synchrony has not yet been studied outside the laboratory. The method presented is based on time series of gaze behavior acquired by mobile eye tracking devices. We applied windowed cross-correlations to the data and used surrogate testing to attain effect sizes even for single interactions (Surrogate Synchrony, SUSY). SUSY thus integrates nomo-thetic with idiographic research goals: The nomothetic interest is to test hypotheses that gaze behavior may be generally synchronized and linked with psychological variables. The idiographic aspect is that effect sizes can be determined even in single-case studies owing to the surrogate analyses, which supports qualitative research. Results of the exemplary dataset suggested that proof-of-concept of this approach was attained. We describe what prerequisites are needed of a setting and technical setup for use in future studies of psychotherapy, counseling, negotiations, or work-related interactions.

Coding Body Language in Sports: The Nonverbal Behavior Coding System for Soccer Penalties

Nonverbal behavior (NVB) plays an important role in sports. However, it has been difficult to measure, as no coding schemes exist to objectively measure NVB in sports. Therefore, the authors adapted the Body Action and Posture Coding System to the context of soccer penalties, validated it, and initially used this system (Nonverbal Behavior Coding System for Soccer Penalties [NBCSP]) to explore NVB in penalties. Study 1 demonstrated that the NBCSP had good to excellent intercoder reliability regarding the occurrence and temporal precision of NVBs. It also showed that the coding system could differentiate certain postures and behaviors as a function of emotional valence (i.e., positive vs. negative emotional states). Study 2 identified differences in NVB for successful and missed shots in a sample of penalties (time spent looking toward the goal, toward the ground, right arm movement, and how upright the body posture was). The authors discuss the utility of the coding system for different sport contexts.

Eye contact during joint attention with a humanoid robot modulates oscillatory brain activity

Eye contact established by a human partner has been shown to affect various cognitive processes of the receiver. However, little is known about humans' responses to eye contact established by a humanoid robot. Here, we aimed at examining humans' oscillatory brain response to eye contact with a humanoid robot. Eye contact (or lack thereof) was embedded in a gaze-cueing task and preceded the phase of gaze-related attentional orienting. In addition to examining the effect of eye contact on the recipient, we also tested its impact on gaze-cueing effects (GCEs). Results showed that participants rated eye contact as more engaging and responded with higher desynchronization of alpha-band activity in left fronto-central and central electrode clusters when the robot established eye contact with them, compared to no eye contact condition. However, eye contact did not modulate GCEs. The results are interpreted in terms of the functional roles involved in alpha central rhythms (potentially interpretable also as mu rhythm), including joint attention and engagement in social interaction.

Overview of Computer Measures of the Referential Process

The Referential Process (RP) has three functions or processes, called Arousal, Symbolizing, and Reflecting/Reorganizing. Taken together, these provide a framework within which to address the question: how do people connect nonverbal experience and verbal forms. The purpose of this paper is to describe the Discourse Attributes Analysis Program (DAAP), which uses certain dictionaries or word lists to produce measures that model the referential process functions. These dictionaries are also described. The referential process may occur in any type of discourse context and the DAAP computer system may be applied to any type of verbal data. The focus in this paper is on transcripts of psychotherapy sessions. DAAP provides numeric and graphic data at several levels of discourse including word-by-word data concerning dictionary matches; average data for each turn of speech; average data for each speaker for each session; numeric data and graphic images for each session; and overall session data for each treatment. Graphic images of the ebb and flow of these computer generated functions of the referential process over the course of a therapy session are presented and interpreted. There are also discussions of new relational measures, as well as the referential process data base, which currently contains numerical data for 22 treatments, and presentations of several applications.

Overview of the Referential Process: The Operation of Language Within and Between People

Human mentation involves multiple formats of thought, which are connected substantially but partially, and may operate within or outside of awareness. The modes of thought include symbolic processes which are discrete representations with properties of reference and generativity, and which may be images or words, and subsymbolic components which are continuous in format and based on analogic relationships. The organization of experience is based on memory schemas, including emotion schemas organized through episodes that involve related sensory and bodily experiences with particular people in particular contexts. The referential process is a set of bidirectional functions that enable connection between the subsymbolic components operating in multiple sensory channels and the discrete single channel verbal code. The process involves three major functions: Arousal, the activation of an emotion schema not yet in symbolic form; Symbolizing, describing an event in which the schema has been activated; and Reflection/Reorganizing, exploring and elaborating the emotional meaning of such an event. The concepts of the theory concern psychological aspects of thought, but are potentially mappable onto the underlying neural structure.

More than words: how multimodal analysis can inform health professions education

The contexts and methods for communicating in healthcare and health professions education (HPE) profoundly affect how we understand information, relate to others, and construct our identities. Multimodal analysis provides a method for exploring how we communicate using multiple modes-e.g., language, gestures, images-in concert with each other and within specific contexts. In this paper, we demonstrate how multimodal analysis helps us investigate the ways our communication practices shape healthcare and HPE. We provide an overview of the theoretical underpinnings, traditions, and methodologies of multimodal analysis. Then, we illustrate how to design and conduct a study using one particular approach to multimodal analysis, multimodal (inter)action analysis, using examples from a study focused on clinical reasoning and patient documentation. Finally, we suggest how multimodal analysis can be used to address a variety of HPE topics and contexts, highlighting the unique contributions multimodal analysis can offer to our field.

Communication training is inadequate: the role of deception, non-verbal communication, and cultural proficiency

In this commentary, we argue that the limited experiential exposure of medical students to different cultures makes the instruction devoted to communication skills inadequate. The relationship of these dynamics to honesty in clinical encounters is explored. Absent significant experiential exposure to differing group cultures to counter the natural tendency to favor one's own, discrimination prevails. Knowledge or awareness of cultural differences does not necessarily equate to communication proficiency. Critically, interactions based on lived experience offer a deeper knowledge and understanding of culturally meaningful nuances than that imparted through other formats. Medical students' lack of experiential exposure to different cultures results in communication miscues. When the stakes are high, people detect those miscues diminishing trust in the doctor-patient relationship. Greater experiential cultural exposure will enhance the facility and use of culturally specific communication cues. At its core, the requisite transformation will require medical students to adapt to other cultures and greater representation by marginalized and stigmatized populations not only among the studentry but staff and faculty. The time is now to ensure that the physicians we produce can care for all Americans. What cannot be taught must be identified by the selection process. Competence with half the population is a failure for American medicine.

Using bodily postures in the treatment of anorexia nervosa: Effects of power posing on interoception and affective states

OBJECTIVE

Power posing involves the adoption of an expansive bodily posture. This study examined whether power posing could benefit individuals with anorexia nervosa (AN) and women with normal weight in regards to interoceptive ability and affective states.

METHOD

Participants included 50 inpatients and outpatients with AN as well as 51 normal-weight women. Interoceptive accuracy (IAcc), measured by the heartbeat tracking task and interoceptive sensibility (IS), measured by confidence ratings, were assessed at baseline, after a single power posing session and after 1 week of daily training. Also, the short-term effects of power posing on subjective feelings of dominance, pleasantness, and arousal were investigated.

RESULTS

Both groups increased in their IAcc after one power posing session. Also, there was a significant main effect of time on feelings of dominance and pleasantness in the short-term. Women with AN displayed lower levels of IS, dominance, and pleasantness as well as higher levels of arousal than women without AN.

DISCUSSION

These findings suggest that power posing has the potential to increase IAcc, subjective feelings of power and pleasant affect in the short-term. Further research should investigate which mechanisms foster the effectiveness of this intervention to tailor it to the needs of women with AN.

Gender Effect on Political Leaders' Nonverbal Communicative Structure during the COVID-19 Crisis

During the COVID-19 pandemic, there has been intense interest in political leaders' nonverbal communicative structures (NCS) during televised appearances. This study analyzes the effect of gender on leaders' NCS and presents theoretical and analytical frameworks of gendered NCS. We analyzed 20 televised appearances by 10 heads of state (five males and five females) from democratic Western countries during the COVID-19 pandemic. The findings revealed that gender had a significant effect on leaders' NCS, indicating that leaders presented NCS that corresponded to their gender. Male leaders' masculine NCS included competition, warning, threatening, and scaring behavior, broad proxemics, tension leakage, and illustrative gestures, while female leaders presented feminine NCS of cooperativeness, emotional communication, empathy, optimism, eye contact, and flexible expressions. Furthermore, the effect of gender on leaders' NCS had an interaction effect with the situation of the pandemic, indicating that countries with a female leader had fewer diseased and severe cases and more calmness and healing NCS. The conclusions present theoretical and analytical frameworks that explain the central effect of gender on contemporary leaders' NCS. This study develops advanced distinctive profiles for male versus female leaders' NCS of emotions, cognition, and behavior during a crisis.

Semi-automated brain responses in communication: A magnetoencephalographic hyperscanning study

Face to face communication is interactive, and involves continuous feedforward and feedback of information, thoughts, and feelings to the opposite party. To accurately assess the neural processing underlying these interactions, synchronous and simultaneous recording of the brain activity from both parties is needed, a method known as hyperscanning. Here, we investigated the neural processing underlying nonverbal face-to-face communication using a magnetoencephalographic (MEG) hyperscanning system, comprising two fiber optically connected MEGs. Eight pairs of subjects participated. Each individual in each pair viewed a combined 80 randomized 20 s trials of 40 real-time and 40 recorded (hereafter, real and simulated, respectively) videos of the opposite party's face. Non-verbal communication through actions such as gaze, eye blinks, and facial expression was intrinsically only possible during real videos. After each trial, subjects individually subjectively discriminated whether the viewed video was real or simulated. Overall subjective discrimination accuracies were slightly but significantly above chance level. Statistical analysis of brain activity revealed a significant three way interaction between theta-band rhythm amplitude, video type, and subjective discrimination response in the right frontal cortex. Additionally, when subjects responded that videos were simulated, theta activity was significantly lower for real videos compared with simulated videos (p = 0.01). This result not only demonstrates the importance of right frontal theta activity during non-verbal communication, but also indicates the existence of unconscious, semi-automated neural processing during non-verbal communication that underlies one's ability to subjectively discriminate whether or not the opposite party is real.

Nonverbal communication in selfies posted on Instagram: Another look at the effect of gender on vertical camera angle

Background

Selfies are a novel social phenomenon that is gradually beginning to receive attention within the cognitive sciences. Several studies have documented biases that may be related to nonverbal communicative intentions. For instance, in selfies posted on the dating platform Tinder males but not females prefer camera views from below (Sedgewick, Flath & Elias, 2017). We re-examined this study to assess whether this bias is confined to dating selection contexts and to compare variability between individuals and between genders.

Methods

Three raters evaluated vertical camera position in 2000 selfies– 1000 by males and 1000 by females—posted in Instagram.

Results

We found that the choices of camera angle do seem to vary depending on the context under which the selfies were uploaded. On Tinder, females appear more likely to choose neutral, frontal presentations than they do on Instagram, whereas males on Tinder appear more likely to opt for camera angles from below than on Instagram.

Conclusions

This result confirms that the composition of selfies is constrained by factors affecting nonverbal communicative intentions.

Gesture, communication, and adult acquired hearing loss

Nonverbal communication, specifically hand and arm movements (commonly known as gesture), has long been recognized and explored as a significant element in human interaction as well as potential compensatory behavior for individuals with communication difficulties. The use of gesture as a compensatory communication method in expressive and receptive human communication disorders has been the subject of much investigation. Yet within the context of adult acquired hearing loss, gesture has received limited research attention and much remains unknown about patterns of nonverbal behaviors in conversations in which hearing loss is a factor. This paper presents key elements of the background of gesture studies and the theories of gesture function and production followed by a review of research focused on adults with hearing loss and the role of gesture and gaze in rehabilitation. The current examination of the visual resource of co-speech gesture in the context of everyday interactions involving adults with acquired hearing loss suggests the need for the development of an evidence base to effect enhancements and changes in the way in which rehabilitation services are conducted.

The influence of conversation parameters on gesture production in aphasia

Conversation is important in everyday life and this importance is not diminished in aphasia. Context parameters such as topic and partner are known to influence the linguistic content of conversations. With gesture being closely linked to language, these parameters may influence gestures used in conversations. This has not been investigated in previous studies. This study explored the spontaneous use of gestures in the conversations of participants with aphasia (PWA) and neurologically healthy participants (NHP). It aimed to examine the influence of conversation topic and partner on gesture production overall and on the production of semantically rich and empty gestures. Twenty PWA and 21 NHP were filmed during conversations with different topics (narrative & procedural) and different partners (familiar & unfamiliar). Analysis 1 investigated the influence of the conversation topic on gesture production overall and on the production of semantically rich and empty gestures. In Analysis 2, the influence of the conversation partner on gesture production was investigated. Both groups produced significantly more gestures in procedural than in narrative conversations. Moreover, PWA and NHP produced significantly more semantically rich gestures in procedural conversations. In terms of the conversation partner, both groups produced significantly more gestures in the conversations with the unfamiliar than in those with the familiar conversation partner. For all findings, there were no group differences and no interactions between group and context parameters. These findings shed light on factors that influence gesture production and suggest that both modalities should be viewed together as a communicative resource for PWA.

Modelling the early expressive communicative trajectories of infants/toddlers with early cochlear implants

For children with normal hearing (NH), early communication skills predict vocabulary, a precursor to grammar. Growth in early communication skills of infants with cochlear implants (CIs) was investigated using the Early Communication Indicator (ECI), a play-based observation measure. Multilevel linear growth modelling on data from six ECI sessions held at three-monthly intervals revealed significant growth overall, with a non-significant slower growth rate than that of children with NH (comparison age centred at 18 months). Analyses of gesture use and of nonword vocalisations revealed the CI group used significantly more of each, with more rapid growth. In contrast, the CI group used significantly fewer single words and multiword utterances, and with slower growth. Maternal education and time to achieve consistent CI use impacted significantly on growth for the CI sample. The results indicate that progression to vocabulary by young CI users can be supported by encouraging their use of prelinguistic communication.

Reproducible research into human chemical communication by cues and pheromones: learning from psychology's renaissance

Despite the lack of evidence that the 'putative human pheromones' androstadienone and estratetraenol ever were pheromones, almost 60 studies have claimed 'significant' results. These are quite possibly false positives and can be best seen as potential examples of the 'reproducibility crisis', sadly common in the rest of the life and biomedical sciences, which has many instances of whole fields based on false positives. Experiments on the effects of olfactory cues on human behaviour are also at risk of false positives because they look for subtle effects but use small sample sizes. Research on human chemical communication, much of it falling within psychology, would benefit from vigorously adopting the proposals made by psychologists to enable better, more reliable science, with an emphasis on enhancing reproducibility. A key change is the adoption of study pre-registration and/or Registered Reports which will also reduce publication bias. As we are mammals, and chemical communication is important to other mammals, it is likely that chemical cues are important in our behaviour and that humans may have pheromones, but new approaches will be needed to reliably demonstrate them. This article is part of the Theo Murphy meeting issue 'Olfactory communication in humans'.

Encoding fear intensity in human sweat

Humans, like other animals, have an excellent sense of smell that can serve social communication. Although ample research has shown that body odours can convey transient emotions like fear, these studies have exclusively treated emotions as categorical, neglecting the question whether emotion quantity can be expressed chemically. Using a unique combination of methods and techniques, we explored a dose-response function: Can experienced fear intensity be encoded in fear sweat? Specifically, fear experience was quantified using multivariate pattern classification (combining physiological data and subjective feelings with partial least-squares-discriminant analysis), whereas a photo-ionization detector quantified volatile molecules in sweat. Thirty-six male participants donated sweat while watching scary film clips and control (calming) film clips. Both traditional univariate and novel multivariate analysis (100% classification accuracy; Q2: 0.76; R2: 0.79) underlined effective fear induction. Using their regression-weighted scores, participants were assigned significantly above chance (83% > 33%) to fear intensity categories (low-medium-high). Notably, the high fear group (n = 12) produced higher doses of armpit sweat, and greater doses of fear sweat emitted more volatile molecules (n = 3). This study brings new evidence to show that fear intensity is encoded in sweat (dose-response function), opening a field that examines intensity coding and decoding of other chemically communicable states/traits. This article is part of the Theo Murphy meeting issue 'Olfactory communication in humans'.

Detecting Faces, Saving Lives

Your phone scans your face to unlock its screen. A social media app offers suggestions of friends to tag in photos. Airline check-in systems verify who you are as you stare into a camera. These are just a few examples of how facial recognition technology (FRT) is now ubiquitous in everyday lives. The industries of law enforcement, Internet search engines, marketing, and security have long harnessed FRT, but the technology is becoming increasingly explored in the health care setting, where its potential benefit-and risks-are much greater.

Contributions of Voice and Nonverbal Communication to Perceived Masculinity-Femininity for Cisgender and Transgender Communicators

Purpose The purpose of this study was twofold: (a) to identify a set of communication-based predictors (including both acoustic and gestural variables) of masculinity-femininity ratings and (b) to explore differences in ratings between audio and audiovisual presentation modes for transgender and cisgender communicators. Method The voices and gestures of a group of cisgender men and women (n = 10 of each) and transgender women (n = 20) communicators were recorded while they recounted the story of a cartoon using acoustic and motion capture recording systems. A total of 17 acoustic and gestural variables were measured from these recordings. A group of observers (n = 20) rated each communicator's masculinity-femininity based on 30- to 45-s samples of the cartoon description presented in three modes: audio, visual, and audio visual. Visual and audiovisual stimuli contained point light displays standardized for size. Ratings were made using a direct magnitude estimation scale without modulus. Communication-based predictors of masculinity-femininity ratings were identified using multiple regression, and analysis of variance was used to determine the effect of presentation mode on perceptual ratings. Results Fundamental frequency, average vowel formant, and sound pressure level were identified as significant predictors of masculinity-femininity ratings for these communicators. Communicators were rated significantly more feminine in the audio than the audiovisual mode and unreliably in the visual-only mode. Conclusions Both study purposes were met. Results support continued emphasis on fundamental frequency and vocal tract resonance in voice and communication modification training with transgender individuals and provide evidence for the potential benefit of modifying sound pressure level, especially when a masculine presentation is desired.

Tracking Early Sentence-Building Progress in Graphic Symbol Communication

Purpose As is the case with children who rely on spoken language, speech-language pathologists must support and track the expressive language development of children with complex communication needs who use graphic symbols to communicate. This research note presents a framework of the progression of expressive English sentence development using graphic symbols and introduces possible approaches for measuring and analyzing graphic symbol use. Method Current issues in measuring graphic symbol utterances are explored, and a range of measures designed to analyze individual graphic symbol utterances as well as larger samples of utterances are presented. Results Both the Graphic Symbol Utterance and Sentence Development Framework and suggested measures are based on years of graphic symbol intervention research, including two large ongoing research studies of preschoolers with severe speech impairments. Our framework adapts the work of Hadley (2014) to depict expressive language progression from early symbol combinations to childlike and adultlike sentences and highlights developmental patterns unique to graphic symbol productions. Adaptations of existing measures (such as mean length of utterance) as well as measures unique to graphic symbol analyses are presented and discussed. Conclusion To accurately track changes in early graphic symbol utterance growth and complexity, a multidimensional approach, which includes analyses such as symbol relevance, word class diversity, and lexical diversity, is recommended.

Investigating pedestrian waiting time at semi-controlled crossing locations: Application of multi-state models for recurrent events analysis

"Semi-controlled" crosswalks are unsignalized, but clearly marked with "yield to pedestrian within crosswalk" signs. Ideally, pedestrians can cross the street immediately after they arrive at the curb. However, real world observations show that pedestrians and vehicles are often involved in non-verbal "negotiations" to decide who should proceed first. This kind of "negotiation" often causes delays for both parties and may lead to unsafe situations. The study in this paper was based on video recordings of the waiting behaviors of 2059 pedestrians interacting with 1003 motorists at selected semi-controlled crosswalks. One such location experienced a conversion from one-way operation to two-way operation, which provided a rare opportunity for a before-and-after study at that location. Multi-state Markov models were introduced as a novel approach to correlate the dynamic process between recurrent events. Time-varying covariates related to pedestrian characteristics, traffic condition, and vehicle dynamics (distance and speed) turned out to be significant. The analytical method developed in this study provides a tool to dynamically model pedestrian waiting decisions with uncertainties. Model results reveal that, after the conversion from one-way to two-way operation, the probability of a pedestrian accepting a lag decreases from 69.7% to just below 60% on the same street. In addition, pedestrians are more hesitant to cross a two-way street than a one-way street. Countermeasures that increase motorist yielding rate or reduce pedestrian confusion will enhance safety such crossing locations.