Socio-affective touch expression database

Pupillary response during social emotion tasks in autism spectrum disorder

Autistic individuals encounter challenges in recognizing emotional expressions of others. Pupillary response has been proposed as an indicator of arousal dysregulation or cognitive load. The pupillary response of autistic individuals during socio-affective tasks remains unclear. This study investigated pupillary response in autistic adults when viewing emotional faces/eyes and recognizing emotions during the Reading the Mind in the Eyes Test (RMET) and watching interpersonal touch scenes in the social touch task. The study included 98 participants diagnosed with autism spectrum disorder and 37 typically developing controls (TD). Pupil size was measured using the Tobii X2-30 Eye Tracker. The results showed that autistic adults had larger maximal pupil sizes, smaller minimal pupil sizes, and greater change rates of pupil size, particularly during the RMET Eyes task. Clinical correlations revealed that attention switching difficulty positively correlated with mean pupil size in TD participants, while social communication deficits positively correlated with mean pupil size in autistic participants. In conclusion, our findings suggest atypical pupillary responses in autistic adults during socio-affective tasks, indicating heightened cognitive demand. Further investigation is necessary to understand the underlying mechanisms and their association with autistic traits.

Phantom Illusion Based Vibrotactile Rendering of Affective Touch Patterns

Physically accurate (authentic) reproduction of affective touch patterns on the forearm is limited by actuator technology. However, in most VR applications a direct comparison with actual touch is not possible. Here, the plausibility is only compared to the user's expectation. Focusing on the approach of plausible instead of authentic touch reproduction enables new rendering techniques, like the utilization of the phantom illusion to create the sensation of moving vibrations. Following this idea, a haptic armband array (4x2 vibrational actuators) was built to investigate the possibilities of recreating plausible affective touch patterns with vibration. The novel aspect of this work is the approach of touch reproduction with a parameterized rendering strategy, enabling the integration in VR. A first user study evaluates suitable parameter ranges for vibrational touch rendering. Duration of vibration and signal shape influence plausibility the most. A second user study found high plausibility ratings in a multimodal scenario and confirmed the expressiveness of the system. Rendering device and strategy are suitable for a various stroking patterns and applicable for emerging research on social affective touch reproduction.

Propensity to trust shapes perceptions of comforting touch between trustworthy human and robot partners

Touching a friend to comfort or be comforted is a common prosocial behaviour, firmly based in mutual trust. Emphasising the interactive nature of trust and touch, we suggest that vulnerability, reciprocity and individual differences shape trust and perceptions of touch. We further investigate whether these elements also apply to companion robots. Participants (n = 152) were exposed to four comics depicting human-human or human-robot exchanges. Across conditions, one character was sad, the other initiated touch to comfort them, and the touchee reciprocated the touch. Participants first rated trustworthiness of a certain character (human or robot in a vulnerable or comforting role), then evaluated the two touch phases (initiation and reciprocity) in terms of interaction realism, touch appropriateness and pleasantness, affective state (valence and arousal) attributed to the characters. Results support an interactive account of trust and touch, with humans being equally trustworthy when comforting or showing vulnerability, and reciprocity of touch buffering sadness. Although these phenomena seem unique to humans, propensity to trust technology reduces the gap between how humans and robots are perceived. Two distinct trust systems emerge: one for human interactions and another for social technologies, both necessitating trust as a fundamental prerequisite for meaningful physical contact.

Rapid Processing of Observed Touch through Social Perceptual Brain Regions: An EEG-fMRI Fusion Study

Seeing social touch triggers a strong social-affective response that involves multiple brain networks, including visual, social perceptual, and somatosensory systems. Previous studies have identified the specific functional role of each system, but little is known about the speed and directionality of the information flow. Is this information extracted via the social perceptual system or from simulation from somatosensory cortex? To address this, we examined the spatiotemporal neural processing of observed touch. Twenty-one human participants (seven males) watched 500-ms video clips showing social and nonsocial touch during electroencephalogram (EEG) recording. Visual and social-affective features were rapidly extracted in the brain, beginning at 90 and 150 ms after video onset, respectively. Combining the EEG data with functional magnetic resonance imaging (fMRI) data from our prior study with the same stimuli reveals that neural information first arises in early visual cortex (EVC), then in the temporoparietal junction and posterior superior temporal sulcus (TPJ/pSTS), and finally in the somatosensory cortex. EVC and TPJ/pSTS uniquely explain EEG neural patterns, while somatosensory cortex does not contribute to EEG patterns alone, suggesting that social-affective information may flow from TPJ/pSTS to somatosensory cortex. Together, these findings show that social touch is processed quickly, within the timeframe of feedforward visual processes, and that the social-affective meaning of touch is first extracted by a social perceptual pathway. Such rapid processing of social touch may be vital to its effective use during social interaction.SIGNIFICANCE STATEMENT Seeing physical contact between people evokes a strong social-emotional response. Previous research has identified the brain systems responsible for this response, but little is known about how quickly and in what direction the information flows. We demonstrated that the brain processes the social-emotional meaning of observed touch quickly, starting as early as 150 ms after the stimulus onset. By combining electroencephalogram (EEG) data with functional magnetic resonance imaging (fMRI) data, we show for the first time that the social-affective meaning of touch is first extracted by a social perceptual pathway and followed by the later involvement of somatosensory simulation. This rapid processing of touch through the social perceptual route may play a pivotal role in effective usage of touch in social communication and interaction.

Affect Recognition in Hand-Object Interaction Using Object-Sensed Tactile and Kinematic Data

We investigate the recognition of the affective states of a person performing an action with an object, by processing the object-sensed data. We focus on sequences of basic actions such as grasping and rotating, which are constituents of daily-life interactions. iCube, a 5 cm cube, was used to collect tactile and kinematics data that consist of tactile maps (without information on the pressure applied to the surface), and rotations. We conduct two studies: classification of i) emotions and ii) the vitality forms. In both, the participants perform a semi-structured task composed of basic actions. For emotion recognition, 237 trials by 11 participants associated with anger, sadness, excitement, and gratitude were used to train models using 10 hand-crafted features. The classifier accuracy reaches up to 82.7%. Interestingly, the same classifier when learned exclusively with the tactile data performs on par with its counterpart modeled with all 10 features. For the second study, 1135 trials by 10 participants were used to classify two vitality forms. The best-performing model differentiated gentle actions from rude ones with an accuracy of 84.85%. The results also confirm that people touch objects differently when performing these basic actions with different affective states and attitudes.

The Language of Social Touch Is Intuitive and Quantifiable

Touch is a powerful communication tool, but we have a limited understanding of the role played by particular physical features of interpersonal touch communication. In this study, adults living in Sweden performed a task in which messages (attention, love, happiness, calming, sadness, and gratitude) were conveyed by a sender touching the forearm of a receiver, who interpreted the messages. Two experiments (N = 32, N = 20) showed that within close relationships, receivers could identify the intuitive touch expressions of the senders, and we characterized the physical features of the touches associated with successful communication. Facial expressions measured with electromyography varied by message but were uncorrelated with communication performance. We developed standardized touch expressions and quantified the physical features with 3D hand tracking. In two further experiments (N = 20, N = 16), these standardized expressions were conveyed by trained senders and were readily understood by strangers unacquainted with the senders. Thus, the possibility emerges of a standardized, intuitively understood language of social touch.

Social Touch, Social Isolation, and Loneliness in Borderline Personality Disorder During the COVID-19 Pandemic

BACKGROUND

Interpersonal impairments in borderline personality disorder (BPD) are characterised by a lack in the sense of belonging and the fear of being excluded. One feature of interactions that can promote a sense of social belonging is interpersonal touch. While some studies suggest that individuals with BPD experience social touch as less pleasurable than healthy individuals (HCs), there are no studies that investigated whether this difference is associated with feeling less socially connected. This question is particularly important during the COVID-19 pandemic, since one central behavioural recommendation is "social distancing". An increase in loneliness has been discussed as a consequence and it has been suggested that individuals with BPD may be particularly burdened. However, the primary goal of "social distancing" is not preventing social contacts, but physical proximity. In our study we investigated the interplay between feeling close to others, contact frequency and the appraisal of social touch in BPD. We were additionally interested in whether these factors contribute to the burden through "physical distancing".

METHODS

We assessed subjective and objective social isolation, the need, importance, and liking of social touch, as well as the burden through "physical distancing" policies in 130 women (61 BPD and 69 HCs).

RESULTS

Participants of the BPD group reported higher loneliness, less social contacts and a lower need for, importance and liking of social touch compared to HCs. Larger social networks, higher frequency of in-person contacts and higher liking and importance of social touch were associated with lower levels of loneliness. Both groups did not differ regarding their burden through "physical distancing". A higher need for and lower importance of social touch predicted a higher burden through "physical distancing".

CONCLUSIONS

A positive appraisal of social touch was associated with less loneliness, independently of an individual's objective social isolation. In BPD, impairments of this fundamental facet of social interaction might hamper forming and strengthening of social bonds and contribute to the patients' interpersonal dysfunction. Changing the attitude towards social touch and in consequence its liking and importance in social interaction might provide one avenue to improve the sense of social connectedness in these patients.

The unreliable influence of multivariate noise normalization on the reliability of neural dissimilarity

Representational similarity analysis (RSA) is a key element in the multivariate pattern analysis toolkit. The central construct of the method is the representational dissimilarity matrix (RDM), which can be generated for datasets from different modalities (neuroimaging, behavior, and computational models) and directly correlated in order to evaluate their second-order similarity. Given the inherent noisiness of neuroimaging signals it is important to evaluate the reliability of neuroimaging RDMs in order to determine whether these comparisons are meaningful. Recently, multivariate noise normalization (NN_M) has been proposed as a widely applicable method for boosting signal estimates for RSA, regardless of choice of dissimilarity metrics, based on evidence that the analysis improves the within-subject reliability of RDMs (Guggenmos et al. 2018; Walther et al. 2016). We revisited this issue with three fMRI datasets and evaluated the impact of NN_M on within- and between-subject reliability and RSA effect sizes using multiple dissimilarity metrics. We also assessed its impact across regions of interest from the same dataset, its interaction with spatial smoothing, and compared it to GLMdenoise, which has also been proposed as a method that improves signal estimates for RSA (Charest et al. 2018). We found that across these tests the impact of NN_M was highly variable, as also seems to be the case for other analysis choices. Overall, we suggest being conservative before adding steps and complexities to the (pre)processing pipeline for RSA.

Calming Effects of Touch in Human, Animal, and Robotic Interaction-Scientific State-of-the-Art and Technical Advances

Small everyday gestures such as a tap on the shoulder can affect the way humans feel and act. Touch can have a calming effect and alter the way stress is handled, thereby promoting mental and physical health. Due to current technical advances and the growing role of intelligent robots in households and healthcare, recent research also addressed the potential of robotic touch for stress reduction. In addition, touch by non-human agents such as animals or inanimate objects may have a calming effect. This conceptual article will review a selection of the most relevant studies reporting the physiological, hormonal, neural, and subjective effects of touch on stress, arousal, and negative affect. Robotic systems capable of non-social touch will be assessed together with control strategies and sensor technologies. Parallels and differences of human-to-human touch and human-to-non-human touch will be discussed. We propose that, under appropriate conditions, touch can act as (social) signal for safety, even when the interaction partner is an animal or a machine. We will also outline potential directions for future research and clinical relevance. Thereby, this review can provide a foundation for further investigations into the beneficial contribution of touch by different agents to regulate negative affect and arousal in humans.

The EmojiGrid as a rating tool for the affective appraisal of touch

In this study we evaluate the convergent validity of a new graphical self-report tool (the EmojiGrid) for the affective appraisal of perceived touch events. The EmojiGrid is a square grid labeled with facial icons (emoji) showing different levels of valence and arousal. The EmojiGrid is language independent and efficient (a single click suffices to report both valence and arousal), making it a practical instrument for studies on affective appraisal. We previously showed that participants can intuitively and reliably report their affective appraisal (valence and arousal) of visual, auditory and olfactory stimuli using the EmojiGrid, even without additional (verbal) instructions. However, because touch events can be bidirectional and dynamic, these previous results cannot be generalized to the touch domain. In this study, participants reported their affective appraisal of video clips showing different interpersonal (social) and object-based touch events, using either the validated 9-point SAM (Self-Assessment Mannikin) scale or the EmojiGrid. The valence ratings obtained with the EmojiGrid and the SAM are in excellent agreement. The arousal ratings show good agreement for object-based touch and moderate agreement for social touch. For social touch and at more extreme levels of valence, the EmojiGrid appears more sensitive to arousal than the SAM. We conclude that the EmojiGrid can also serve as a valid and efficient graphical self-report instrument to measure human affective response to a wide range of tactile signals.

A Comparison of Functional Networks Derived From Representational Similarity, Functional Connectivity, and Univariate Analyses

The invention of representational similarity analysis [RSA, following multi-voxel pattern analysis (MVPA)] has allowed cognitive neuroscientists to identify the representational structure of multiple brain regions, moving beyond functional localization. By comparing these structures, cognitive neuroscientists can characterize how brain areas form functional networks. Univariate analysis (UNIVAR) and functional connectivity analysis (FCA) are two other popular methods to identify functional networks. Despite their popularity, few studies have examined the relationship between networks from RSA with those from UNIVAR and FCA. Thus, the aim of the current study is to examine the similarities between neural networks derived from RSA with those from UNIVAR and FCA to explore how these methods relate to each other. We analyzed the data of a previously published study with the three methods and compared the results by performing (partial) correlation and multiple regression analysis. Our findings reveal that neural networks resulting from RSA, UNIVAR, and FCA methods are highly similar to each other even after ruling out the effect of anatomical proximity between the network nodes. Nevertheless, the neural network from each method shows unique organization that cannot be explained by any of the other methods. Thus, we conclude that the RSA, UNIVAR and FCA methods provide similar but not identical information on how brain regions are organized in functional networks.

Intact neural representations of affective meaning of touch but lack of embodied resonance in autism: a multi-voxel pattern analysis study

Background

Humans can easily grasp the affective meaning of touch when observing social interactions. Several neural systems support this ability, including the theory of mind (ToM) network and the somatosensory system linked to embodied resonance, but it is unclear how these systems are affected in autism spectrum disorder (ASD). Individuals with ASD exhibit impairments in the use of nonverbal communication such as social and reciprocal touch. Despite the importance of touch in social communication and the reported touch aversion in ASD, surprisingly little is known about the neural systems underlying impairments in touch communication in ASD.

Methods

The present study applies a dynamic and socially meaningful stimulus set combined with functional magnetic resonance imaging (fMRI) to pinpoint atypicalities in the neural circuitry underlying socio-affective touch observation in adults with ASD. Twenty-one adults with ASD and 21 matched neurotypical adults evaluated the valence and arousal of 75 video fragments displaying touch interactions. Subsequently, they underwent fMRI while watching the same videos. Using multi-voxel pattern analysis (MVPA) and multiple regression analysis, we examined which brain regions represent the socio-affective meaning of observed touch. To further understand the brain-behavior relationship, we correlated the strength of affective representations in the somatosensory cortex with individuals' attitude towards social touch in general and with a quantitative index of autism traits as measured by the Social Responsiveness Scale.

Results

Results revealed that the affective meaning of touch was well represented in the temporoparietal junction, a core mentalizing area, in both groups. Conversely, only the neurotypical group represented affective touch in the somatosensory cortex, a region involved in self-experienced touch. Lastly, irrespective of the group, individuals with a more positive attitude towards receiving, witnessing, and providing social touch and with a higher score on social responsivity showed more differentiated representations of the affective meaning of touch in these somatosensory areas.

Conclusions

Together, our findings imply that male adults with ASD show intact cognitive understanding (i.e., "knowing") of observed socio-affective touch interactions, but lack of spontaneous embodied resonance (i.e., "feeling").

Validation and psychometric properties of the Russian version of the Touch Experiences and Attitudes Questionnaire (TEAQ-37 Rus)

It has been demonstrated that nurturing and affiliative touch is essential for human emotional and physical well-being throughout our entire life. Within the last 30 years a system of low-threshold mechanosensitive C fibers innervating the hairy skin was discovered and described; this system is hypothesized to represent the neurobiological substrate for the affective and rewarding properties of touch. This discovery opens new perspectives for multidisciplinary research of the role of affiliative social touch in health and disease, and calls for establishing novel psychometric tools assessing individual differences in the domain of affective touch. The main objective of the study was to construct and validate a Russian version of the Touch Experiences and Attitudes Questionnaire (TEAQ), a self-report measure recently developed to quantify individual experience and attitude to social and affective touch. A pool of 117 items was translated into Russian and all the items were assessed for appropriateness for Russian culture (232 participants). After exploring the factor structure (468 participants), we composed a 37-item questionnaire (TEAQ-37 Rus) characterized by good reliability and a clear 5-factor structure, covering the aspects of attitude to intimate touch, attitude to friendly touch, attitude to self-care, current intimate touch experiences, and childhood touch experiences. Confirmatory factor analysis (551 participants) has demonstrated good consistency and reliability of the 5-factor structure of the TEAQ-37 Rus. Cross-validation research demonstrated moderate positive correlations between predisposition to social touch and emotional intelligence; positive correlations with extraversion and openness facets of the Big Five personality model were also found. As predicted, participants with higher TEAQ-37 Rus scores rated all observed kinds of touch as more pleasant, with a particular preference for slow touch. We anticipate that this questionnaire will be a valuable tool for researchers of social touch, touch perception abnormalities, and the importance of touch experiences for emotional and mental health.

Social touch and human development

Social touch is a powerful force in human development, shaping social reward, attachment, cognitive, communication, and emotional regulation from infancy and throughout life. In this review, we consider the question of how social touch is defined from both bottom-up and top-down perspectives. In the former category, there is a clear role for the C-touch (CT) system, which constitutes a unique submodality that mediates affective touch and contrasts with discriminative touch. Top-down factors such as culture, personal relationships, setting, gender, and other contextual influences are also important in defining and interpreting social touch. The critical role of social touch throughout the lifespan is considered, with special attention to infancy and young childhood, a time during which social touch and its neural, behavioral, and physiological contingencies contribute to reinforcement-based learning and impact a variety of developmental trajectories. Finally, the role of social touch in an example of disordered development –autism spectrum disorder—is reviewed.

The multidimensional representational space of observed socio-affective touch experiences

Observed touch interactions provide useful information on how others communicate with the external world. Previous studies revealed shared neural circuits between the direct experience and the passive observation of simple touch, such as being stroked/slapped. Here, we investigate the complexity of the neural representations underlying the understanding of others' socio-affective touch interactions. Importantly, we use a recently developed touch database that contains a larger range of more complex social and non-social touch interactions. Participants judged affective aspects of each touch event and were scanned while watching the same videos. Using correlational multivariate pattern analysis methods, we obtained neural similarity matrices in 18 regions of interest from five different networks: somatosensory, pain, the theory of mind, visual and motor regions. Among them, four networks except motor cortex represent the social nature of the touch, whereas fine-detailed affective information is reflected in more targeted areas such as social brain regions and somatosensory cortex. Lastly, individual social touch preference at the behavioral level was correlated with the involvement of somatosensory areas on representing affective information, suggesting that individuals with higher social touch preference exhibit stronger vicarious emotional responses to others' social touch experiences. Together, these results highlight the overall complexity and the individual modulation of the distributed neural representations underlying the processing of observed socio-affective touch.

•

∙Neural bases of observed touch are investigated with touch videos and MVPA.

•

∙Social touch evokes stronger activation in the theory of mind (ToM) network.

•

∙The ToM network represents affective meanings of observed social touch events.

•

∙Affective representations of observed touch are present in somatosensory areas.

•

∙Affective representations in S1 relate to individual's attitude towards touch.