Changing motor perception by sensorimotor conflicts and body ownership

In Case of Doubt, One Follows One's Self: The Implicit Guidance of the Embodied Self-Avatar

The sense of embodiment in virtual reality (VR) is commonly understood as the subjective experience that one's physical body is substituted by a virtual counterpart, and is typically achieved when the avatar's body, seen from a first-person view, moves like one's physical body. Embodiment can also be experienced in other circumstances (e.g., in third-person view) or with imprecise or distorted visuo-motor coupling. It was moreover observed, in various cases of small or progressive temporal and spatial manipulations of avatars' movements, that participants may spontaneously follow the movement shown by the avatar. The present work investigates whether, in some specific contexts, participants would follow what their avatar does even when large movement discrepancies occur, thereby extending the scope of understanding of the self-avatar follower effect beyond subtle changes of motion or speed manipulations. We conducted an experimental study in which we introduced uncertainty about which movement to perform at specific times and analyzed participants' movements and subjective feedback after their avatar showed them an incorrect movement. Results show that, when in doubt, participants were influenced by their avatar's movements, leading them to perform that particular error twice more often than normal. Importantly, results of the embodiment score indicate that participants experienced a dissociation with their avatar at those times. Overall, these observations not only demonstrate the possibility of provoking situations in which participants follow the guidance of their avatar for large motor distortions, despite their awareness about the avatar movement disruption and on the possible influence it had on their choice, and, importantly, exemplify how the cognitive mechanism of embodiment is deeply rooted in the necessity of having a body.

Disentangling the Neural Correlates of Agency, Ownership and Multisensory Processing

The experience of the self as an embodied agent in the world is an essential aspect of human consciousness. This experience arises from the feeling of control over one's bodily actions, termed the Sense of Agency, and the feeling that the body belongs to the self, Body Ownership. Despite long-standing philosophical and scientific interest in the relationship between the body and brain, the neural systems involved in Body Ownership and Sense of Agency, and especially their interactions, are not yet understood. In this preregistered study using the Moving Rubber Hand Illusion inside an MR-scanner, we aimed to uncover the relationship between Body Ownership and Sense of Agency in the human brain. Importantly, by using both visuomotor and visuotactile stimulations and measuring online trial-by-trial fluctuations in the illusion magnitude, we were able to disentangle brain systems related to objective sensory stimulation and subjective judgments of the bodily-self. Our results indicate that at both the behavioral and neural levels, Body Ownership and Sense of Agency are strongly interrelated. Multisensory regions in the occipital and fronto-parietal regions encoded convergence of sensory stimulation conditions. The subjective judgments of the bodily-self were related to BOLD fluctuations in the Somatosensory cortex and in regions not activated by the sensory conditions, such as the insular cortex and precuneus. Our results highlight the convergence of multisensory processing in specific neural systems for both Body Ownership and Sense of Agency with partially dissociable regions for subjective judgments in regions of the Default Mode Network.

Avatar error in your favor: Embodied avatars can fix users' mistakes without them noticing

In immersive Virtual Reality (VR), users can experience the subjective feeling of embodiment for the avatar representing them in a virtual world. This is known to be strongly supported by a high Sense of Agency (SoA) for the movements of the avatar that follows the user. In general, users do not self-attribute actions of their avatar that are different from the one they actually performed. The situation is less clear when actions of the avatar satisfies the intention of the user despite distortions and noticeable differences between user and avatar movements. Here, a within-subject experiment was condutected to determine wether a finger swap helping users to achieve a task would be more tolerated than one penalizing them. In particular, in a context of fast-paced finger movements and with clear correct or incorrect responses, we swapped the finger animation of the avatar (e.g. user moves the index finger, the avatar moves the middle one) to either automatically correct for spontaneous mistakes or to introduce incorrect responses. Subjects playing a VR game were asked to report when they noticed the introduction of a finger swap. Results based on 3256 trials (∼24% of swaps noticed) show that swaps helping users have significantly fewer odds of being noticed (and with higher confidence) than the ones penalizing users. This demonstrates how the context and the intention for motor action are important factors for the SoA and for embodiment, opening new perspectives on how to design and study interactions in immersive VR.

The sense of agency in emerging technologies for human–computer integration: A review

Human–computer integration is an emerging area in which the boundary between humans and technology is blurred as users and computers work collaboratively and share agency to execute tasks. The sense of agency (SoA) is an experience that arises by a combination of a voluntary motor action and sensory evidence whether the corresponding body movements have somehow influenced the course of external events. The SoA is not only a key part of our experiences in daily life but also in our interaction with technology as it gives us the feeling of “I did that” as opposed to “the system did that,” thus supporting a feeling of being in control. This feeling becomes critical with human–computer integration, wherein emerging technology directly influences people’s body, their actions, and the resulting outcomes. In this review, we analyse and classify current integration technologies based on what we currently know about agency in the literature, and propose a distinction between body augmentation, action augmentation, and outcome augmentation. For each category, we describe agency considerations and markers of differentiation that illustrate a relationship between assistance level (low, high), agency delegation (human, technology), and integration type (fusion, symbiosis). We conclude with a reflection on the opportunities and challenges of integrating humans with computers, and finalise with an expanded definition of human–computer integration including agency aspects which we consider to be particularly relevant. The aim this review is to provide researchers and practitioners with guidelines to situate their work within the integration research agenda and consider the implications of any technologies on SoA, and thus overall user experience when designing future technology.

Non-motor cues do not generate the perception of self-agency: A critique of cue-integration

How does one know that (s)he is the causal agent of their motor actions? Earlier theories of sense of agency have attributed the capacity for perception of self-agency to the comparator process of the motor-control/action system. However, with the advent of the findings implying a role of non-motor cues (like affective states, beliefs, primed concepts, and social instructions or previews of actions) in the sense of agency literature, the perception of self-agency is hypothesized to be generated even by non-motor cues (based on their relative reliability or weighting estimate); and, this theory is come to be known as the cue-integration of sense of agency. However, the cue-integration theory motivates skepticism about whether it is falsifiable and whether it is plausible that non-motor cues that are sensorily unrelated to typical sensory processes of self-agency have the capacity to produce a perception of self-agency. To substantiate this skepticism, I critically analyze the experimental operationalizations of cue-integration-with the (classic) vicarious agency experiment as a case study-to show that (1) the participants in these experiments are ambiguous about their causal agency over motor actions, (2) thus, these participants resort to reports of self-agency as heuristic judgments (under ambiguity) rather than due to cue-integration per se, and (3) there might not have occurred cue-integration based self-agency reports if these experimental operationalizations had eliminated ambiguity about the causal agency. Thus, I conclude that the reports of self-agency (observed in typical non-motor cues based cue-integration experiments) are not instances of perceptual effect-that are hypothesized to be produced by non-motor cues-but are of heuristic judgment effect.

Reaching articular limits can negatively impact embodiment in virtual reality

Providing Virtual Reality(VR) users with a 3D representation of their body complements the experience of immersion and presence in the virtual world with the experience of being physically located and more personally involved. A full-body avatar representation is known to induce a Sense of Embodiment (SoE) for this virtual body, which is associated with improvements in task performance, motivation and motor learning. Recent experimental research on embodiment provides useful guidelines, indicating the extent of discrepancy tolerated by users and, conversely, the limits and disruptive events that lead to a break in embodiment (BiE). Based on previous works on the limit of agency under movement distortion, this paper describes, studies and analyses the impact of a very common yet overlooked embodiment limitation linked to articular limits when performing a reaching movement. We demonstrate that perceiving the articular limit when fully extending the arm provides users with an additional internal proprioceptive feedback which, if not matched in the avatar's movement, leads to the disruptive realization of an incorrect posture mapping. This study complements previous works on self-contact and visuo-haptic conflicts and emphasizes the risk of disrupting the SoE when distorting users' movements or using a poorly-calibrated avatar.

The role of motor memory dynamics in structuring bodily self-consciousness

Bodily self-consciousness has been considered a sensorimotor root of self-consciousness. If this is the case, how does sensorimotor memory, which is important for the prediction of sensory consequences of volitional actions, influence awareness of bodily self-consciousness? This question is essential for understanding the effective acquisition and recovery of self-consciousness following its impairment, but it has remained unexamined. Here, we investigated how body ownership and agency recovered following body schema distortion in a virtual reality environment along with two kinds of motor memories: memories that were rapidly updated and memories that were gradually updated. We found that, although agency and body ownership recovered in parallel, the recovery of body ownership was predicted by fast memories and that of agency was predicted by slow memories. Thus, the bodily self was represented in multiple motor memories with different dynamics. This finding demystifies the controversy about the causal relationship between body ownership and agency.

Trust as Extended Control: Human-Machine Interactions as Active Inference

In order to interact seamlessly with robots, users must infer the causes of a robot's behavior-and be confident about that inference (and its predictions). Hence, trust is a necessary condition for human-robot collaboration (HRC). However, and despite its crucial role, it is still largely unknown how trust emerges, develops, and supports human relationship to technological systems. In the following paper we review the literature on trust, human-robot interaction, HRC, and human interaction at large. Early models of trust suggest that it is a trade-off between benevolence and competence; while studies of human to human interaction emphasize the role of shared behavior and mutual knowledge in the gradual building of trust. We go on to introduce a model of trust as an agent' best explanation for reliable sensory exchange with an extended motor plant or partner. This model is based on the cognitive neuroscience of active inference and suggests that, in the context of HRC, trust can be casted in terms of virtual control over an artificial agent. Interactive feedback is a necessary condition to the extension of the trustor's perception-action cycle. This model has important implications for understanding human-robot interaction and collaboration-as it allows the traditional determinants of human trust, such as the benevolence and competence attributed to the trustee, to be defined in terms of hierarchical active inference, while vulnerability can be described in terms of information exchange and empowerment. Furthermore, this model emphasizes the role of user feedback during HRC and suggests that boredom and surprise may be used in personalized interactions as markers for under and over-reliance on the system. The description of trust as a sense of virtual control offers a crucial step toward grounding human factors in cognitive neuroscience and improving the design of human-centered technology. Furthermore, we examine the role of shared behavior in the genesis of trust, especially in the context of dyadic collaboration, suggesting important consequences for the acceptability and design of human-robot collaborative systems.

Modulation of Bodily Self-Consciousness by Self and External Touch

The full body illusion (FBI) is a bodily illusion based on the application of multisensory conflicts that induce changes in bodily self-consciousness (BSC). This has been used to study cognitive brain mechanisms underlying body ownership and related aspects of self-consciousness. Typically, such paradigms employ external passive multisensory stimulation, thus neglecting the possible contributions of self-generated action and haptic cues to body ownership. In this article, the effects of both external and voluntary self-touch on BSC were examined with a robotics-based FBI paradigm. We compared the effects of classical passive visuotactile stimulation and active self-touch (in which experimental participants had a sense of agency over the tactile stimulation) on the FBI. We evaluated these effects using a questionnaire, crossmodal congruency task, and measurements of changes in self-location. The results indicated that both synchronous passive visuotactile stimulation and synchronous active self-touch induced illusory ownership over a virtual body, without significant differences in their magnitudes. However, the FBI induced by active self-touch was associated with a larger drift in self-location towards the virtual body. These results show that movement-related signals arising from self-touch impact the BSC not only for hand ownership but also for torso-centered body ownership and related aspects of BSC.

Assessing the Relationship between Sense of Agency, the Bodily-Self and Stress: Four Virtual-Reality Experiments in Healthy Individuals

The bodily-self, our experience of being a body, arises from the interaction of several processes. For example, embodied Sense of Agency (SoA), the feeling of controlling our body’s actions, is a fundamental facet of the bodily-self. SoA is disturbed in psychosis, with stress promoting its inception. However, there is little knowledge regarding the relationship between SoA, stress, and other facets of the bodily-self. In four experiments manipulating embodied SoA using a virtual hand (VH), we examined (1) How is embodied SoA related to other facets of the bodily-self?; and (2) How is SoA impacted by stress? We found that increased alteration of the VH significantly decreased subjective ratings of SoA and body ownership (Exp. 1), supporting the close relation between SoA and body ownership. Interoceptive accuracy and SoA were positively correlated (Exp. 3), connecting awareness to one’s actions and cardiac signals. Contrary to our expectations, SoA was not related to trait anxiety (Exp. 3), nor did induced stress impair SoA (Exp. 4). Finally, we found a negative correlation between self-reported prodromal symptoms and SoA. These results strongly support the connection between SoA and the bodily-self. Whereas, SoA was not impaired by stress, and weakly related to psychotic symptoms.

Self-Body Recognition through a Mirror: Easing Spatial-Consistency Requirements for Rubber Hand Illusion

Considering that humans recognize mirror images as copies of the real world despite misinterpreting optical reflections, spatial disagreement may be accepted in rubber hand illusion (RHI) settings when a mirror is used to show a fake hand. The present study performed two experiments to reveal how self-body recognition of a fake hand via a mirror affects RHI. First, we tested whether illusory ownership of a fake hand seen in a mirror could be induced in our experimental environment (screening experiment). Subjective evaluations using an RHI questionnaire demonstrated that embodiment of the rubber hand was evoked in the presence or absence of a mirror. We then examined whether using a mirror image for RHI allows disagreement in orientation (45 ∘ ) between the rubber and actual hands (main experiment). The participants experienced RHI even when the actual and rubber hands were incongruent in terms of orientation. These findings suggest that using a mirror masks subtle spatial incongruency or degrades the contribution of visual cues for spatial recognition and facilitates multisensory integration for bodily illusions.

UnReal? Investigating the Sense of Reality and Psychotic Symptoms with Virtual Reality

Distortions of reality, such as hallucinations, are common symptoms of many psychiatric conditions. Accordingly, sense of reality (SoR), the ability to discriminate between true and false perceptions, is a central criterion in the assessment of neurological and psychiatric health. Despite the critical role of the SoR in daily life, little is known about how this is formed in the mind. Here, we propose a novel theoretical and methodological framework to study the SoR and its relation to psychotic symptoms. In two experiments, we employed a specialized immersive virtual reality (VR) environment allowing for well-controlled manipulations of visual reality. We first tested the impact of manipulating visual reality on objective perceptual thresholds (just noticeable differences). In a second experiment, we tested how these manipulations affected subjective judgments of reality. The results revealed that the objective perceptual thresholds were robust and replicable, demonstrating that SoR is a stable psychometric property that can be measured experimentally. Furthermore, reality alterations reduced subjective reality judgments across all manipulated visual aspects. Finally, reduced sensitivity to changes in visual reality was related to self-reported prodromal psychotic symptoms. These results provide evidence for the relevance of SoR in the assessment of psychosis and other mental disorders in which reality is distorted.

Effects of Human Synchronous Hand Movements in Eliciting a Sense of Agency and Ownership

The self is built as an entity independent from the external world using the human ability to experience the senses of agency and ownership. Humans usually experience these senses during movement. Nevertheless, researchers recently reported that another person’s synchronous mirror-symmetrical movements elicited both agency and ownership in research participants. However, it is unclear whether this elicitation was caused by the synchronicity or the mirror symmetry of the movements. To address this question, we investigated the effect of interpersonal synchronization on the self-reported sense of agency and ownership in two conditions, using movements with and without mirror symmetry. Participants performed rhythmic hand movements while viewing the experimenter’s synchronous or random hand movements, and then reported their perceptions of agency and ownership in a questionnaire. We observed that agency and ownership were significantly elicited by the experimenter’s synchronous hand movements in both conditions. The results suggested that the synchronous movements of another person—rather than mirror- or non-mirror-symmetrical movements—appear to elicit the experience of a sense of agency and ownership. The results also suggested that people could experience these senses not only from their own movements but also from another person’s synchronous movements.

Illusory movement perception improves motor control for prosthetic hands

To effortlessly complete an intentional movement, the brain needs feedback from the body regarding the movement's progress. This largely nonconscious kinesthetic sense helps the brain to learn relationships between motor commands and outcomes to correct movement errors. Prosthetic systems for restoring function have predominantly focused on controlling motorized joint movement. Without the kinesthetic sense, however, these devices do not become intuitively controllable. We report a method for endowing human amputees with a kinesthetic perception of dexterous robotic hands. Vibrating the muscles used for prosthetic control via a neural-machine interface produced the illusory perception of complex grip movements. Within minutes, three amputees integrated this kinesthetic feedback and improved movement control. Combining intent, kinesthesia, and vision instilled participants with a sense of agency over the robotic movements. This feedback approach for closed-loop control opens a pathway to seamless integration of minds and machines.

The boundaries of the self: The sense of agency across different sensorimotor aspects

The sense of agency (SoA) is the sensation of control over our actions. SoA is thought to rely mainly upon the comparison of predictions regarding the sensory outcomes of one's actions and the actual sensory outcomes. Previous studies have shown that when a discrepancy is introduced between one's actions and the sensory feedback, the reported SoA is reduced. Experimental manipulations of SoA are typically induced by introducing a discrepancy between a motor action and visual feedback of a specific sensorimotor aspect. For example, introducing a delay or a spatial deviation between the action and its sensory feedback reduces SoA. However, it is yet unclear whether the sensorimotor prediction processes underlying SoA are related between different aspects. Here in one exploratory and one preregistered experiment we tested the sense of agency across temporal, spatial, and anatomical aspects in a within-subject design. Using a novel virtual-reality task allowing the manipulation of the visual feedback of a motor action across different aspects, we show that the sensitivity of agency is different across aspects, agency judgments are correlated across aspects within subjects and bias toward attributing the viewed action to the self or to an external source is correlated as well. Our results suggest that sensorimotor prediction mechanisms underlying SoA are related between different aspects and that people have a predisposition for the directionality of agency judgments. These findings reveal the psychophysical attributes of SoA across sensorimotor aspects. Data and preregistration are available at https://goo.gl/SkbGrb.

The Relationship Between the Virtual Hand Illusion and Motor Performance

Bodily self-consciousness consists of agency (i.e., the feeling of controlling one’s actions and causing external events) and body ownership (i.e., the feeling that one’s body belongs to one’s self). If a visual presentation of a virtual (fake) hand matches the active movement of a real hand, both the agency and body ownership of the virtual hand are induced [i.e., the active virtual hand illusion (VHI)]. However, previous active VHI studies have rarely considered the effects of goal-related movement errors (i.e., motor performance) on the senses of agency and ownership. Hence, the current study aimed to clarify the relationship between the active VHI and motor performance. To induce the VHI, 18 healthy subjects (three men and 15 women; 20.7 ± 7.3 years) were required to continuously move a virtual hand around a circle at a predetermined speed (i.e., spatial and temporal goals) using their active hand movements. While moving the virtual hand actively, five visual feedback delays were introduced: 90, 210, 330, 450, and 570 ms. It was found that the subjective ratings of both the agency and body ownership of the virtual hand decreased as a function of the delay intervals, whereas most of the spatial and temporal movement errors linearly increased. Using multiple regression analyses, we examined whether the agency and ownership ratings could be explained effectively by both the delay and movement errors. The results demonstrated that the agency was determined not only by the delay but also by the movement variability, whereas the body ownership was mostly determined by the delay. These findings suggest a possibility that the goal-related motor performance of the active VHI influences the agency judgment more strongly, while its effect on the ownership judgment is weaker.

Rethinking Body Ownership in Schizophrenia: Experimental and Meta-analytical Approaches Show no Evidence for Deficits

Schizophrenia is a severe psychiatric disorder, in which patients experience an abnormal sense of self. While deficits in sensorimotor self-representation (agency) are well documented in schizophrenia, less is known about other aspects of bodily self-representation (body ownership). Here, we tested a large cohort (N = 59) of chronic schizophrenia patients and matched controls (N = 30) on a well-established body illusion paradigm, the Full Body Illusion (FBI). In this paradigm, changes in body ownership are induced through prolonged multisensory stimulation, in which participants are stroked on their back while seeing the stroking on the back of a virtual body. When the felt and seen stroking are synchronous, participants typically feel higher identification with the seen body as well as a drift in self-location towards it. However, when the stroking is asynchronous, no such changes occur. Our results show no evidence for abnormal body ownership in schizophrenia patients. A meta-analysis of previous work corroborates this result. Thus, while schizophrenia patients may be impaired in the sense of agency, their multisensory bodily self-representation, as tested here, seems to be unaffected by the illness.

Characterizing first and third person viewpoints and their alternation for embodied interaction in virtual reality

Empirical research on the bodily self has shown that the body representation is malleable, and prone to manipulation when conflicting sensory stimuli are presented. Using Virtual Reality (VR) we assessed the effects of manipulating multisensory feedback (full body control and visuo-tactile congruence) and visual perspective (first and third person perspective) on the sense of embodying a virtual body that was exposed to a virtual threat. We also investigated how subjects behave when the possibility of alternating between first and third person perspective at will was presented. Our results support that illusory ownership of a virtual body can be achieved in both first and third person perspectives under congruent visuo-motor-tactile condition. However, subjective body ownership and reaction to threat were generally stronger for first person perspective and alternating condition than for third person perspective. This suggests that the possibility of alternating perspective is compatible with a strong sense of embodiment, which is meaningful for the design of new embodied VR experiences.

Embodiment in a virtual body that speaks produces agency over the speaking but does not necessarily influence subsequent real speaking

Previous results have shown that body ownership, induced through first-person perspective (1PP) over a virtual body (VB) that moves synchronously with real body movements, can lead to illusory agency over VB utterances even though the participant does not speak. It was also found that when participants later speak they follow the fundamental frequency (FF) of the voice of their VB, indicating a new motor plan for speaking. To eliminate the contribution of veridical agency over the VB movements, we conducted a study where we induced body ownership using visuotactile (VT) synchrony rather than visuomotor. Participants saw a life-sized VB from 1PP and reflected in a virtual mirror, that spoke with corresponding lip movements. Half of the 36 experimental participants experienced synchronous (Sync) passive VT on their hands and abdomen, and the other half asynchronous (Async). We found that both VT Sync and Async conditions resulted in a strong subjective illusion of body ownership and agency over the VB, but not, however, changes in voice FF in subsequent speaking. This shows that although illusory agency may be associated with body ownership, a change in motor plan is likely to be a generalisation from veridical agency over whole body movements.

Visuotactile integration modulates motor performance in a perceptual decision-making task

Body ownership is critically dependent on multimodal integration as for instance revealed in the Rubber Hand Illusion (RHI) and a number of studies which have addressed the neural correlates of the processes underlying this phenomenon. Both experimental and clinical research have shown that the structures underlying body ownership seem to significantly overlap with those of motor control including the parietal and ventral premotor cortices, Temporal Parietal Junction (TPJ) and the insula. This raises the question of whether this structural overlap between body ownership and motor control structures is of any functional significance. Here, we investigate the specific question of whether experimentally induced ownership over a virtual limb can modulate the performance of that limb in a simple sensorimotor task. Using a Virtual reality (VR) environment we modulate body ownership in three experimental conditions with respect to the (in)congruence of stimulus configurations. Our results show that the degree of ownership directly modulates motor performance. This implies that body ownership is not exclusively a perceptual and/or subjective multimodal state but that it is tightly coupled to systems for decision-making and motor control.

The Effect of Visual, Spatial and Temporal Manipulations on Embodiment and Action

The feeling of owning and controlling the body relies on the integration and interpretation of sensory input from multiple sources with respect to existing representations of the bodily self. Illusion paradigms involving multisensory manipulations have demonstrated that while the senses of ownership and agency are strongly related, these two components of bodily experience may be dissociable and differentially affected by alterations to sensory input. Importantly, however, much of the current literature has focused on the application of sensory manipulations to external objects or virtual representations of the self that are visually incongruent with the viewer’s own body and which are not part of the existing body representation. The current experiment used MIRAGE-mediated reality to investigate how manipulating the visual, spatial and temporal properties of the participant’s own hand (as opposed to a fake/virtual limb) affected embodiment and action. Participants viewed two representations of their right hand inside a MIRAGE multisensory illusions box with opposing visual (normal or grossly distorted), temporal (synchronous or asynchronous) and spatial (precise real location or false location) manipulations applied to each hand. Subjective experiences of ownership and agency towards each hand were measured alongside an objective measure of perceived hand location using a pointing task. The subjective sense of agency was always anchored to the synchronous hand, regardless of physical appearance and location. Subjective ownership also moved with the synchronous hand, except when both the location and appearance of the synchronous limb were incongruent with that of the real limb. Objective pointing measures displayed a similar pattern, however movement synchrony was not sufficient to drive a complete shift in perceived hand location, indicating a greater reliance on the spatial location of the real hand. The results suggest that while the congruence of self-generated movement is a sufficient driver for the sense of agency, the sense of ownership is additionally sensitive to cues about the visual appearance and spatial location of one’s own body.

Consciousness is more than meets the eye: a call for a multisensory study of subjective experience

Over the last 30 years, our understanding of the neurocognitive bases of consciousness has improved, mostly through studies employing vision. While studying consciousness in the visual modality presents clear advantages, we believe that a comprehensive scientific account of subjective experience must not neglect other exteroceptive and interoceptive signals as well as the role of multisensory interactions for perceptual and self-consciousness. Here, we briefly review four distinct lines of work which converge in documenting how multisensory signals are processed across several levels and contents of consciousness. Namely, how multisensory interactions occur when consciousness is prevented because of perceptual manipulations (i.e. subliminal stimuli) or because of low vigilance states (i.e. sleep, anesthesia), how interactions between exteroceptive and interoceptive signals give rise to bodily self-consciousness, and how multisensory signals are combined to form metacognitive judgments. By describing the interactions between multisensory signals at the perceptual, cognitive, and metacognitive levels, we illustrate how stepping out the visual comfort zone may help in deriving refined accounts of consciousness, and may allow cancelling out idiosyncrasies of each sense to delineate supramodal mechanisms involved during consciousness.

Analysis of Mental Workload in Online Shopping: Are Augmented and Virtual Reality Consistent?

A market research company (Nielsen) reported that consumers in the Asia-Pacific region have become the most active group in online shopping. Focusing on augmented reality (AR), which is one of three major techniques used to change the method of shopping in the future, this study used a mixed design to discuss the influences of the method of online shopping, user gender, cognitive style, product value, and sensory channel on mental workload in virtual reality (VR) and AR situations. The results showed that males' mental workloads were significantly higher than females'. For males, high-value products' mental workload was significantly higher than that of low-value products. In the VR situation, the visual mental workload of field-independent and field-dependent consumers showed a significant difference, but the difference was reduced under audio-visual conditions. In the AR situation, the visual mental workload of field-independent and field-dependent consumers showed a significant difference, but the difference increased under audio-visual conditions. This study provided a psychological study of online shopping with AR and VR technology with applications in the future. Based on the perspective of embodied cognition, AR online shopping may be potential focus of research and market application. For the future design of online shopping platforms and the updating of user experience, this study provides a reference.