1
|
Onishi Y, Ogawa K, Tanaka K, Nakanishi H. Embodied, visible, and courteous: exploring robotic social touch with virtual idols. Front Robot AI 2024; 11:1240408. [PMID: 38590970 PMCID: PMC10999612 DOI: 10.3389/frobt.2024.1240408] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 02/15/2024] [Indexed: 04/10/2024] Open
Abstract
In recent years, virtual idols have garnered considerable attention because they can perform activities similar to real idols. However, as they are fictitious idols with nonphysical presence, they cannot perform physical interactions such as handshake. Combining a robotic hand with a display showing virtual idols is the one of the methods to solve this problem. Nonetheless a physical handshake is possible, the form of handshake that can effectively induce the desirable behavior is unclear. In this study, we adopted a robotic hand as an interface and aimed to imitate the behavior of real idols. To test the effects of this behavior, we conducted step-wise experiments. The series of experiments revealed that the handshake by the robotic hand increased the feeling of intimacy toward the virtual idol, and it became more enjoyable to respond to a request from the virtual idol. In addition, viewing the virtual idols during the handshake increased the feeling of intimacy with the virtual idol. Moreover, the method of the hand-shake peculiar to idols, which tried to keep holding the user's hand after the conversation, increased the feeling of intimacy to the virtual idol.
Collapse
Affiliation(s)
- Yuya Onishi
- Interaction Science Laboratories, Advanced Telecommunication Research Institute International, Sorakugun, Japan
| | - Kosuke Ogawa
- Department of Mechanical Engineering, Osaka University, Suita, Japan
| | - Kazuaki Tanaka
- Faculty of Information and Human Sciences, Kyoto Institute of Technology, Kyoto, Japan
| | | |
Collapse
|
2
|
Abdulazeem N, Hu Y. Human Factors Considerations for Quantifiable Human States in Physical Human-Robot Interaction: A Literature Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:7381. [PMID: 37687837 PMCID: PMC10490212 DOI: 10.3390/s23177381] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Revised: 08/11/2023] [Accepted: 08/16/2023] [Indexed: 09/10/2023]
Abstract
As the global population rapidly ages with longer life expectancy and declining birth rates, the need for healthcare services and caregivers for older adults is increasing. Current research envisions addressing this shortage by introducing domestic service robots to assist with daily activities. The successful integration of robots as domestic service providers in our lives requires them to possess efficient manipulation capabilities, provide effective physical assistance, and have adaptive control frameworks that enable them to develop social understanding during human-robot interaction. In this context, human factors, especially quantifiable ones, represent a necessary component. The objective of this paper is to conduct an unbiased review encompassing the studies on human factors studied in research involving physical interactions and strong manipulation capabilities. We identified the prevalent human factors in physical human-robot interaction (pHRI), noted the factors typically addressed together, and determined the frequently utilized assessment approaches. Additionally, we gathered and categorized proposed quantification approaches based on the measurable data for each human factor. We also formed a map of the common contexts and applications addressed in pHRI for a comprehensive understanding and easier navigation of the field. We found out that most of the studies in direct pHRI (when there is direct physical contact) focus on social behaviors with belief being the most commonly addressed human factor type. Task collaboration is moderately investigated, while physical assistance is rarely studied. In contrast, indirect pHRI studies (when the physical contact is mediated via a third item) often involve industrial settings, with physical ergonomics being the most frequently investigated human factor. More research is needed on the human factors in direct and indirect physical assistance applications, including studies that combine physical social behaviors with physical assistance tasks. We also found that while the predominant approach in most studies involves the use of questionnaires as the main method of quantification, there is a recent trend that seeks to address the quantification approaches based on measurable data.
Collapse
Affiliation(s)
| | - Yue Hu
- Active & Interactive Robotics Lab, Department of Mechanical and Mechatronics Engineering, University of Waterloo, 200 University Ave. W., Waterloo, ON N2L 3G1, Canada;
| |
Collapse
|
3
|
Xu S, Xu C, McIntyre S, Olausson H, Gerling GJ. 3D Visual Tracking to Quantify Physical Contact Interactions in Human-to-Human Touch. Front Physiol 2022; 13:841938. [PMID: 35755449 PMCID: PMC9219726 DOI: 10.3389/fphys.2022.841938] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Accepted: 04/04/2022] [Indexed: 11/13/2022] Open
Abstract
Across a plethora of social situations, we touch others in natural and intuitive ways to share thoughts and emotions, such as tapping to get one’s attention or caressing to soothe one’s anxiety. A deeper understanding of these human-to-human interactions will require, in part, the precise measurement of skin-to-skin physical contact. Among prior efforts, each measurement approach exhibits certain constraints, e.g., motion trackers do not capture the precise shape of skin surfaces, while pressure sensors impede skin-to-skin contact. In contrast, this work develops an interference-free 3D visual tracking system using a depth camera to measure the contact attributes between the bare hand of a toucher and the forearm of a receiver. The toucher’s hand is tracked as a posed and positioned mesh by fitting a hand model to detected 3D hand joints, whereas a receiver’s forearm is extracted as a 3D surface updated upon repeated skin contact. Based on a contact model involving point clouds, the spatiotemporal changes of hand-to-forearm contact are decomposed as six, high-resolution, time-series contact attributes, i.e., contact area, indentation depth, absolute velocity, and three orthogonal velocity components, together with contact duration. To examine the system’s capabilities and limitations, two types of experiments were performed. First, to evaluate its ability to discern human touches, one person delivered cued social messages, e.g., happiness, anger, sympathy, to another person using their preferred gestures. The results indicated that messages and gestures, as well as the identities of the touchers, were readily discerned from their contact attributes. Second, the system’s spatiotemporal accuracy was validated against measurements from independent devices, including an electromagnetic motion tracker, sensorized pressure mat, and laser displacement sensor. While validated here in the context of social communication, this system is extendable to human touch interactions such as maternal care of infants and massage therapy.
Collapse
Affiliation(s)
- Shan Xu
- School of Engineering and Applied Science, University of Virginia, Charlottesville, VA, United States
| | - Chang Xu
- School of Engineering and Applied Science, University of Virginia, Charlottesville, VA, United States
| | - Sarah McIntyre
- Center for Social and Affective Neuroscience (CSAN), Linköping University, Linköping, Sweden
| | - Håkan Olausson
- Center for Social and Affective Neuroscience (CSAN), Linköping University, Linköping, Sweden
| | - Gregory J Gerling
- School of Engineering and Applied Science, University of Virginia, Charlottesville, VA, United States
| |
Collapse
|
4
|
Shiomi M, Zheng X, Minato T, Ishiguro H. Implementation and Evaluation of a Grip Behavior Model to Express Emotions for an Android Robot. Front Robot AI 2021; 8:755150. [PMID: 34722641 PMCID: PMC8548368 DOI: 10.3389/frobt.2021.755150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2021] [Accepted: 09/29/2021] [Indexed: 11/13/2022] Open
Abstract
In this study, we implemented a model with which a robot expressed such complex emotions as heartwarming (e.g., happy and sad) or horror (fear and surprise) by its touches and experimentally investigated the effectiveness of the modeled touch behaviors. Robots that can express emotions through touching behaviors increase their interaction capabilities with humans. Although past studies achieved ways to express emotions through a robot’s touch, such studies focused on expressing such basic emotions as happiness and sadness and downplayed these complex emotions. Such studies only proposed a model that expresses these emotions by touch behaviors without evaluations. Therefore, we conducted the experiment to evaluate the model with participants. In the experiment, they evaluated the perceived emotions and empathies from a robot’s touch while they watched a video stimulus with the robot. Our results showed that the touch timing before the climax received higher evaluations than touch timing after for both the scary and heartwarming videos.
Collapse
Affiliation(s)
- Masahiro Shiomi
- Advanced Telecommunications Research Institute International, Kyoto, Japan
| | - Xiqian Zheng
- Advanced Telecommunications Research Institute International, Kyoto, Japan.,Graduate School of Engineering Science, Osaka University, Osaka, Japan
| | - Takashi Minato
- Advanced Telecommunications Research Institute International, Kyoto, Japan.,Guardian Robot Project, RIKEN, Kyoto, Japan
| | - Hiroshi Ishiguro
- Graduate School of Engineering Science, Osaka University, Osaka, Japan
| |
Collapse
|
5
|
Higashino K, Kimoto M, Iio T, Shimohara K, Shiomi M. Tactile stimulus is essential to increase motivation for touch interaction in virtual environment. Adv Robot 2021. [DOI: 10.1080/01691864.2021.1967780] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Kana Higashino
- ATR-ISL, Kyoto, Japan
- Faculty of Science and Engineering, Doshisha Univ., Kyoto, Japan
| | - Mitsuhiko Kimoto
- ATR-ISL, Kyoto, Japan
- Faculty of Science and Technology, Keio Univ., Kanagawa, Japan
| | - Takamasa Iio
- ATR-ISL, Kyoto, Japan
- Faculty of Culture and Information Science, Doshisha Univ., Kyoto, Japan
| | | | | |
Collapse
|
6
|
Stock-Homburg R. Survey of Emotions in Human–Robot Interactions: Perspectives from Robotic Psychology on 20 Years of Research. Int J Soc Robot 2021. [DOI: 10.1007/s12369-021-00778-6] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
AbstractKnowledge production within the interdisciplinary field of human–robot interaction (HRI) with social robots has accelerated, despite the continued fragmentation of the research domain. Together, these features make it hard to remain at the forefront of research or assess the collective evidence pertaining to specific areas, such as the role of emotions in HRI. This systematic review of state-of-the-art research into humans’ recognition and responses to artificial emotions of social robots during HRI encompasses the years 2000–2020. In accordance with a stimulus–organism–response framework, the review advances robotic psychology by revealing current knowledge about (1) the generation of artificial robotic emotions (stimulus), (2) human recognition of robotic artificial emotions (organism), and (3) human responses to robotic emotions (response), as well as (4) other contingencies that affect emotions as moderators.
Collapse
|
7
|
Sumioka H, Shiomi M, Honda M, Nakazawa A. Technical Challenges for Smooth Interaction With Seniors With Dementia: Lessons From Humanitude™. Front Robot AI 2021; 8:650906. [PMID: 34150858 PMCID: PMC8207295 DOI: 10.3389/frobt.2021.650906] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2021] [Accepted: 05/20/2021] [Indexed: 11/13/2022] Open
Abstract
Due to cognitive and socio-emotional decline and mental diseases, senior citizens, especially people with dementia (PwD), struggle to interact smoothly with their caregivers. Therefore, various care techniques have been proposed to develop good relationships with seniors. Among them, Humanitude is one promising technique that provides caregivers with useful interaction skills to improve their relationships with PwD, from four perspectives: face-to-face interaction, verbal communication, touch interaction, and helping care receivers stand up (physical interaction). Regardless of advances in elderly care techniques, since current social robots interact with seniors in the same manner as they do with younger adults, they lack several important functions. For example, Humanitude emphasizes the importance of interaction at a relatively intimate distance to facilitate communication with seniors. Unfortunately, few studies have developed an interaction model for clinical care communication. In this paper, we discuss the current challenges to develop a social robot that can smoothly interact with PwDs and overview the interaction skills used in Humanitude as well as the existing technologies.
Collapse
Affiliation(s)
- Hidenobu Sumioka
- Advanced Telecommunications Research Institute International, Kyoto, Japan
| | - Masahiro Shiomi
- Advanced Telecommunications Research Institute International, Kyoto, Japan
| | - Miwako Honda
- National Hospital Organization Tokyo Medical Center, Tokyo, Japan
| | | |
Collapse
|
8
|
Zheng X, Shiomi M, Minato T, Ishiguro H. Modeling the Timing and Duration of Grip Behavior to Express Emotions for a Social Robot. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2020.3036372] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
9
|
|
10
|
Zheng X, Shiomi M, Minato T, Ishiguro H. How Can Robots Make People Feel Intimacy Through Touch? JOURNAL OF ROBOTICS AND MECHATRONICS 2020. [DOI: 10.20965/jrm.2020.p0051] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This study investigates the effects of the touch characteristics that change the intimacy perceived by humans in human-robot touch interaction with an android robot having a human-like feminine appearance. Past studies on human-robot touch interaction focused on understanding which types of human touches are used to express emotions to robots. However, they less focused on how a robot’s touch characteristics can affect humans’ perceived intimacy. In this study, first, we concentrated on two types of touch characteristics (type and place) and their effects on the perceived intimacy of a commonly used emotion in human-robot interaction, namely happiness. The results showed that the touch types are useful for changing the perceived intimacy, although the touched place did not exhibit any significant effects. Based on the results of our first experiment, we investigated the effects of different touch characteristics (length and part). We concluded that the touch part is useful to change the perceived intimacy, although the touch length did not exhibit any significant effects. Finally, the results suggested that a pat (type) by the fingers (part) is a better combination to express intimacy with our robot.
Collapse
|
11
|
Nakagawa K, Matsumura R, Shiomi M. Effect of Robot’s Play-Biting in Non-Verbal Communication. JOURNAL OF ROBOTICS AND MECHATRONICS 2020. [DOI: 10.20965/jrm.2020.p0086] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This paper focuses on “play-biting” as a touch communication method used by robots. We investigated an appropriate play-biting behavior and its effect on interaction. The touching action has positive effects in human-robot interactions. However, as biting is a defenseless act, it may cause a negative effect as well. Therefore, we first examine biting manner and the appearance of the robot using a virtual play-biting system in Experiment 1. Next, based on the result of experiment, the play-biting system is implemented in a stuffed animal robot. We verified the impressions created by the robot and its effect on mitigating stress in Experiment 2. Consequently, the play-biting communication gave positive and lively impression, and effect of reducing the physiological index of stress, in comparison to only touching the robot.
Collapse
|
12
|
Shiomi M, Hirano T, Kimoto M, Iio T, Shimohara K. Gaze-Height and Speech-Timing Effects on Feeling Robot-Initiated Touches. JOURNAL OF ROBOTICS AND MECHATRONICS 2020. [DOI: 10.20965/jrm.2020.p0068] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This paper reports the effects of communication cues on robot-initiated touch interactions at close distance by focusing on two factors: gaze-height for making eye contact and speech timing before and after touches. Although both factors are essential to achieve acceptable touches in human-human touch interaction, their effectiveness remains unknown in human-robot touch interaction contexts. To investigate the effects of these factors, we conducted an experiment whose results showed that being touched with before-touch timing is preferred to being touched with after-touch timing, although gaze-height did not significantly improve the feelings of robot-initiated touch.
Collapse
|
13
|
Shiomi M, Sumioka H, Ishiguro H. Survey of Social Touch Interaction Between Humans and Robots. JOURNAL OF ROBOTICS AND MECHATRONICS 2020. [DOI: 10.20965/jrm.2020.p0128] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
In human-human interaction, social touch provides several merits, from both physical and mental perspectives. The physical existence of robots helps them reproduce human-like social touch, during their interaction with people. Such social touch shows positive effects, similar to those observed in human-human interaction. Therefore, social touch is a growing research topic in the field of human-robot interaction. This survey provides an overview of the work conducted so far on this topic.
Collapse
|