Nonverbal Sound in Human-Robot Interaction: a Systematic Review

Nonverbal sound offers great potential to enhance robots’ interactions with humans, and a growing body of research has begun to explore nonverbal sound for tasks such as sound source localization, explicit communication, and improving sociability. However, nonverbal sound has a broad interpretation and design space that can draw from areas such as machine learning, music theory, and foley. We sought to identify and compare use cases and approaches for nonverbal sound in human-robot interaction through a systematic review. A search of sound and robotics-related publisher databases yielded 148 peer-reviewed articles presenting systems, studies, and taxonomies. Differences in taxonomy and overlap of terminology with adjacent research fields such as speech, gaze, and gesture posed difficulties for the search, which we attempted to address through a multi-stage search process. Based on the reviewed articles, we developed a pair of taxonomies using scientific communication principles and analyzed study designs and measures for the creation of nonverbal robot sound. We discuss recommendations for the field, including the use of the new taxonomies; methods for design, generation, and validation; and paths for future research. Roboticists may benefit from incorporating nonverbal sound as a key component in multimodal human-robot interaction.