1
|
Gong Y, Zhang K, Lei IM, Wang Y, Zhong J. Advances in Piezoelectret Materials-Based Bidirectional Haptic Communication Devices. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024:e2405308. [PMID: 38895922 DOI: 10.1002/adma.202405308] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/14/2024] [Revised: 06/10/2024] [Indexed: 06/21/2024]
Abstract
Bidirectional haptic communication devices accelerate the revolution of virtual/augmented reality and flexible/wearable electronics. As an emerging kind of flexible piezoelectric materials, piezoelectret materials can effortlessly convert mechanical force into electrical signals and respond to electrical fields in a deformation manner, exhibiting enormous potential in the construction of bidirectional haptic communication devices. Existing reviews on piezoelectret materials primarily focus on flexible energy harvesters and sensors, and the recent development of piezoelectret-based bidirectional haptic communication devices has not been comprehensively reviewed. Herein, a comprehensive overview of the materials construction, along with the recent advances in bidirectional haptic communication devices, is provided. First, the development timeline, key characteristics, and various fabrication methods of piezoelectret materials are introduced. Subsequently, following the underlying mechanisms of bidirectional electromechanical signal conversion of piezoelectret, strategies to improve the d33 coefficients of materials are proposed. The principles of haptic perception and feedback are also highlighted, and representative works and progress in this area are summarized. Finally, the challenges and opportunities associated with improving the overall practicability of piezoelectret materials-based bidirectional haptic communication devices are discussed.
Collapse
Affiliation(s)
- Yanting Gong
- Department of Electromechanical Engineering and Centre for Artificial Intelligence and Robotics, University of Macau, Macau, SAR, 999078, China
| | - Kaijun Zhang
- Department of Electromechanical Engineering and Centre for Artificial Intelligence and Robotics, University of Macau, Macau, SAR, 999078, China
| | - Iek Man Lei
- Department of Electromechanical Engineering and Centre for Artificial Intelligence and Robotics, University of Macau, Macau, SAR, 999078, China
| | - Yan Wang
- Department of Chemical Engineering, Guangdong Technion-Israel Institute of Technology (GTIIT), Shantou, Guangdong, 515063, China
| | - Junwen Zhong
- Department of Electromechanical Engineering and Centre for Artificial Intelligence and Robotics, University of Macau, Macau, SAR, 999078, China
| |
Collapse
|
2
|
Su J, Zhang H, Li H, He K, Tu J, Zhang F, Liu Z, Lv Z, Cui Z, Li Y, Li J, Tang LZ, Chen X. Skin-Inspired Multi-Modal Mechanoreceptors for Dynamic Haptic Exploration. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024; 36:e2311549. [PMID: 38363810 DOI: 10.1002/adma.202311549] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 02/02/2024] [Indexed: 02/18/2024]
Abstract
Active sensing is a fundamental aspect of human and animal interactions with the environment, providing essential information about the hardness, texture, and tackiness of objects. This ability stems from the presence of diverse mechanoreceptors in the skin, capable of detecting a wide range of stimuli and from the sensorimotor control of biological mechanisms. In contrast, existing tactile sensors for robotic applications typically excel in identifying only limited types of information, lacking the versatility of biological mechanoreceptors and the requisite sensing strategies to extract tactile information proactively. Here, inspired by human haptic perception, a skin-inspired artificial 3D mechanoreceptor (SENS) capable of detecting multiple mechanical stimuli is developed to bridge sensing and action in a closed-loop sensorimotor system for dynamic haptic exploration. A tensor-based non-linear theoretical model is established to characterize the 3D deformation (e.g., tensile, compressive, and shear deformation) of SENS, providing guidance for the design and optimization of multimode sensing properties with high fidelity. Based on SENS, a closed-loop robotic system capable of recognizing objects with improved accuracy (≈96%) is further demonstrated. This dynamic haptic exploration approach shows promise for a wide range of applications such as autonomous learning, healthcare, and space and deep-sea exploration.
Collapse
Affiliation(s)
- Jiangtao Su
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
| | - Hang Zhang
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
- Singapore-HUJ Alliance for Research and Enterprise (SHARE), The Smart Grippers for Soft Robotics (SGSR) Programme, Campus for Research Excellence and Technological Enterprise (CREATE), Singapore, 138602, Singapore
| | - Haicheng Li
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
| | - Ke He
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
- Singapore-HUJ Alliance for Research and Enterprise (SHARE), The Smart Grippers for Soft Robotics (SGSR) Programme, Campus for Research Excellence and Technological Enterprise (CREATE), Singapore, 138602, Singapore
| | - Jiaqi Tu
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
| | - Feilong Zhang
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
| | - Zhihua Liu
- Institute of Materials Research and Engineering, the Agency for Science, Technology and Research, Singapore, 138634, Singapore
| | - Zhisheng Lv
- Institute of Materials Research and Engineering, the Agency for Science, Technology and Research, Singapore, 138634, Singapore
| | - Zequn Cui
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
| | - Yanzhen Li
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
| | - Jiaofu Li
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
| | - Leng Ze Tang
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
| | - Xiaodong Chen
- Innovative Centre for Flexible Devices (iFLEX), Max Planck-NTU Joint Lab for Artificial Senses, School of Materials Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore
- Institute for Digital Molecular Analytics and Science (IDMxS), Nanyang Technological University, 59 Nanyang Drive, Singapore, 636921, Singapore
| |
Collapse
|
3
|
Seminara L, Dosen S, Mastrogiovanni F, Bianchi M, Watt S, Beckerle P, Nanayakkara T, Drewing K, Moscatelli A, Klatzky RL, Loeb GE. A hierarchical sensorimotor control framework for human-in-the-loop robotic hands. Sci Robot 2023; 8:eadd5434. [PMID: 37196072 DOI: 10.1126/scirobotics.add5434] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/19/2023]
Abstract
Human manual dexterity relies critically on touch. Robotic and prosthetic hands are much less dexterous and make little use of the many tactile sensors available. We propose a framework modeled on the hierarchical sensorimotor controllers of the nervous system to link sensing to action in human-in-the-loop, haptically enabled, artificial hands.
Collapse
Affiliation(s)
- Lucia Seminara
- Department of Electrical, Electronic, and Telecommunication Engineering and Naval Architecture, University of Genoa, Genoa, Italy
| | - Strahinja Dosen
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Fulvio Mastrogiovanni
- Department of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genoa, Genoa, Italy
| | - Matteo Bianchi
- Research Center "E. Piaggio" and Department of Information Engineering, University of Pisa, Pisa, Italy
| | - Simon Watt
- School of Human and Behavioural Sciences, Bangor University, Bangor, UK
| | - Philipp Beckerle
- Department of Electrical Engineering, Friedrich-Alexander Universität Erlangen-Nürnberg, Nürnberg, Germany
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander Universität Erlangen-Nürnberg, Nürnberg, Germany
| | | | - Knut Drewing
- Department of Experimental Psychology, HapLab, University of Giessen, Giessen, Germany
| | - Alessandro Moscatelli
- Laboratory of Neuromotor Physiology, Fondazione Santa Lucia IRCCS, Rome, Italy
- Department of Systems Medicine and Centre of Space Biomedicine, University of Rome Tor Vergata, Rome, Italy
| | - Roberta L Klatzky
- Department of Psychology and Human-Computer Interaction Institute, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Gerald E Loeb
- Alfred E. Mann Department of Biomedical Engineering, Keck School of Medicine, and Viterbi School of Engineering, University of Southern California, Los Angeles, CA, USA
| |
Collapse
|
4
|
Navarro-Guerrero N, Toprak S, Josifovski J, Jamone L. Visuo-haptic object perception for robots: an overview. Auton Robots 2023. [DOI: 10.1007/s10514-023-10091-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/28/2023]
Abstract
AbstractThe object perception capabilities of humans are impressive, and this becomes even more evident when trying to develop solutions with a similar proficiency in autonomous robots. While there have been notable advancements in the technologies for artificial vision and touch, the effective integration of these two sensory modalities in robotic applications still needs to be improved, and several open challenges exist. Taking inspiration from how humans combine visual and haptic perception to perceive object properties and drive the execution of manual tasks, this article summarises the current state of the art of visuo-haptic object perception in robots. Firstly, the biological basis of human multimodal object perception is outlined. Then, the latest advances in sensing technologies and data collection strategies for robots are discussed. Next, an overview of the main computational techniques is presented, highlighting the main challenges of multimodal machine learning and presenting a few representative articles in the areas of robotic object recognition, peripersonal space representation and manipulation. Finally, informed by the latest advancements and open challenges, this article outlines promising new research directions.
Collapse
|
5
|
Ibrahim MK, Bashar BS, Al-Nabi NRA, Ismail MM. Robot-assisted for medical surgery: A literature review. AIP CONFERENCE PROCEEDINGS 2023. [DOI: 10.1063/5.0119586] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
|
6
|
Abbass Y, Dosen S, Seminara L, Valle M. Full-hand electrotactile feedback using electronic skin and matrix electrodes for high-bandwidth human-machine interfacing. PHILOSOPHICAL TRANSACTIONS. SERIES A, MATHEMATICAL, PHYSICAL, AND ENGINEERING SCIENCES 2022; 380:20210017. [PMID: 35762222 DOI: 10.1098/rsta.2021.0017] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Accepted: 01/16/2022] [Indexed: 06/15/2023]
Abstract
Tactile feedback is relevant in a broad range of human-machine interaction systems (e.g. teleoperation, virtual reality and prosthetics). The available tactile feedback interfaces comprise few sensing and stimulation units, which limits the amount of information conveyed to the user. The present study describes a novel technology that relies on distributed sensing and stimulation to convey comprehensive tactile feedback to the user of a robotic end effector. The system comprises six flexible sensing arrays (57 sensors) integrated on the fingers and palm of a robotic hand, embedded electronics (64 recording channels), a multichannel stimulator and seven flexible electrodes (64 stimulation pads) placed on the volar side of the subject's hand. The system was tested in seven subjects asked to recognize contact positions and identify contact sliding on the electronic skin, using distributed anode configuration (DAC) and single dedicated anode configuration. The experiments demonstrated that DAC resulted in substantially better performance. Using DAC, the system successfully translated the contact patterns into electrotactile profiles that the subjects could recognize with satisfactory accuracy ([Formula: see text] for static and [Formula: see text] for dynamic patterns). The proposed system is an important step towards the development of a high-density human-machine interfacing between the user and a robotic hand. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.
Collapse
Affiliation(s)
- Yahya Abbass
- Department of Electrical, Electronic, Telecommunications Engineering, and Naval Architecture (DITEN), University of Genoa, 16145 Genova, Italy
| | - Strahinja Dosen
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Lucia Seminara
- Department of Electrical, Electronic, Telecommunications Engineering, and Naval Architecture (DITEN), University of Genoa, 16145 Genova, Italy
| | - Maurizio Valle
- Department of Electrical, Electronic, Telecommunications Engineering, and Naval Architecture (DITEN), University of Genoa, 16145 Genova, Italy
| |
Collapse
|
7
|
A Comprehensive Study of Mobile Robot: History, Developments, Applications, and Future Research Perspectives. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12146951] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
Intelligent mobile robots that can move independently were laid out in the real world around 100 years ago during the second world war after advancements in computer science. Since then, mobile robot research has transformed robotics and information engineering. For example, robots were crucial in military applications, especially in teleoperations, when they emerged during the second world war era. Furthermore, after the implementation of artificial intelligence (AI) in robotics, they became autonomous or more intelligent. Currently, mobile robots have been implemented in many applications like defense, security, freight, pattern recognition, medical treatment, mail delivery, infrastructure inspection and developments, passenger travel, and many more because they are more intelligent nowadays with artificial intelligence technology. To study the developments of mobile robots, we have studied an extensive literature survey of the last 50 years. In this article, we discuss a complete century of mobile robotics research, major sensors used in robotics, some major applications of mobile robots, and their impact on our lives and in applied engineering.
Collapse
|
8
|
Carfì A, Patten T, Kuang Y, Hammoud A, Alameh M, Maiettini E, Weinberg AI, Faria D, Mastrogiovanni F, Alenyà G, Natale L, Perdereau V, Vincze M, Billard A. Hand-Object Interaction: From Human Demonstrations to Robot Manipulation. Front Robot AI 2021; 8:714023. [PMID: 34660702 PMCID: PMC8517111 DOI: 10.3389/frobt.2021.714023] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2021] [Accepted: 09/14/2021] [Indexed: 11/13/2022] Open
Abstract
Human-object interaction is of great relevance for robots to operate in human environments. However, state-of-the-art robotic hands are far from replicating humans skills. It is, therefore, essential to study how humans use their hands to develop similar robotic capabilities. This article presents a deep dive into hand-object interaction and human demonstrations, highlighting the main challenges in this research area and suggesting desirable future developments. To this extent, the article presents a general definition of the hand-object interaction problem together with a concise review for each of the main subproblems involved, namely: sensing, perception, and learning. Furthermore, the article discusses the interplay between these subproblems and describes how their interaction in learning from demonstration contributes to the success of robot manipulation. In this way, the article provides a broad overview of the interdisciplinary approaches necessary for a robotic system to learn new manipulation skills by observing human behavior in the real world.
Collapse
Affiliation(s)
- Alessandro Carfì
- Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genoa, Genoa, Italy
| | - Timothy Patten
- Vision for Robotics Laboratory, Institut für Automatisierungs- und Regelungstechnik, Technische Universität Wien, Vienna, Austria
| | - Yingyi Kuang
- Robotics, Vision and Intelligent Systems, College of Engineering and Physical Sciences, Aston University, Birmingham, United Kingdom
| | - Ali Hammoud
- Institut des Systèmes Intelligents et de Robotique, Sorbonne Université, Paris, France
| | - Mohamad Alameh
- Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genoa, Genoa, Italy
| | - Elisa Maiettini
- Humanoid Sensing and Perception, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Abraham Itzhak Weinberg
- Robotics, Vision and Intelligent Systems, College of Engineering and Physical Sciences, Aston University, Birmingham, United Kingdom
| | - Diego Faria
- Robotics, Vision and Intelligent Systems, College of Engineering and Physical Sciences, Aston University, Birmingham, United Kingdom
| | - Fulvio Mastrogiovanni
- Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genoa, Genoa, Italy
| | - Guillem Alenyà
- Institut de Robòtica i Informàtica Industrial, CSIC-UPC, Barcelona, Spain
| | - Lorenzo Natale
- Humanoid Sensing and Perception, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Véronique Perdereau
- Institut des Systèmes Intelligents et de Robotique, Sorbonne Université, Paris, France
| | - Markus Vincze
- Vision for Robotics Laboratory, Institut für Automatisierungs- und Regelungstechnik, Technische Universität Wien, Vienna, Austria
| | - Aude Billard
- Learning Algorithms and Systems Laboratory, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| |
Collapse
|
9
|
Ryan CP, Bettelani GC, Ciotti S, Parise C, Moscatelli A, Bianchi M. The interaction between motion and texture in the sense of touch. J Neurophysiol 2021; 126:1375-1390. [PMID: 34495782 DOI: 10.1152/jn.00583.2020] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Besides providing information on elementary properties of objects, like texture, roughness, and softness, the sense of touch is also important in building a representation of object movement and the movement of our hands. Neural and behavioral studies shed light on the mechanisms and limits of our sense of touch in the perception of texture and motion, and of its role in the control of movement of our hands. The interplay between the geometrical and mechanical properties of the touched objects, such as shape and texture, the movement of the hand exploring the object, and the motion felt by touch, will be discussed in this article. Interestingly, the interaction between motion and textures can generate perceptual illusions in touch. For example, the orientation and the spacing of the texture elements on a static surface induces the illusion of surface motion when we move our hand on it or can elicit the perception of a curved trajectory during sliding, straight hand movements. In this work we present a multiperspective view that encompasses both the perceptual and the motor aspects, as well as the response of peripheral and central nerve structures, to analyze and better understand the complex mechanisms underpinning the tactile representation of texture and motion. Such a better understanding of the spatiotemporal features of the tactile stimulus can reveal novel transdisciplinary applications in neuroscience and haptics.
Collapse
Affiliation(s)
- Colleen P Ryan
- Department of Systems Medicine and Centre of Space Bio-Medicine, University of Rome "Tor Vergata", Rome, Italy.,Department of Neuromotor Physiology, Istituto di Ricovero e Cura a Carattere Scientifico Santa Lucia Foundation, Rome, Italy
| | - Gemma C Bettelani
- Research Center E. Piaggio, University of Pisa, Pisa, Italy.,Department of Information Engineering, University of Pisa, Pisa, Italy
| | - Simone Ciotti
- Department of Systems Medicine and Centre of Space Bio-Medicine, University of Rome "Tor Vergata", Rome, Italy.,Department of Neuromotor Physiology, Istituto di Ricovero e Cura a Carattere Scientifico Santa Lucia Foundation, Rome, Italy.,Department of Information Engineering, University of Pisa, Pisa, Italy
| | | | - Alessandro Moscatelli
- Department of Systems Medicine and Centre of Space Bio-Medicine, University of Rome "Tor Vergata", Rome, Italy.,Department of Neuromotor Physiology, Istituto di Ricovero e Cura a Carattere Scientifico Santa Lucia Foundation, Rome, Italy
| | - Matteo Bianchi
- Research Center E. Piaggio, University of Pisa, Pisa, Italy.,Department of Information Engineering, University of Pisa, Pisa, Italy
| |
Collapse
|
10
|
Ren L, Li B, Wei G, Wang K, Song Z, Wei Y, Ren L, Qingping Liu. Biology and bioinspiration of soft robotics: Actuation, sensing, and system integration. iScience 2021; 24:103075. [PMID: 34568796 PMCID: PMC8449090 DOI: 10.1016/j.isci.2021.103075] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/19/2023] Open
Abstract
Organisms in nature grow with senses, nervous, and actuation systems coordinated in ingenious ways to sustain metabolism and other essential life activities. The understanding of biological structures and functions guide the construction of soft robotics with unprecedented performances. However, despite the progress in soft robotics, there still remains a big gap between man-made soft robotics and natural lives in terms of autonomy, adaptability, self-repair, durability, energy efficiency, etc. Here, the actuation and sensing strategies in the natural biological world are summarized along with their man-made counterparts applied in soft robotics. The development trends of bioinspired soft robotics toward closed loop and embodiment are proposed. Challenges for obtaining autonomous soft robotics similar to natural organisms are outlined to provide a perspective in this field.
Collapse
Affiliation(s)
- Luquan Ren
- Key Laboratory of Bionic Engineering (Ministry of Education), Jilin University, Changchun 130022, China
| | - Bingqian Li
- Key Laboratory of Bionic Engineering (Ministry of Education), Jilin University, Changchun 130022, China
| | - Guowu Wei
- School of Science, Engineering and Environment, University of Salford, M5 4WT Salford, UK
| | - Kunyang Wang
- Key Laboratory of Bionic Engineering (Ministry of Education), Jilin University, Changchun 130022, China
| | - Zhengyi Song
- Key Laboratory of Bionic Engineering (Ministry of Education), Jilin University, Changchun 130022, China
| | - Yuyang Wei
- School of Mechanical, Aerospace and Civil Engineering, University of Manchester, M13 9PL Manchester, UK
| | - Lei Ren
- Key Laboratory of Bionic Engineering (Ministry of Education), Jilin University, Changchun 130022, China.,School of Mechanical, Aerospace and Civil Engineering, University of Manchester, M13 9PL Manchester, UK
| | - Qingping Liu
- Key Laboratory of Bionic Engineering (Ministry of Education), Jilin University, Changchun 130022, China
| |
Collapse
|
11
|
Tactile Object Recognition for Humanoid Robots Using New Designed Piezoresistive Tactile Sensor and DCNN. SENSORS 2021; 21:s21186024. [PMID: 34577230 PMCID: PMC8473115 DOI: 10.3390/s21186024] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/04/2021] [Revised: 09/04/2021] [Accepted: 09/06/2021] [Indexed: 11/17/2022]
Abstract
A tactile sensor array is a crucial component for applying physical sensors to a humanoid robot. This work focused on developing a palm-size tactile sensor array (56.0 mm × 56.0 mm) to apply object recognition for the humanoid robot hand. This sensor was based on a PCB technology operating with the piezoresistive principle. A conductive polymer composites sheet was used as a sensing element and the matrix array of this sensor was 16 × 16 pixels. The sensitivity of this sensor was evaluated and the sensor was installed on the robot hand. The tactile images, with resolution enhancement using bicubic interpolation obtained from 20 classes, were used to train and test 19 different DCNNs. InceptionResNetV2 provided superior performance with 91.82% accuracy. However, using the multimodal learning method that included InceptionResNetV2 and XceptionNet, the highest recognition rate of 92.73% was achieved. Moreover, this recognition rate improved when the object exploration was applied to demonstrate.
Collapse
|
12
|
Siddiqui MS, Coppola C, Solak G, Jamone L. Grasp Stability Prediction for a Dexterous Robotic Hand Combining Depth Vision and Haptic Bayesian Exploration. Front Robot AI 2021; 8:703869. [PMID: 34458325 PMCID: PMC8387702 DOI: 10.3389/frobt.2021.703869] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Accepted: 07/12/2021] [Indexed: 11/13/2022] Open
Abstract
Grasp stability prediction of unknown objects is crucial to enable autonomous robotic manipulation in an unstructured environment. Even if prior information about the object is available, real-time local exploration might be necessary to mitigate object modelling inaccuracies. This paper presents an approach to predict safe grasps of unknown objects using depth vision and a dexterous robot hand equipped with tactile feedback. Our approach does not assume any prior knowledge about the objects. First, an object pose estimation is obtained from RGB-D sensing; then, the object is explored haptically to maximise a given grasp metric. We compare two probabilistic methods (i.e. standard and unscented Bayesian Optimisation) against random exploration (i.e. uniform grid search). Our experimental results demonstrate that these probabilistic methods can provide confident predictions after a limited number of exploratory observations, and that unscented Bayesian Optimisation can find safer grasps, taking into account the uncertainty in robot sensing and grasp execution.
Collapse
Affiliation(s)
- Muhammad Sami Siddiqui
- ARQ (Advanced Robotics at Queen Mary), School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom
| | - Claudio Coppola
- ARQ (Advanced Robotics at Queen Mary), School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom
| | - Gokhan Solak
- ARQ (Advanced Robotics at Queen Mary), School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom
| | - Lorenzo Jamone
- ARQ (Advanced Robotics at Queen Mary), School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom
| |
Collapse
|
13
|
Komeno N, Matsubara T. Tactile Perception Based on Injected Vibration in Soft Sensor. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3075664] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
14
|
Araujo RLC, Benevenuto DSÁ, Zilberstein B, Sallum RA, Aguiar-Jr S, Cavazzola LT, Nacul M, Melani AGF, Tomasich FDS. Overview and perspectives about the robotic surgical certification process in Brazil: the new statement and a national web-survey. Rev Col Bras Cir 2020; 47:e20202714. [PMID: 33111834 DOI: 10.1590/0100-6991e-20202714] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2020] [Accepted: 08/27/2020] [Indexed: 11/21/2022] Open
Abstract
OBJECTIVE to appraise the general profile of the Brazilian robotic surgeon and the acknowledgment of the new certification process for robotic surgery upon the Associação Médica Brasileira (AMB - Brazilian Medical Association) statement. According to the AMB statement, medical societies and proctors have to achieve leading roles in training and certification of surgeons, acting in partnership with industry. METHODS a national web-based survey was promoted by the Colégio Brasileiro de Cirurgiões (CBC - Brazilian College of Surgeons) among their members. RESULTS the 294 answers were split into two groups: 133 (45.3%) who had robotic console certification, and 161 (54.8%) who did not have it. The overall median age was 46, but the non-robotic group presented more surgeons with at least 30 years of experience than to the robotic group (32.3% versus 23.3%, p=0.033). Surgeons with robotic certification more frequently work in a city with at least one million inhabitants than surgeons who were not certified (85.7 versus 63.4%, p<0.001). The majority of surgeons in both groups have similar positioning for all main points of the statement. However, the agreement proportions for the preceptors responsibility during the procedures were higher among non-robotic surgeons that expected the preceptor to assume co-responsibility for the procedure (85% versus 60.9%, p<0.001), and intervene during the procedure as much as necessary (97.5% versus 91.7%, p=0.033). CONCLUSION the overall agreement of the answers to the AMB statement seems to be a promising pathway to increase the participation of the medical entities into the robotic certification in Brazil.
Collapse
Affiliation(s)
- Raphael L C Araujo
- - Universidade Federal de São Paulo, Department of Digestive Surgery - São Paulo - São Paulo - Brasil.,- Hospital Israelita Albert Einstein, Department of Oncology - São Paulo - São Paulo - Brasil
| | | | - Bruno Zilberstein
- - Faculdade de Medicina SL Mandic, Service of Digestive Surgery - Campinas - São Paulo - Brasil.,- Universidade de São Paulo, Department of Digestive Surgery - São Paulo - SP - Brasil
| | - Rubens A Sallum
- - Universidade de São Paulo, Department of Digestive Surgery - São Paulo - SP - Brasil
| | - Samuel Aguiar-Jr
- - Hospital AC Camargo, Department of Colorectal Surgery - São Paulo - SP - Brasil
| | - Leandro Totti Cavazzola
- - Universidade Federal do Rio Grande do Sul, Department of Surgery - Porto Alegre - RS - Brasil.,- Hospital das Clínicas de Porto Alegre, Service of General Surgery - Porto Alegre - RS - Brasil
| | - Miguel Nacul
- - Hospital Moinhos de Vento, Service of Surgery - Porto Alegre - RS - Brasil
| | - Armando G F Melani
- - Americas Serviços Medicos, Service of Colorectal Surgery - Rio de Janeiro - RJ - Brasil.,- IRCAD America Latina, IRCAD - Rio de Janeiro - RJ - Brasil
| | - FlÁvio D S Tomasich
- - Universidade Federal do Paraná, Department of Surgery - Curitiba - PR - Brasil
| |
Collapse
|
15
|
Vermesan O, Bahr R, Ottella M, Serrano M, Karlsen T, Wahlstrøm T, Sand HE, Ashwathnarayan M, Gamba MT. Internet of Robotic Things Intelligent Connectivity and Platforms. Front Robot AI 2020; 7:104. [PMID: 33501271 PMCID: PMC7805974 DOI: 10.3389/frobt.2020.00104] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2019] [Accepted: 07/02/2020] [Indexed: 11/27/2022] Open
Abstract
The Internet of Things (IoT) and Industrial IoT (IIoT) have developed rapidly in the past few years, as both the Internet and "things" have evolved significantly. "Things" now range from simple Radio Frequency Identification (RFID) devices to smart wireless sensors, intelligent wireless sensors and actuators, robotic things, and autonomous vehicles operating in consumer, business, and industrial environments. The emergence of "intelligent things" (static or mobile) in collaborative autonomous fleets requires new architectures, connectivity paradigms, trustworthiness frameworks, and platforms for the integration of applications across different business and industrial domains. These new applications accelerate the development of autonomous system design paradigms and the proliferation of the Internet of Robotic Things (IoRT). In IoRT, collaborative robotic things can communicate with other things, learn autonomously, interact safely with the environment, humans and other things, and gain qualities like self-maintenance, self-awareness, self-healing, and fail-operational behavior. IoRT applications can make use of the individual, collaborative, and collective intelligence of robotic things, as well as information from the infrastructure and operating context to plan, implement and accomplish tasks under different environmental conditions and uncertainties. The continuous, real-time interaction with the environment makes perception, location, communication, cognition, computation, connectivity, propulsion, and integration of federated IoRT and digital platforms important components of new-generation IoRT applications. This paper reviews the taxonomy of the IoRT, emphasizing the IoRT intelligent connectivity, architectures, interoperability, and trustworthiness framework, and surveys the technologies that enable the application of the IoRT across different domains to perform missions more efficiently, productively, and completely. The aim is to provide a novel perspective on the IoRT that involves communication among robotic things and humans and highlights the convergence of several technologies and interactions between different taxonomies used in the literature.
Collapse
Affiliation(s)
| | | | | | - Martin Serrano
- Insight Centre for Data Analytics, National University of Ireland Galway, Galway, Ireland
| | | | | | | | | | | |
Collapse
|
16
|
Liu H, Guo D, Sun F, Yang W, Furber S, Sun T. Embodied tactile perception and learning. BRAIN SCIENCE ADVANCES 2020. [DOI: 10.26599/bsa.2020.9050012] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022] Open
Abstract
Various living creatures exhibit embodiment intelligence, which is reflected by a collaborative interaction of the brain, body, and environment. The actual behavior of embodiment intelligence is generated by a continuous and dynamic interaction between a subject and the environment through information perception and physical manipulation. The physical interaction between a robot and the environment is the basis for realizing embodied perception and learning. Tactile information plays a critical role in this physical interaction process. It can be used to ensure safety, stability, and compliance, and can provide unique information that is difficult to capture using other perception modalities. However, due to the limitations of existing sensors and perception and learning methods, the development of robotic tactile research lags significantly behind other sensing modalities, such as vision and hearing, thereby seriously restricting the development of robotic embodiment intelligence. This paper presents the current challenges related to robotic tactile embodiment intelligence and reviews the theory and methods of robotic embodied tactile intelligence. Tactile perception and learning methods for embodiment intelligence can be designed based on the development of new large‐scale tactile array sensing devices, with the aim to make breakthroughs in the neuromorphic computing technology of tactile intelligence.
Collapse
Affiliation(s)
- Huaping Liu
- Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
| | - Di Guo
- Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
| | - Fuchun Sun
- Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
| | - Wuqiang Yang
- Department of Electrical and Electronic Engineering, The University of Manchester, Manchester M13 9 PL, U.K
| | - Steve Furber
- Department of Computer Science, The University of Manchester, Manchester M13 9 PL, U.K
| | - Tengchen Sun
- Beijing Tashan Technology Co., Ltd., Beijing 102300, China
| |
Collapse
|