1
|
You B, Liu H. Multimodal information bottleneck for deep reinforcement learning with multiple sensors. Neural Netw 2024; 176:106347. [PMID: 38688069 DOI: 10.1016/j.neunet.2024.106347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2023] [Revised: 01/17/2024] [Accepted: 04/25/2024] [Indexed: 05/02/2024]
Abstract
Reinforcement learning has achieved promising results on robotic control tasks but struggles to leverage information effectively from multiple sensory modalities that differ in many characteristics. Recent works construct auxiliary losses based on reconstruction or mutual information to extract joint representations from multiple sensory inputs to improve the sample efficiency and performance of reinforcement learning algorithms. However, the representations learned by these methods could capture information irrelevant to learning a policy and may degrade the performance. We argue that compressing information in the learned joint representations about raw multimodal observations is helpful, and propose a multimodal information bottleneck model to learn task-relevant joint representations from egocentric images and proprioception. Our model compresses and retains the predictive information in multimodal observations for learning a compressed joint representation, which fuses complementary information from visual and proprioceptive feedback and meanwhile filters out task-irrelevant information in raw multimodal observations. We propose to minimize the upper bound of our multimodal information bottleneck objective for computationally tractable optimization. Experimental evaluations on several challenging locomotion tasks with egocentric images and proprioception show that our method achieves better sample efficiency and zero-shot robustness to unseen white noise than leading baselines. We also empirically demonstrate that leveraging information from egocentric images and proprioception is more helpful for learning policies on locomotion tasks than solely using one single modality.
Collapse
Affiliation(s)
- Bang You
- Department of Computer Science and Technology, Beijing National Research Centre for Information Science and Technology, Tsinghua University, Beijing 100084, China
| | - Huaping Liu
- Department of Computer Science and Technology, Beijing National Research Centre for Information Science and Technology, Tsinghua University, Beijing 100084, China.
| |
Collapse
|
2
|
Bai Y, Shao S, Zhang J, Zhao X, Fang C, Wang T, Wang Y, Zhao H. A Review of Brain-Inspired Cognition and Navigation Technology for Mobile Robots. CYBORG AND BIONIC SYSTEMS 2024; 5:0128. [PMID: 38938902 PMCID: PMC11210290 DOI: 10.34133/cbsystems.0128] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Accepted: 04/23/2024] [Indexed: 06/29/2024] Open
Abstract
Brain-inspired navigation technologies combine environmental perception, spatial cognition, and target navigation to create a comprehensive navigation research system. Researchers have used various sensors to gather environmental data and enhance environmental perception using multimodal information fusion. In spatial cognition, a neural network model is used to simulate the navigation mechanism of the animal brain and to construct an environmental cognition map. However, existing models face challenges in achieving high navigation success rate and efficiency. In addition, the limited incorporation of navigation mechanisms borrowed from animal brains necessitates further exploration. On the basis of the brain-inspired navigation process, this paper launched a systematic study on brain-inspired environment perception, brain-inspired spatial cognition, and goal-based navigation in brain-inspired navigation, which provides a new classification of brain-inspired cognition and navigation techniques and a theoretical basis for subsequent experimental studies. In the future, brain-inspired navigation technology should learn from more perfect brain-inspired mechanisms to improve its generalization ability and be simultaneously applied to large-scale distributed intelligent body cluster navigation. The multidisciplinary nature of brain-inspired navigation technology presents challenges, and multidisciplinary scholars must cooperate to promote the development of this technology.
Collapse
Affiliation(s)
- Yanan Bai
- School of Computer Science and Engineering,
Northeastern University, Shenyang 110819, China
- State Key Laboratory of Robotics, Shenyang Institute of Automation,
Chinese Academy of Sciences, Shenyang 110016, China
- Institutes for Robotics and Intelligent Manufacturing,
Chinese Academy of Sciences, Shenyang 110169, China
| | - Shiliang Shao
- State Key Laboratory of Robotics, Shenyang Institute of Automation,
Chinese Academy of Sciences, Shenyang 110016, China
- Institutes for Robotics and Intelligent Manufacturing,
Chinese Academy of Sciences, Shenyang 110169, China
| | - Jin Zhang
- School of Computer Science and Engineering,
Northeastern University, Shenyang 110819, China
- State Key Laboratory of Robotics, Shenyang Institute of Automation,
Chinese Academy of Sciences, Shenyang 110016, China
- Institutes for Robotics and Intelligent Manufacturing,
Chinese Academy of Sciences, Shenyang 110169, China
| | - Xianzhe Zhao
- School of Computer Science and Engineering,
Northeastern University, Shenyang 110819, China
- State Key Laboratory of Robotics, Shenyang Institute of Automation,
Chinese Academy of Sciences, Shenyang 110016, China
- Institutes for Robotics and Intelligent Manufacturing,
Chinese Academy of Sciences, Shenyang 110169, China
| | - Chuxi Fang
- School of Computer Science and Engineering,
Northeastern University, Shenyang 110819, China
- State Key Laboratory of Robotics, Shenyang Institute of Automation,
Chinese Academy of Sciences, Shenyang 110016, China
- Institutes for Robotics and Intelligent Manufacturing,
Chinese Academy of Sciences, Shenyang 110169, China
| | - Ting Wang
- State Key Laboratory of Robotics, Shenyang Institute of Automation,
Chinese Academy of Sciences, Shenyang 110016, China
- Institutes for Robotics and Intelligent Manufacturing,
Chinese Academy of Sciences, Shenyang 110169, China
| | - Yongliang Wang
- Department of Artificial Intelligence,
University of Groningen, Groningen 9747 AG, Netherlands
| | - Hai Zhao
- School of Computer Science and Engineering,
Northeastern University, Shenyang 110819, China
| |
Collapse
|
3
|
Chang S, Koo JH, Yoo J, Kim MS, Choi MK, Kim DH, Song YM. Flexible and Stretchable Light-Emitting Diodes and Photodetectors for Human-Centric Optoelectronics. Chem Rev 2024; 124:768-859. [PMID: 38241488 DOI: 10.1021/acs.chemrev.3c00548] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2024]
Abstract
Optoelectronic devices with unconventional form factors, such as flexible and stretchable light-emitting or photoresponsive devices, are core elements for the next-generation human-centric optoelectronics. For instance, these deformable devices can be utilized as closely fitted wearable sensors to acquire precise biosignals that are subsequently uploaded to the cloud for immediate examination and diagnosis, and also can be used for vision systems for human-interactive robotics. Their inception was propelled by breakthroughs in novel optoelectronic material technologies and device blueprinting methodologies, endowing flexibility and mechanical resilience to conventional rigid optoelectronic devices. This paper reviews the advancements in such soft optoelectronic device technologies, honing in on various materials, manufacturing techniques, and device design strategies. We will first highlight the general approaches for flexible and stretchable device fabrication, including the appropriate material selection for the substrate, electrodes, and insulation layers. We will then focus on the materials for flexible and stretchable light-emitting diodes, their device integration strategies, and representative application examples. Next, we will move on to the materials for flexible and stretchable photodetectors, highlighting the state-of-the-art materials and device fabrication methods, followed by their representative application examples. At the end, a brief summary will be given, and the potential challenges for further development of functional devices will be discussed as a conclusion.
Collapse
Affiliation(s)
- Sehui Chang
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), Gwangju 61005, Republic of Korea
| | - Ja Hoon Koo
- Department of Semiconductor Systems Engineering, Sejong University, Seoul 05006, Republic of Korea
- Institute of Semiconductor and System IC, Sejong University, Seoul 05006, Republic of Korea
| | - Jisu Yoo
- Department of Materials Science and Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan 44919, Republic of Korea
| | - Min Seok Kim
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), Gwangju 61005, Republic of Korea
| | - Moon Kee Choi
- Department of Materials Science and Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan 44919, Republic of Korea
- Graduate School of Semiconductor Materials and Devices Engineering, Center for Future Semiconductor Technology (FUST), UNIST, Ulsan 44919, Republic of Korea
- Center for Nanoparticle Research, Institute for Basic Science (IBS), Seoul 08826, Republic of Korea
| | - Dae-Hyeong Kim
- Center for Nanoparticle Research, Institute for Basic Science (IBS), Seoul 08826, Republic of Korea
- School of Chemical and Biological Engineering, Institute of Chemical Processes, Seoul National University (SNU), Seoul 08826, Republic of Korea
- Department of Materials Science and Engineering, SNU, Seoul 08826, Republic of Korea
- Interdisciplinary Program for Bioengineering, SNU, Seoul 08826, Republic of Korea
| | - Young Min Song
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), Gwangju 61005, Republic of Korea
- Department of Materials Science and Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan 44919, Republic of Korea
- Artificial Intelligence (AI) Graduate School, GIST, Gwangju 61005, Republic of Korea
| |
Collapse
|
4
|
McCarthy WP, Kirsh D, Fan JE. Consistency and Variation in Reasoning About Physical Assembly. Cogn Sci 2023; 47:e13397. [PMID: 38146204 DOI: 10.1111/cogs.13397] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Revised: 10/27/2023] [Accepted: 12/06/2023] [Indexed: 12/27/2023]
Abstract
The ability to reason about how things were made is a pervasive aspect of how humans make sense of physical objects. Such reasoning is useful for a range of everyday tasks, from assembling a piece of furniture to making a sandwich and knitting a sweater. What enables people to reason in this way even about novel objects, and how do people draw upon prior experience with an object to continually refine their understanding of how to create it? To explore these questions, we developed a virtual task environment to investigate how people come up with step-by-step procedures for recreating block towers whose composition was not readily apparent, and analyzed how the procedures they used to build them changed across repeated attempts. Specifically, participants (N = 105) viewed 2D silhouettes of eight unique block towers in a virtual environment simulating rigid-body physics, and aimed to reconstruct each one in less than 60 s. We found that people built each tower more accurately and quickly across repeated attempts, and that this improvement reflected both group-level convergence upon a tiny fraction of all possible viable procedures, as well as error-dependent updating across successive attempts by the same individual. Taken together, our study presents a scalable approach to measuring consistency and variation in how people infer solutions to physical assembly problems.
Collapse
Affiliation(s)
| | - David Kirsh
- Department of Cognitive Science, University of California San Diego
| | - Judith E Fan
- Department of Psychology, University of California San Diego
- Department of Psychology, Stanford University
| |
Collapse
|
5
|
Zhao H, Zhang Y, Han L, Qian W, Wang J, Wu H, Li J, Dai Y, Zhang Z, Bowen CR, Yang Y. Intelligent Recognition Using Ultralight Multifunctional Nano-Layered Carbon Aerogel Sensors with Human-Like Tactile Perception. NANO-MICRO LETTERS 2023; 16:11. [PMID: 37943399 PMCID: PMC10635924 DOI: 10.1007/s40820-023-01216-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Accepted: 09/11/2023] [Indexed: 11/10/2023]
Abstract
Humans can perceive our complex world through multi-sensory fusion. Under limited visual conditions, people can sense a variety of tactile signals to identify objects accurately and rapidly. However, replicating this unique capability in robots remains a significant challenge. Here, we present a new form of ultralight multifunctional tactile nano-layered carbon aerogel sensor that provides pressure, temperature, material recognition and 3D location capabilities, which is combined with multimodal supervised learning algorithms for object recognition. The sensor exhibits human-like pressure (0.04-100 kPa) and temperature (21.5-66.2 °C) detection, millisecond response times (11 ms), a pressure sensitivity of 92.22 kPa-1 and triboelectric durability of over 6000 cycles. The devised algorithm has universality and can accommodate a range of application scenarios. The tactile system can identify common foods in a kitchen scene with 94.63% accuracy and explore the topographic and geomorphic features of a Mars scene with 100% accuracy. This sensing approach empowers robots with versatile tactile perception to advance future society toward heightened sensing, recognition and intelligence.
Collapse
Affiliation(s)
- Huiqi Zhao
- CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro-Nano Energy and Sensor, Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing, 101400, People's Republic of China
- School of Nanoscience and Technology, University of Chinese Academy of Sciences, Beijing, 100049, People's Republic of China
| | - Yizheng Zhang
- Tencent Robotics X, Shenzhen, 518054, People's Republic of China
| | - Lei Han
- Tencent Robotics X, Shenzhen, 518054, People's Republic of China
| | - Weiqi Qian
- CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro-Nano Energy and Sensor, Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing, 101400, People's Republic of China
- School of Nanoscience and Technology, University of Chinese Academy of Sciences, Beijing, 100049, People's Republic of China
| | - Jiabin Wang
- CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro-Nano Energy and Sensor, Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing, 101400, People's Republic of China
- Center on Nanoenergy Research, School of Physical Science and Technology, Guangxi University, Nanning, 530004, People's Republic of China
| | - Heting Wu
- CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro-Nano Energy and Sensor, Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing, 101400, People's Republic of China
| | - Jingchen Li
- Tencent Robotics X, Shenzhen, 518054, People's Republic of China
| | - Yuan Dai
- Tencent Robotics X, Shenzhen, 518054, People's Republic of China.
| | - Zhengyou Zhang
- Tencent Robotics X, Shenzhen, 518054, People's Republic of China
| | - Chris R Bowen
- Department of Mechanical Engineering, University of Bath, Bath, BA2 7AK, UK
| | - Ya Yang
- CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro-Nano Energy and Sensor, Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing, 101400, People's Republic of China.
- School of Nanoscience and Technology, University of Chinese Academy of Sciences, Beijing, 100049, People's Republic of China.
- Center on Nanoenergy Research, School of Physical Science and Technology, Guangxi University, Nanning, 530004, People's Republic of China.
| |
Collapse
|
6
|
A Recognition Method for Soft Objects Based on the Fusion of Vision and Haptics. Biomimetics (Basel) 2023; 8:biomimetics8010086. [PMID: 36810417 PMCID: PMC9944461 DOI: 10.3390/biomimetics8010086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 02/07/2023] [Accepted: 02/14/2023] [Indexed: 02/22/2023] Open
Abstract
For humans and animals to recognise an object, the integration of multiple sensing methods is essential when one sensing modality is only able to acquire limited information. Among the many sensing modalities, vision has been intensively studied and proven to have superior performance for many problems. Nevertheless, there are many problems which are difficult to solve by solitary vision, such as in a dark environment or for objects with a similar outlook but different inclusions. Haptic sensing is another commonly used means of perception, which can provide local contact information and physical features that are difficult to obtain by vision. Therefore, the fusion of vision and touch is beneficial to improve the robustness of object perception. To address this, an end-to-end visual-haptic fusion perceptual method has been proposed. In particular, the YOLO deep network is used to extract vision features, while haptic explorations are used to extract haptic features. Then, visual and haptic features are aggregated using a graph convolutional network, and the object is recognised based on a multi-layer perceptron. Experimental results show that the proposed method excels in distinguishing soft objects that have similar appearance but varied interior fillers, comparing a simple convolutional network and a Bayesian filter. The resultant average recognition accuracy was improved to 0.95 from vision only (mAP is 0.502). Moreover, the extracted physical features could be further used for manipulation tasks targeting soft objects.
Collapse
|
7
|
Khan MS, Olds JL. When neuro-robots go wrong: A review. Front Neurorobot 2023; 17:1112839. [PMID: 36819005 PMCID: PMC9935594 DOI: 10.3389/fnbot.2023.1112839] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Accepted: 01/19/2023] [Indexed: 02/05/2023] Open
Abstract
Neuro-robots are a class of autonomous machines that, in their architecture, mimic aspects of the human brain and cognition. As such, they represent unique artifacts created by humans based on human understanding of healthy human brains. European Union's Convention on Roboethics 2025 states that the design of all robots (including neuro-robots) must include provisions for the complete traceability of the robots' actions, analogous to an aircraft's flight data recorder. At the same time, one can anticipate rising instances of neuro-robotic failure, as they operate on imperfect data in real environments, and the underlying AI behind such neuro-robots has yet to achieve explainability. This paper reviews the trajectory of the technology used in neuro-robots and accompanying failures. The failures demand an explanation. While drawing on existing explainable AI research, we argue explainability in AI limits the same in neuro-robots. In order to make robots more explainable, we suggest potential pathways for future research.
Collapse
|
8
|
Tan H, Sun Z, Zhu X. Editorial: Neuro-inspired sensing and computing: Novel materials, devices, and systems. Front Comput Neurosci 2023; 17:1126493. [PMID: 36714052 PMCID: PMC9878689 DOI: 10.3389/fncom.2023.1126493] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2022] [Accepted: 01/02/2023] [Indexed: 01/15/2023] Open
Affiliation(s)
- Hongwei Tan
- Department of Applied Physics, Aalto University, Espoo, Finland,*Correspondence: Hongwei Tan ✉
| | - Zhong Sun
- Institute for Artificial Intelligence, School of Integrated Circuits, Beijing Advanced Innovation Center for Integrated Circuits, Peking University, Beijing, China,Zhong Sun ✉
| | - Xiaojian Zhu
- Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, China,Xiaojian Zhu ✉
| |
Collapse
|
9
|
Marchionna L, Pugliese G, Martini M, Angarano S, Salvetti F, Chiaberge M. Deep Instance Segmentation and Visual Servoing to Play Jenga with a Cost-Effective Robotic System. SENSORS (BASEL, SWITZERLAND) 2023; 23:752. [PMID: 36679543 PMCID: PMC9866192 DOI: 10.3390/s23020752] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 01/04/2023] [Accepted: 01/06/2023] [Indexed: 06/17/2023]
Abstract
The game of Jenga is a benchmark used for developing innovative manipulation solutions for complex tasks. Indeed, it encourages the study of novel robotics methods to successfully extract blocks from a tower. A Jenga game involves many traits of complex industrial and surgical manipulation tasks, requiring a multi-step strategy, the combination of visual and tactile data, and the highly precise motion of a robotic arm to perform a single block extraction. In this work, we propose a novel, cost-effective architecture for playing Jenga with e.Do, a 6DOF anthropomorphic manipulator manufactured by Comau, a standard depth camera, and an inexpensive monodirectional force sensor. Our solution focuses on a visual-based control strategy to accurately align the end-effector with the desired block, enabling block extraction by pushing. To this aim, we trained an instance segmentation deep learning model on a synthetic custom dataset to segment each piece of the Jenga tower, allowing for visual tracking of the desired block's pose during the motion of the manipulator. We integrated the visual-based strategy with a 1D force sensor to detect whether the block could be safely removed by identifying a force threshold value. Our experimentation shows that our low-cost solution allows e.DO to precisely reach removable blocks and perform up to 14 consecutive extractions in a row.
Collapse
Affiliation(s)
- Luca Marchionna
- Department of Electronics and Telecommunications (DET), Politecnico di Torino, 10129 Torino, Italy
| | - Giulio Pugliese
- Department of Electronics and Telecommunications (DET), Politecnico di Torino, 10129 Torino, Italy
| | - Mauro Martini
- Department of Electronics and Telecommunications (DET), Politecnico di Torino, 10129 Torino, Italy
- PIC4SeR Interdepartmental Centre for Service Robotics, 10129 Torino, Italy
| | - Simone Angarano
- Department of Electronics and Telecommunications (DET), Politecnico di Torino, 10129 Torino, Italy
- PIC4SeR Interdepartmental Centre for Service Robotics, 10129 Torino, Italy
| | - Francesco Salvetti
- Department of Electronics and Telecommunications (DET), Politecnico di Torino, 10129 Torino, Italy
- PIC4SeR Interdepartmental Centre for Service Robotics, 10129 Torino, Italy
| | - Marcello Chiaberge
- Department of Electronics and Telecommunications (DET), Politecnico di Torino, 10129 Torino, Italy
- PIC4SeR Interdepartmental Centre for Service Robotics, 10129 Torino, Italy
| |
Collapse
|
10
|
Attention-based adaptive context network for anchor-free instance segmentation. INT J MACH LEARN CYB 2022. [DOI: 10.1007/s13042-022-01648-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2022]
|
11
|
Abstract
Vision and learning have long been considered to be two areas of research linked only distantly. However, recent developments in vision research have changed the conceptual definition of vision from a signal-evaluating process to a goal-oriented interpreting process, and this shift binds learning, together with the resulting internal representations, intimately to vision. In this review, we consider various types of learning (perceptual, statistical, and rule/abstract) associated with vision in the past decades and argue that they represent differently specialized versions of the fundamental learning process, which must be captured in its entirety when applied to complex visual processes. We show why the generalized version of statistical learning can provide the appropriate setup for such a unified treatment of learning in vision, what computational framework best accommodates this kind of statistical learning, and what plausible neural scheme could feasibly implement this framework. Finally, we list the challenges that the field of statistical learning faces in fulfilling the promise of being the right vehicle for advancing our understanding of vision in its entirety. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
Collapse
Affiliation(s)
- József Fiser
- Department of Cognitive Science, Center for Cognitive Computation, Central European University, Vienna 1100, Austria;
| | - Gábor Lengyel
- Department of Brain and Cognitive Sciences, University of Rochester, Rochester, New York 14627, USA
| |
Collapse
|
12
|
Wang H, Liu X, Qiu N, Guo N, Wan F, Song C. DeepClaw 2.0: A Data Collection Platform for Learning Human Manipulation. Front Robot AI 2022; 9:787291. [PMID: 35368430 PMCID: PMC8964492 DOI: 10.3389/frobt.2022.787291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 01/31/2022] [Indexed: 11/23/2022] Open
Abstract
Besides direct interaction, human hands are also skilled at using tools to manipulate objects for typical life and work tasks. This paper proposes DeepClaw 2.0 as a low-cost, open-sourced data collection platform for learning human manipulation. We use an RGB-D camera to visually track the motion and deformation of a pair of soft finger networks on a modified kitchen tong operated by human teachers. These fingers can be easily integrated with robotic grippers to bridge the structural mismatch between humans and robots during learning. The deformation of soft finger networks, which reveals tactile information in contact-rich manipulation, is captured passively. We collected a comprehensive sample dataset involving five human demonstrators in ten manipulation tasks with five trials per task. As a low-cost, open-sourced platform, we also developed an intuitive interface that converts the raw sensor data into state-action data for imitation learning problems. For learning-by-demonstration problems, we further demonstrated our dataset’s potential by using real robotic hardware to collect joint actuation data or using a simulated environment when limited access to the hardware.
Collapse
Affiliation(s)
- Haokun Wang
- Robotics and Autonomous Systems Thrust, System Hub, Hong Kong University of Science and Technology, Hong Kong, Hong Kong SAR, China
| | - Xiaobo Liu
- Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, China
| | - Nuofan Qiu
- Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, China
| | - Ning Guo
- Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, China
| | - Fang Wan
- School of Design, Southern University of Science and Technology, Shenzhen, China
- *Correspondence: Fang Wan, ; Chaoyang Song,
| | - Chaoyang Song
- Shenzhen Key Laboratory of Biomimetic Robotics and Intelligent Systems, Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen, China
- Guangdong Provincial Key Laboratory of Human-Augmentation and Rehabilitation Robotics in Universities, Southern University of Science and Technology, Shenzhen, China
- *Correspondence: Fang Wan, ; Chaoyang Song,
| |
Collapse
|
13
|
Abd MA, Ingicco J, Hutchinson DT, Tognoli E, Engeberg ED. Multichannel haptic feedback unlocks prosthetic hand dexterity. Sci Rep 2022; 12:2323. [PMID: 35149695 PMCID: PMC8837642 DOI: 10.1038/s41598-022-04953-1] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2021] [Accepted: 12/20/2021] [Indexed: 01/13/2023] Open
Abstract
Loss of tactile sensations is a major roadblock preventing upper limb-absent people from multitasking or using the full dexterity of their prosthetic hands. With current myoelectric prosthetic hands, limb-absent people can only control one grasp function at a time even though modern artificial hands are mechanically capable of individual control of all five digits. In this paper, we investigated whether people could precisely control the grip forces applied to two different objects grasped simultaneously with a dexterous artificial hand. Toward that end, we developed a novel multichannel wearable soft robotic armband to convey artificial sensations of touch to the robotic hand users. Multiple channels of haptic feedback enabled subjects to successfully grasp and transport two objects simultaneously with the dexterous artificial hand without breaking or dropping them, even when their vision of both objects was obstructed. Simultaneous transport of the objects provided a significant time savings to perform the deliveries in comparison to a one-at-a-time approach. This paper demonstrated that subjects were able to integrate multiple channels of haptic feedback into their motor control strategies to perform a complex simultaneous object grasp control task with an artificial limb, which could serve as a paradigm shift in the way prosthetic hands are operated.
Collapse
Affiliation(s)
- Moaed A Abd
- Ocean and Mechanical Engineering Department, Florida Atlantic University, Boca Raton, FL, USA
| | - Joseph Ingicco
- Ocean and Mechanical Engineering Department, Florida Atlantic University, Boca Raton, FL, USA
| | | | - Emmanuelle Tognoli
- The Center for Complex Systems & Brain Sciences, Florida Atlantic University, Boca Raton, FL, USA
| | - Erik D Engeberg
- Ocean and Mechanical Engineering Department, Florida Atlantic University, Boca Raton, FL, USA. .,The Center for Complex Systems & Brain Sciences, Florida Atlantic University, Boca Raton, FL, USA.
| |
Collapse
|
14
|
Efficient Reinforcement Learning from Demonstration via Bayesian Network-Based Knowledge Extraction. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2021; 2021:7588221. [PMID: 34603434 PMCID: PMC8486502 DOI: 10.1155/2021/7588221] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Accepted: 08/19/2021] [Indexed: 11/17/2022]
Abstract
Reinforcement learning from demonstration (RLfD) is considered to be a promising approach to improve reinforcement learning (RL) by leveraging expert demonstrations as the additional decision-making guidance. However, most existing RLfD methods only regard demonstrations as low-level knowledge instances under a certain task. Demonstrations are generally used to either provide additional rewards or pretrain the neural network-based RL policy in a supervised manner, usually resulting in poor generalization capability and weak robustness performance. Considering that human knowledge is not only interpretable but also suitable for generalization, we propose to exploit the potential of demonstrations by extracting knowledge from them via Bayesian networks and develop a novel RLfD method called Reinforcement Learning from demonstration via Bayesian Network-based Knowledge (RLBNK). The proposed RLBNK method takes advantage of node influence with the Wasserstein distance metric (NIW) algorithm to obtain abstract concepts from demonstrations and then a Bayesian network conducts knowledge learning and inference based on the abstract data set, which will yield the coarse policy with corresponding confidence. Once the coarse policy's confidence is low, another RL-based refine module will further optimize and fine-tune the policy to form a (near) optimal hybrid policy. Experimental results show that the proposed RLBNK method improves the learning efficiency of corresponding baseline RL algorithms under both normal and sparse reward settings. Furthermore, we demonstrate that our RLBNK method delivers better generalization capability and robustness than baseline methods.
Collapse
|
15
|
Cui J, Trinkle J. Toward next-generation learned robot manipulation. Sci Robot 2021; 6:6/54/eabd9461. [PMID: 34043539 DOI: 10.1126/scirobotics.abd9461] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2020] [Accepted: 04/29/2021] [Indexed: 11/03/2022]
Abstract
The ever-changing nature of human environments presents great challenges to robot manipulation. Objects that robots must manipulate vary in shape, weight, and configuration. Important properties of the robot, such as surface friction and motor torque constants, also vary over time. Before robot manipulators can work gracefully in homes and businesses, they must be adaptive to such variations. This survey summarizes types of variations that robots may encounter in human environments and categorizes, compares, and contrasts the ways in which learning has been applied to manipulation problems through the lens of adaptability. Promising avenues for future research are proposed at the end.
Collapse
Affiliation(s)
- Jinda Cui
- Department of Computer Science and Engineering, Lehigh University, Bethlehem, PA 18015, USA.
| | - Jeff Trinkle
- Department of Computer Science and Engineering, Lehigh University, Bethlehem, PA 18015, USA
| |
Collapse
|
16
|
Abstract
In the last century, robots have been revolutionizing our lives, augmenting human actions with greater precision and repeatability. Unfortunately, most robotic systems can only operate in controlled environments. While increasing the complexity of the centralized controller is an instinctive direction to enable robots that are capable of autonomously adapting to their environment, there are ample examples in nature where adaptivity emerges from simpler decentralized processes. Here we perform experiments and simulations on a modular and scalable robotic platform in which each unit is stochastically updating its own behavior to explore requirements needed for a decentralized learning strategy capable of achieving locomotion in a continuously changing environment or when undergoing damage. One of the main challenges in robotics is the development of systems that can adapt to their environment and achieve autonomous behavior. Current approaches typically aim to achieve this by increasing the complexity of the centralized controller by, e.g., direct modeling of their behavior, or implementing machine learning. In contrast, we simplify the controller using a decentralized and modular approach, with the aim of finding specific requirements needed for a robust and scalable learning strategy in robots. To achieve this, we conducted experiments and simulations on a specific robotic platform assembled from identical autonomous units that continuously sense their environment and react to it. By letting each unit adapt its behavior independently using a basic Monte Carlo scheme, the assembled system is able to learn and maintain optimal behavior in a dynamic environment as long as its memory is representative of the current environment, even when incurring damage. We show that the physical connection between the units is enough to achieve learning, and no additional communication or centralized information is required. As a result, such a distributed learning approach can be easily scaled to larger assemblies, blurring the boundaries between materials and robots, paving the way for a new class of modular “robotic matter” that can autonomously learn to thrive in dynamic or unfamiliar situations, for example, encountered by soft robots or self-assembled (micro)robots in various environments spanning from the medical realm to space explorations.
Collapse
|
17
|
Yang L, Han X, Guo W, Wan F, Pan J, Song C. Learning-Based Optoelectronically Innervated Tactile Finger for Rigid-Soft Interactive Grasping. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3065186] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
18
|
Burn Image Recognition of Medical Images Based on Deep Learning: From CNNs to Advanced Networks. Neural Process Lett 2021. [DOI: 10.1007/s11063-021-10459-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
19
|
Yang GZ. Robot learning-Beyond imitation. Sci Robot 2021; 4:4/26/eaaw3520. [PMID: 33137765 DOI: 10.1126/scirobotics.aaw3520] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2018] [Accepted: 01/08/2019] [Indexed: 11/02/2022]
Abstract
This special issue covers different aspects and applications of robot learning and outlines current progress and opportunities, as well as the challenges ahead.
Collapse
Affiliation(s)
- Guang-Zhong Yang
- Guang-Zhong Yang is the Editor of Science Robotics and Director and Co-founder of the Hamlyn Centre, Imperial College London, London, UK.
| |
Collapse
|
20
|
Development and Application of a Tandem Force Sensor. SENSORS 2020; 20:s20216042. [PMID: 33114199 PMCID: PMC7660700 DOI: 10.3390/s20216042] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/02/2020] [Revised: 10/18/2020] [Accepted: 10/20/2020] [Indexed: 11/16/2022]
Abstract
In robot teaching for contact tasks, it is necessary to not only accurately perceive the traction force exerted by hands, but also to perceive the contact force at the robot end. This paper develops a tandem force sensor to detect traction and contact forces. As a component of the tandem force sensor, a cylindrical traction force sensor is developed to detect the traction force applied by hands. Its structure is designed to be suitable for humans to operate, and the mechanical model of its cylinder-shaped elastic structural body has been analyzed. After calibration, the cylindrical traction force sensor is proven to be able to detect forces/moments with small errors. Then, a tandem force sensor is developed based on the developed cylindrical traction force sensor and a wrist force sensor. The robot teaching experiment of drawer switches were made and the results confirm that the developed traction force sensor is simple to operate and the tandem force sensor can achieve the perception of the traction and contact forces.
Collapse
|
21
|
Marjaninejad A, Urbina-Melendez D, Valero-Cuevas F. Simple Kinematic Feedback Enhances Autonomous Learning in Bio-Inspired Tendon-Driven Systems. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:4687-4693. [PMID: 33019039 DOI: 10.1109/embc44109.2020.9176182] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Error feedback is known to improve performance by correcting control signals in response to perturbations. Here we show how adding simple error feedback can also accelerate and robustify autonomous learning in a tendon-driven robot. We have implemented two versions of the General-to-Particular (G2P) autonomous learning algorithm using a tendon-driven leg with two joints and three tendons: one with and one without real-time kinematic feedback. We have performed a rigorous study on the performance of each system, for both simulation and physical implementation cases, over a wide range of tasks. As expected, feedback improved performance in simulation and hardware. However, we see these improvements even in the presence of sensory delays of up to 100 ms and when experiencing substantial contact collisions. Importantly, feedback accelerates learning and enhances G2P's continual refinement of the initial inverse map by providing the system with more relevant data to train on. This allows the system to perform well even after only 60 seconds of initial motor babbling.
Collapse
|
22
|
Cui S, Wang R, Wei J, Hu J, Wang S. Self-Attention Based Visual-Tactile Fusion Learning for Predicting Grasp Outcomes. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.3010720] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
23
|
Abstract
The efficient multi-modal fusion of data streams from different sensors is a crucial ability that a robotic perception system should exhibit to ensure robustness against disturbances. However, as the volume and dimensionality of sensory-feedback increase it might be difficult to manually design a multimodal-data fusion system that can handle heterogeneous data. Nowadays, multi-modal machine learning is an emerging field with research focused mainly on analyzing vision and audio information. Although, from the robotics perspective, haptic sensations experienced from interaction with an environment are essential to successfully execute useful tasks. In our work, we compared four learning-based multi-modal fusion methods on three publicly available datasets containing haptic signals, images, and robots’ poses. During tests, we considered three tasks involving such data, namely grasp outcome classification, texture recognition, and—most challenging—multi-label classification of haptic adjectives based on haptic and visual data. Conducted experiments were focused not only on the verification of the performance of each method but mainly on their robustness against data degradation. We focused on this aspect of multi-modal fusion, as it was rarely considered in the research papers, and such degradation of sensory feedback might occur during robot interaction with its environment. Additionally, we verified the usefulness of data augmentation to increase the robustness of the aforementioned data fusion methods.
Collapse
|
24
|
Lee MA, Zhu Y, Zachares P, Tan M, Srinivasan K, Savarese S, Fei-Fei L, Garg A, Bohg J. Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks. IEEE T ROBOT 2020. [DOI: 10.1109/tro.2019.2959445] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
25
|
Homayounfar N, Xiong Y, Liang J, Ma WC, Urtasun R. LevelSet R-CNN: A Deep Variational Method for Instance Segmentation. COMPUTER VISION – ECCV 2020 2020. [DOI: 10.1007/978-3-030-58592-1_33] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
|
26
|
Marjaninejad A, Urbina-Meléndez D, Cohn BA, Valero-Cuevas FJ. Autonomous Functional Movements in a Tendon-Driven Limb via Limited Experience. NAT MACH INTELL 2019; 1:144-154. [PMID: 31161156 PMCID: PMC6544439 DOI: 10.1038/s42256-019-0029-0] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2018] [Accepted: 02/05/2019] [Indexed: 11/09/2022]
Abstract
Robots will become ubiquitously useful only when they can use few attempts to teach themselves to perform different tasks, even with complex bodies and in dynamical environments. Vertebrates, in fact, use sparse trial-and-error to learn multiple tasks despite their intricate tendon-driven anatomies-which are particularly hard to control because they are simultaneously nonlinear, under-determined, and over-determined. We demonstrate-for the first time in simulation and hardware-how a model-free, open-loop approach allows few-shot autonomous learning to produce effective movements in a 3-tendon 2-joint limb. We use a short period of motor babbling (to create an initial inverse map) followed by building functional habits by reinforcing high-reward behavior and refinements of the inverse map in a movement's neighborhood. This biologically-plausible algorithm, which we call G2P (General-to-Particular), can potentially enable quick, robust and versatile adaptation in robots as well as shed light on the foundations of the enviable functional versatility of organisms.
Collapse
Affiliation(s)
- Ali Marjaninejad
- Department of Biomedical, University of Southern California, Los Angeles, CA, USA
- Department of Electrical (Systems), University of Southern California, Los Angeles, CA, USA
| | | | - Brian A. Cohn
- Department of Computer Science, University of Southern California, Los Angeles, CA, USA
| | - Francisco J. Valero-Cuevas
- Department of Biomedical, University of Southern California, Los Angeles, CA, USA
- Department of Electrical (Systems), University of Southern California, Los Angeles, CA, USA
- Department of Aerospace & Mechanical Engineering, University of Southern California, Los Angeles, CA, USA
- Department of Computer Science, University of Southern California, Los Angeles, CA, USA
- Division of Biokinesiology & Physical Therapy University of Southern California, Los Angeles, CA, USA
| |
Collapse
|