1
|
Lee J, Miri S, Bayro A, Kim M, Jeong H, Yeo WH. Biosignal-integrated robotic systems with emerging trends in visual interfaces: A systematic review. BIOPHYSICS REVIEWS 2024; 5:011301. [PMID: 38510371 PMCID: PMC10903439 DOI: 10.1063/5.0185568] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 01/29/2024] [Indexed: 03/22/2024]
Abstract
Human-machine interfaces (HMI) are currently a trendy and rapidly expanding area of research. Interestingly, the human user does not readily observe the interface between humans and machines. Instead, interactions between the machine and electrical signals from the user's body are obscured by complex control algorithms. The result is effectively a one-way street, wherein data is only transmitted from human to machine. Thus, a gap remains in the literature: how can information be effectively conveyed to the user to enable mutual understanding between humans and machines? Here, this paper reviews recent advancements in biosignal-integrated wearable robotics, with a particular emphasis on "visualization"-the presentation of relevant data, statistics, and visual feedback to the user. This review article covers various signals of interest, such as electroencephalograms and electromyograms, and explores novel sensor architectures and key materials. Recent developments in wearable robotics are examined from control and mechanical design perspectives. Additionally, we discuss current visualization methods and outline the field's future direction. While much of the HMI field focuses on biomedical and healthcare applications, such as rehabilitation of spinal cord injury and stroke patients, this paper also covers less common applications in manufacturing, defense, and other domains.
Collapse
Affiliation(s)
| | - Sina Miri
- Department of Mechanical and Industrial Engineering, The University of Illinois at Chicago, Chicago, Illinois 60607, USA
| | - Allison Bayro
- School of Biological and Health Systems Engineering, Ira A. Fulton Schools of Engineering, Arizona State University, Tempe, Arizona 85287, USA
| | - Myunghee Kim
- Department of Mechanical and Industrial Engineering, The University of Illinois at Chicago, Chicago, Illinois 60607, USA
| | - Heejin Jeong
- Authors to whom correspondence should be addressed:; ; and
| | - Woon-Hong Yeo
- Authors to whom correspondence should be addressed:; ; and
| |
Collapse
|
2
|
Wu T, Ren J, Cheng C, Liu X, Peng H, Lu H. An Admittance Control Method Based on Parameters Fuzzification for Humanoid Steering Wheel Manipulation. Biomimetics (Basel) 2023; 8:495. [PMID: 37887626 PMCID: PMC10603995 DOI: 10.3390/biomimetics8060495] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Revised: 10/10/2023] [Accepted: 10/13/2023] [Indexed: 10/28/2023] Open
Abstract
Developing a human bionic manipulator to achieve certain humanoid behavioral skills is a rising problem. Enabling robots to operate the steering wheel to drive the vehicle is a challenging task. To address the problem, this work designs a novel 7-DOF (degree of freedom) humanoid manipulator based on the arm structure of human bionics. The 3-2-2 structural arrangement of the motors and the structural modifications at the wrist allow the manipulator to act more similar to a man. Meanwhile, to manipulate the steering wheel stably and compliantly, an admittance control approach is firstly applied for this case. Considering that the system parameters vary in configuration, we further introduce parameter fuzzification for admittance control. Designed simulations were carried out in Coppeliasim to verify the proposed control approach. As the result shows, the improved method could realize compliant force control under extreme configurations. It demonstrates that the humanoid manipulator can twist the steering wheel stably even in extreme configurations. It is the first exploration to operate a steering wheel similar to a human with a manipulator by using admittance control.
Collapse
Affiliation(s)
- Tuochang Wu
- College of Intelligence Science and Technology, National University of Defense Technology, 109 Deya Road, Kaifu District, Changsha 410073, China; (T.W.); (C.C.)
| | - Junkai Ren
- College of Intelligence Science and Technology, National University of Defense Technology, 109 Deya Road, Kaifu District, Changsha 410073, China; (T.W.); (C.C.)
| | - Chuang Cheng
- College of Intelligence Science and Technology, National University of Defense Technology, 109 Deya Road, Kaifu District, Changsha 410073, China; (T.W.); (C.C.)
| | - Xun Liu
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China;
| | - Hui Peng
- College of Computer Science and Technology, Central South University, Changsha 410017, China;
| | - Huimin Lu
- College of Intelligence Science and Technology, National University of Defense Technology, 109 Deya Road, Kaifu District, Changsha 410073, China; (T.W.); (C.C.)
| |
Collapse
|
3
|
Bilyea AJ, French SH, Abdullah HA. Modeling contact forces during human-robot interactions for performing activities of daily living. Proc Inst Mech Eng H 2023:9544119231179568. [PMID: 37300488 DOI: 10.1177/09544119231179568] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
This study aims to characterize contact forces between humans and tools during activities of daily living (ADL) to provide information to a personal care robot. The study was conducted on non-impaired subjects to capture various static and dynamic force levels when making contact with three different robotic tools, each designed to perform an ADL task: hair brushing, face wiping, and face shaving. The static trial of the study involved 21 participants. Forces were collected at fixed locations for each task and used to develop models for each participant. Extraction of the maximum force levels was performed for both the maximum and desired levels of force. The dynamic trial involved 24 individuals. Participants were asked to maintain a comfortable level of force for the duration of their contact with the tool as the robot moved along its path to perform the ADL task. For the static and dynamic trials, higher forces were observed during hair brushing compared to the other two tasks. It was observed that the hair brushing task force at a specific contact point has an overall maximum of 55.66 N, while the maximum forces detected in the face wiping and face shaving tasks were 36.40 and 11.11 N, respectively. The forces collected were analyzed, and no trends were found relating the contact forces to the gender, height, or weight of the subjects. Based on the analysis of the results, recommendations have been made to enhance the force safety limits for the personal care robot working environment.
Collapse
Affiliation(s)
- Aubrianna Jn Bilyea
- Robotics Institute, School of Engineering, University of Guelph, Guelph, ON, Canada
| | - Steven H French
- Robotics Institute, School of Engineering, University of Guelph, Guelph, ON, Canada
| | - Hussein A Abdullah
- Robotics Institute, School of Engineering, University of Guelph, Guelph, ON, Canada
| |
Collapse
|
4
|
Arcangeli D, Dubois O, Roby-Brami A, Famié S, de Marco G, Arnold G, Jarrassé N, Parry R. Human Exteroception during Object Handling with an Upper Limb Exoskeleton. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23115158. [PMID: 37299885 DOI: 10.3390/s23115158] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 05/17/2023] [Accepted: 05/18/2023] [Indexed: 06/12/2023]
Abstract
Upper limb exoskeletons may confer significant mechanical advantages across a range of tasks. The potential consequences of the exoskeleton upon the user's sensorimotor capacities however, remain poorly understood. The purpose of this study was to examine how the physical coupling of the user's arm to an upper limb exoskeleton influenced the perception of handheld objects. In the experimental protocol, participants were required to estimate the length of a series of bars held in their dominant right hand, in the absence of visual feedback. Their performance in conditions with an exoskeleton fixed to the forearm and upper arm was compared to conditions without the upper limb exoskeleton. Experiment 1 was designed to verify the effects of attaching an exoskeleton to the upper limb, with object handling limited to rotations of the wrist only. Experiment 2 was designed to verify the effects of the structure, and its mass, with combined movements of the wrist, elbow, and shoulder. Statistical analysis indicated that movements performed with the exoskeleton did not significantly affect perception of the handheld object in experiment 1 (BF01 = 2.3) or experiment 2 (BF01 = 4.3). These findings suggest that while the integration of an exoskeleton complexifies the architecture of the upper limb effector, this does not necessarily impede transmission of the mechanical information required for human exteroception.
Collapse
Affiliation(s)
- Dorine Arcangeli
- LINP2, UPL, UFR STAPS, Université Paris Nanterre, 200 Avenue de la République, 92001 Nanterre, France
- CAYLAR, 14 Avenue du Québec, 91140 Villebonne sur Yvette, France
| | - Océane Dubois
- ISIR, Sorbonne University, CNRS UMR 7222, ERL INSERM U 1150, 75005 Paris, France
| | - Agnès Roby-Brami
- ISIR, Sorbonne University, CNRS UMR 7222, ERL INSERM U 1150, 75005 Paris, France
| | - Sylvain Famié
- LINP2, UPL, UFR STAPS, Université Paris Nanterre, 200 Avenue de la République, 92001 Nanterre, France
| | - Giovanni de Marco
- LINP2, UPL, UFR STAPS, Université Paris Nanterre, 200 Avenue de la République, 92001 Nanterre, France
| | - Gabriel Arnold
- CAYLAR, 14 Avenue du Québec, 91140 Villebonne sur Yvette, France
| | - Nathanaël Jarrassé
- ISIR, Sorbonne University, CNRS UMR 7222, ERL INSERM U 1150, 75005 Paris, France
| | - Ross Parry
- LINP2, UPL, UFR STAPS, Université Paris Nanterre, 200 Avenue de la République, 92001 Nanterre, France
- ISIR, Sorbonne University, CNRS UMR 7222, ERL INSERM U 1150, 75005 Paris, France
| |
Collapse
|
5
|
Chojecki P, Strazdas D, Przewozny D, Gard N, Runde D, Hoerner N, Al-Hamadi A, Eisert P, Bosse S. Assessing the Value of Multimodal Interfaces: A Study on Human-Machine Interaction in Weld Inspection Workstations. SENSORS (BASEL, SWITZERLAND) 2023; 23:5043. [PMID: 37299770 PMCID: PMC10255088 DOI: 10.3390/s23115043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Revised: 05/09/2023] [Accepted: 05/15/2023] [Indexed: 06/12/2023]
Abstract
Multimodal user interfaces promise natural and intuitive human-machine interactions. However, is the extra effort for the development of a complex multisensor system justified, or can users also be satisfied with only one input modality? This study investigates interactions in an industrial weld inspection workstation. Three unimodal interfaces, including spatial interaction with buttons augmented on a workpiece or a worktable, and speech commands, were tested individually and in a multimodal combination. Within the unimodal conditions, users preferred the augmented worktable, but overall, the interindividual usage of all input technologies in the multimodal condition was ranked best. Our findings indicate that the implementation and the use of multiple input modalities is valuable and that it is difficult to predict the usability of individual input modalities for complex systems.
Collapse
Affiliation(s)
- Paul Chojecki
- Fraunhofer HHI, 10587 Berlin, Germany; (D.P.); (N.G.); (D.R.); (N.H.); (P.E.); (S.B.)
| | - Dominykas Strazdas
- Neuro-Information Technology, Otto-von-Guericke-University Magdeburg, 39106 Magdeburg, Germany; (D.S.); (A.A.-H.)
| | - David Przewozny
- Fraunhofer HHI, 10587 Berlin, Germany; (D.P.); (N.G.); (D.R.); (N.H.); (P.E.); (S.B.)
| | - Niklas Gard
- Fraunhofer HHI, 10587 Berlin, Germany; (D.P.); (N.G.); (D.R.); (N.H.); (P.E.); (S.B.)
| | - Detlef Runde
- Fraunhofer HHI, 10587 Berlin, Germany; (D.P.); (N.G.); (D.R.); (N.H.); (P.E.); (S.B.)
| | - Niklas Hoerner
- Fraunhofer HHI, 10587 Berlin, Germany; (D.P.); (N.G.); (D.R.); (N.H.); (P.E.); (S.B.)
| | - Ayoub Al-Hamadi
- Neuro-Information Technology, Otto-von-Guericke-University Magdeburg, 39106 Magdeburg, Germany; (D.S.); (A.A.-H.)
| | - Peter Eisert
- Fraunhofer HHI, 10587 Berlin, Germany; (D.P.); (N.G.); (D.R.); (N.H.); (P.E.); (S.B.)
| | - Sebastian Bosse
- Fraunhofer HHI, 10587 Berlin, Germany; (D.P.); (N.G.); (D.R.); (N.H.); (P.E.); (S.B.)
| |
Collapse
|
6
|
Tsolakis N, Gasteratos A. Sensor-Driven Human-Robot Synergy: A Systems Engineering Approach. SENSORS (BASEL, SWITZERLAND) 2022; 23:21. [PMID: 36616620 PMCID: PMC9823401 DOI: 10.3390/s23010021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/23/2022] [Revised: 11/30/2022] [Accepted: 12/18/2022] [Indexed: 06/17/2023]
Abstract
Knowledge-based synergistic automation is a potential intermediate option between the opposite extremes of manual and fully automated robotic labor in agriculture. Disruptive information and communication technologies (ICT) and sophisticated solutions for human-robot interaction (HRI) endow a skilled farmer with enhanced capabilities to perform agricultural tasks more efficiently and productively. This research aspires to apply systems engineering principles to assess the design of a conceptual human-robot synergistic platform enabled by a sensor-driven ICT sub-system. In particular, this paper firstly presents an overview of a use case, including a human-robot synergistic platform comprising a drone, a mobile platform, and wearable equipment. The technology framework constitutes a paradigm of human-centric worker-robot logistics synergy for high-value crops, which is applicable in operational environments of outdoor in-field harvesting and handling operations. Except for the physical sub-system, the ICT sub-system of the robotic framework consists of an extended sensor network for enabling data acquisition to extract the context (e.g., worker's status, environment awareness) and plan and schedule the robotic agents of the framework. Secondly, this research explicitly presents the underpinning Design Structure Matrix (DSM) that systematically captures the interrelations between the sensors in the platform and data/information signals for enabling synergistic operations. The employed Systems Engineering approach provides a comprehensible analysis of the baseline structure existing in the examined human-robot synergy platform. In particular, the applied DSM allows for understanding and synthesizing a sensor sub-system's architecture and enriching its efficacy by informing targeted interventions and reconfiguring the developed robotic solution modules depending on the required farming tasks at an orchard. Human-centric solutions for the agrarian sector demand careful study of the features that the particular agri-field possesses; thus, the insight DSM provides to system designers can turn out to be useful in the investigation of other similar data-driven applications.
Collapse
Affiliation(s)
- Naoum Tsolakis
- Department of Supply Chain Management, International Hellenic University, 570 01 Thessaloniki, Greece
- Institute for Bio-Economy and Agri-Technology (IBO), Centre of Research and Technology-Hellas (CERTH), 6th km Charilaou-Thermi Rd., 570 01 Thessaloniki, Greece
| | - Antonios Gasteratos
- Department of Production and Management Engineering, Democritus University of Thrace, Vas. Sophias 12, 671 32 Xanthi, Greece
| |
Collapse
|