1
|
Zhang Y, Doyle T. Integrating intention-based systems in human-robot interaction: a scoping review of sensors, algorithms, and trust. Front Robot AI 2023; 10:1233328. [PMID: 37876910 PMCID: PMC10591094 DOI: 10.3389/frobt.2023.1233328] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Accepted: 09/18/2023] [Indexed: 10/26/2023] Open
Abstract
The increasing adoption of robot systems in industrial settings and teaming with humans have led to a growing interest in human-robot interaction (HRI) research. While many robots use sensors to avoid harming humans, they cannot elaborate on human actions or intentions, making them passive reactors rather than interactive collaborators. Intention-based systems can determine human motives and predict future movements, but their closer interaction with humans raises concerns about trust. This scoping review provides an overview of sensors, algorithms, and examines the trust aspect of intention-based systems in HRI scenarios. We searched MEDLINE, Embase, and IEEE Xplore databases to identify studies related to the forementioned topics of intention-based systems in HRI. Results from each study were summarized and categorized according to different intention types, representing various designs. The literature shows a range of sensors and algorithms used to identify intentions, each with their own advantages and disadvantages in different scenarios. However, trust of intention-based systems is not well studied. Although some research in AI and robotics can be applied to intention-based systems, their unique characteristics warrant further study to maximize collaboration performance. This review highlights the need for more research on the trust aspects of intention-based systems to better understand and optimize their role in human-robot interactions, at the same time establishes a foundation for future research in sensor and algorithm designs for intention-based systems.
Collapse
Affiliation(s)
- Yifei Zhang
- Department of Electrical and Computer Engineering, McMaster University, Hamilton, ON, Canada
| | - Thomas Doyle
- Department of Electrical and Computer Engineering, McMaster University, Hamilton, ON, Canada
- School of Biomedical Engineering, McMaster University, Hamilton, ON, Canada
- Vector Institute for Artificial Intelligence, Toronto, ON, Canada
| |
Collapse
|
2
|
Oh HW, Hong YD. Divergent Component of Motion-Based Gait Intention Detection Method Using Motion Information From Single Leg. J INTELL ROBOT SYST 2023. [DOI: 10.1007/s10846-023-01843-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/30/2023]
|
3
|
Zhang Z, Wang Z, Lei H, Gu W. Gait phase recognition of lower limb exoskeleton system based on the integrated network model. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103693] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
4
|
The Middleware for an Exoskeleton Assisting Upper Limb Movement. SENSORS 2022; 22:s22082986. [PMID: 35458977 PMCID: PMC9032928 DOI: 10.3390/s22082986] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/17/2022] [Revised: 03/31/2022] [Accepted: 04/12/2022] [Indexed: 12/01/2022]
Abstract
This article presents the possibilities of newly developed middleware dedicated for distributed and modular control systems. The software enables the exchange of information locally, within one control module, and globally, between many modules. The executed information exchange system speed tests confirmed the correct operation of the software. The middleware was used in the control system of the active upper-limb exoskeleton. The upper-limb rehabilitation exoskeleton structure with six degrees of mechanical freedom is presented. The tests were performed using the prototype with three joints. The drives’ models of individual joints were developed and simulated. As a result, the courses of the motion trajectory were shown for different kinds of pressure on the force sensors, and different methods of signal filtering. The tests confirmed a correct operation of middleware and drives control system.
Collapse
|
5
|
Adaptive Adjustment Strategy for Walking Characteristics of Single-Legged Exoskeleton Robots. MACHINES 2022. [DOI: 10.3390/machines10020134] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
In order to achieve the normal walking of hemiplegic patients, this paper proposes a single-legged exoskeleton robot according to the bionics principle, and presents an adaptive adjustment strategy for walking characteristics. The least square regression analysis is used to fit the angle data of healthy leg joints by cubic polynomials, and then the parametric design of the fitted curve is carried out to obtain the influence of the user’s stride frequency and stride length on the joint angle, so that the gait of the exoskeleton can be adjusted in real time according to the stride length and stride frequency of the healthy leg to realize normal walking. In order to verify the effectiveness of the adaptive adjustment strategy proposed in this paper, the angle of leg joints under normal gait is collected in advance. In addition, an adult male is chosen as the subject to walk on the horizontal ground wearing the single-legged exoskeleton as the experiment. The experimental results show that the designed exoskeleton is reasonable, and the adaptive adjustment strategy proposed in this paper can make the exoskeleton adapt well and follow the gait of healthy legs to achieve a more natural walking state.
Collapse
|
6
|
Tiboni M, Borboni A, Vérité F, Bregoli C, Amici C. Sensors and Actuation Technologies in Exoskeletons: A Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:884. [PMID: 35161629 PMCID: PMC8839165 DOI: 10.3390/s22030884] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/31/2021] [Revised: 01/16/2022] [Accepted: 01/19/2022] [Indexed: 02/06/2023]
Abstract
Exoskeletons are robots that closely interact with humans and that are increasingly used for different purposes, such as rehabilitation, assistance in the activities of daily living (ADLs), performance augmentation or as haptic devices. In the last few decades, the research activity on these robots has grown exponentially, and sensors and actuation technologies are two fundamental research themes for their development. In this review, an in-depth study of the works related to exoskeletons and specifically to these two main aspects is carried out. A preliminary phase investigates the temporal distribution of scientific publications to capture the interest in studying and developing novel ideas, methods or solutions for exoskeleton design, actuation and sensors. The distribution of the works is also analyzed with respect to the device purpose, body part to which the device is dedicated, operation mode and design methods. Subsequently, actuation and sensing solutions for the exoskeletons described by the studies in literature are analyzed in detail, highlighting the main trends in their development and spread. The results are presented with a schematic approach, and cross analyses among taxonomies are also proposed to emphasize emerging peculiarities.
Collapse
Affiliation(s)
- Monica Tiboni
- Department of Mechanical and Industrial Engineering, University of Brescia, Via Branze, 38, 25123 Brescia, Italy; (M.T.); (C.A.)
| | - Alberto Borboni
- Department of Mechanical and Industrial Engineering, University of Brescia, Via Branze, 38, 25123 Brescia, Italy; (M.T.); (C.A.)
| | - Fabien Vérité
- Agathe Group INSERM U 1150, UMR 7222 CNRS, ISIR (Institute of Intelligent Systems and Robotics), Sorbonne Université, 75005 Paris, France;
| | - Chiara Bregoli
- Institute of Condensed Matter Chemistry and Technologies for Energy (ICMATE), National Research Council (CNR), Via Previati 1/E, 23900 Lecco, Italy;
| | - Cinzia Amici
- Department of Mechanical and Industrial Engineering, University of Brescia, Via Branze, 38, 25123 Brescia, Italy; (M.T.); (C.A.)
| |
Collapse
|
7
|
Zaroug A, Garofolini A, Lai DTH, Mudie K, Begg R. Prediction of gait trajectories based on the Long Short Term Memory neural networks. PLoS One 2021; 16:e0255597. [PMID: 34351994 PMCID: PMC8341582 DOI: 10.1371/journal.pone.0255597] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2021] [Accepted: 07/20/2021] [Indexed: 11/19/2022] Open
Abstract
The forecasting of lower limb trajectories can improve the operation of assistive devices and minimise the risk of tripping and balance loss. The aim of this work was to examine four Long Short Term Memory (LSTM) neural network architectures (Vanilla, Stacked, Bidirectional and Autoencoders) in predicting the future trajectories of lower limb kinematics, i.e. Angular Velocity (AV) and Linear Acceleration (LA). Kinematics data of foot, shank and thigh (LA and AV) were collected from 13 male and 3 female participants (28 ± 4 years old, 1.72 ± 0.07 m in height, 66 ± 10 kg in mass) who walked for 10 minutes at preferred walking speed (4.34 ± 0.43 km.h-1) and at an imposed speed (5km.h-1, 15.4% ± 7.6% faster) on a 0% gradient treadmill. The sliding window technique was adopted for training and testing the LSTM models with total kinematics time-series data of 10,500 strides. Results based on leave-one-out cross validation, suggested that the LSTM autoencoders is the top predictor of the lower limb kinematics trajectories (i.e. up to 0.1s). The normalised mean squared error was evaluated on trajectory predictions at each time-step and it obtained 2.82-5.31% for the LSTM autoencoders. The ability to predict future lower limb motions may have a wide range of applications including the design and control of bionics allowing improved human-machine interface and mitigating the risk of falls and balance loss.
Collapse
Affiliation(s)
- Abdelrahman Zaroug
- Institute for Health and Sport, Victoria University, Melbourne, Victoria, Australia
| | | | - Daniel T. H. Lai
- Institute for Health and Sport, Victoria University, Melbourne, Victoria, Australia
- College of Engineering and Science, Victoria University, Melbourne, Victoria, Australia
| | - Kurt Mudie
- Defence Science and Technology Group, Melbourne, Victoria, Australia
| | - Rezaul Begg
- Institute for Health and Sport, Victoria University, Melbourne, Victoria, Australia
| |
Collapse
|
8
|
Zhou YM, Hohimer C, Proietti T, O'Neill CT, Walsh CJ. Kinematics-Based Control of an Inflatable Soft Wearable Robot for Assisting the Shoulder of Industrial Workers. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3061365] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
9
|
Labarrière F, Thomas E, Calistri L, Optasanu V, Gueugnon M, Ornetti P, Laroche D. Machine Learning Approaches for Activity Recognition and/or Activity Prediction in Locomotion Assistive Devices-A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2020; 20:E6345. [PMID: 33172158 PMCID: PMC7664393 DOI: 10.3390/s20216345] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/25/2020] [Revised: 10/22/2020] [Accepted: 11/04/2020] [Indexed: 01/16/2023]
Abstract
Locomotion assistive devices equipped with a microprocessor can potentially automatically adapt their behavior when the user is transitioning from one locomotion mode to another. Many developments in the field have come from machine learning driven controllers on locomotion assistive devices that recognize/predict the current locomotion mode or the upcoming one. This review synthesizes the machine learning algorithms designed to recognize or to predict a locomotion mode in order to automatically adapt the behavior of a locomotion assistive device. A systematic review was conducted on the Web of Science and MEDLINE databases (as well as in the retrieved papers) to identify articles published between 1 January 2000 to 31 July 2020. This systematic review is reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines and is registered on Prospero (CRD42020149352). Study characteristics, sensors and algorithms used, accuracy and robustness were also summarized. In total, 1343 records were identified and 58 studies were included in this review. The experimental condition which was most often investigated was level ground walking along with stair and ramp ascent/descent activities. The machine learning algorithms implemented in the included studies reached global mean accuracies of around 90%. However, the robustness of those algorithms seems to be more broadly evaluated, notably, in everyday life. We also propose some guidelines for homogenizing future reports.
Collapse
Affiliation(s)
- Floriant Labarrière
- INSERM, UMR1093-CAPS, Université de Bourgogne Franche Comté, UFR des Sciences du Sport, F-21000 Dijon, France; (F.L.); (E.T.); (P.O.)
| | - Elizabeth Thomas
- INSERM, UMR1093-CAPS, Université de Bourgogne Franche Comté, UFR des Sciences du Sport, F-21000 Dijon, France; (F.L.); (E.T.); (P.O.)
| | - Laurine Calistri
- PROTEOR, 6 rue de la Redoute, CS 37833, CEDEX 21078 Dijon, France;
| | - Virgil Optasanu
- ICB, UMR 6303 CNRS, Université de Bourgogne Franche Comté 9 Av. Alain Savary, CEDEX 21078 Dijon, France;
| | - Mathieu Gueugnon
- INSERM, CIC 1432, Module Plurithematique, Plateforme d’Investigation Technologique, CHU Dijon-Bourgogne, Centre d’Investigation Clinique, Module Plurithématique, Plateforme d’Investigation Technologique, 21079 Dijon, France;
| | - Paul Ornetti
- INSERM, UMR1093-CAPS, Université de Bourgogne Franche Comté, UFR des Sciences du Sport, F-21000 Dijon, France; (F.L.); (E.T.); (P.O.)
- INSERM, CIC 1432, Module Plurithematique, Plateforme d’Investigation Technologique, CHU Dijon-Bourgogne, Centre d’Investigation Clinique, Module Plurithématique, Plateforme d’Investigation Technologique, 21079 Dijon, France;
- Department of Rheumatology, Dijon University Hospital, 21079 Dijon, France
| | - Davy Laroche
- INSERM, UMR1093-CAPS, Université de Bourgogne Franche Comté, UFR des Sciences du Sport, F-21000 Dijon, France; (F.L.); (E.T.); (P.O.)
- INSERM, CIC 1432, Module Plurithematique, Plateforme d’Investigation Technologique, CHU Dijon-Bourgogne, Centre d’Investigation Clinique, Module Plurithématique, Plateforme d’Investigation Technologique, 21079 Dijon, France;
| |
Collapse
|
10
|
Zaroug A, Lai DTH, Mudie K, Begg R. Lower Limb Kinematics Trajectory Prediction Using Long Short-Term Memory Neural Networks. Front Bioeng Biotechnol 2020; 8:362. [PMID: 32457881 PMCID: PMC7227385 DOI: 10.3389/fbioe.2020.00362] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2020] [Accepted: 03/31/2020] [Indexed: 12/03/2022] Open
Abstract
This study determined whether the kinematics of lower limb trajectories during walking could be extrapolated using long short-term memory (LSTM) neural networks. It was hypothesised that LSTM auto encoders could reliably forecast multiple time-step trajectories of the lower limb kinematics, specifically linear acceleration (LA) and angular velocity (AV). Using 3D motion capture, lower limb position-time coordinates were sampled (100 Hz) from six male participants (age 22 ± 2 years, height 1.77 ± 0.02 m, body mass 82 ± 4 kg) who walked for 10 min at 5 km/h on a 0% gradient motor-driven treadmill. These data were fed into an LSTM model with a sliding window of four kinematic variables with 25 samples or time steps: LA and AV for thigh and shank. The LSTM was tested to forecast five samples (i.e., time steps) of the four kinematic input variables. To attain generalisation, the model was trained on a dataset of 2,665 strides from five participants and evaluated on a test set of 1 stride from a sixth participant. The LSTM model learned the lower limb kinematic trajectories using the training samples and tested for generalisation across participants. The forecasting horizon suggested higher model reliability in predicting earlier future trajectories. The mean absolute error (MAE) was evaluated on each variable across the single tested stride, and for the five-sample forecast, it obtained 0.047 m/s2 thigh LA, 0.047 m/s2 shank LA, 0.028 deg/s thigh AV and 0.024 deg/s shank AV. All predicted trajectories were highly correlated with the measured trajectories, with correlation coefficients greater than 0.98. The motion prediction model may have a wide range of applications, such as mitigating the risk of falls or balance loss and improving the human-machine interface for wearable assistive devices.
Collapse
Affiliation(s)
- Abdelrahman Zaroug
- Institute for Health and Sport, Victoria University, Melbourne, VIC, Australia
| | - Daniel T. H. Lai
- Institute for Health and Sport, Victoria University, Melbourne, VIC, Australia
- College of Engineering and Science, Victoria University, Melbourne, VIC, Australia
| | - Kurt Mudie
- Defence Science and Technology Group, Melbourne, VIC, Australia
| | - Rezaul Begg
- Institute for Health and Sport, Victoria University, Melbourne, VIC, Australia
| |
Collapse
|