Visual-inertial hand motion tracking with robustness against occlusion, interference, and contact

Stretchable glove for accurate and robust hand pose reconstruction based on comprehensive motion data

We propose a compact wearable glove capable of estimating both the finger bone lengths and the joint angles of the wearer with a simple stretch-based sensing mechanism. The soft sensing glove is designed to easily stretch and to be one-size-fits-all, both measuring the size of the hand and estimating the finger joint motions of the thumb, index, and middle fingers. The system was calibrated and evaluated using comprehensive hand motion data that reflect the extensive range of natural human hand motions and various anatomical structures. The data were collected with a custom motion-capture setup and transformed into the joint angles through our post-processing method. The glove system is capable of reconstructing arbitrary and even unconventional hand poses with accuracy and robustness, confirmed by evaluations on the estimation of bone lengths (mean error: 2.1 mm), joint angles (mean error: 4.16°), and fingertip positions (mean 3D error: 4.02 mm), and on overall hand pose reconstructions in various applications. The proposed glove allows us to take advantage of the dexterity of the human hand with potential applications, including but not limited to teleoperation of anthropomorphic robot hands or surgical robots, virtual and augmented reality, and collection of human motion data.

A Multilayer Perceptron-Based Spherical Visual Compass Using Global Features

This paper presents a visual compass method utilizing global features, specifically spherical moments. One of the primary challenges faced by photometric methods employing global features is the variation in the image caused by the appearance and disappearance of regions within the camera's field of view as it moves. Additionally, modeling the impact of translational motion on the values of global features poses a significant challenge, as it is dependent on scene depths, particularly for non-planar scenes. To address these issues, this paper combines the utilization of image masks to mitigate abrupt changes in global feature values and the application of neural networks to tackle the modeling challenge posed by translational motion. By employing masks at various locations within the image, multiple estimations of rotation corresponding to the motion of each selected region can be obtained. Our contribution lies in offering a rapid method for implementing numerous masks on the image with real-time inference speed, rendering it suitable for embedded robot applications. Extensive experiments have been conducted on both real-world and synthetic datasets generated using Blender. The results obtained validate the accuracy, robustness, and real-time performance of the proposed method compared to a state-of-the-art method.

Biomimetic Wearable Sensors: Emerging Combination of Intelligence and Electronics

Owing to the advancement of interdisciplinary concepts, for example, wearable electronics, bioelectronics, and intelligent sensing, during the microelectronics industrial revolution, nowadays, extensively mature wearable sensing devices have become new favorites in the noninvasive human healthcare industry. The combination of wearable sensing devices with bionics is driving frontier developments in various fields, such as personalized medical monitoring and flexible electronics, due to the superior biocompatibilities and diverse sensing mechanisms. It is noticed that the integration of desired functions into wearable device materials can be realized by grafting biomimetic intelligence. Therefore, herein, the mechanism by which biomimetic materials satisfy and further enhance system functionality is reviewed. Next, wearable artificial sensory systems that integrate biomimetic sensing into portable sensing devices are introduced, which have received significant attention from the industry owing to their novel sensing approaches and portabilities. To address the limitations encountered by important signal and data units in biomimetic wearable sensing systems, two paths forward are identified and current challenges and opportunities are presented in this field. In summary, this review provides a further comprehensive understanding of the development of biomimetic wearable sensing devices from both breadth and depth perspectives, offering valuable guidance for future research and application expansion of these devices.

Machine Learning-Enabled Intelligent Gesture Recognition and Communication System Using Printed Strain Sensors

Gesture contains abundant and complicated information in daily life; as a consequence, gesture recognition attracts a wide range of application prospects and academic values as an important way of achieving human-machine interactions (HMIs). Here, we report an intelligent system consisting of a smart glove made by printed CNT-graphene/PDMS strain sensors. The smart glove shows excellent fitness, comfort, and lightness for human hands. Inspired by machine learning strategies, several objects and gestures can be well classified and implemented by a customized artificial neural network. Several data sets of different sign language gestures and object-grabbing gestures were established, and the result shows that the intelligent system can achieve an average accuracy of 97% and up to 99.4% for a number of gesture groups. Moreover, a robot hand is connected to this system, which is able to react to the motion of human hands with certain gestures where simple sign communication is achieved. These features provide a feasible practical application scheme for gesture recognition in HMIs.

Roles of Low-Dimensional Nanomaterials in Pursuing Human-Machine-Thing Natural Interaction

A wide variety of low-dimensional nanomaterials with excellent properties can meet almost all the requirements of functional materials for information sensing, processing, and feedback devices. Low-dimensional nanomaterials are becoming the star of hope on the road to pursuing human-machine-thing natural interactions, benefiting from the breakthroughs in precise preparation, performance regulation, structural design, and device construction in recent years. This review summarizes several types of low-dimensional nanomaterials commonly used in human-machine-thing natural interactions and outlines the differences in properties and application areas of different materials. According to the sequence of information flow in the human-machine-thing interaction process, the representative research progress of low-dimensional nanomaterials-based information sensing, processing, and feedback devices is reviewed and the key roles played by low-dimensional nanomaterials are discussed. Finally, the development trends and existing challenges of low-dimensional nanomaterials in the field of human-machine-thing natural interaction technology are discussed.

Sensor-based intervention to enhance movement control of the spine in low back pain: Protocol for a quasi-randomized controlled trial

Introduction

Chronic low back pain is a common condition that imposes an enormous burden on individuals and society. Physical exercise with education is the most effective treatment, but generally results in small, albeit significant improvements. However, which type of exercise is most effective remains unknown. Core stability training is often used to improve muscle strength and spinal stability in these patients. The majority of the core stability exercises mentioned in intervention studies involve no spinal movements (static motor control exercises). It is questionable if these exercises would improve controlled movements of the spine. Sensor-based exergames controlled with spinal movements could help improve movement control of the spine. The primary aim of this study is to compare the effects of such sensor-based exergames to static motor control exercises on spinal movement control.

Methods and analysis

In this quasi-randomized controlled trial, 60 patients with chronic low back pain who are already enrolled in a multidisciplinary rehabilitation programme will be recruited. Patients will be randomly allocated into one of two groups: the Sensor-Based Movement Control group (n = 30) or the Static Motor Control group (n = 30). Both groups will receive 8 weeks of two supervised therapy sessions and four home exercises per week in addition to the rehabilitation programme. At baseline (week 1) and after the intervention (week 10), movement control of the spine will be assessed using a tracking task and clinical movement control test battery. Questionnaires on pain, disability, fear avoidance and quality of life will be taken at baseline, after intervention and at 6- and 12 months follow-up. Repeated measures ANOVAs will be used to evaluate if a significant Group x Time interaction effect exists for the movement control evaluations.

Discussion

Sensor-based spinal controlled exergames are a novel way to train spinal movement control using meaningful and engaging feedback. The results of this study will inform clinicians and researchers on the efficacy of movement control training for patients with low back pain.

Ethics and dissemination

Ethical approval for this study protocol was obtained from the METC Brabant (protocol number NL76811.028.21).

Trial registration

Open Science Framework Registries (https://osf.io/v3mw9/), registration number: 10.17605/OSF.IO/V3MW9, registered on 1 September 2021.

Current State of Robotics in Hand Rehabilitation after Stroke: A Systematic Review

Among the methods of hand function rehabilitation after stroke, robot-assisted rehabilitation is widely used, and the use of hand rehabilitation robots can provide functional training of the hand or assist the paralyzed hand with activities of daily living. However, patients with hand disorders consistently report that the needs of some users are not being met. The purpose of this review is to understand the reasons why these user needs are not being adequately addressed, to explore research on hand rehabilitation robots, to review their current state of research in recent years, and to summarize future trends in the hope that it will be useful to researchers in this research area. This review summarizes the techniques in this paper in a systematic way. We first provide a comprehensive review of research institutions, commercial products, and literature. Thus, the state of the art and deficiencies of functional hand rehabilitation robots are sought and guide the development of subsequent hand rehabilitation robots. This review focuses specifically on the actuation and control of hand functional rehabilitation robots, as user needs are primarily focused on actuation and control strategies. We also review hand detection technologies and compare them with patient needs. The results show that the trends in recent years are more inclined to pursue new lightweight materials to improve hand adaptability, investigating intelligent control methods for human-robot interaction in hand functional rehabilitation robots to improve control robustness and accuracy, and VR virtual task positioning to improve the effectiveness of active rehabilitation training.

Textile-Based Flexible Capacitive Pressure Sensors: A Review

Flexible capacitive pressure sensors have been widely used in electronic skin, human movement and health monitoring, and human–machine interactions. Recently, electronic textiles afford a valuable alternative to traditional capacitive pressure sensors due to their merits of flexibility, light weight, air permeability, low cost, and feasibility to fit various surfaces. The textile-based functional layers can serve as electrodes, dielectrics, and substrates, and various devices with semi-textile or all-textile structures have been well developed. This paper provides a comprehensive review of recent developments in textile-based flexible capacitive pressure sensors. The latest research progresses on textile devices with sandwich structures, yarn structures, and in-plane structures are introduced, and the influences of different device structures on performance are discussed. The applications of textile-based sensors in human wearable devices, robotic sensing, and human–machine interaction are then summarized. Finally, evolutionary trends, future directions, and challenges are highlighted.