Object recognition in performed basic daily activities with a handcrafted data glove prototype

Glove-Net: Enhancing Grasp Classification with Multisensory Data and Deep Learning Approach

Grasp classification is pivotal for understanding human interactions with objects, with wide-ranging applications in robotics, prosthetics, and rehabilitation. This study introduces a novel methodology utilizing a multisensory data glove to capture intricate grasp dynamics, including finger posture bending angles and fingertip forces. Our dataset comprises data collected from 10 participants engaging in grasp trials with 24 objects using the YCB object set. We evaluate classification performance under three scenarios: utilizing grasp posture alone, utilizing grasp force alone, and combining both modalities. We propose Glove-Net, a hybrid CNN-BiLSTM architecture for classifying grasp patterns within our dataset, aiming to harness the unique advantages offered by both CNNs and BiLSTM networks. This model seamlessly integrates CNNs' spatial feature extraction capabilities with the temporal sequence learning strengths inherent in BiLSTM networks, effectively addressing the intricate dependencies present within our grasping data. Our study includes findings from an extensive ablation study aimed at optimizing model configurations and hyperparameters. We quantify and compare the classification accuracy across these scenarios: CNN achieved 88.09%, 69.38%, and 93.51% testing accuracies for posture-only, force-only, and combined data, respectively. LSTM exhibited accuracies of 86.02%, 70.52%, and 92.19% for the same scenarios. Notably, the hybrid CNN-BiLSTM proposed model demonstrated superior performance with accuracies of 90.83%, 73.12%, and 98.75% across the respective scenarios. Through rigorous numerical experimentation, our results underscore the significance of multimodal grasp classification and highlight the efficacy of the proposed hybrid Glove-Net architectures in leveraging multisensory data for precise grasp recognition. These insights advance understanding of human-machine interaction and hold promise for diverse real-world applications.

Manipulator Control System Based on Flexible Sensor Technology

The research on the remote control of manipulators based on flexible sensor technology is gradually extensive. In order to achieve stable, accurate, and efficient control of the manipulator, it is necessary to reasonably design the structure of the sensor with excellent tensile strength and flexibility. The acquisition of manual information by high-performance sensors is the basis of manipulator control. This paper starts with the manufacturing of materials of the flexible sensor for the manipulator, introduces the substrate, sensor, and flexible electrode materials, respectively, and summarizes the performance of different flexible sensors. From the perspective of manufacturing, it introduces their basic principles and compares their advantages and disadvantages. Then, according to the different ways of wearing, the two control methods of data glove control and surface EMG control are respectively introduced, the principle, control process, and detection accuracy are summarized, and the problems of material microstructure, reducing the cost, optimizing the circuit design and so on are emphasized in this field. Finally, the commercial application in this field is explained and the future research direction is proposed from two aspects: how to ensure real-time control and better receive the feedback signal from the manipulator.

Master-Slave Control System for Virtual-Physical Interactions Using Hands

Among the existing technologies for hand protection, master-slave control technology has been extensively researched and applied within the field of safety engineering to mitigate the occurrence of safety incidents. However, it has been identified through research that traditional master-slave control technologies no longer meet current production and lifestyle needs, and they have even begun to pose new safety risks. To resolve the safety risks exposed by traditional master-slave control, this research fuses master-slave control technology for hands with virtual reality technology, and the design of a master-slave control system for hands based on virtual reality technology is investigated. This study aims to realize the design of a master-slave control system for virtual-physical interactions using hands that captures the position, orientation, and finger joint angles of the user's hand in real time and synchronizes the motion of the slave interactive device with that of a virtual hand. With amplitude limiting, jitter elimination, and a complementary filtering algorithm, the original motion data collected by the designed glove are turned into a Kalman-filtering-algorithm-based driving database, which drives the synchronous interaction of the virtual hand and a mechanical hand. As for the experimental results, the output data for the roll, pitch, and yaw were in the stable ranges of -0.1° to 0.1°, -0.15° to 0.15°, and -0.15° to 0.15°, respectively, which met the accuracy requirements for the system's operation under different conditions. More importantly, these data prove that, in terms of accuracy and denoising, the data-processing algorithm was relatively compatible with the hardware platform of the system. Based on the algorithm for the virtual-physical interaction model, the authors introduced the concept of an auxiliary hand into the research, put forward an algorithmic process and a judgement condition for the stable grasp of the virtual hand's, and solved a model-penetrating problem while enhancing the immersive experience during virtual-physical interactions. In an interactive experiment, a dynamic accuracy test was run on the system. As shown by the experimental data and the interactive effect, the system was satisfactorily stable and interactive.

Textronic Glove Translating Polish Sign Language

Communication between people is a basic social skill used to exchange information. It is often used for self-express and to meet basic human needs, such as the need for closeness, belonging, and security. This process takes place at different levels, using different means, with specific effects. It generally means a two-way flow of information in the immediate area of contact with another person. When people are communicating using the same language, the flow of information is much easier compared to the situation when two people use two different languages from different language families. The process of social communication with the deaf is difficult as well. It is therefore essential to use modern technologies to facilitate communication with deaf and non-speaking people. This article presents the results of work on a prototype of a glove using textronic elements produced using a physical vacuum deposition process. The signal from the sensors, in the form of resistance changes, is read by the microcontroller, and then it is processed and displayed on a smartphone screen in the form of single letters. During the experiment, 520 letters were signed by each author. The correctness of interpreting the signs was 86.5%. Each letter was recognized within approximately 3 s. One of the main results of the article was also the selection of an appropriate material (Velostat, membrane) that can be used as a sensor for the proposed application solution. The proposed solution can enable communication with the deaf using the finger alphabet, which can be used to spell single words or the most important key words.

A Noble Classification Framework for Data Glove Classification of a Large Number of Hand Movements

The recognition of hand movements is an important method for human-computer interaction (HCI) technology, and it is widely used in virtual reality and other HCI areas. While many valuable efforts have been made, efficient ways to capture over 20 types of hand movements with high accuracy by one data glove are still lacking. This paper addresses a new classification framework for 52 hand movements. This classification framework includes the following two parts: the movement detection algorithm and the movement classification algorithm. The fine K-nearest neighbor (Fine KNN) is the core of the movement detection algorithm. The movement classification algorithm is composed of downsampling in data preparation and a new deep learning network named the DBDF network. Bidirectional Long Short-Term Memory (BiLSTM) is the main part of the DBDF network. The results of experiments using the Ninapro DB1 dataset demonstrate that our work can classify more types of hand movements than related algorithms with a precision of 93.15%.