A wearable motion capture device able to detect dynamic motion of human limbs

Developing excellent plantar pressure sensors for monitoring human motions by using highly compressible and resilient PMMA conductive iongels

Based on real-time detection of plantar pressure, gait recognition could provide important health information for rehabilitation administration, fatigue prevention, and sports training assessment. So far, such researches are extremely limited due to lacking of reliable, stable and comfortable plantar pressure sensors. Herein, a strategy for preparing high compression strength and resilience conductive iongels has been proposed by implanting physically entangled polymer chains with covalently cross-linked networks. The resulting iongels have excellent mechanical properties including nice compliance (young's modulus < 300 kPa), high compression strength (>10 MPa at a strain of 90 %), and good resilience (self-recovery within seconds). And capacitive pressure sensor composed by them possesses excellent sensitivity, good linear response even under very small stress (∼kPa), and long-term durability (cycles > 100,000) under high-stress conditions (133 kPa). Then, capacitive pressure sensor arrays have been prepared for high-precision detection of plantar pressure spatial distribution, which also exhibit excellent sensing performances and long-term stability. Further, an extremely sensitive and fast response plantar pressure monitoring system has been designed for monitoring plantar pressure of foot at different postures including upright, forward and backward. The system achieves real-time tracking and monitoring of changes of plantar pressure during different static and dynamic posture processes. And the characteristics of plantar pressure information can be digitally and photography displayed. Finally, we propose an intelligent framework for real-time detection of plantar pressure by combining electronic insoles with data analysis system, which presents excellent applications in sport trainings and safety precautions.

A data-efficient and easy-to-use lip language interface based on wearable motion capture and speech movement reconstruction

Lip language recognition urgently needs wearable and easy-to-use interfaces for interference-free and high-fidelity lip-reading acquisition and to develop accompanying data-efficient decoder-modeling methods. Existing solutions suffer from unreliable lip reading, are data hungry, and exhibit poor generalization. Here, we propose a wearable lip language decoding technology that enables interference-free and high-fidelity acquisition of lip movements and data-efficient recognition of fluent lip language based on wearable motion capture and continuous lip speech movement reconstruction. The method allows us to artificially generate any wanted continuous speech datasets from a very limited corpus of word samples from users. By using these artificial datasets to train the decoder, we achieve an average accuracy of 92.0% across individuals (n = 7) for actual continuous and fluent lip speech recognition for 93 English sentences, even observing no training burn on users because all training datasets are artificially generated. Our method greatly minimizes users' training/learning load and presents a data-efficient and easy-to-use paradigm for lip language recognition.

AI-Enabled Soft Sensing Array for Simultaneous Detection of Muscle Deformation and Mechanomyography for Metaverse Somatosensory Interaction

Motion recognition (MR)-based somatosensory interaction technology, which interprets user movements as input instructions, presents a natural approach for promoting human-computer interaction, a critical element for advancing metaverse applications. Herein, this work introduces a non-intrusive muscle-sensing wearable device, that in conjunction with machine learning, enables motion-control-based somatosensory interaction with metaverse avatars. To facilitate MR, the proposed device simultaneously detects muscle mechanical activities, including dynamic muscle shape changes and vibrational mechanomyogram signals, utilizing a flexible 16-channel pressure sensor array (weighing ≈0.38 g). Leveraging the rich information from multiple channels, a recognition accuracy of ≈96.06% is achieved by classifying ten lower-limb motions executed by ten human subjects. In addition, this work demonstrates the practical application of muscle-sensing-based somatosensory interaction, using the proposed wearable device, for enabling the real-time control of avatars in a virtual space. This study provides an alternative approach to traditional rigid inertial measurement units and electromyography-based methods for achieving accurate human motion capture, which can further broaden the applications of motion-interactive wearable devices for the coming metaverse age.

Haptic Sensing and Feedback Techniques toward Virtual Reality

Haptic interactions between human and machines are essential for information acquisition and object manipulation. In virtual reality (VR) system, the haptic sensing device can gather information to construct virtual elements, while the haptic feedback part can transfer feedbacks to human with virtual tactile sensation. Therefore, exploring high-performance haptic sensing and feedback interface imparts closed-loop haptic interaction to VR system. This review summarizes state-of-the-art VR-related haptic sensing and feedback techniques based on the hardware parts. For the haptic sensor, we focus on mechanism scope (piezoresistive, capacitive, piezoelectric, and triboelectric) and introduce force sensor, gesture translation, and touch identification in the functional view. In terms of the haptic feedbacks, methodologies including mechanical, electrical, and elastic actuators are surveyed. In addition, the interactive application of virtual control, immersive entertainment, and medical rehabilitation is also summarized. The challenges of virtual haptic interactions are given including the accuracy, durability, and technical conflicts of the sensing devices, bottlenecks of various feedbacks, as well as the closed-loop interaction system. Besides, the prospects are outlined in artificial intelligence of things, wise information technology of medicine, and multimedia VR areas.

Advances and Opportunities of Mobile Health in the Postpandemic Era: Smartphonization of Wearable Devices and Wearable Deviceization of Smartphones

Mobile health (mHealth) with continuous real-time monitoring is leading the era of digital medical convergence. Wearable devices and smartphones optimized as personalized health management platforms enable disease prediction, prevention, diagnosis, and even treatment. Ubiquitous and accessible medical services offered through mHealth strengthen universal health coverage to facilitate service use without discrimination. This viewpoint investigates the latest trends in mHealth technology, which are comprehensive in terms of form factors and detection targets according to body attachment location and type. Insights and breakthroughs from the perspective of mHealth sensing through a new form factor and sensor-integrated display overcome the problems of existing mHealth by proposing a solution of smartphonization of wearable devices and the wearable deviceization of smartphones. This approach maximizes the infinite potential of stagnant mHealth technology and will present a new milestone leading to the popularization of mHealth. In the postpandemic era, innovative mHealth solutions through the smartphonization of wearable devices and the wearable deviceization of smartphones could become the standard for a new paradigm in the field of digital medicine.

Synergizing Machine Learning Algorithm with Triboelectric Nanogenerators for Advanced Self-Powered Sensing Systems

The advancement of the Internet of Things (IoT) has increased the demand for large-scale intelligent sensing systems. The periodic replacement of power sources for ubiquitous sensing systems leads to significant resource waste and environmental pollution. Human staffing costs associated with replacement also increase the economic burden. The triboelectric nanogenerators (TENGs) provide both an energy harvesting scheme and the possibility of self-powered sensing. Based on contact electrification from different materials, TENGs provide a rich material selection to collect complex and diverse data. As the data collected by TENGs become increasingly numerous and complex, different approaches to machine learning (ML) and deep learning (DL) algorithms have been proposed to efficiently process output signals. In this paper, the latest advances in ML algorithms assisting solid-solid TENG and liquid-solid TENG sensors are reviewed based on the sample size and complexity of the data. The pros and cons of various algorithms are analyzed and application scenarios of various TENG sensing systems are presented. The prospects of synergizing hardware (TENG sensors) with software (ML algorithms) in a complex environment and their main challenges for future developments are discussed.

Strong, Tough, and Anti-Swelling Supramolecular Conductive Hydrogels for Amphibious Motion Sensors

Conductive polymer hydrogels (CPHs) are widely employed in emerging flexible electronic devices because they possess both the electrical conductivity of conductors and the mechanical properties of hydrogels. However, the poor compatibility between conductive polymers and the hydrogel matrix, as well as the swelling behavior in humid environments, greatly compromises the mechanical and electrical properties of CPHs, limiting their applications in wearable electronic devices. Herein, a supramolecular strategy to develop a strong and tough CPH with excellent anti-swelling properties by incorporating hydrogen, coordination bonds, and cation-π interactions between a rigid conducting polymer and a soft hydrogel matrix is reported. Benefiting from the effective interactions between the polymer networks, the obtained supramolecular hydrogel has homogeneous structural integrity, exhibiting remarkable tensile strength (1.63 MPa), superior elongation at break (453%), and remarkable toughness (5.5 MJ m-3 ). As a strain sensor, the hydrogel possesses high electrical conductivity (2.16 S m-1 ), a wide strain linear detection range (0-400%), and excellent sensitivity (gauge factor = 4.1), sufficient to monitor human activities with different strain windows. Furthermore, this hydrogel with high swelling resistance has been successfully applied to underwater sensors for monitoring frog swimming and underwater communication. These results reveal new possibilities for amphibious applications of wearable sensors.

Metaverse Wearables for Immersive Digital Healthcare: A Review

The recent exponential growth of metaverse technology has been instrumental in reshaping a myriad of sectors, not least digital healthcare. This comprehensive review critically examines the landscape and future applications of metaverse wearables toward immersive digital healthcare. The key technologies and advancements that have spearheaded the metamorphosis of metaverse wearables are categorized, encapsulating all-encompassed extended reality, such as virtual reality, augmented reality, mixed reality, and other haptic feedback systems. Moreover, the fundamentals of their deployment in assistive healthcare (especially for rehabilitation), medical and nursing education, and remote patient management and treatment are investigated. The potential benefits of integrating metaverse wearables into healthcare paradigms are multifold, encompassing improved patient prognosis, enhanced accessibility to high-quality care, and high standards of practitioner instruction. Nevertheless, these technologies are not without their inherent challenges and untapped opportunities, which span privacy protection, data safeguarding, and innovation in artificial intelligence. In summary, future research trajectories and potential advancements to circumvent these hurdles are also discussed, further augmenting the incorporation of metaverse wearables within healthcare infrastructures in the post-pandemic era.

Sit-to-walk strategy classification in healthy adults using hip and knee joint angles at gait initiation

Forward continuation, balance, and sit-to-stand-and-walk (STSW) are three common movement strategies during sit-to-walk (STW) executions. Literature identifies these strategies through biomechanical parameters using gold standard laboratory equipment, which is expensive, bulky, and requires significant post-processing. STW strategy becomes apparent at gait-initiation (GI) and the hip/knee are primary contributors in STW, therefore, this study proposes to use the hip/knee joint angles at GI as an alternate method of strategy classification. To achieve this, K-means clustering was implemented using three clusters corresponding to the three STW strategies; and two feature sets corresponding to the hip/knee angles (derived from motion capture data); from an open access online database (age: 21-80 years; n = 10). The results identified forward continuation with the lowest hip/knee extension, followed by balance and then STSW, at GI. Using this classification, strategy biomechanics were investigated by deriving the established biomechanical quantities from literature. The biomechanical parameters that significantly varied between strategies (P < 0.05) were time, horizontal centre of mass (COM) momentum, braking impulse, centre of pressure (COP) range and velocities, COP-COM separation, hip/knee torque and movement fluency. This alternate method of strategy classification forms a generalized framework for describing STW executions and is consistent with literature, thus validating the joint angle classification method.

A Fully Self-Powered Wearable Leg Movement Sensing System for Human Health Monitoring

Energy-autonomous wearable human activity monitoring is imperative for daily healthcare, benefiting from long-term sustainable uses. Herein, a fully self-powered wearable system, enabling real-time monitoring and assessments of human multimodal health parameters including knee joint movement, metabolic energy, locomotion speed, and skin temperature, which are fully self-powered by highly-efficient flexible thermoelectric generators (f-TEGs) is proposed and developed. The wearable system is composed of f-TEGs, fabric strain sensors, ultra-low-power edge computing, and Bluetooth. The f-TEGs worn on the leg not only harvest energy from body heat and supply power sustainably for the whole monitoring system, but also serve as zero-power motion sensors to detect limb movement and skin temperature. The fabric strain sensor made by printing PEDOT: PSS on pre-stretched nylon fiber-wrapped rubber band enables high-fidelity and ultralow-power measurements on highly-dynamic knee movements. Edge computing is elaborately designed to estimate multimodal health parameters including time-varying metabolic energy in real-time, which are wirelessly transmitted via Bluetooth. The whole monitoring system is operated automatically and intelligently, works sustainably in both static and dynamic states, and is fully self-powered by the f-TEGs.

Upper limb modeling and motion extraction based on multi-space-fusion

Modeling and motion extraction of human upper limbs are essential for interpreting the natural behavior of upper limb. Owing to the high degrees of freedom (DOF) and highly dynamic nature, existing upper limb modeling methods have limited applications. This study proposes a generic modeling and motion extraction method, named Primitive-Based triangular body segment method (P-BTBS), which follows the physiology of upper limbs, allows high accuracy of motion angles, and describes upper-limb motions with high accuracy. For utilizing the upper-limb modular motion model, the motion angles and bones can be selected as per the research topics (The generic nature of the study targets). Additionally, P-BTBS is suitable in most scenarios for estimating spatial coordinates (The generic nature of equipment and technology). Experiments in continuous motions with seven DOFs and upper-limb motion description validated the excellent performance and robustness of P-BTBS in extracting motion information and describing upper-limb motions, respectively. P-BTBS provides a new perspective and mathematical tool for human understanding and exploration of upper-limb motions, which theoretically supports upper-limb research.

Multi-view emotional expressions dataset using 2D pose estimation

Human body expressions convey emotional shifts and intentions of action and, in some cases, are even more effective than other emotion models. Despite many datasets of body expressions incorporating motion capture available, there is a lack of more widely distributed datasets regarding naturalized body expressions based on the 2D video. In this paper, therefore, we report the multi-view emotional expressions dataset (MEED) using 2D pose estimation. Twenty-two actors presented six emotional (anger, disgust, fear, happiness, sadness, surprise) and neutral body movements from three viewpoints (left, front, right). A total of 4102 videos were captured. The MEED consists of the corresponding pose estimation results (i.e., 397,809 PNG files and 397,809 JSON files). The size of MEED exceeds 150 GB. We believe this dataset will benefit the research in various fields, including affective computing, human-computer interaction, social neuroscience, and psychiatry.

Neural network-based Bluetooth synchronization of multiple wearable devices

Bluetooth-enabled wearables can be linked to form synchronized networks to provide insightful and representative data that is exceptionally beneficial in healthcare applications. However, synchronization can be affected by inevitable variations in the component's performance from their ideal behavior. Here, we report an application-level solution that embeds a Neural network to analyze and overcome these variations. The neural network examines the timing at each wearable node, recognizes time shifts, and fine-tunes a virtual clock to make them operate in unison and thus achieve synchronization. We demonstrate the integration of multiple Kinematics Detectors to provide synchronized motion capture at a high frequency (200 Hz) that could be used for performing spatial and temporal interpolation in movement assessments. The technique presented in this work is general and independent from the physical layer used, and it can be potentially applied to any wireless communication protocol.

Peak Tibiofemoral Contact Forces Estimated Using IMU-Based Approaches Are Not Significantly Different from Motion Capture-Based Estimations in Patients with Knee Osteoarthritis

Altered tibiofemoral contact forces represent a risk factor for osteoarthritis onset and progression, making optimization of the knee force distribution a target of treatment strategies. Musculoskeletal model-based simulations are a state-of-the-art method to estimate joint contact forces, but they typically require laboratory-based input and skilled operators. To overcome these limitations, ambulatory methods, relying on inertial measurement units, have been proposed to estimated ground reaction forces and, consequently, knee contact forces out-of-the-lab. This study proposes the use of a full inertial-capture-based musculoskeletal modelling workflow with an underlying probabilistic principal component analysis model trained on 1787 gait cycles in patients with knee osteoarthritis. As validation, five patients with knee osteoarthritis were instrumented with 17 inertial measurement units and 76 opto-reflective markers. Participants performed multiple overground walking trials while motion and inertial capture methods were synchronously recorded. Moderate to strong correlations were found for the inertial capture-based knee contact forces compared to motion capture with root mean square error between 0.15 and 0.40 of body weight. The results show that our workflow can inform and potentially assist clinical practitioners to monitor knee joint loading in physical therapy sessions and eventually assess long-term therapeutic effects in a clinical context.

Wearables for Running Gait Analysis: A Systematic Review

BACKGROUND

Running gait assessment has traditionally been performed using subjective observation or expensive laboratory-based objective technologies, such as three-dimensional motion capture or force plates. However, recent developments in wearable devices allow for continuous monitoring and analysis of running mechanics in any environment. Objective measurement of running gait is an important (clinical) tool for injury assessment and provides measures that can be used to enhance performance.

OBJECTIVES

We aimed to systematically review the available literature investigating how wearable technology is being used for running gait analysis in adults.

METHODS

A systematic search of the literature was conducted in the following scientific databases: PubMed, Scopus, Web of Science and SPORTDiscus. Information was extracted from each included article regarding the type of study, participants, protocol, wearable device(s), main outcomes/measures, analysis and key findings.

RESULTS

A total of 131 articles were reviewed: 56 investigated the validity of wearable technology, 22 examined the reliability and 77 focused on applied use. Most studies used inertial measurement units (n = 62) [i.e. a combination of accelerometers, gyroscopes and magnetometers in a single unit] or solely accelerometers (n = 40), with one using gyroscopes alone and 31 using pressure sensors. On average, studies used one wearable device to examine running gait. Wearable locations were distributed among the shank, shoe and waist. The mean number of participants was 26 (± 27), with an average age of 28.3 (± 7.0) years. Most studies took place indoors (n = 93), using a treadmill (n = 62), with the main aims seeking to identify running gait outcomes or investigate the effects of injury, fatigue, intrinsic factors (e.g. age, sex, morphology) or footwear on running gait outcomes. Generally, wearables were found to be valid and reliable tools for assessing running gait compared to reference standards.

CONCLUSIONS

This comprehensive review highlighted that most studies that have examined running gait using wearable sensors have done so with young adult recreational runners, using one inertial measurement unit sensor, with participants running on a treadmill and reporting outcomes of ground contact time, stride length, stride frequency and tibial acceleration. Future studies are required to obtain consensus regarding terminology, protocols for testing validity and the reliability of devices and suitability of gait outcomes.

CLINICAL TRIAL REGISTRATION

CRD42021235527.

A Wearable-Sensor System with AI Technology for Real-Time Biomechanical Feedback Training in Hammer Throw

Developing real-time biomechanical feedback systems for in-field applications will transfer human motor skills' learning/training from subjective (experience-based) to objective (science-based). The translation will greatly improve the efficiency of human motor skills' learning and training. Such a translation is especially indispensable for the hammer-throw training which still relies on coaches' experience/observation and has not seen a new world record since 1986. Therefore, we developed a wearable wireless sensor system combining with artificial intelligence for real-time biomechanical feedback training in hammer throw. A framework was devised for developing such practical wearable systems. A printed circuit board was designed to miniaturize the size of the wearable device, where an Arduino microcontroller, an XBee wireless communication module, an embedded load cell and two micro inertial measurement units (IMUs) could be inserted/connected onto the board. The load cell was for measuring the wire tension, while the two IMUs were for determining the vertical displacements of the wrists and the hip. After calibration, the device returned a mean relative error of 0.87% for the load cell and the accuracy of 6% for the IMUs. Further, two deep neural network models were built to estimate selected joint angles of upper and lower limbs related to limb coordination based on the IMUs' measurements. The estimation errors for both models were within an acceptable range, i.e., approximately ±12° and ±4°, respectively, demonstrating strong correlation existed between the limb coordination and the IMUs' measurements. The results of the current study suggest a remarkable novelty: the difficulty-to-measure human motor skills, especially in those sports involving high speed and complex motor skills, can be tracked by wearable sensors with neglect movement constraints to the athletes. Therefore, the application of artificial intelligence in a wearable system has shown great potential of establishing real-time biomechanical feedback training in various sports. To our best knowledge, this is the first practical research of combing wearables and machine learning to provide biomechanical feedback in hammer throw. Hopefully, more wearable biomechanical feedback systems integrating artificial intelligence would be developed in the future.

Machine-Learning Assisted Handwriting Recognition Using Graphene Oxide-Based Hydrogel

Machine-learning assisted handwriting recognition is crucial for development of next-generation biometric technologies. However, most of the currently reported handwriting recognition systems are lacking in flexible sensing and machine learning capabilities, both of which are essential for implementation of intelligent systems. Herein, assisted by machine learning, we develop a new handwriting recognition system, which can be applied as both a recognizer for written texts and an encryptor for confidential information. This flexible and intelligent handwriting recognition system combines a printed circuit board with graphene oxide-based hydrogel sensors. It offers fast response and good sensitivity and allows high-precision recognition of handwritten content from a single letter to words and signatures. By analyzing 690 acquired handwritten signatures obtained from seven participants, we successfully demonstrate a fast recognition time (less than 1 s) and a high recognition rate (∼91.30%). Our developed handwriting recognition system has great potential in advanced human-machine interactions, wearable communication devices, soft robotics manipulators, and augmented virtual reality.

A new 3D, microfluidic-oriented, multi-functional, and highly stretchable soft wearable sensor

Increasing demand for wearable devices has resulted in the development of soft sensors; however, an excellent soft sensor for measuring stretch, twist, and pressure simultaneously has not been proposed yet. This paper presents a novel, fully 3D, microfluidic-oriented, gel-based, and highly stretchable resistive soft sensor. The proposed sensor is multi-functional and could be used to measure stretch, twist, and pressure, which is the potential of using a fully 3D structure in the sensor. Unlike previous methods, in which almost all of them used EGaIn as the conductive material, in this case, we used a low-cost, safe (biocompatible), and ubiquitous conductive gel instead. To show the functionality of the proposed sensor, FEM simulations and a set of designed experiments were done, which show linear (99%), accurate (> 94.9%), and durable (tested for a whole of four hours) response of the proposed sensor. Then, the sensor was put through its paces on a female test subject's knee, elbow, and wrist to show the potential application of the sensor as a body motion sensor. Also, a fully 3D active foot insole was developed, fabricated, and evaluated to evaluate the pressure functionality of the sensor. The result shows good discrimination and pressure measurement for different foot sole areas. The proposed sensor has the potential to be used in real-world applications like rehabilitation, wearable devices, soft robotics, smart clothing, gait analysis, AR/VR, etc.

A Time-Division Position-Sensitive Detector Image System for High-Speed Multitarget Trajectory Tracking

High-speed trajectory tracking with real-time processing capability is particularly important in the fields of pilotless automobiles, guidance systems, robotics, and filmmaking. The conventional optical approach to high-speed trajectory tracking involves charge coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) image sensors, which suffer from trade-offs between resolution and framerates, complexity of the system, and enormous data-analysis processes. Here, a high-speed trajectory tracking system is designed by using a time-division position-sensitive detector (TD-PSD) based on a graphene-silicon Schottky heterojunction. Benefiting from the high-speed optoelectronic response and sub-micrometer positional accuracy of the TD-PSD, multitarget real-time trajectory tracking is realized, with a maximum image output framerate of up to 62 000 frames per second. Moreover, multichannel trajectory tracking and image-distortion correction functionalities are realized by TD-PSD systems through frequency-related image preprocessing, which significantly improves the capacity of real-time information processing and image quality in complicated light environments.

A Systematic Review of Wearable Sensor-Based Technologies for Fall Risk Assessment in Older Adults

Falls have been recognized as the major cause of accidental death and injury in people aged 65 and above. The timely prediction of fall risks can help identify older adults prone to falls and implement preventive interventions. Recent advancements in wearable sensor-based technologies and big data analysis have spurred the development of accurate, affordable, and easy-to-use approaches to fall risk assessment. The objective of this study was to systematically assess the current state of wearable sensor-based technologies for fall risk assessment among community-dwelling older adults. Twenty-five of 614 identified research articles were included in this review. A comprehensive comparison was conducted to evaluate these approaches from several perspectives. In general, these approaches provide an accurate and effective surrogate for fall risk assessment. The accuracy of fall risk prediction can be influenced by various factors such as sensor location, sensor type, features utilized, and data processing and modeling techniques. Features constructed from the raw signals are essential for predictive model development. However, more investigations are needed to identify distinct, clinically interpretable features and develop a general framework for fall risk assessment based on the integration of sensor technologies and data modeling.

Motion Analysis of Football Kick Based on an IMU Sensor

A greater variety of technologies are being applied in sports and health with the advancement of technology, but most optoelectronic systems have strict environmental restrictions and are usually costly. To visualize and perform quantitative analysis on the football kick, we introduce a 3D motion analysis system based on a six-axis inertial measurement unit (IMU) to reconstruct the motion trajectory, in the meantime analyzing the velocity and the highest point of the foot during the backswing. We build a signal processing system in MATLAB and standardize the experimental process, allowing users to reconstruct the foot trajectory and obtain information about the motion within a short time. This paper presents a system that directly analyzes the instep kicking motion rather than recognizing different motions or obtaining biomechanical parameters. For the instep kicking motion of path length around 3.63 m, the root mean square error (RMSE) is about 0.07 m. The RMSE of the foot velocity is 0.034 m/s, which is around 0.45% of the maximum velocity. For the maximum velocity of the foot and the highest point of the backswing, the error is approximately 4% and 2.8%, respectively. With less complex hardware, our experimental results achieve excellent velocity accuracy.

Noncontact Human-Machine Interface Using Complementary Information Fusion Based on MEMS and Triboelectric Sensors

Current noncontact human-machine interfaces (HMIs) either suffer from high power consumption, complex signal processing circuits, and algorithms, or cannot support multidimensional interaction. Here, a minimalist, low-power, and multimodal noncontact interaction interface is realized by fusing the complementary information obtained from a microelectromechanical system (MEMS) humidity sensor and a triboelectric sensor. The humidity sensor composed of a two-port aluminum nitride (AlN) bulk wave resonator operating in its length extensional mode and a layer of graphene oxide (GO) film with uniform and controllable thickness, possesses an ultra-tiny form factor (200 × 400 µm² ), high signal strength (Q = 1729.5), and low signal noise level (±0.31%RH), and is able to continuously and steadily interact with an approaching finger. Meanwhile, the facile triboelectric sensor made of two annular aluminum electrodes enables the interaction interface to rapidly recognize the multidirectional finger movements. By leveraging the resonant frequency changes of the humidity sensor and output voltage waveforms of the triboelectric sensor, the proposed interaction interface is successfully demonstrated as a game control interface to manipulate a car in virtual reality (VR) space and a password input interface to enter high-security 3D passwords, indicating its great potential in diversified applications in the future Metaverse.

A Kinematic Information Acquisition Model That Uses Digital Signals from an Inertial and Magnetic Motion Capture System

This paper presents a model that enables the transformation of digital signals generated by an inertial and magnetic motion capture system into kinematic information. First, the operation and data generated by the used inertial and magnetic system are described. Subsequently, the five stages of the proposed model are described, concluding with its implementation in a virtual environment to display the kinematic information. Finally, the applied tests are presented to evaluate the performance of the model through the execution of four exercises on the upper limb: flexion and extension of the elbow, and pronation and supination of the forearm. The results show a mean squared error of 3.82° in elbow flexion-extension movements and 3.46° in forearm pronation-supination movements. The results were obtained by comparing the inertial and magnetic system versus an optical motion capture system, allowing for the identification of the usability and functionality of the proposed model.

Application Effect of Motion Capture Technology in Basketball Resistance Training and Shooting Hit Rate in Immersive Virtual Reality Environment

With the progress of society, sports have become the mainstream of social development. Strengthening the athletic ability of basketball players can effectively improve their shooting percentage. Firstly, virtual reality (VR) technology and motion capture technology are summarized. Secondly, the resistance training and shooting training of basketball players are analyzed and explained. Finally, the algorithm of motion capture technology is designed to capture and optimize the movements of athletes. In addition, a comprehensive evaluation of the shooting percentage of basketball players is carried out. The results show that the motion capture technology proposed here effectively captures the shooting action of basketball players, and the shooting percentage of players is promoted through resistance training. Among all athletes, the highest shooting percentage improvement is around 14% and the lowest is around 4%. In all groups, athletes of different heights have the largest difference in the improvement of shooting percentage. Therefore, this work plays an important role in improving the shooting rate of basketball players through VR technology. It provides technical support for improving the shooting percentage of basketball players and contributes to the progress of athletes' comprehensive athletic ability.

Self-Powered Gesture Recognition Wristband Enabled by Machine Learning for Full Keyboard and Multicommand Input

Virtual reality is a brand-new technology that can be applied extensively. To realize virtual reality, certain types of human-computer interaction equipment are necessary. Existing virtual reality technologies often rely on cameras, data gloves, game pads, and other equipment. These equipment are either bulky, inconvenient to carry and use, or expensive to popularize. Therefore, the development of a convenient and low-cost high-precision human-computer interaction device can contribute positively to the development of virtual reality technology. In this study, a gesture recognition wristband that can realize a full keyboard and multicommand input is developed. The wristband is convenient to wear, low in cost, and does not affect other daily operations of the hand. This wristband is based on physiological anatomy as well as aided by active sensor and machine learning technology; it can achieve a maximum accuracy of 92.6% in recognizing 26 letters. This wristband offers broad application prospects in the fields of gesture command recognition, assistive devices for the disabled, and wearable electronics.

Progress of Advanced Devices and Internet of Things Systems as Enabling Technologies for Smart Homes and Health Care

In the Internet of Things (IoT) era, various devices (e.g., sensors, actuators, energy harvesters, etc.) and systems have been developed toward the realization of smart homes/buildings and personal health care. These advanced devices can be categorized into ambient devices and wearable devices based on their usage scenarios, to enable motion tracking, health monitoring, daily care, home automation, fall detection, intelligent interaction, assistance, living convenience, and security in smart homes. With the rapidly increasing number of such advanced devices and IoT systems, achieving fully self-sustained and multimodal intelligent systems is becoming more and more important to realize a sustainable and all-in-one smart home platform. Hence, in this Review, we systematically present the recent progress of the development of advanced materials, fabrication techniques, devices, and systems for enabling smart home and health care applications. First, advanced polymer, fiber, and fabric materials as well as their respective fabrication techniques for large-scale manufacturing are discussed. After that, functional devices classified into ambient devices (at home ambiance such as door, floor, table, chair, bed, toilet, window, wall, etc.) and wearable devices (on body parts such as finger, wrist, arm, throat, face, back, etc.) are presented for diverse monitoring and auxiliary applications. Next, the current developments of self-sustained systems and intelligent systems are reviewed in detail, indicating two promising research directions in this field. Last, conclusions and outlook pinpointed on the existing challenges and opportunities are provided for the research community to consider.

Smart-Sleeve: A Wearable Textile Pressure Sensor Array for Human Activity Recognition

Human activity recognition is becoming increasingly important. As contact with oneself and the environment accompanies almost all human activities, a Smart-Sleeve, made of soft and stretchable textile pressure sensor matrix, is proposed to sense human contact with the surroundings and identify performed activities in this work. Additionally, a dataset including 18 activities, performed by 14 subjects in 10 repetitions, is generated. The Smart-Sleeve is evaluated over six classical machine learning classifiers (support vector machine, k-nearest neighbor, logistic regression, random forest, decision tree and naive Bayes) and a convolutional neural network model. For classical machine learning, a new normalization approach is proposed to overcome signal differences caused by different body sizes and statistical, geometric, and symmetry features are used. All classification techniques are compared in terms of classification accuracy, precision, recall, and F-measure. Average accuracies of 82.02% (support vector machine) and 82.30% (convolutional neural network) can be achieved in 10-fold cross-validation, and 72.66% (support vector machine) and 74.84% (convolutional neural network) in leave-one-subject-out validation, which shows that the Smart-Sleeve and the proposed data processing method are suitable for human activity recognition.

Translational Applications of Wearable Sensors in Education: Implementation and Efficacy

Background: Adding new approaches to teaching curriculums can be both expensive and complex to learn. The aim of this research was to gain insight into students’ literacy and confidence in learning sports science with new wearable technologies, specifically a novel program known as STEMfit. Methods: A three-phase design was carried out, with 36 students participating and exposed to wearable devices and associated software. This was to determine whether the technology hardware (phase one) and associated software (phase two) were used in a positive way that demonstrated user confidence. Results: Hardware included choosing a scalable wearable device that worked in conjunction with familiar and readily available software (Microsoft Excel) that extracted data through VBA coding. This allowed for students to experience and provide survey feedback on the usability and confidence gained when interacting with the STEMfit program. Outcomes indicated strong acceptance of the program, with high levels of motivation, resulting in a positive uptake of wearable technology as a teaching tool by students. The initial finding of this study offers an opportunity to further test the STEMfit program on other student cohorts as well as testing the scalability of the system into other year groups at the university level.

Unsupervised IMU-based evaluation of at-home exercise programmes: a feasibility study

Background

The benefits to be obtained from home-based physical therapy programmes are dependent on the proper execution of physiotherapy exercises during unsupervised treatment. Wearable sensors and appropriate movement-related metrics may be used to determine at-home exercise performance and compliance to a physical therapy program.

Methods

A total of thirty healthy volunteers (mean age of 31 years) had their movements captured using wearable inertial measurement units (IMUs), after video recordings of five different exercises with varying levels of complexity were demonstrated to them. Participants were then given wearable sensors to enable a second unsupervised data capture at home. Movement performance between the participants’ recordings was assessed with metrics of movement smoothness, intensity, consistency and control.

Results

In general, subjects executed all exercises similarly when recording at home and as compared with their performance in the lab. However, participants executed all movements faster compared to the physiotherapist’s demonstrations, indicating the need of a wearable system with user feedback that will set the pace of movement.

Conclusion

In light of the Covid-19 pandemic and the imperative transition towards remote consultation and tele-rehabilitation, this work aims to promote new tools and methods for the assessment of adherence to home-based physical therapy programmes. The studied IMU-derived features have shown adequate sensitivity to evaluate home-based programmes in an unsupervised manner. Cost-effective wearables, such as the one presented in this study, can support therapeutic exercises that ought to be performed with appropriate speed, intensity, smoothness and range of motion.

Artificial Intelligence-Enabled Caregiving Walking Stick Powered by Ultra-Low-Frequency Human Motion

The increasing population of the elderly and motion-impaired people brings a huge challenge to our social system. However, the walking stick as their essential tool has rarely been investigated into its potential capabilities beyond basic physical support, such as activity monitoring, tracing, and accident alert. Here, we report a walking stick powered by ultra-low-frequency human motion and equipped with deep-learning-enabled advanced sensing features to provide a healthcare-monitoring platform for motion-impaired users. A linear-to-rotary structure is designed to achieve highly efficient energy harvesting from the linear motion of a walking stick with ultralow frequency. Besides, two kinds of self-powered triboelectric sensors are proposed and integrated to extract the motion features of the walking stick. Augmented sensing functionalities with high accuracies have been enabled by deep-learning-based data analysis, including identity recognition, disability evaluation, and motion status distinguishing. Furthermore, a self-sustainable Internet of Things (IoT) system with global positioning system tracing and environmental temperature and humidity amenity sensing functions is obtained. Combined with the aforementioned functionalities, this walking stick is demonstrated in various usage scenarios as a caregiver for real-time well-being status and activity monitoring. The caregiving walking stick shows the potential of being an intelligent aid for motion-impaired users to help them live life with adequate autonomy and safety.

Graded Microstructured Flexible Pressure Sensors with High Sensitivity and an Ultrabroad Pressure Range for Epidermal Pulse Monitoring

Precisely detecting epidermal pulse waves with pressure sensors is crucial for pulse-based personalized health-monitoring technologies. However, developing a pressure sensor that simultaneously demonstrates high sensitivity and an ultrabroad pressure range and a convenient fabrication process for large-scale production is a considerable challenge. Herein, by utilizing a commercial conductive fabric (CF) and a silica gel film, we develop a high-performance pressure sensor (HPPS) for the monitoring of human physiological signals. Based on convenient turnover formwork technology, the silica gel film was fabricated by replicating the microstructure of the sandpaper surface. This microstructure and the plain weave structure on the CF surface together provide a sharp increase in the contact-separation area and structural compressibility, which are beneficial for the enhancement of output performance. Made of these two materials, the graded microstructured HPPS holds high sensitivity (4.5 mV/Pa), an ultrabroad pressure range (0-30 kPa), a wide working frequency bandwidth (up to 35 Hz), decent stability (>50,000 cycles), and a simple fabrication process that is suitable for large-scale production. Given these noticeable features, the developed HPPS not only succeeds in precisely detecting subtle pulse waves on various positions of different people but can also objectively capture changes in cardiovascular parameters caused by exercise training at different intensities in real time. These findings exhibit the enormous potential application of HPPS in tracking an individual's health status and comprehensively evaluating exercise intensity.

Wearable Assistive Robotics: A Perspective on Current Challenges and Future Trends

Wearable assistive robotics is an emerging technology with the potential to assist humans with sensorimotor impairments to perform daily activities. This assistance enables individuals to be physically and socially active, perform activities independently, and recover quality of life. These benefits to society have motivated the study of several robotic approaches, developing systems ranging from rigid to soft robots with single and multimodal sensing, heuristics and machine learning methods, and from manual to autonomous control for assistance of the upper and lower limbs. This type of wearable robotic technology, being in direct contact and interaction with the body, needs to comply with a variety of requirements to make the system and assistance efficient, safe and usable on a daily basis by the individual. This paper presents a brief review of the progress achieved in recent years, the current challenges and trends for the design and deployment of wearable assistive robotics including the clinical and user need, material and sensing technology, machine learning methods for perception and control, adaptability and acceptability, datasets and standards, and translation from lab to the real world.

A Motion Capturing and Energy Harvesting Hybridized Lower-Limb System for Rehabilitation and Sports Applications

Lower-limb motion monitoring is highly desired in various application scenarios ranging from rehabilitation to sports training. However, there still lacks a cost-effective, energy-saving, and computational complexity-reducing solution for this specific demand. Here, a motion capturing and energy harvesting hybridized lower-limb (MC-EH-HL) system with 3D printing is demonstrated. It enables low-frequency biomechanical energy harvesting with a sliding block-rail piezoelectric generator (S-PEG) and lower-limb motion sensing with a ratchet-based triboelectric nanogenerator (R-TENG). A unique S-PEG is proposed with particularly designed mechanical structures to convert lower-limb 3D motion into 1D linear sliding on the rail. On the one hand, high output power is achieved with the S-PEG working at a very low frequency, which realizes self-sustainable systems for wireless sensing under the Internet of Things framework. On the other hand, the R-TENG gives rise to digitalized triboelectric output, matching the rotation angles to the pulse numbers. Additional physical parameters can be estimated to enrich the sensory dimension. Accordingly, demonstrative rehabilitation, human-machine interfacing in virtual reality, and sports monitoring are presented. This developed hybridized system exhibits an economic and energy-efficient solution to support the need for lower-limb motion tracking in various scenarios, paving the way for self-sustainable multidimensional motion tracking systems in near future.

Vibrotactile-Based Operational Guidance System for Space Science Experiments

On-orbit astronauts and scientists on the ground need to cooperate closely, to complete space science experiments efficiently. However, for the increasingly diverse space science experiments, scientists are unable to train astronauts on the ground about the details of each experiment. The traditional interaction of visual and auditory channels is not enough for scientists to directly guide astronauts to experimentalize. An intuitive and transparent interaction interface between scientists and astronauts has to be built to meet the requirements of space science experiments. Therefore, this paper proposed a vibrotactile guidance system for cooperation between scientists and astronauts. We utilized Kinect V2 sensors to track the movements of the participants of space science experiments, process data in the virtual experimental environment developed by Unity 3D, and provide astronauts with different guidance instructions using the wearable vibrotactile device. Compared with other schemes using only visual and auditory channels, our approach provides more direct and more efficient guidance information that astronauts perceive is what they need to perform different tasks. Three virtual space science experiment tasks verified the feasibility of the vibrotactile operational guidance system. Participants were able to complete the experimental task with a short period of training, and the experimental results show that the method has an application prospect.

A Wearable IMU System for Flexible Teleoperation of a Collaborative Industrial Robot

Increasing the accessibility of collaborative robotics requires interfaces that support intuitive teleoperation. One possibility for an intuitive interface is offered by wearable systems that measure the operator’s movement and use the information for robot control. Such wearable systems should preserve the operator’s movement capabilities and, thus, their ability to flexibly operate in the workspace. This paper presents a novel wireless wearable system that uses only inertial measurement units (IMUs) to determine the orientation of the operator’s upper body parts. An algorithm was developed to transform the measured orientations to movement commands for an industrial collaborative robot. The algorithm includes a calibration procedure, which aligns the coordinate systems of all IMUs, the operator, and the robot, and the transformation of the operator’s relative hand motions to the movement of the robot’s end effector, which takes into account the operator’s orientation relative to the robot. The developed system is demonstrated with an example of an industrial application in which a workpiece needs to be inserted into a fixture. The robot’s motion is compared between the developed system and a standard robot controller. The results confirm that the developed system is intuitive, allows for flexible control, and is robust enough for use in industrial collaborative robotic applications.

Mechanical Parameters and Trajectory of Two Chinese Cervical Manipulations Compared by a Motion Capture System

Objective: To compare the mechanical parameters and trajectory while operating the oblique pulling manipulation and the cervical rotation–traction manipulation.

Methods: An experimental research measuring kinematics parameter and recording motion trajectories of two cervical manipulations were carried out. A total of 48 healthy volunteers participated in this study, who were randomly divided into two groups of 24 representing each of the two manipulations. A clinician performed two manipulations in two groups separately. A motion capture system was used to monitor and analyze kinematics parameters during the operation.

Results: The two cervical manipulations have similar thrust time, displacement, mean velocity, max velocity, and max acceleration. There were no significant differences in active and passive amplitudes between the two cervical rotation manipulations. The thrust amplitudes of the oblique pulling manipulation and the cervical rotation–traction manipulation were 5.735 ± 3.041° and 2.142 ± 1.742°, respectively. The thrust amplitudes of the oblique pulling manipulation was significantly greater than that of the cervical rotation–traction manipulation (P < 0.001).

Conclusion: Compared with the oblique pulling manipulation, the cervical rotation–traction manipulation has a less thrust amplitudes.

Estimation of Human Center of Mass Position through the Inertial Sensors-Based Methods in Postural Tasks: An Accuracy Evaluation

The estimation of the body’s center of mass (CoM) trajectory is typically obtained using force platforms, or optoelectronic systems (OS), bounding the assessment inside a laboratory setting. The use of magneto-inertial measurement units (MIMUs) allows for more ecological evaluations, and previous studies proposed methods based on either a single sensor or a sensors’ network. In this study, we compared the accuracy of two methods based on MIMUs. Body CoM was estimated during six postural tasks performed by 15 healthy subjects, using data collected by a single sensor on the pelvis (Strapdown Integration Method, SDI), and seven sensors on the pelvis and lower limbs (Biomechanical Model, BM). The accuracy of the two methods was compared in terms of RMSE and estimation of posturographic parameters, using an OS as reference. The RMSE of the SDI was lower in tasks with little or no oscillations, while the BM outperformed in tasks with greater CoM displacement. Moreover, higher correlation coefficients were obtained between the posturographic parameters obtained with the BM and the OS. Our findings showed that the estimation of CoM displacement based on MIMU was reasonably accurate, and the use of the inertial sensors network methods should be preferred to estimate the kinematic parameters.