A Mouth and Tongue Interactive Device to Control Wearable Robotic Limbs in Tasks where Human Limbs Are Occupied

The Wearable Robotic Limb (WRL) is a type of robotic arm worn on the human body, aiming to enhance the wearer's operational capabilities. However, proposing additional methods to control and perceive the WRL when human limbs are heavily occupied with primary tasks presents a challenge. Existing interactive methods, such as voice, gaze, and electromyography (EMG), have limitations in control precision and convenience. To address this, we have developed an interactive device that utilizes the mouth and tongue. This device is lightweight and compact, allowing wearers to achieve continuous motion and contact force control of the WRL. By using a tongue controller and mouth gas pressure sensor, wearers can control the WRL while also receiving sensitive contact feedback through changes in mouth pressure. To facilitate bidirectional interaction between the wearer and the WRL, we have devised an algorithm that divides WRL control into motion and force-position hybrid modes. In order to evaluate the performance of the device, we conducted an experiment with ten participants tasked with completing a pin-hole assembly task with the assistance of the WRL system. The results show that the device enables continuous control of the position and contact force of the WRL, with users perceiving feedback through mouth airflow resistance. However, the experiment also revealed some shortcomings of the device, including user fatigue and its impact on breathing. After experimental investigation, it was observed that fatigue levels can decrease with training. Experimental studies have revealed that fatigue levels can decrease with training. Furthermore, the limitations of the device have shown potential for improvement through structural enhancements. Overall, our mouth and tongue interactive device shows promising potential in controlling the WRL during tasks where human limbs are occupied.

Vibrotactile feedback as a countermeasure for spatial disorientation

Spaceflight can make astronauts susceptible to spatial disorientation which is one of the leading causes of fatal aircraft accidents. In our experiment, blindfolded participants used a joystick to balance themselves while inside a multi-axis rotation device (MARS) in either the vertical or horizontal roll plane. On Day 1, in the vertical roll plane (Earth analog condition) participants could use gravitational cues and therefore had a good sense of their orientation. On Day 2, in the horizontal roll plane (spaceflight analog condition) participants could not use gravitational cues and rapidly became disoriented and showed minimal learning and poor performance. One potential countermeasure for spatial disorientation is vibrotactile feedback that conveys body orientation provided by small vibrating devices applied to the skin. Orientation-dependent vibrotactile feedback provided to one group enhanced performance in the spaceflight condition but the participants reported a conflict between the accurate vibrotactile cues and their erroneous perception of their orientation. Specialized vibrotactile training on Day 1 provided to another group resulted in significantly better learning and performance in the spaceflight analog task with vibrotactile cueing. In this training, participants in the Earth analog condition on Day 1 were required to disengage from the task of aligning with the gravitational vertical encoded by natural vestibular/somatosensory afference and had to align with randomized non-vertical directions of balance signaled by vibrotactile feedback. At the end of Day 2, we deactivated the vibrotactile feedback after both vibration-cued groups had practiced with it in the spaceflight analog condition. They performed as well as the group who did not have any vibrotactile feedback. We conclude that after appropriate training, vibrotactile orientation feedback augments dynamic spatial orientation and does not lead to any negative dependence.

Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device

Mixed reality technology can give humans an intuitive visual experience, and combined with the multi-source information of the human body, it can provide a comfortable human-robot interaction experience. This paper applies a mixed reality device (Hololens2) to provide interactive communication between the wearer and the wearable robotic limb (supernumerary robotic limb, SRL). Hololens2 can obtain human body information, including eye gaze, hand gestures, voice input, etc. It can also provide feedback information to the wearer through augmented reality and audio output, which is the communication bridge needed in human-robot interaction. Implementing a wearable robotic arm integrated with HoloLens2 is proposed to augment the wearer's capabilities. Taking two typical practical tasks of cable installation and electrical connector soldering in aircraft manufacturing as examples, the task models and interaction scheme are designed. Finally, human augmentation is evaluated in terms of task completion time statistics.

Cognitive assistance for action selection: Challenges and approaches

Cognitive assistance systems aim at compensating shortcomings of natural cognition concerning specific activities. Notable progress has been made regarding data acquisition, analysis, and the exploration of technical means for supporting human action selection and execution. The related challenges and potential solutions can be associated to four largely independent questions: What actions should be executed, when this must or should be done, whether assistance is needed for a specific action, and if so, how the action should be supported. A broad range of technological and methodical approaches can be taken for tackling each of these issues, including recent advances and new challenges in the automatized analysis of task-related mental representation structures.

Human Augmentation Technologies for Employee Well-Being: A Research and Development Agenda

The COVID-19 pandemic has changed the style of work. In adapting to the changing work environment, human augmentation technologies (HAT) can provide employees with new options to support their work. However, the agenda for research and development of HAT for the new normal is still unclear. In this study, we set two research questions: (i) what type of technology demand has emerged among employees due to the COVID-19 pandemic; and (ii) what is the nature of job satisfaction experienced by employees during the COVID-19 pandemic? This study aims to clarify the technology demand and job satisfaction of employees during the COVID-19 pandemic. We analyzed data from in-depth interviews with employees based on a grounded theory approach to answer the research questions and proposed an agenda for the research and development of HAT to enhance employees' well-being in this new normal based on the crosspoint of technology demand and job satisfaction. The theoretical contribution of this study is the development of models of technology demand and job satisfaction of employees during the COVID-19 pandemic. The practical contribution is the development of a crosspoint framework to enable the development of HAT to support work while considering their impact on employees' well-being.

Simultaneous control of natural and extra degrees of freedom by isometric force and electromyographic activity in the muscle-to-force null space

OBJECTIVE

Muscle activation patterns in the muscle-to-force null space, i.e., patterns that do not generate task-relevant forces, may provide an opportunity for motor augmentation by allowing to control additional end-effectors simultaneously to natural limbs. Here we tested the feasibility of muscular null space control for augmentation by assessing simultaneous control of natural and extra degrees of freedom.

APPROACH

We instructed eight participants to control translation and rotation of a virtual 3D end-effector by simultaneous generation of isometric force at the hand and null space activity extracted in real-time from the electromyographic signals recorded from 15 shoulder and arm muscles. First, we identified the null space components that each participant could control more naturally by voluntary co-contraction. Then, participants performed several blocks of a reaching and holding task. They displaced an ellipsoidal cursor to reach one of nine targets by generating force, and simultaneously rotated the cursor to match the target orientation by activating null space components. We developed an information-theoretic metric, an index of difficulty defined as the sum of a spatial and a temporal term, to assess individual null space control ability for both reaching and holding.

MAIN RESULTS

On average, participants could reach the targets in most trials already in the first block (72%) and they improved with practice (maximum 93%) but holding performance remained lower (maximum 43%). As there was a high inter-individual variability in performance, we performed a simulation with different spatial and temporal task conditions to estimate those for which each individual participants would have performed best.

SIGNIFICANCE

Muscular null space control is feasible and may be used to control additional virtual or robotics end-effectors. However, decoding of motor commands must be optimized according to individual null space control ability.

Interface, interaction, and intelligence in generalized brain-computer interfaces

A brain-computer interface (BCI) establishes a direct communication channel between a brain and an external device. With recent advances in neurotechnology and artificial intelligence (AI), the brain signals in BCI communication have been advanced from sensation and perception to higher-level cognition activities. While the field of BCI has grown rapidly in the past decades, the core technologies and innovative ideas behind seemingly unrelated BCI systems have never been summarized from an evolutionary point of view. Here, we review various BCI paradigms and present an evolutionary model of generalized BCI technology which comprises three stages: interface, interaction, and intelligence (I3). We also highlight challenges, opportunities, and future perspectives in the development of new BCI technology.

Towards human motor augmentation by voluntary decoupling beta activity in the neural drive to muscle and force production

Objective.Effective human motor augmentation should rely on biological signals that can be volitionally modulated without compromising natural motor control.Approach.We provided human subjects with real-time information on the power of two separate spectral bands of the spiking activity of motor neurons innervating the tibialis anterior muscle: the low-frequency band (<7 Hz), which is directly translated into natural force control, and the beta band (13-30 Hz), which is outside the dynamics of the neuromuscular system.Main Results.Subjects could gain control over the powers in these two bands to navigate a cursor towards specific targets in a 2D space (experiment 1) and to up- and down-modulate beta activity while keeping steady force contractions (experiment 2).Significance.Results indicate that beta projections to the spinal motor neuron pool can be voluntarily controlled partially decoupled from natural muscle contractions and, therefore, they could be valid control signals for implementing effective human motor augmentation platforms.

A Human-Algorithm Integration System for Hip Fracture Detection on Plain Radiography: System Development and Validation Study

Background

Hip fracture is the most common type of fracture in elderly individuals. Numerous deep learning (DL) algorithms for plain pelvic radiographs (PXRs) have been applied to improve the accuracy of hip fracture diagnosis. However, their efficacy is still undetermined.

Objective

The objective of this study is to develop and validate a human-algorithm integration (HAI) system to improve the accuracy of hip fracture diagnosis in a real clinical environment.

Methods

The HAI system with hip fracture detection ability was developed using a deep learning algorithm trained on trauma registry data and 3605 PXRs from August 2008 to December 2016. To compare their diagnostic performance before and after HAI system assistance using an independent testing dataset, 34 physicians were recruited. We analyzed the physicians’ accuracy, sensitivity, specificity, and agreement with the algorithm; we also performed subgroup analyses according to physician specialty and experience. Furthermore, we applied the HAI system in the emergency departments of different hospitals to validate its value in the real world.

Results

With the support of the algorithm, which achieved 91% accuracy, the diagnostic performance of physicians was significantly improved in the independent testing dataset, as was revealed by the sensitivity (physician alone, median 95%; HAI, median 99%; P<.001), specificity (physician alone, median 90%; HAI, median 95%; P<.001), accuracy (physician alone, median 90%; HAI, median 96%; P<.001), and human-algorithm agreement [physician alone κ, median 0.69 (IQR 0.63-0.74); HAI κ, median 0.80 (IQR 0.76-0.82); P<.001. With the help of the HAI system, the primary physicians showed significant improvement in their diagnostic performance to levels comparable to those of consulting physicians, and both the experienced and less-experienced physicians benefited from the HAI system. After the HAI system had been applied in 3 departments for 5 months, 587 images were examined. The sensitivity, specificity, and accuracy of the HAI system for detecting hip fractures were 97%, 95.7%, and 96.08%, respectively.

Conclusions

HAI currently impacts health care, and integrating this technology into emergency departments is feasible. The developed HAI system can enhance physicians’ hip fracture diagnostic performance.

Industrial exoskeletons: Need for intervention effectiveness research

Exoskeleton devices are being introduced across several industry sectors to augment, amplify, or reinforce the performance of a worker's existing body components-primarily the lower back and the upper extremity. Industrial exoskeletons may play a role in reducing work-related musculoskeletal disorders arising from lifting and handling heavy materials or from supporting heavy tools in overhead work. However, wearing an exoskeleton may pose a number of risks that are currently not well-studied. There are only a few studies about the safety and health implications of wearable exoskeletons and most of those studies involve only a small number of participants. Before the widespread implementation of industrial exoskeletons occurs, there is need for prospective interventional studies to evaluate the safety and health effectiveness of exoskeletons across various industry sectors. Developing a research strategy to fill current safety and health knowledge gaps, understanding the benefits, risks, and barriers to adoption of industrial exoskeletons, determining whether exoskeleton can be considered a type of personal protective equipment, and advancing consensus standards that address exoskeleton safety, should be major interests of both the occupational safety and health research and practice communities.

Design of a Low Profile, Unpowered Ankle Exoskeleton That Fits Under Clothes: Overcoming Practical Barriers to Widespread Societal Adoption

Here, we present the design of a novel unpowered ankle exoskeleton that is low profile, lightweight, quiet, and low cost to manufacture, intrinsically adapts to different walking speeds, and does not restrict non-sagittal joint motion; while still providing assistive ankle torque that can reduce demands on the biological calf musculature. This paper is an extension of the previously-successful ankle exoskeleton concept by Collins, Wiggin, and Sawicki. We created a device that blends the torque assistance of the prior exoskeleton with the form-factor benefits of clothing. Our design integrates a low profile under-the-foot clutch and a soft conformal shank interface, coupled by an ankle assistance spring that operates in parallel with the user's calf muscles. We fabricated and characterized technical performance of a prototype through benchtop testing and then validated device functionality in two gait analysis case studies. To our knowledge, this is the first ankle plantarflexion assistance exoskeleton that could be feasibly worn under typical daily clothing, without restricting ankle motion, and without components protruding substantially from the shoe, leg, waist, or back. Our new design highlights the potential for performance-enhancing exoskeletons that are inexpensive, unobtrusive, and can be used on a wide scale to benefit a broad range of individuals throughout society, such as the elderly, individuals with impaired plantarflexor muscle strength, or recreational users. In summary, this paper demonstrates how an unpowered ankle exoskeleton could be redesigned to more seamlessly integrate into daily life, while still providing performance benefits for common locomotion tasks.