Wheelchair control for disabled patients using EMG/EOG based human machine interface: a review

A Robust Gaze Estimation Approach via Exploring Relevant Electrooculogram Features and Optimal Electrodes Placements

Gaze estimation, as a technique that reflects individual attention, can be used for disability assistance and assisting physicians in diagnosing diseases such as autism spectrum disorder (ASD), Parkinson's disease, and attention deficit hyperactivity disorder (ADHD). Various techniques have been proposed for gaze estimation and achieved high resolution. Among these approaches, electrooculography (EOG)-based gaze estimation, as an economical and effective method, offers a promising solution for practical applications.

OBJECTIVE

In this paper, we systematically investigated the possible EOG electrode locations which are spatially distributed around the orbital cavity. Afterward, quantities of informative features to characterize physiological information of eye movement from the temporal-spectral domain are extracted from the seven differential channels.

METHODS AND PROCEDURES

To select the optimum channels and relevant features, and eliminate irrelevant information, a heuristical search algorithm (i.e., forward stepwise strategy) is applied. Subsequently, a comparative analysis of the impacts of electrode placement and feature contributions on gaze estimation is evaluated via 6 classic models with 18 subjects.

RESULTS

Experimental results showed that the promising performance was achieved both in the Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) within a wide gaze that ranges from -50° to +50°. The MAE and RMSE can be improved to 2.80° and 3.74° ultimately, while only using 10 features extracted from 2 channels. Compared with the prevailing EOG-based techniques, the performance improvement of MAE and RMSE range from 0.70° to 5.48° and 0.66° to 5.42°, respectively.

CONCLUSION

We proposed a robust EOG-based gaze estimation approach by systematically investigating the optimal channel/feature combination. The experimental results indicated not only the superiority of the proposed approach but also its potential for clinical application. Clinical and translational impact statement: Accurate gaze estimation is a key step for assisting disabilities and accurate diagnosis of various diseases including ASD, Parkinson's disease, and ADHD. The proposed approach can accurately estimate the points of gaze via EOG signals, and thus has the potential for various related medical applications.

Current development of biosensing technologies towards diagnosis of mental diseases

The biosensor is an instrument that converts the concentration of biomarkers into electrical signals for detection. Biosensing technology is non-invasive, lightweight, automated, and biocompatible in nature. These features have significantly advanced medical diagnosis, particularly in the diagnosis of mental disorder in recent years. The traditional method of diagnosing mental disorders is time-intensive, expensive, and subject to individual interpretation. It involves a combination of the clinical experience by the psychiatrist and the physical symptoms and self-reported scales provided by the patient. Biosensors on the other hand can objectively and continually detect disease states by monitoring abnormal data in biomarkers. Hence, this paper reviews the application of biosensors in the detection of mental diseases, and the diagnostic methods are divided into five sub-themes of biosensors based on vision, EEG signal, EOG signal, and multi-signal. A prospective application in clinical diagnosis is also discussed.

Research on Intelligent Wheelchair Attitude-Based Adjustment Method Based on Action Intention Recognition

At present, research on intelligent wheelchairs mostly focuses on motion control, while research on attitude-based adjustment is relatively insufficient. The existing methods for adjusting wheelchair posture generally lack collaborative control and good human-machine collaboration. This article proposes an intelligent wheelchair posture-adjustment method based on action intention recognition by studying the relationship between the force changes on the contact surface between the human body and the wheelchair and the action intention. This method is applied to a multi-part adjustable electric wheelchair, which is equipped with multiple force sensors to collect pressure information from various parts of the passenger's body. The upper level of the system converts the pressure data into the form of a pressure distribution map, extracts the shape features using the VIT deep learning model, identifies and classifies them, and ultimately identifies the action intentions of the passengers. Based on different action intentions, the electric actuator is controlled to adjust the wheelchair posture. After testing, this method can effectively collect the body pressure data of passengers, with an accuracy of over 95% for the three common intentions of lying down, sitting up, and standing up. The wheelchair can adjust its posture based on the recognition results. By adjusting the wheelchair posture through this method, users do not need to wear additional equipment and are less affected by the external environment. The target function can be achieved with simple learning, which has good human-machine collaboration and can solve the problem of some people having difficulty adjusting the wheelchair posture independently during wheelchair use.

A Hardware-Based Configurable Algorithm for Eye Blink Signal Detection Using a Single-Channel BCI Headset

Eye blink artifacts in electroencephalographic (EEG) signals have been used in multiple applications as an effective method for human-computer interaction. Hence, an effective and low-cost blinking detection method would be an invaluable aid for the development of this technology. A configurable hardware algorithm, described using hardware description language, for eye blink detection based on EEG signals from a one-channel brain-computer interface (BCI) headset was developed and implemented, showing better performance in terms of effectiveness and detection time than manufacturer-provided software.

Soft Wireless Headband Bioelectronics and Electrooculography for Persistent Human-Machine Interfaces

Recent advances in wearable technologies have enabled ways for people to interact with external devices, known as human-machine interfaces (HMIs). Among them, electrooculography (EOG), measured by wearable devices, is used for eye movement-enabled HMI. Most prior studies have utilized conventional gel electrodes for EOG recording. However, the gel is problematic due to skin irritation, while separate bulky electronics cause motion artifacts. Here, we introduce a low-profile, headband-type, soft wearable electronic system with embedded stretchable electrodes, and a flexible wireless circuit to detect EOG signals for persistent HMIs. The headband with dry electrodes is printed with flexible thermoplastic polyurethane. Nanomembrane electrodes are prepared by thin-film deposition and laser cutting techniques. A set of signal processing data from dry electrodes demonstrate successful real-time classification of eye motions, including blink, up, down, left, and right. Our study shows that the convolutional neural network performs exceptionally well compared to other machine learning methods, showing 98.3% accuracy with six classes: the highest performance till date in EOG classification with only four electrodes. Collectively, the real-time demonstration of continuous wireless control of a two-wheeled radio-controlled car captures the potential of the bioelectronic system and the algorithm for targeting various HMI and virtual reality applications.

Recent Progress of Tactile and Force Sensors for Human-Machine Interaction

Human-Machine Interface (HMI) plays a key role in the interaction between people and machines, which allows people to easily and intuitively control the machine and immersively experience the virtual world of the meta-universe by virtual reality/augmented reality (VR/AR) technology. Currently, wearable skin-integrated tactile and force sensors are widely used in immersive human-machine interactions due to their ultra-thin, ultra-soft, conformal characteristics. In this paper, the recent progress of tactile and force sensors used in HMI are reviewed, including piezoresistive, capacitive, piezoelectric, triboelectric, and other sensors. Then, this paper discusses how to improve the performance of tactile and force sensors for HMI. Next, this paper summarizes the HMI for dexterous robotic manipulation and VR/AR applications. Finally, this paper summarizes and proposes the future development trend of HMI.

Ultrasoft Porous 3D Conductive Dry Electrodes for Electrophysiological Sensing and Myoelectric Control

Biopotential electrodes have found broad applications in health monitoring, human-machine interactions, and rehabilitation. Here, we report the fabrication and applications of ultrasoft breathable dry electrodes that can address several challenges for their long-term wearable applications - skin compatibility, wearability, and long-term stability. The proposed electrodes rely on porous and conductive silver nanowire based nanocomposites as the robust mechanical and electrical interface. The highly conductive and conformable structure eliminates the necessity of conductive gel while establishing a sufficiently low electrode-skin impedance for high-fidelity electrophysiological sensing. The introduction of gas-permeable structures via a simple and scalable method based on sacrificial templates improves breathability and skin compatibility for applications requiring long-term skin contact. Such conformable and breathable dry electrodes allow for efficient and unobtrusive monitoring of heart, muscle, and brain activities. In addition, based on the muscle activities captured by the electrodes and a musculoskeletal model, electromyogram-based neural-machine interfaces were realized, illustrating the great potential for prosthesis control, neurorehabilitation, and virtual reality.

Evaluating surface EMG control of motorized wheelchairs for amyotrophic lateral sclerosis patients

Background

This study evaluated a novel control method for patients unable to independently control powered wheelchairs. Patients with amyotrophic lateral sclerosis often require a wheelchair but struggle with sufficient hand dexterity required for joystick control making them a population that needs this type of control method.

Methods

The study employed a novel control mechanism, using electromyography surface sensors applied to temporalis muscles able to measure the myoelectric voltage. Pattern and magnitude control of muscle contraction allowed for steering intention recognition and were used to manipulate their power wheelchair joystick. Four patients ages 51 to 69, two female and two male with amyotrophic lateral sclerosis, conducted Wheelchair Skills Test developed by Dalhousie University and were surveyed on the experience’s Clinical Global Impression of Change.

Results

Findings showed independent steering was capable for patients without hand function and provided recommendations for improved human-machine interface. All patients demonstrated the ability to engage the system, with varying precision, for driving their wheelchair in a controlled environment.

Conclusions

Three patients in the pilot trial reported the highest score of clinical global impression of change, all of whom had lost independent control of their wheelchair joystick. Patient four retained impaired hand dexterity for joystick control and reported negative impression of change, comparatively. Feedback from the study will be leveraged to improve training outcomes.

Trial registration Subjects provided signed informed consent according to the Declaration of Helsinki to enter the study that was approved by the Mayo Clinic Institutional Review Board in Rochester, Minnesota. The study is registered on ClinicalTrials.gov under identifier NCT04800926 as of March 14, 2021 retrospectively registered.

The Symmetry of the Muscle Tension Signal in the Upper Limbs When Propelling a Wheelchair and Innovative Control Systems for Propulsion System Gear Ratio or Propulsion Torque: A Pilot Study

Innovative wheelchair designs require new means of controlling the drive units or the propulsion transmission systems. The article proposes a signal to control the gear ratio or the amount of additional propulsion torque coming from an electric motor. The innovative control signal in this application is the signal generated by the maximum voluntary contraction (MVC) of the muscles of the upper limbs, transformed by the central processing unit (CPU) into muscle activity (MA) when using a wheelchair. The paper includes research on eight muscles of the upper limbs that are active when propelling a wheelchair. Asymmetry in the value for MVC was found between the left and right limbs, while the belly of the long radial extensor muscle of the wrist was determined to be the muscle with the least asymmetry for the users under study. This pilot research demonstrates that the difference in mean MVCmax values between the left and the right limbs can range from 20% to 49%, depending on the muscle being tested. The finding that some muscle groups demonstrate less difference in MVC values suggests that it is possible to design systems for regulating the gear ratio or additional propelling force based on the MVC signal from the muscle of one limb, as described in the patent application from 2022, no. P.440187.

Wearable Triboelectric Sensors Enabled Gait Analysis and Waist Motion Capture for IoT-Based Smart Healthcare Applications

Gait and waist motions always contain massive personnel information and it is feasible to extract these data via wearable electronics for identification and healthcare based on the Internet of Things (IoT). There also remains a demand to develop a cost-effective human-machine interface to enhance the immersion during the long-term rehabilitation. Meanwhile, triboelectric nanogenerator (TENG) revealing its merits in both wearable electronics and IoT tends to be a possible solution. Herein, the authors present wearable TENG-based devices for gait analysis and waist motion capture to enhance the intelligence and performance of the lower-limb and waist rehabilitation. Four triboelectric sensors are equidistantly sewed onto a fabric belt to recognize the waist motion, enabling the real-time robotic manipulation and virtual game for immersion-enhanced waist training. The insole equipped with two TENG sensors is designed for walking status detection and a 98.4% identification accuracy for five different humans aiming at rehabilitation plan selection is achieved by leveraging machine learning technology to further analyze the signals. Through a lower-limb rehabilitation robot, the authors demonstrate that the sensory system performs well in user recognition, motion monitoring, as well as robot and gaming-aided training, showing its potential in IoT-based smart healthcare applications.

Face-Computer Interface (FCI): Intent Recognition Based on Facial Electromyography (fEMG) and Online Human-Computer Interface With Audiovisual Feedback

Patients who have lost limb control ability, such as upper limb amputation and high paraplegia, are usually unable to take care of themselves. Establishing a natural, stable, and comfortable human-computer interface (HCI) for controlling rehabilitation assistance robots and other controllable equipments will solve a lot of their troubles. In this study, a complete limbs-free face-computer interface (FCI) framework based on facial electromyography (fEMG) including offline analysis and online control of mechanical equipments was proposed. Six facial movements related to eyebrows, eyes, and mouth were used in this FCI. In the offline stage, 12 models, eight types of features, and three different feature combination methods for model inputing were studied and compared in detail. In the online stage, four well-designed sessions were introduced to control a robotic arm to complete drinking water task in three ways (by touch screen, by fEMG with and without audio feedback) for verification and performance comparison of proposed FCI framework. Three features and one model with an average offline recognition accuracy of 95.3%, a maximum of 98.8%, and a minimum of 91.4% were selected for use in online scenarios. In contrast, the way with audio feedback performed better than that without audio feedback. All subjects completed the drinking task in a few minutes with FCI. The average and smallest time difference between touch screen and fEMG under audio feedback were only 1.24 and 0.37 min, respectively.

Eye-Movement-Controlled Wheelchair Based on Flexible Hydrogel Biosensor and WT-SVM

To assist patients with restricted mobility to control wheelchair freely, this paper presents an eye-movement-controlled wheelchair prototype based on a flexible hydrogel biosensor and Wavelet Transform-Support Vector Machine (WT-SVM) algorithm. Considering the poor deformability and biocompatibility of rigid metal electrodes, we propose a flexible hydrogel biosensor made of conductive HPC/PVA (Hydroxypropyl cellulose/Polyvinyl alcohol) hydrogel and flexible PDMS (Polydimethylsiloxane) substrate. The proposed biosensor is affixed to the wheelchair user's forehead to collect electrooculogram (EOG) and strain signals, which are the basis to recognize eye movements. The low Young's modulus (286 KPa) and exceptional breathability (18 g m-2 h-1 of water vapor transmission rate) of the biosensor ensures a conformal and unobtrusive adhesion between it and the epidermis. To improve the recognition accuracy of eye movements (straight, upward, downward, left, and right), the WT-SVM algorithm is introduced to classify EOG and strain signals according to different features (amplitude, duration, interval). The average recognition accuracy reaches 96.3%, thus the wheelchair can be manipulated precisely.