1
|
Palumbo A, Ielpo N, Calabrese B, Garropoli R, Gramigna V, Ammendolia A, Marotta N. An Innovative Device Based on Human-Machine Interface (HMI) for Powered Wheelchair Control for Neurodegenerative Disease: A Proof-of-Concept. SENSORS (BASEL, SWITZERLAND) 2024; 24:4774. [PMID: 39123822 PMCID: PMC11314770 DOI: 10.3390/s24154774] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/24/2024] [Revised: 07/16/2024] [Accepted: 07/19/2024] [Indexed: 08/12/2024]
Abstract
In the global context, advancements in technology and science have rendered virtual, augmented, and mixed-reality technologies capable of transforming clinical care and medical environments by offering enhanced features and improved healthcare services. This paper aims to present a mixed reality-based system to control a robotic wheelchair for people with limited mobility. The test group comprised 11 healthy subjects (six male, five female, mean age 35.2 ± 11.7 years). A novel platform that integrates a smart wheelchair and an eye-tracking-enabled head-mounted display was proposed to reduce the cognitive requirements needed for wheelchair movement and control. The approach's effectiveness was demonstrated by evaluating our system in realistic scenarios. The demonstration of the proposed AR head-mounted display user interface for controlling a smart wheelchair and the results provided in this paper could highlight the potential of the HoloLens 2-based innovative solutions and bring focus to emerging research topics, such as remote control, cognitive rehabilitation, the implementation of patient autonomy with severe disabilities, and telemedicine.
Collapse
Affiliation(s)
- Arrigo Palumbo
- Department of Medical and Surgical Sciences, Magna Graecia University of Catanzaro, 88100 Catanzaro, Italy
| | - Nicola Ielpo
- Department of Medical and Surgical Sciences, Magna Graecia University of Catanzaro, 88100 Catanzaro, Italy
| | - Barbara Calabrese
- Istituto Tecnico Industriale Statale “Enrico Fermi”, Via Piero Della Francesca, 87012 Castrovillari, Italy
| | - Remo Garropoli
- Garropoli Computer Science Consulting, 87100 Cosenza, Italy;
| | - Vera Gramigna
- Department of Medical and Surgical Sciences, Magna Graecia University of Catanzaro, 88100 Catanzaro, Italy
| | - Antonio Ammendolia
- Physical Medicine and Rehabilitation Unit, Department of Medical and Surgical Sciences, University Hospital “Mater Domini”, University of Catanzaro Magna Graecia, Via Campanella, 88100 Catanzaro, Italy;
- Research Center on Musculoskeletal Health, MusculoSkeletalHealth@UMG, University of Catanzaro “Magna Graecia”, 88100 Catanzaro, Italy;
| | - Nicola Marotta
- Research Center on Musculoskeletal Health, MusculoSkeletalHealth@UMG, University of Catanzaro “Magna Graecia”, 88100 Catanzaro, Italy;
- Department of Experimental and Clinical Medicine, Magna Graecia University of Catanzaro, 88100 Catanzaro, Italy
| |
Collapse
|
2
|
Zeng Z, Tao L, Zhu H, Zhu Y, Meng L, Fan J, Chen C, Chen W. A Robust Gaze Estimation Approach via Exploring Relevant Electrooculogram Features and Optimal Electrodes Placements. IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE 2023; 12:56-65. [PMID: 38088999 PMCID: PMC10712680 DOI: 10.1109/jtehm.2023.3320713] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 07/16/2023] [Accepted: 09/21/2023] [Indexed: 12/18/2023]
Abstract
Gaze estimation, as a technique that reflects individual attention, can be used for disability assistance and assisting physicians in diagnosing diseases such as autism spectrum disorder (ASD), Parkinson's disease, and attention deficit hyperactivity disorder (ADHD). Various techniques have been proposed for gaze estimation and achieved high resolution. Among these approaches, electrooculography (EOG)-based gaze estimation, as an economical and effective method, offers a promising solution for practical applications. OBJECTIVE In this paper, we systematically investigated the possible EOG electrode locations which are spatially distributed around the orbital cavity. Afterward, quantities of informative features to characterize physiological information of eye movement from the temporal-spectral domain are extracted from the seven differential channels. METHODS AND PROCEDURES To select the optimum channels and relevant features, and eliminate irrelevant information, a heuristical search algorithm (i.e., forward stepwise strategy) is applied. Subsequently, a comparative analysis of the impacts of electrode placement and feature contributions on gaze estimation is evaluated via 6 classic models with 18 subjects. RESULTS Experimental results showed that the promising performance was achieved both in the Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) within a wide gaze that ranges from -50° to +50°. The MAE and RMSE can be improved to 2.80° and 3.74° ultimately, while only using 10 features extracted from 2 channels. Compared with the prevailing EOG-based techniques, the performance improvement of MAE and RMSE range from 0.70° to 5.48° and 0.66° to 5.42°, respectively. CONCLUSION We proposed a robust EOG-based gaze estimation approach by systematically investigating the optimal channel/feature combination. The experimental results indicated not only the superiority of the proposed approach but also its potential for clinical application. Clinical and translational impact statement: Accurate gaze estimation is a key step for assisting disabilities and accurate diagnosis of various diseases including ASD, Parkinson's disease, and ADHD. The proposed approach can accurately estimate the points of gaze via EOG signals, and thus has the potential for various related medical applications.
Collapse
Affiliation(s)
- Zheng Zeng
- Center for Intelligent Medical Electronics, School of Information Science and TechnologyFudan UniversityShanghai200433China
| | - Linkai Tao
- Department of Industrial DesignEindhoven University of Technology5600 MBEindhovenThe Netherlands
| | - Hangyu Zhu
- Center for Intelligent Medical Electronics, School of Information Science and TechnologyFudan UniversityShanghai200433China
| | - Yunfeng Zhu
- Center for Intelligent Medical Electronics, School of Information Science and TechnologyFudan UniversityShanghai200433China
| | - Long Meng
- Center for Intelligent Medical Electronics, School of Information Science and TechnologyFudan UniversityShanghai200433China
| | - Jiahao Fan
- Center for Intelligent Medical Electronics, School of Information Science and TechnologyFudan UniversityShanghai200433China
| | - Chen Chen
- Human Phenome Institute, Fudan UniversityShanghai201203China
| | - Wei Chen
- Center for Intelligent Medical Electronics, School of Information Science and TechnologyFudan UniversityShanghai200433China
- Human Phenome Institute, Fudan UniversityShanghai201203China
| |
Collapse
|
3
|
Cho G, Yang W, Lee D, You D, Lee H, Kim S, Lee S, Nam W. Characterization of signal features for real-time sEMG onset detection. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104774] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/17/2023]
|
4
|
Zheng Y, Liu C, Lai NYG, Wang Q, Xia Q, Sun X, Zhang S. Current development of biosensing technologies towards diagnosis of mental diseases. Front Bioeng Biotechnol 2023; 11:1190211. [PMID: 37456720 PMCID: PMC10342212 DOI: 10.3389/fbioe.2023.1190211] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Accepted: 06/16/2023] [Indexed: 07/18/2023] Open
Abstract
The biosensor is an instrument that converts the concentration of biomarkers into electrical signals for detection. Biosensing technology is non-invasive, lightweight, automated, and biocompatible in nature. These features have significantly advanced medical diagnosis, particularly in the diagnosis of mental disorder in recent years. The traditional method of diagnosing mental disorders is time-intensive, expensive, and subject to individual interpretation. It involves a combination of the clinical experience by the psychiatrist and the physical symptoms and self-reported scales provided by the patient. Biosensors on the other hand can objectively and continually detect disease states by monitoring abnormal data in biomarkers. Hence, this paper reviews the application of biosensors in the detection of mental diseases, and the diagnostic methods are divided into five sub-themes of biosensors based on vision, EEG signal, EOG signal, and multi-signal. A prospective application in clinical diagnosis is also discussed.
Collapse
Affiliation(s)
- Yuhan Zheng
- Faculty of Science and Engineering, University of Nottingham Ningbo China, Ningbo, China
- Ningbo Research Center, Ningbo Innovation Center, Zhejiang University, Ningbo, China
- Robotics Institute, Ningbo University of Technology, Ningbo, China
| | - Chen Liu
- Faculty of Science and Engineering, University of Nottingham Ningbo China, Ningbo, China
- Ningbo Research Center, Ningbo Innovation Center, Zhejiang University, Ningbo, China
| | - Nai Yeen Gavin Lai
- Faculty of Science and Engineering, University of Nottingham Ningbo China, Ningbo, China
| | - Qingfeng Wang
- Nottingham Ningbo China Beacons of Excellence Research and Innovation Institute, University of Nottingham Ningbo China, Ningbo, China
| | - Qinghua Xia
- Ningbo Research Center, Ningbo Innovation Center, Zhejiang University, Ningbo, China
| | - Xu Sun
- Faculty of Science and Engineering, University of Nottingham Ningbo China, Ningbo, China
- Nottingham Ningbo China Beacons of Excellence Research and Innovation Institute, University of Nottingham Ningbo China, Ningbo, China
| | - Sheng Zhang
- Faculty of Science and Engineering, University of Nottingham Ningbo China, Ningbo, China
- Ningbo Research Center, Ningbo Innovation Center, Zhejiang University, Ningbo, China
- School of Mechanical Engineering, Zhejiang University, Hangzhou, China
| |
Collapse
|
5
|
Cui J, Huang Z, Li X, Cui L, Shang Y, Tong L. Research on Intelligent Wheelchair Attitude-Based Adjustment Method Based on Action Intention Recognition. MICROMACHINES 2023; 14:1265. [PMID: 37374850 DOI: 10.3390/mi14061265] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 06/12/2023] [Accepted: 06/14/2023] [Indexed: 06/29/2023]
Abstract
At present, research on intelligent wheelchairs mostly focuses on motion control, while research on attitude-based adjustment is relatively insufficient. The existing methods for adjusting wheelchair posture generally lack collaborative control and good human-machine collaboration. This article proposes an intelligent wheelchair posture-adjustment method based on action intention recognition by studying the relationship between the force changes on the contact surface between the human body and the wheelchair and the action intention. This method is applied to a multi-part adjustable electric wheelchair, which is equipped with multiple force sensors to collect pressure information from various parts of the passenger's body. The upper level of the system converts the pressure data into the form of a pressure distribution map, extracts the shape features using the VIT deep learning model, identifies and classifies them, and ultimately identifies the action intentions of the passengers. Based on different action intentions, the electric actuator is controlled to adjust the wheelchair posture. After testing, this method can effectively collect the body pressure data of passengers, with an accuracy of over 95% for the three common intentions of lying down, sitting up, and standing up. The wheelchair can adjust its posture based on the recognition results. By adjusting the wheelchair posture through this method, users do not need to wear additional equipment and are less affected by the external environment. The target function can be achieved with simple learning, which has good human-machine collaboration and can solve the problem of some people having difficulty adjusting the wheelchair posture independently during wheelchair use.
Collapse
Affiliation(s)
- Jianwei Cui
- Institute of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| | - Zizheng Huang
- Institute of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| | - Xiang Li
- Institute of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| | - Linwei Cui
- Institute of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| | - Yucheng Shang
- Institute of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| | - Liyan Tong
- Institute of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| |
Collapse
|
6
|
López-Ahumada R, Jiménez-Naharro R, Gómez-Bravo F. A Hardware-Based Configurable Algorithm for Eye Blink Signal Detection Using a Single-Channel BCI Headset. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23115339. [PMID: 37300066 DOI: 10.3390/s23115339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 04/20/2023] [Accepted: 05/29/2023] [Indexed: 06/12/2023]
Abstract
Eye blink artifacts in electroencephalographic (EEG) signals have been used in multiple applications as an effective method for human-computer interaction. Hence, an effective and low-cost blinking detection method would be an invaluable aid for the development of this technology. A configurable hardware algorithm, described using hardware description language, for eye blink detection based on EEG signals from a one-channel brain-computer interface (BCI) headset was developed and implemented, showing better performance in terms of effectiveness and detection time than manufacturer-provided software.
Collapse
Affiliation(s)
- Rafael López-Ahumada
- Departamento de Ingeniería Electrónica Sistemas Informáticos y Automática, Universidad de Huelva, 21007 Huelva, Spain
| | - Raúl Jiménez-Naharro
- Departamento de Ingeniería Electrónica Sistemas Informáticos y Automática, Universidad de Huelva, 21007 Huelva, Spain
| | - Fernando Gómez-Bravo
- Centro Científico Tecnológico de Huelva (CCTH), University of Huelva, 21007 Huelva, Spain
| |
Collapse
|
7
|
Ban S, Lee YJ, Kwon S, Kim YS, Chang JW, Kim JH, Yeo WH. Soft Wireless Headband Bioelectronics and Electrooculography for Persistent Human-Machine Interfaces. ACS APPLIED ELECTRONIC MATERIALS 2023; 5:877-886. [PMID: 36873262 PMCID: PMC9979786 DOI: 10.1021/acsaelm.2c01436] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Accepted: 01/29/2023] [Indexed: 06/18/2023]
Abstract
Recent advances in wearable technologies have enabled ways for people to interact with external devices, known as human-machine interfaces (HMIs). Among them, electrooculography (EOG), measured by wearable devices, is used for eye movement-enabled HMI. Most prior studies have utilized conventional gel electrodes for EOG recording. However, the gel is problematic due to skin irritation, while separate bulky electronics cause motion artifacts. Here, we introduce a low-profile, headband-type, soft wearable electronic system with embedded stretchable electrodes, and a flexible wireless circuit to detect EOG signals for persistent HMIs. The headband with dry electrodes is printed with flexible thermoplastic polyurethane. Nanomembrane electrodes are prepared by thin-film deposition and laser cutting techniques. A set of signal processing data from dry electrodes demonstrate successful real-time classification of eye motions, including blink, up, down, left, and right. Our study shows that the convolutional neural network performs exceptionally well compared to other machine learning methods, showing 98.3% accuracy with six classes: the highest performance till date in EOG classification with only four electrodes. Collectively, the real-time demonstration of continuous wireless control of a two-wheeled radio-controlled car captures the potential of the bioelectronic system and the algorithm for targeting various HMI and virtual reality applications.
Collapse
Affiliation(s)
- Seunghyeb Ban
- School
of Engineering and Computer Science, Washington
State University, Vancouver, Washington 98686, United States
- IEN
Center for Human-Centric Interfaces and Engineering at the Institute
for Electronics and Nanotechnology, Georgia
Institute of Technology, Atlanta, Georgia 30332, United States
| | - Yoon Jae Lee
- IEN
Center for Human-Centric Interfaces and Engineering at the Institute
for Electronics and Nanotechnology, Georgia
Institute of Technology, Atlanta, Georgia 30332, United States
- School
of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332, United States
| | - Shinjae Kwon
- IEN
Center for Human-Centric Interfaces and Engineering at the Institute
for Electronics and Nanotechnology, Georgia
Institute of Technology, Atlanta, Georgia 30332, United States
- George
W. Woodruff School of Mechanical Engineering, College of Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332, United States
| | - Yun-Soung Kim
- BioMedical
Engineering and Imaging Institute, Icahn
School of Medicine at Mount Sinai, New York, New York 10029, United States
| | - Jae Won Chang
- Department
of Otolaryngology Head and Neck Surgery, School of Medicine, Chungnam National University Hospital, Daejeon 35015, Republic of Korea
| | - Jong-Hoon Kim
- School
of Engineering and Computer Science, Washington
State University, Vancouver, Washington 98686, United States
- Department
of Mechanical Engineering, University of
Washington, Seattle, Washington 98195, United States
| | - Woon-Hong Yeo
- IEN
Center for Human-Centric Interfaces and Engineering at the Institute
for Electronics and Nanotechnology, Georgia
Institute of Technology, Atlanta, Georgia 30332, United States
- George
W. Woodruff School of Mechanical Engineering, College of Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332, United States
- Wallace
H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University School of Medicine, Atlanta, Georgia 30332, United States
- Parker
H. Petit Institute for Bioengineering and Biosciences, Institute for
Materials, Neural Engineering Center, Institute for Robotics and Intelligent
Machines, Georgia Institute of Technology, Atlanta, Georgia 30332, United States
| |
Collapse
|
8
|
Xu J, Pan J, Cui T, Zhang S, Yang Y, Ren TL. Recent Progress of Tactile and Force Sensors for Human-Machine Interaction. SENSORS (BASEL, SWITZERLAND) 2023; 23:1868. [PMID: 36850470 PMCID: PMC9961639 DOI: 10.3390/s23041868] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Revised: 01/23/2023] [Accepted: 02/01/2023] [Indexed: 06/18/2023]
Abstract
Human-Machine Interface (HMI) plays a key role in the interaction between people and machines, which allows people to easily and intuitively control the machine and immersively experience the virtual world of the meta-universe by virtual reality/augmented reality (VR/AR) technology. Currently, wearable skin-integrated tactile and force sensors are widely used in immersive human-machine interactions due to their ultra-thin, ultra-soft, conformal characteristics. In this paper, the recent progress of tactile and force sensors used in HMI are reviewed, including piezoresistive, capacitive, piezoelectric, triboelectric, and other sensors. Then, this paper discusses how to improve the performance of tactile and force sensors for HMI. Next, this paper summarizes the HMI for dexterous robotic manipulation and VR/AR applications. Finally, this paper summarizes and proposes the future development trend of HMI.
Collapse
Affiliation(s)
- Jiandong Xu
- School of Integrated Circuits and Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China
| | - Jiong Pan
- School of Integrated Circuits and Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China
| | - Tianrui Cui
- School of Integrated Circuits and Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China
| | - Sheng Zhang
- Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
| | - Yi Yang
- School of Integrated Circuits and Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China
| | - Tian-Ling Ren
- School of Integrated Circuits and Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing 100084, China
- Center for Flexible Electronics Technology, Tsinghua University, Beijing 100084, China
| |
Collapse
|
9
|
Bouyam C, Punsawad Y. Human–machine interface-based wheelchair control using piezoelectric sensors based on face and tongue movements. Heliyon 2022; 8:e11679. [DOI: 10.1016/j.heliyon.2022.e11679] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Revised: 08/29/2022] [Accepted: 11/10/2022] [Indexed: 11/20/2022] Open
|
10
|
Yao S, Zhou W, Hinson R, Dong P, Wu S, Ives J, Hu X, Huang H, Zhu Y. Ultrasoft Porous 3D Conductive Dry Electrodes for Electrophysiological Sensing and Myoelectric Control. ADVANCED MATERIALS TECHNOLOGIES 2022; 7:2101637. [PMID: 36276406 PMCID: PMC9581336 DOI: 10.1002/admt.202101637] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/09/2021] [Indexed: 05/12/2023]
Abstract
Biopotential electrodes have found broad applications in health monitoring, human-machine interactions, and rehabilitation. Here, we report the fabrication and applications of ultrasoft breathable dry electrodes that can address several challenges for their long-term wearable applications - skin compatibility, wearability, and long-term stability. The proposed electrodes rely on porous and conductive silver nanowire based nanocomposites as the robust mechanical and electrical interface. The highly conductive and conformable structure eliminates the necessity of conductive gel while establishing a sufficiently low electrode-skin impedance for high-fidelity electrophysiological sensing. The introduction of gas-permeable structures via a simple and scalable method based on sacrificial templates improves breathability and skin compatibility for applications requiring long-term skin contact. Such conformable and breathable dry electrodes allow for efficient and unobtrusive monitoring of heart, muscle, and brain activities. In addition, based on the muscle activities captured by the electrodes and a musculoskeletal model, electromyogram-based neural-machine interfaces were realized, illustrating the great potential for prosthesis control, neurorehabilitation, and virtual reality.
Collapse
Affiliation(s)
- Shanshan Yao
- Department of Mechanical Engineering, Stony Brook University, Stony Brook, New York 11794, USA
- Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, North Carolina 27695, USA
| | - Weixin Zhou
- Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, North Carolina 27695, USA
| | - Robert Hinson
- Joint Department of Biomedical Engineering at University of North Carolina-Chapel Hill and NC State University, Chapel Hill/Raleigh, North Carolina 27599/27695, USA
| | - Penghao Dong
- Department of Mechanical Engineering, Stony Brook University, Stony Brook, New York 11794, USA
| | - Shuang Wu
- Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, North Carolina 27695, USA
| | - Jasmine Ives
- Department of Mechanical Engineering, Stony Brook University, Stony Brook, New York 11794, USA
| | - Xiaogang Hu
- Joint Department of Biomedical Engineering at University of North Carolina-Chapel Hill and NC State University, Chapel Hill/Raleigh, North Carolina 27599/27695, USA
| | - He Huang
- Joint Department of Biomedical Engineering at University of North Carolina-Chapel Hill and NC State University, Chapel Hill/Raleigh, North Carolina 27599/27695, USA
| | - Yong Zhu
- Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, North Carolina 27695, USA
| |
Collapse
|
11
|
Manero AC, McLinden SL, Sparkman J, Oskarsson B. Evaluating surface EMG control of motorized wheelchairs for amyotrophic lateral sclerosis patients. J Neuroeng Rehabil 2022; 19:88. [PMID: 35965311 PMCID: PMC9375912 DOI: 10.1186/s12984-022-01066-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Accepted: 08/02/2022] [Indexed: 12/04/2022] Open
Abstract
Background This study evaluated a novel control method for patients unable to independently control powered wheelchairs. Patients with amyotrophic lateral sclerosis often require a wheelchair but struggle with sufficient hand dexterity required for joystick control making them a population that needs this type of control method. Methods The study employed a novel control mechanism, using electromyography surface sensors applied to temporalis muscles able to measure the myoelectric voltage. Pattern and magnitude control of muscle contraction allowed for steering intention recognition and were used to manipulate their power wheelchair joystick. Four patients ages 51 to 69, two female and two male with amyotrophic lateral sclerosis, conducted Wheelchair Skills Test developed by Dalhousie University and were surveyed on the experience’s Clinical Global Impression of Change. Results Findings showed independent steering was capable for patients without hand function and provided recommendations for improved human-machine interface. All patients demonstrated the ability to engage the system, with varying precision, for driving their wheelchair in a controlled environment. Conclusions Three patients in the pilot trial reported the highest score of clinical global impression of change, all of whom had lost independent control of their wheelchair joystick. Patient four retained impaired hand dexterity for joystick control and reported negative impression of change, comparatively. Feedback from the study will be leveraged to improve training outcomes. Trial registration Subjects provided signed informed consent according to the Declaration of Helsinki to enter the study that was approved by the Mayo Clinic Institutional Review Board in Rochester, Minnesota. The study is registered on ClinicalTrials.gov under identifier NCT04800926 as of March 14, 2021 retrospectively registered.
Collapse
|
12
|
The Symmetry of the Muscle Tension Signal in the Upper Limbs When Propelling a Wheelchair and Innovative Control Systems for Propulsion System Gear Ratio or Propulsion Torque: A Pilot Study. Symmetry (Basel) 2022. [DOI: 10.3390/sym14051002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023] Open
Abstract
Innovative wheelchair designs require new means of controlling the drive units or the propulsion transmission systems. The article proposes a signal to control the gear ratio or the amount of additional propulsion torque coming from an electric motor. The innovative control signal in this application is the signal generated by the maximum voluntary contraction (MVC) of the muscles of the upper limbs, transformed by the central processing unit (CPU) into muscle activity (MA) when using a wheelchair. The paper includes research on eight muscles of the upper limbs that are active when propelling a wheelchair. Asymmetry in the value for MVC was found between the left and right limbs, while the belly of the long radial extensor muscle of the wrist was determined to be the muscle with the least asymmetry for the users under study. This pilot research demonstrates that the difference in mean MVCmax values between the left and the right limbs can range from 20% to 49%, depending on the muscle being tested. The finding that some muscle groups demonstrate less difference in MVC values suggests that it is possible to design systems for regulating the gear ratio or additional propelling force based on the MVC signal from the muscle of one limb, as described in the patent application from 2022, no. P.440187.
Collapse
|
13
|
An sEMG based adaptive method for human-exoskeleton collaboration in variable walking environments. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103477] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
|
14
|
Zhang Q, Jin T, Cai J, Xu L, He T, Wang T, Tian Y, Li L, Peng Y, Lee C. Wearable Triboelectric Sensors Enabled Gait Analysis and Waist Motion Capture for IoT-Based Smart Healthcare Applications. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2022; 9:e2103694. [PMID: 34796695 PMCID: PMC8811828 DOI: 10.1002/advs.202103694] [Citation(s) in RCA: 58] [Impact Index Per Article: 29.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Revised: 10/20/2021] [Indexed: 05/04/2023]
Abstract
Gait and waist motions always contain massive personnel information and it is feasible to extract these data via wearable electronics for identification and healthcare based on the Internet of Things (IoT). There also remains a demand to develop a cost-effective human-machine interface to enhance the immersion during the long-term rehabilitation. Meanwhile, triboelectric nanogenerator (TENG) revealing its merits in both wearable electronics and IoT tends to be a possible solution. Herein, the authors present wearable TENG-based devices for gait analysis and waist motion capture to enhance the intelligence and performance of the lower-limb and waist rehabilitation. Four triboelectric sensors are equidistantly sewed onto a fabric belt to recognize the waist motion, enabling the real-time robotic manipulation and virtual game for immersion-enhanced waist training. The insole equipped with two TENG sensors is designed for walking status detection and a 98.4% identification accuracy for five different humans aiming at rehabilitation plan selection is achieved by leveraging machine learning technology to further analyze the signals. Through a lower-limb rehabilitation robot, the authors demonstrate that the sensory system performs well in user recognition, motion monitoring, as well as robot and gaming-aided training, showing its potential in IoT-based smart healthcare applications.
Collapse
Affiliation(s)
- Quan Zhang
- Shanghai Key Laboratory of Intelligent Manufacturing and RoboticsSchool of Mechatronic Engineering and AutomationShanghai UniversityShanghai200444China
- School of Artificial IntelligenceShanghai UniversityShanghai200444China
| | - Tao Jin
- Shanghai Key Laboratory of Intelligent Manufacturing and RoboticsSchool of Mechatronic Engineering and AutomationShanghai UniversityShanghai200444China
- School of Artificial IntelligenceShanghai UniversityShanghai200444China
| | - Jianguo Cai
- Key Laboratory of C and PC Structures of Ministry of EducationNational Prestress Engineering Research CenterSoutheast UniversityNanjing211189China
| | - Liang Xu
- Shanghai Key Laboratory of Intelligent Manufacturing and RoboticsSchool of Mechatronic Engineering and AutomationShanghai UniversityShanghai200444China
- School of Artificial IntelligenceShanghai UniversityShanghai200444China
| | - Tianyiyi He
- Department of Electrical and Computer EngineeringNational University of Singapore4 Engineering Drive 3Singapore117583Singapore
- Center for Intelligent Sensors and MEMS (CISM)National University of Singapore4 Engineering Drive 3Singapore117583Singapore
| | - Tianhong Wang
- Shanghai Key Laboratory of Intelligent Manufacturing and RoboticsSchool of Mechatronic Engineering and AutomationShanghai UniversityShanghai200444China
- School of Artificial IntelligenceShanghai UniversityShanghai200444China
| | - Yingzhong Tian
- Shanghai Key Laboratory of Intelligent Manufacturing and RoboticsSchool of Mechatronic Engineering and AutomationShanghai UniversityShanghai200444China
| | - Long Li
- Shanghai Key Laboratory of Intelligent Manufacturing and RoboticsSchool of Mechatronic Engineering and AutomationShanghai UniversityShanghai200444China
- School of Artificial IntelligenceShanghai UniversityShanghai200444China
| | - Yan Peng
- School of Artificial IntelligenceShanghai UniversityShanghai200444China
| | - Chengkuo Lee
- Department of Electrical and Computer EngineeringNational University of Singapore4 Engineering Drive 3Singapore117583Singapore
- Center for Intelligent Sensors and MEMS (CISM)National University of Singapore4 Engineering Drive 3Singapore117583Singapore
- National University of Singapore Suzhou Research Institute (NUSRI)Suzhou Industrial ParkSuzhou215123China
| |
Collapse
|
15
|
Dasgupta A, Routray A. Piecewise empirical mode Bayesian estimation – A new method to denoise electrooculograms. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
16
|
Zhu B, Zhang D, Chu Y, Zhao X, Zhang L, Zhao L. Face-Computer Interface (FCI): Intent Recognition Based on Facial Electromyography (fEMG) and Online Human-Computer Interface With Audiovisual Feedback. Front Neurorobot 2021; 15:692562. [PMID: 34335220 PMCID: PMC8322851 DOI: 10.3389/fnbot.2021.692562] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2021] [Accepted: 06/21/2021] [Indexed: 11/13/2022] Open
Abstract
Patients who have lost limb control ability, such as upper limb amputation and high paraplegia, are usually unable to take care of themselves. Establishing a natural, stable, and comfortable human-computer interface (HCI) for controlling rehabilitation assistance robots and other controllable equipments will solve a lot of their troubles. In this study, a complete limbs-free face-computer interface (FCI) framework based on facial electromyography (fEMG) including offline analysis and online control of mechanical equipments was proposed. Six facial movements related to eyebrows, eyes, and mouth were used in this FCI. In the offline stage, 12 models, eight types of features, and three different feature combination methods for model inputing were studied and compared in detail. In the online stage, four well-designed sessions were introduced to control a robotic arm to complete drinking water task in three ways (by touch screen, by fEMG with and without audio feedback) for verification and performance comparison of proposed FCI framework. Three features and one model with an average offline recognition accuracy of 95.3%, a maximum of 98.8%, and a minimum of 91.4% were selected for use in online scenarios. In contrast, the way with audio feedback performed better than that without audio feedback. All subjects completed the drinking task in a few minutes with FCI. The average and smallest time difference between touch screen and fEMG under audio feedback were only 1.24 and 0.37 min, respectively.
Collapse
Affiliation(s)
- Bo Zhu
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China.,Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China.,University of Chinese Academy of Sciences, Beijing, China
| | - Daohui Zhang
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China.,Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| | - Yaqi Chu
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China.,Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China.,University of Chinese Academy of Sciences, Beijing, China
| | - Xingang Zhao
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China.,Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, China
| | - Lixin Zhang
- Rehabilitation Center, Shengjing Hospital of China Medical University, Shenyang, China
| | - Lina Zhao
- Rehabilitation Center, Shengjing Hospital of China Medical University, Shenyang, China
| |
Collapse
|
17
|
Wang X, Xiao Y, Deng F, Chen Y, Zhang H. Eye-Movement-Controlled Wheelchair Based on Flexible Hydrogel Biosensor and WT-SVM. BIOSENSORS 2021; 11:198. [PMID: 34208524 PMCID: PMC8234407 DOI: 10.3390/bios11060198] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/12/2021] [Revised: 05/31/2021] [Accepted: 06/07/2021] [Indexed: 11/17/2022]
Abstract
To assist patients with restricted mobility to control wheelchair freely, this paper presents an eye-movement-controlled wheelchair prototype based on a flexible hydrogel biosensor and Wavelet Transform-Support Vector Machine (WT-SVM) algorithm. Considering the poor deformability and biocompatibility of rigid metal electrodes, we propose a flexible hydrogel biosensor made of conductive HPC/PVA (Hydroxypropyl cellulose/Polyvinyl alcohol) hydrogel and flexible PDMS (Polydimethylsiloxane) substrate. The proposed biosensor is affixed to the wheelchair user's forehead to collect electrooculogram (EOG) and strain signals, which are the basis to recognize eye movements. The low Young's modulus (286 KPa) and exceptional breathability (18 g m-2 h-1 of water vapor transmission rate) of the biosensor ensures a conformal and unobtrusive adhesion between it and the epidermis. To improve the recognition accuracy of eye movements (straight, upward, downward, left, and right), the WT-SVM algorithm is introduced to classify EOG and strain signals according to different features (amplitude, duration, interval). The average recognition accuracy reaches 96.3%, thus the wheelchair can be manipulated precisely.
Collapse
Affiliation(s)
| | | | - Fangming Deng
- School of Electrical and Automation Engineering, East China Jiaotong University, Nanchang 330013, China; (X.W.); (Y.X.); (Y.C.); (H.Z.)
| | | | | |
Collapse
|