1
|
Amri Bin Suhaimi MS, Matsushita K, Kitamura T, Laksono PW, Sasaki M. Object Grasp Control of a 3D Robot Arm by Combining EOG Gaze Estimation and Camera-Based Object Recognition. Biomimetics (Basel) 2023; 8:biomimetics8020208. [PMID: 37218794 DOI: 10.3390/biomimetics8020208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 05/11/2023] [Accepted: 05/12/2023] [Indexed: 05/24/2023] Open
Abstract
The purpose of this paper is to quickly and stably achieve grasping objects with a 3D robot arm controlled by electrooculography (EOG) signals. A EOG signal is a biological signal generated when the eyeballs move, leading to gaze estimation. In conventional research, gaze estimation has been used to control a 3D robot arm for welfare purposes. However, it is known that the EOG signal loses some of the eye movement information when it travels through the skin, resulting in errors in EOG gaze estimation. Thus, EOG gaze estimation is difficult to point out the object accurately, and the object may not be appropriately grasped. Therefore, developing a methodology to compensate, for the lost information and increase spatial accuracy is important. This paper aims to realize highly accurate object grasping with a robot arm by combining EMG gaze estimation and the object recognition of camera image processing. The system consists of a robot arm, top and side cameras, a display showing the camera images, and an EOG measurement analyzer. The user manipulates the robot arm through the camera images, which can be switched, and the EOG gaze estimation can specify the object. In the beginning, the user gazes at the screen's center position and then moves their eyes to gaze at the object to be grasped. After that, the proposed system recognizes the object in the camera image via image processing and grasps it using the object centroid. The object selection is based on the object centroid closest to the estimated gaze position within a certain distance (threshold), thus enabling highly accurate object grasping. The observed size of the object on the screen can differ depending on the camera installation and the screen display state. Therefore, it is crucial to set the distance threshold from the object centroid for object selection. The first experiment is conducted to clarify the distance error of the EOG gaze estimation in the proposed system configuration. As a result, it is confirmed that the range of the distance error is 1.8-3.0 cm. The second experiment is conducted to evaluate the performance of the object grasping by setting two thresholds from the first experimental results: the medium distance error value of 2 cm and the maximum distance error value of 3 cm. As a result, it is found that the grasping speed of the 3 cm threshold is 27% faster than that of the 2 cm threshold due to more stable object selection.
Collapse
Affiliation(s)
- Muhammad Syaiful Amri Bin Suhaimi
- Faculty of Information and Communication Technology, Universiti Tunku Abdul Rahman, Jalan Universiti, Bandar Barat, Kampar 31900, Malaysia
- Graduate School of Engineering, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan
| | - Kojiro Matsushita
- Graduate School of Engineering, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan
- Intelligent Production Technology Research & Development Center for Aerospace (IPTeCA), Tokai National Higher Education and Research System, Gifu 501-1193, Japan
| | - Takahide Kitamura
- Graduate School of Engineering, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan
- Intelligent Production Technology Research & Development Center for Aerospace (IPTeCA), Tokai National Higher Education and Research System, Gifu 501-1193, Japan
| | - Pringgo Widyo Laksono
- Graduate School of Engineering, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan
- Industrial Engineering, Faculty of Engineering, Universitas Sebelas Maret, Surakarta 57126, Indonesia
| | - Minoru Sasaki
- Graduate School of Engineering, Gifu University, 1-1 Yanagido, Gifu 501-1193, Japan
- Intelligent Production Technology Research & Development Center for Aerospace (IPTeCA), Tokai National Higher Education and Research System, Gifu 501-1193, Japan
| |
Collapse
|
2
|
Mohanty M, Dash M, Biswal P, Sabut S. Classification of ventricular arrhythmias using empirical mode decomposition and machine learning algorithms. PROGRESS IN ARTIFICIAL INTELLIGENCE 2021. [DOI: 10.1007/s13748-021-00250-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
3
|
Hooge ITC, Niehorster DC, Nyström M, Andersson R, Hessels RS. Is human classification by experienced untrained observers a gold standard in fixation detection? Behav Res Methods 2018; 50:1864-1881. [PMID: 29052166 PMCID: PMC7875941 DOI: 10.3758/s13428-017-0955-x] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Manual classification is still a common method to evaluate event detection algorithms. The procedure is often as follows: Two or three human coders and the algorithm classify a significant quantity of data. In the gold standard approach, deviations from the human classifications are considered to be due to mistakes of the algorithm. However, little is known about human classification in eye tracking. To what extent do the classifications from a larger group of human coders agree? Twelve experienced but untrained human coders classified fixations in 6 min of adult and infant eye-tracking data. When using the sample-based Cohen's kappa, the classifications of the humans agreed near perfectly. However, we found substantial differences between the classifications when we examined fixation duration and number of fixations. We hypothesized that the human coders applied different (implicit) thresholds and selection rules. Indeed, when spatially close fixations were merged, most of the classification differences disappeared. On the basis of the nature of these intercoder differences, we concluded that fixation classification by experienced untrained human coders is not a gold standard. To bridge the gap between agreement measures (e.g., Cohen's kappa) and eye movement parameters (fixation duration, number of fixations), we suggest the use of the event-based F1 score and two new measures: the relative timing offset (RTO) and the relative timing deviation (RTD).
Collapse
Affiliation(s)
- Ignace T C Hooge
- Experimental Psychology, Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands.
| | - Diederick C Niehorster
- Lund University Humanities Lab, Lund University, Helgonabacken 12, 22362, Lund, Sweden
- Department of Psychology, Lund University, Lund, Sweden
| | - Marcus Nyström
- Lund University Humanities Lab, Lund University, Helgonabacken 12, 22362, Lund, Sweden
| | - Richard Andersson
- Eye Information Group, IT University of Copenhagen, Copenhagen, Denmark
- Department of Philosophy and Cognitive Sciences, Lund University, Lund, Sweden
| | - Roy S Hessels
- Experimental Psychology, Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands
- Developmental Psychology, Utrecht University, Helgonabacken 1, 3584 CS, Utrecht, The Netherlands
| |
Collapse
|
4
|
Becerra-García R, García-Bermúdez R, Joya-Caparrós G, Fernández-Higuera A, Velázquez-Rodríguez C, Velázquez-Mariño M, Cuevas-Beltrán F, García-Lagos F, Rodráguez-Labrada R. Data mining process for identification of non-spontaneous saccadic movements in clinical electrooculography. Neurocomputing 2017. [DOI: 10.1016/j.neucom.2016.10.077] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
5
|
Wu SL, Liao LD, Lu SW, Jiang WL, Chen SA, Lin CT. Controlling a human-computer interface system with a novel classification method that uses electrooculography signals. IEEE Trans Biomed Eng 2013; 60:2133-41. [PMID: 23446030 DOI: 10.1109/tbme.2013.2248154] [Citation(s) in RCA: 57] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Electrooculography (EOG) signals can be used to control human-computer interface (HCI) systems, if properly classified. The ability to measure and process these signals may help HCI users to overcome many of the physical limitations and inconveniences in daily life. However, there are currently no effective multidirectional classification methods for monitoring eye movements. Here, we describe a classification method used in a wireless EOG-based HCI device for detecting eye movements in eight directions. This device includes wireless EOG signal acquisition components, wet electrodes and an EOG signal classification algorithm. The EOG classification algorithm is based on extracting features from the electrical signals corresponding to eight directions of eye movement (up, down, left, right, up-left, down-left, up-right, and down-right) and blinking. The recognition and processing of these eight different features were achieved in real-life conditions, demonstrating that this device can reliably measure the features of EOG signals. This system and its classification procedure provide an effective method for identifying eye movements. Additionally, it may be applied to study eye functions in real-life conditions in the near future.
Collapse
Affiliation(s)
- Shang-Lin Wu
- Institute of Electrical Control Engineering, National Chiao Tung University, Hsinchu 300, Taiwan.
| | | | | | | | | | | |
Collapse
|
6
|
Kara S, Güven A. Neural network-based diagnosing for optic nerve disease from visual-evoked potential. J Med Syst 2007; 31:391-6. [PMID: 17918693 DOI: 10.1007/s10916-007-9081-0] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
In this paper, we purpose a diagnostic procedure to identify the optic nerve disease from visual evoked potential (VEP) signals using an Artificial Neural Network (ANN). Multilayer feed forward ANN trained with a Levenberg Marquart backpropagation algorithm was implemented. The correct classification rate was 96.87% for subjects having optic nerve disease and 96.66% for healthy subjects. The end results are classified as healthy and diseased. Testing results were found to be compliant with the expected results that are derived from the physician's direct diagnosis, angiography, VEP and pattern electroretinography. The stated results show that the proposed method could point out the ability of design of a new intelligent assistance diagnosis system.
Collapse
Affiliation(s)
- Sadik Kara
- Department of Electrical and Electronics Eng., Erciyes University, 38039 Kayseri, Turkey.
| | | |
Collapse
|
7
|
Abstract
Monitoring eye movements is clinically important in diagnosis of diseases of the central nervous system. Electrooculography (EOG) is one method of obtaining such records which uses skin electrodes, and utilizes the anterior posterior polarization of the eye. A new EOG diagnostic system has been developed that utilizes two off-the-shelf portable notebook computers, one projector and simple electronic hardware. It can be operated under Windows 95, 98, NT, and has significant advantages over any other similar equipment, including programmability, portability, improved safety and low cost. Especially, portability of the instrument is extremely important for acutely ill or handicapped patients. The purpose of this paper is to introduce the techniques of computer animation, data acquisition, real time analysis of measured data, and database management to implement a portable, programmable and inexpensive contacting EOG instrument. It is very convenient to replace the present expensive, inflexible and large-sized commercially available EOG instruments. A lot of interesting stimulation patterns for clinical application can be created easily in different shape, time sequence, and colour by programming in Delphi language. With the help of Winstar (a software package that is used to control I/O and interrupt functions of the computer under Windows 95, 98, NT), the I/O communication between two notebook computers and A/D interface module can be effectively programmed. In addition, the new EOG diagnostic system is battery operated and it has the advantages of low noise as well as isolation from electricity. Two kinds of EOG tests, pursuit and saccade, were performed on 20 normal subjects with this new portable and programmable instrument. Based on the test result, the performance of the new instrument is superior to the other commercially available instruments. In conclusion, we hope that it will be more convenient for doctors and researchers to do the clinical EOG diagnosis and basic medical science research by using this new creation.
Collapse
Affiliation(s)
- S C Chen
- Department of Electrical Engineering National Cheng Kung University, Tainan, Taiwan
| | | | | |
Collapse
|