Evaluation of the JACO robotic arm: clinico-economic study for powered wheelchair users with upper-extremity disabilities

HoLLiECares - Development of a multi-functional robot for professional care

Germany's healthcare sector suffers from a shortage of nursing staff, and robotic solutions are being explored as a means to provide quality care. While many robotic systems have already been established in various medical fields (e.g., surgical robots, logistics robots), there are only a few very specialized robotic applications in the care sector. In this work, a multi-functional robot is applied in a hospital, capable of performing activities in the areas of transport and logistics, interactive assistance, and documentation. The service robot platform HoLLiE was further developed, with a focus on implementing innovative solutions for handling non-rigid objects, motion planning for non-holonomic motions with a wheelchair, accompanying and providing haptic support to patients, optical recognition and control of movement exercises, and automated speech recognition. Furthermore, the potential of a robot platform in a nursing context was evaluated by field tests in two hospitals. The results show that a robot can take over or support certain tasks. However, it was noted that robotic tasks should be carefully selected, as robots are not able to provide empathy and affection that are often required in nursing. The remaining challenges still exist in the implementation and interaction of multi-functional capabilities, ensuring ease of use for a complex robotic system, grasping highly heterogeneous objects, and fulfilling formal and infrastructural requirements in healthcare (e.g., safety, security, and data protection).

Wheelchair-mounted robotic arms: a survey of occupational therapists' practices and perspectives

PURPOSE

Despite the benefits of wheelchair-mounted robotic arms (WMRAs), occupational therapists are not yet widely involved in the recommendation or implementation of these assistive devices. The purpose of this study was to investigate and compare the current practices and perspectives of occupational therapists who had and had not recommended a WMRA on the recommendation, training, and implementation of WMRAs.

METHODS

This was a descriptive cross-sectional study. An online survey was sent to Canadian, European, and American occupational therapists who had or had not worked with WMRAs. Respondents were asked close-ended questions about their experience, role, barriers, motivations, and future needs regarding WMRAs. We compared results between respondents who had and had not recommended WMRAs using descriptive statistics.

RESULTS

Ninety-three North American and European occupational therapists completed the survey. Of those, 29 (31.2%) had recommended a WMRA, mostly the JACO robotic arm (n = 26, 89.7%) in rehabilitation centres (n = 18, 62.1%). Their perspectives on their role and barriers related to WMRAs were similar to those who had never recommended a WMRA. All respondents recognised the relevance of occupational therapists' contribution, and most reported interest in WMRAs (n = 76, 81.7%). However, many barriers emerged, mainly related to limited funding (n = 49, 76.6%), lack of training and knowledge (n = 38, 59.4%), and resource constraints (n = 37, 54.4%). Future needs identified matched these barriers.

CONCLUSION

This survey provides novel insight into occupational therapists' perspectives on WMRAs. It highlights that health professionals need to have easier access to funding, formal training, and resources to support their involvement with WMRAs.Implications for rehabilitationMost occupational therapists are interested in working with WMRAs, considering the potential of these devices to support individuals with upper extremity impairments in their daily activities. They also recognise their unique contribution to the assessment, recommendation, and implementation process among multidisciplinary teams.WMRA recommendation is relevant in various clinical settings and with a wide range of client populations. Nevertheless, it appears that occupational therapists working with adults, in rehabilitation centres or specialised clinics, may have more opportunities to get involved in this process and to attend formal training on this technology, as compared to other settings.Many barriers remain, impeding occupational therapists' role in the recommendation and implementation of WMRAs. Addressing these barriers may increase the number of devices that are successfully adopted and utilised by individuals with upper extremity impairments. In particular, future research and health policies should focus on access to sufficient funding, formal training, and resources for occupational therapists relative to their role in recommending and implementing WMRAs.

Using workspace restrictiveness for adaptive velocity adjustment of assistive robots and upper limb exoskeletons

Individuals with severe disabilities can benefit from assistive robotic systems (ARS) for performing activities of daily living. However, limited control interfaces are available for individuals who cannot use their hands for the control, and most of these interfaces require high effort to perform simple tasks. Therefore, autonomous and intelligent control strategies were proposed for assisting with the control in complex tasks. In this paper, we presented an autonomous and adaptive method for adjusting an assistive robot's velocity in different regions of its workspace and reducing the robot velocity where fine control is required. Two participants controlled a JACO assistive robot to grasp and lift a bottle with and without the velocity adjustment method. The task was performed 9.1% faster with velocity adjustment. Furthermore, analyzing the robot trajectory showed that the method recognized highly restrictive regions and reduced the robot end-effector velocity accordingly.Clinical relevance- The autonomous velocity adjustment method can ease the control of ARSs and improve their usability, leading to a higher quality of life for individuals with severe disabilities who can benefit from ARSs.

Efficient Self-Attention Model for Speech Recognition-Based Assistive Robots Control

Assistive robots are tools that people living with upper body disabilities can leverage to autonomously perform Activities of Daily Living (ADL). Unfortunately, conventional control methods still rely on low-dimensional, easy-to-implement interfaces such as joysticks that tend to be unintuitive and cumbersome to use. In contrast, vocal commands may represent a viable and intuitive alternative. This work represents an important step toward providing a viable vocal interface for people living with upper limb disabilities by proposing a novel lightweight vocal command recognition system. The proposed model leverages the MobileNet2 architecture, augmenting it with a novel approach to the self-attention mechanism, achieving a new state-of-the-art performance for Keyword Spotting (KWS) on the Google Speech Commands Dataset (GSCD). Moreover, this work presents a new dataset, referred to as the French Speech Commands Dataset (FSCD), comprising 4963 vocal command utterances. Using the GSCD as the source, we used Transfer Learning (TL) to adapt the model to this cross-language task. TL has been shown to significantly improve the model performance on the FSCD. The viability of the proposed approach is further demonstrated through real-life control of a robotic arm by four healthy participants using both the proposed vocal interface and a joystick.

Adaptive control of a wheelchair mounted robotic arm with neuromorphically integrated velocity readings and online-learning

Wheelchair-mounted robotic arms support people with upper extremity disabilities with various activities of daily living (ADL). However, the associated cost and the power consumption of responsive and adaptive assistive robotic arms contribute to the fact that such systems are in limited use. Neuromorphic spiking neural networks can be used for a real-time machine learning-driven control of robots, providing an energy efficient framework for adaptive control. In this work, we demonstrate a neuromorphic adaptive control of a wheelchair-mounted robotic arm deployed on Intel’s Loihi chip. Our algorithm design uses neuromorphically represented and integrated velocity readings to derive the arm’s current state. The proposed controller provides the robotic arm with adaptive signals, guiding its motion while accounting for kinematic changes in real-time. We pilot-tested the device with an able-bodied participant to evaluate its accuracy while performing ADL-related trajectories. We further demonstrated the capacity of the controller to compensate for unexpected inertia-generating payloads using online learning. Videotaped recordings of ADL tasks performed by the robot were viewed by caregivers; data summarizing their feedback on the user experience and the potential benefit of the system is reported.

Assistive Robots for Patients With Amyotrophic Lateral Sclerosis: Exploratory Task-Based Evaluation Study With an Early-Stage Demonstrator

Background

Although robotic manipulators have great potential in promoting motor independence of people with motor impairments, only few systems are currently commercially available. In addition to technical, economic, and normative barriers, a key challenge for their distribution is the current lack of evidence regarding their usefulness, acceptance, and user-specific requirements.

Objective

Against this background, a semiautonomous robot system was developed in the research and development project, robot-assisted services for individual and resource-oriented intensive and palliative care of people with amyotrophic lateral sclerosis (ROBINA), to support people with amyotrophic lateral sclerosis (ALS) in various everyday activities.

Methods

The developed early-stage demonstrator was evaluated in a task-based laboratory study of 11 patients with ALS. On the basis of a multimethod design consisting of standardized questionnaires, open-ended questions, and observation protocols, participants were asked about its relevance to everyday life, usability, and design requirements.

Results

Most participants considered the system to provide relevant support within the test scenarios and for their everyday life. On the basis of the System Usability Scale, the overall usability of the robot-assisted services for individual and resource-oriented intensive and palliative care of people with ALS system was rated as excellent, with a median of 90 (IQR 75-95) points. Moreover, 3 central areas of requirements for the development of semiautonomous robotic manipulators were identified and discussed: requirements for semiautonomous human-robot collaboration, requirements for user interfaces, and requirements for the adaptation of robotic capabilities regarding everyday life.

Conclusions

Robotic manipulators can contribute to increase the autonomy of people with ALS. A key issue for future studies is how the existing ability level and the required robotic capabilities can be balanced to ensure both high user satisfaction and effective and efficient task performance.

Development and evaluation of demonstration information recording approach for wheelchair mounted robotic arm

To offer simple and convenient assistance for the elderly and disabled, researchers focus on programming by demonstration approach to improve the intelligence and adaptability of wheelchair mounted robotic arm assistive robot. But how to easily and quickly obtain the demonstration information is still an urgent problem to be solved. Based on the systematic analysis of the daily living tasks in need of robot assistance, this paper proposes the key-point-based programming by demonstration recording approach to quickly obtain the demonstration information and develops a specified demonstration interface to simplify the operation process. A corresponding evaluation approach is also proposed from the demonstration trajectories and demonstration process two aspects. Additionally, tasks of “holding water glass task”, “eating task”, and “opening door task” are carried out and experimental results, as well as comparative evaluations confirm the validity of the proposed approach with high efficiency. This study can not only offer a convenient and feasible way to obtain the demonstration information of daily living tasks, but also lay a good foundation for the assistive robot to learn relative motion skills, especially for the demonstrated dexterous manipulation skills, and semi-autonomously accomplish complex, multi-step tasks following the user’s instructions in the daily home environment.

A force-based human machine interface to drive a motorized upper limb exoskeleton. a pilot study

Muscular dystrophy is a strongly invalidating disease that causes the progressive loss of motor skills. The use of assistive devices, especially those in support of the upper limb, can increase the ability to perform daily-life activities and foster a partial recovery of the lost motor functionalities. However, for the use of these devices to be truly effective and accepted by patients, their activation must coincide with the user's intention to move. This work describes a new human-machine interface based on the integration of a six-axis force sensor to drive an upper limb motorized exoskeleton. This novel system can detect the patient's intention to move and produce displacements of the robotic device that are of magnitude and direction consistent with the user's wishes. The integration of the force-sensor interface in the BRIDGE/EMPATIA exoskeletal system was successful, and tests performed on both healthy and dystrophic subjects showed promising results, especially for the execution of planar movements.

The impact of interdisciplinarity and user involvement on the design and usability of an assistive upper limb exoskeleton - a case study on the EXOTIC

This study describes an interdisciplinary approach to develop a 5 degrees of freedom assistive upper limb exoskeleton (ULE) for users with severe to complete functional tetraplegia. Four different application levels were identified for the ULE ranging from basic technical application to interaction with users, interaction with caregivers and interaction with the society, each level posing requirements for the design and functionality of the ULE. These requirements were addressed through an interdisciplinary collaboration involving users, clinicians and researchers within social sciences and humanities, mechanical engineering, control engineering media technology and biomedical engineering. The results showed that the developed ULE, the EXOTIC, had a high level of usability, safety and adoptability. Further, the results showed that several topics are important to explicitly address in relation to the facilitation of interdisciplinary collaboration including, defining a common language, a joint visualization of the end goal and a physical frame for the collaboration, such as a shared laboratory. The study underlined the importance of interdisciplinarity and we believe that future collaboration amongst interdisciplinary researchers and centres, also at an international level, can strongly facilitate the usefulness and adoption of assistive exoskeletons and similar technologies.

IMU-Based Hand Gesture Interface Implementing a Sequence-Matching Algorithm for the Control of Assistive Technologies

Assistive technologies (ATs) often have a high-dimensionality of possible movements (e.g., assistive robot with several degrees of freedom or a computer), but the users have to control them with low-dimensionality sensors and interfaces (e.g., switches). This paper presents the development of an open-source interface based on a sequence-matching algorithm for the control of ATs. Sequence matching allows the user to input several different commands with low-dimensionality sensors by not only recognizing their output, but also their sequential pattern through time, similarly to Morse code. In this paper, the algorithm is applied to the recognition of hand gestures, inputted using an inertial measurement unit worn by the user. An SVM-based algorithm, that is aimed to be robust, with small training sets (e.g., five examples per class) is developed to recognize gestures in real-time. Finally, the interface is applied to control a computer’s mouse and keyboard. The interface was compared against (and combined with) the head movement-based AssystMouse software. The hand gesture interface showed encouraging results for this application but could also be used with other body parts (e.g., head and feet) and could control various ATs (e.g., assistive robotic arm and prosthesis).

The iHand clinical trial protocol: multi-center uncontrolled intervention study to examine the therapeutic effect of a soft-robotic glove as assistive device to support people with impaired hand strength during activities of daily living (Preprint)

Background

Decline of hand function, especially reduced hand strength, is a common problem that can be caused by many disorders and results in difficulties performing activities of daily living. A wearable soft robotic glove may be a solution, enabling use of the affected arm and hand repeatedly during functional daily activities and providing intensive and task-specific training simultaneously with assistance of hand function.

Objective

We aim to investigate the therapeutic effect of an assistive soft robotic glove (Carbonhand).

Methods

This multicenter uncontrolled intervention study consists of 3 preassessments (T0, T1, and T2), a postassessment (T3), and a follow-up assessment (T4). Participants are patients who experience hand function limitations. For the intervention, participants will use the glove during activities of daily living at home for 6 weeks, with a recommended use of at least 180 minutes per week. The primary outcome measure is handgrip strength, and secondary outcome measures are related to functional arm and hand abilities, amount of glove use, and quality of life.

Results

The first participant was included on June 25, 2019. Currently, the study has been extended due to the COVID-19 pandemic; data collection and analysis are expected to be completed in 2022.

Conclusions

The Carbonhand system is a wearable assistive device, allowing performance of functional activities to be enhanced directly during functional daily activities. At the same time, active movement of the user is encouraged as much as possible, which has potential to provide highly intensive and task-specific training. As such, it is one of the first assistive devices to incorporate assist-as-needed principles. This is the first powered clinical trial that investigates the unique application of an assistive grip-supporting soft robotic glove outside of clinical settings with the aim to have a therapeutic effect.

Trial Registration

Netherlands Trial Register NTR NL7561; https://www.trialregister.nl/trial/7561

International Registered Report Identifier (IRRID)

DERR1-10.2196/34200

Continuous Tongue Robot Mapping for Paralyzed Individuals Improves the Functional Performance of Tongue-Based Robotic Assistance

Individuals with tetraplegia have a challenging life due to a lack of independence and autonomy. Assistive robots have the potential to assist with the activities of daily living and thus improve the quality of life. However, an efficient and reliable control interface for severely disabled individuals is still missing. An intraoral tongue-computer interface (ITCI) for people with tetraplegia has previously been introduced and tested for controlling a robotic manipulator in a study deploying discrete tongue robot mapping. To improve the efficiency of the interface, the current study proposed the use of virtual buttons based on the ITCI and evaluated them in combination with a joystick-like control implementation, enabling continuous control commands. Twelve able-bodied volunteers participated in a three-day experiment. They controlled an assistive robotic manipulator through the tongue to perform two tasks: Pouring water in a cup (PW) and picking up a roll of tape (PUT). Four different tongue-robot mapping methods were compared. The results showed that using continuous commands reduced the task completion time by 16% and the number of commands of the PUT test by 20% compared with discrete commands. The highest success rate for completing the tasks was 77.8% for the PUT test and 100% for the PW test, both achieved by the control methods with continuous commands. Thus, the study demonstrated that incorporating continuous commands can improve the performance of the ITCI system for controlling robotic manipulators.

Dimensionality Reduction and Motion Clustering During Activities of Daily Living: Decoupling Hand Location and Orientation

This article is the second in a two-part series analyzing human arm and hand motion during a wide range of unstructured tasks. In this work, we track the hand of healthy individuals as they perform a variety of activities of daily living (ADLs) in three ways decoupled from hand orientation: end-point locations of the hand trajectory, whole path trajectories of the hand, and straight-line paths generated using start and end points of the hand. These data are examined by a clustering procedure to reduce the wide range of hand use to a smaller representative set. Hand orientations are subsequently analyzed for the end-point location clustering results and subsets of orientations are identified in three reference frames: global, torso, and forearm. Data driven methods that are used include dynamic time warping (DTW), DTW barycenter averaging (DBA), and agglomerative hierarchical clustering with Ward's linkage. Analysis of the end-point locations, path trajectory, and straight-line path trajectory identified 5, 5, and 7 ADL task categories, respectively, while hand orientation analysis identified up to 4 subsets of orientations for each task location, discretized and classified to the facets of a rhombicuboctahedron. Together these provide insight into our hand usage in daily life and inform an implementation in prosthetic or robotic devices using sequential control.

Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features

The functional independence of individuals with upper limb impairment could be enhanced by teleoperated robots that can assist with activities of daily living. However, robot control is not always intuitive for the operator. In this work, eye gaze was leveraged as a natural way to infer human intent and advance action recognition for shared autonomy control schemes. We introduced a classifier structure for recognizing low-level action primitives that incorporates novel three-dimensional gaze-related features. We defined an action primitive as a triplet comprised of a verb, target object, and hand object. A recurrent neural network was trained to recognize a verb and target object, and was tested on three different activities. For a representative activity (making a powdered drink), the average recognition accuracy was 77% for the verb and 83% for the target object. Using a non-specific approach to classifying and indexing objects in the workspace, we observed a modest level of generalizability of the action primitive classifier across activities, including those for which the classifier was not trained. The novel input features of gaze object angle and its rate of change were especially useful for accurately recognizing action primitives and reducing the observational latency of the classifier.

Exploring physiological signals on people with Duchenne muscular dystrophy for an active trunk support: a case study

Background

Arm support devices are available to support people with Duchenne muscular dystrophy (DMD), but active trunk support devices are lacking. An active trunk support device can potentially extend the reach of the arm and stabilize the unstable trunk of people with DMD. In a previous study, we showed that healthy people were able to control an active trunk support using four different control interfaces (based on joystick, force on feet, force on sternum and surface electromyography). All four control interfaces had different advantages and disadvantages. The aim of this study was to explore which of the four inputs is detectably used by people with DMD to control an active trunk support.

Results

The results were subject-dependent in both experiments. In the active experiment, the joystick was the most promising control interface. Regarding the static experiment, surface electromyography and force on feet worked for two out of the three subjects.

Conclusions

To our knowledge, this is the first time that people with DMD have engaged in a control task using signals other than those related to their arm muscles. According to our findings, the control interfaces have to be customised to every DMD subject.

Mechanical design of a new device to assist eating in people with movement disorders

Many people living with neurological disorders, such as cerebral palsy, stroke, muscular dystrophy or dystonia, experience upper limb impairments (muscle spasticity, loss of selective motor control, muscle weakness or tremors) and are unable to eat independently. This article presents the development of a new device to assist with eating, aimed at stabilizing the movement of people who have movement disorders. The design was guided by insights gathered through focus groups, with occupational therapists and engineers, about the challenges faced by individuals who have movement disorders and difficulty in eating autonomously. The proposed assistive device prototype is designed to be fixed on a table and to support a spoon. The mechanism is designed so that the spoon maintains a position parallel to the ground for the user. Dampers and inertia allow stabilizing the user's motion. A preliminary trial with five individuals living with cerebral palsy is presented to assess the prototype's performance and to guide future iterations of the prototype. Task completion time generally decreased and movement fluidity generally improved when using the assistive device prototype. The prototype showed good potential in stabilizing the spoon for the user and improving movement fluidity.

Development of Cybernic Robot Arm to realize Support Action cooperated with Hemiplegic Person's Arm

Hemiplegics have difficulty conducting various daily tasks because they must perform such tasks using only their unaffected arm. If a robot arm that can replace their missing upper-limb function is made available, it will greatly contribute to an improvement in their lives. To realize such a robot arm, cooperative actions of the robot arm and the unaffected arm are important. The purpose of this research is to develop a "cybernic robot arm" that can provide work support through cooperation with the unaffected arm for realization of upper-limb tasks in the daily life of hemiplegics. The developed system can detect the external force applied to a gripped object using a tactile force sensor embedded in the robot fingers. The system estimates the motion of the unaffected arm and the user's intention regarding its operation based on the external force, and provides a support action in cooperation with the unaffected arm based on the estimated information. In addition, to confirm the applicability of the developed system, we developed a support function for opening tasks that are difficult for hemiplegics to carry out on their own, and conducted an experiment confirming the applicability of the system. The results confirm that the proposed system provides proper support by cooperating with the user's arm, allowing an opening task to be performed. The developed system will support various upper-limb tasks that are difficult for hemiplegics.

Proof of Concept of an Assistive Robotic Arm Control Using Artificial Stereovision and Eye-Tracking

Assistive robotic arms have become popular to help users with upper limb disabilities achieve autonomy in their daily tasks, such as drinking and grasping objects in general. Usually, these robotic arms are controlled with an adapted joystick. Joysticks are user-friendly when it comes to a general approach to an object. However, they are not as intuitive when having to accurately approach an object, especially when obstacles are present. Alternatively, the combined use of artificial stereovision and eye-tracking seems to be a promising solution, as the user's vision is usually dissociated from their upper limb disability. Hence, the objective of this study was to develop a proof of concept for the control of an assistive robotic arm using a low-cost combination of stereovision and eye-tracking. Using the developed control system, a typically developed person was able to control the robotic arm successfully reaching and grasping an object for 92% of the trials without obstacles with an average time of 13.8 seconds. Then, another set of trials with one obstacle had a success rate of 91% with an average time of 17.3 seconds. Finally, the last set of trials with two obstacles had a success rate of 98% with an average time of 18.4 seconds. Furthermore, the cost of an eye-tracker and stereovision remains below 400$.

Functional Tasks Performed by People with Severe Muscular Atrophy Using an sEMG Controlled Robotic Manipulator

For paralyzed people activities of daily living like eating or drinking are impossible without external assistance. Robotic assistance systems can give these people a part of their independence back. Especially if the operation with a joystick is not possible anymore due to a missing hand function, people need innovative interfaces to control assistive robots in 3D. Besides brain computer interfaces an approach based on surface electromyography (sEMG) can present an opportunity for people with a strong muscular atrophy. In this work we show that two people with proceeded spinal muscular atrophy can perform functional tasks using an sEMG controlled robotic manipulator. The interface provides a continuous control of three degrees of freedom of the endeffector of the robot. The performance was assessed with two clinical measures of upper limb functionality: the Box and Blocks Test and the Action Research Arm Test. Additionally, the participant could show that they can drink by themselves with the provided system.

Impacts of robotic arm use on individuals with upper extremity disabilities: A scoping review

BACKGROUND.

Robotic arms may help users perform various activities. Even though robotic arms are commercially available, their impacts are still poorly understood.

PURPOSE.

This scoping review aimed to identify the potential impacts of using robotic arms for individuals with upper-extremity disabilities and appraise the scientific quality of the selected studies.

METHOD.

A search for studies published between 1970 and 2016 was conducted in PubMed, Embase, Compendex, and Scopus. The Canadian Model of Occupational Performance and Engagement was used to classify activities in which impacts were evaluated. The quality of each study was rated using McMaster University's critical review form for quantitative studies.

FINDINGS.

Thirty-six studies were reviewed, which evaluated self-care (21), productivity (33), and leisure (8). The short-term impacts were more commonly documented than long-term impacts. The impacts identified were mostly positive. The studies' mean quality score was 8.8/15.

IMPLICATIONS.

Additional studies with more rigorous conditions are needed to produce higher-quality scientific evidence of the long-term impacts of robotic arm use.

Semi-Autonomous Tongue Control of an Assistive Robotic Arm for Individuals with Quadriplegia

Individuals suffering from quadriplegia can achieve increased independence by using an assistive robotic manipulator (ARM). However, due to their disability, the interfaces that can be used to operate such devices become limited. A versatile intraoral tongue control interface (ITCI) has previously been develop for this user group, as the tongue is usually spared from disability. A previous study has shown that the ITCI can provide direct and continuous control of 6-7 degrees of freedom (DoF) of an ARM, due to a high number of provided inputs (18). In the present pilot study we investigated whether semi-automation might further improve the efficiency of the ITCI, when controlling an ARM. This was achieved by adding a camera to the end effector of the ARM and using computer vision algorithms to guide the ARM to grasp a target object. Three ITCI and one joystick control scheme were tested and compared: 1) manual Cartesian control with a base frame reference point, 2) manual Cartesian control with an end effector reference point 3) manual Cartesian control with an end effector reference point and an autonomous grasp function 4) regular JACO² joystick control. The results indicated that end effector control was superior to the base frame control in total task time, number of commands issued and path efficiency. The addition of the automatic grasp function did not improve the performance, but resulted in fewer collisions/displacements of the target object when grasping.

Autonomous Multi-Sensory Robotic Assistant for a Drinking Task

Assistive robots have the potential to support people with disabilities in their Activities of Daily Life. The drinking task has a high priority and requires constant assistance by caregivers to be executed regularly. Due to incapacitating disabilities such as tetraplegia, which is the paralysis of all limbs, affected people cannot use classic control interfaces such as joysticks. This paper presents a robotic solution to enable independent, straw-less drinking using a smart cup and no physically attached elements on the user. The system's hardware and software components are presented and the overarching control scheme described. The cup approaches the mouth utilising a user-friendly and vision-based robot control based on head pose estimation. Once contact has been established, the user can drink by tilting the cup with a force sensor-based control setup. Two experimental studies have been conducted, where the participants (mostly able-bodied and one tetraplegic), could separately experience the cup's contactless approach and the contact-based sequence. First results show a high user acceptance rate and consistent positive feedback. The evaluation of internal data showed a high reliability of the safety-critical components with the test groups perceiving the system as intuitive and easy to use.

Voice Control Interface Prototype for Assistive Robots for People Living with Upper Limb Disabilities

This paper presents a voice control interface prototype for assistive robots aiming to help people living with upper limb disabilities to perform daily activities autonomously. Assistive robotic devices can be used to help people with upper-body disabilities gain more autonomy in their daily life. However, it is very difficult or even impossible for certain users to control the robot with conventional control systems (e.g. joystick, sip-and-puff). This paper presents the design and preliminary evaluation of a voice command system prototype for the control of assistive robotic arms' movements. This work aims at making the control of assistive robots more intuitive and fluid, and to perform various tasks in less time and with a lesser effort. The prototype of the voice command interface developed is first presented, followed by two experiments with five able-bodied subjects in order to assess the system's performance and guide future development.

A high-resolution tongue-based joystick to enable robot control for individuals with severe disabilities

Assistive robotic arms have shown the potential to improve the quality of life of people with severe disabilities. However, a high performance and intuitive control interface for robots with 6-7 DOFs is still missing for these individuals. An inductive tongue computer interface (ITCI) was recently tested for control of robots and the study illustrated potential in this field. The paper describes the investigation of the possibility of developing a high performance tongue-based joystick-like controller for robots through two studies. The first compared different methods for mapping the 18 sensor signals to a 2D coordinate, as a touchpad. The second evaluated the performance of a novel approach for emulating an analog joystick by the ITCI based on the ISO9241-411 standard. Two subjects performed a multi-directional tapping test using a standard analog joystick, the ITCI system held in hand and operated by the other hand, and finally by tongue when mounted inside the mouth. Throughput was measured as the evaluation parameter. The results show that the contact on the touchpads can be localized by almost 1 mm accuracy. The effective throughput of ITCI system for the multi-directional tapping test was 2.03 bps while keeping it in the hand and 1.31 bps when using it inside the mouth.

Preliminary Design of an Active Stabilization Assistive Eating Device for People Living with Movement Disorders

This paper presents the development of a new active assistive eating device, which aims to stabilize the movements of people living with movement disorders, such as spasticity and ataxia. Many people living with upper-body incapacities are unable to eat on their own, due to movement disorders (ex. tremors, spastic motions, lack of muscular tone), resulting from various ailments like Cerebral palsy, Parkinson's disease, Dystonia, Multiple sclerosis, strokes, and Muscular dystrophy). Our past work focused on the development of a purely mechanical device, which involved damping of the system via passive mechanical dampers. This paper extends said work by using active stabilization of user movements. The active assistance enables the design of intelligent algorithms that can assist human movements more efficiently. This active version has the benefits of being easily adjustable; the level of damping can be adjusted in real-time, depending on the user movement; different control modes are offered, and the guiding of user movements is also allowed. Firstly, the mechanical design of the device is presented, followed by the damping arrangement, the electronic design, the control algorithms and finally, the preliminary experiments are mentioned.

Designing a brain computer interface for control of an assistive robotic manipulator using steady state visually evoked potentials

An assistive robotic manipulator (ARM) can provide independence and improve the quality of life for patients suffering from tetraplegia. However, to properly control such device to a satisfactory level without any motor functions requires a very high performing brain-computer interface (BCI). Steady-state visual evoked potentials (SSVEP) based BCI are among the best performing. Thus, this study investigates the design of a system for a full workspace control of a 7 degrees of freedom ARM. A SSVEP signal is elicited by observing a visual stimulus flickering at a specific frequency and phase. This study investigates the best combination of unique frequencies and phases to provide a 16-target BCI by testing three different systems off line. Furthermore, a fourth system is developed to investigate the impact of the stimulating monitor refresh rate. Experiments conducted on two subjects suggest that a 16-target BCI created by four unique frequencies and 16-unique phases provide the best performance. Subject 1 reaches a maximum estimated ITR of 235 bits/min while subject 2 reaches 140 bits/min. The findings suggest that the optimal SSVEP stimuli to generate 16 targets are a low number of frequencies and a high number of unique phases. Moreover, the findings do not suggest any need for considering the monitor refresh rate if stimuli are modulated using a sinusoidal signal sampled at the refresh rate.

Low-cost low-tech obstacle pushing/gliding wheelchair accessory

Purpose: Some wheelchair users continue to struggle in maneuvering a wheelchair and navigating through manual doors. Several smart wheelchairs and robotic manipulators were developed to minimize such challenges facing disabled people. Disappointingly, a majority of these high-tech solutions are restricted to laboratories and are not extensively available as commercial products. Previously, a low-tech wheelchair accessory (arc-shaped with many wheels) for pushing doors was modelled and simulated. This work demonstrates the fabrication and testing of the first-generation prototype of the accessory.Materials and methods: The accessory has side portions with a straight arrangement of wheels and a front portion with a straight-arc-straight arrangement of wheels. The accessory was fabricated using conventional manufacturing, off-the-shelf components, and 3D printed ABS fasteners. Stress analysis simulations were done for the fasteners that attach the front accessory to the wheelchair frame. The proof-of-concept of the prototype installed onto a powered wheelchair was tested with a door and an obstacle, each with ∼50 N resistance force.Results: Prototype tests demonstrate the ability of the accessory along with the mechanical robustness of the 3D printed fasteners to push open doors allowing easy navigation through doors and to push/glide against obstacles. The accessory is foldable and detachable.Conclusion: The low-cost of the accessory makes it affordable to many users intending to improve their quality of life. The current study provides an engineering perspective of the accessory, and a clinical perspective is crucial. Other potential applications of the wheelchair accessory include use with scooters, walkers and stretchers.Implications for rehabilitationLow-cost, low-tech accessory is foldable and detachable.Accessory is effective for pushing doors and pushing/gliding against obstacles.Protective nature of the front accessory could prove highly beneficial to some wheelchair users.

The Helping Hand: An Assistive Manipulation Framework Using Augmented Reality and Tongue-Drive Interfaces

A human-in-the-loop system is proposed to enable collaborative manipulation tasks for person with physical disabilities. Studies show that the cognitive burden of subject reduces with increased autonomy of assistive system. Our framework obtains high-level intent from the user to specify manipulation tasks. The system processes sensor input to interpret the user's environment. Augmented reality glasses provide ego-centric visual feedback of the interpretation and summarize robot affordances on a menu. A tongue drive system serves as the input modality for triggering a robotic arm to execute the tasks. Assistance experiments compare the system to Cartesian control and to state-of-the-art approaches. Our system achieves competitive results with faster completion time by simplifying manipulation tasks.

An sEMG-based Interface to give People with Severe Muscular Atrophy control over Assistive Devices

Injuries, accidents, strokes, and other diseases can significantly degrade the capabilities to perform even the most simple activities in daily life. While assistive technology becomes more and more available to the people affected, there is still a big need for user interfaces suitable for people without functional hand movement. A large share of these cases involves neuromuscular diseases, which lead to severely reduced muscle function. However, even though affected people are no longer able to functionally move their limbs, residual muscle function can still be existent. Previous work has shown that this residual muscular activity can suffice, to create an EMG-based user interface, and e.g., allow for control of assistive devices. In this paper, we enhance this user interface with additional EMG-features and an improved training paradigm in order to increase information extraction from recordings of strongly atrophic muscles. The interface was tested and validated by subjects with severe spinal muscular atrophy. Results show that the used methods improve the decoding and thereby allow for a considerable increase in performance when controlling a robotic manipulator in a 3D reaching task.

Evaluation of Control Interfaces for Active Trunk Support

A feasibility study was performed to evaluate the control interfaces for a novel trunk support assistive device (Trunk Drive), namely, joystick, force on sternum, force on feet, and electromyography (EMG) to be used by adult men with Duchene muscular dystrophy. The objective of this paper was to evaluate the performance of the different control interfaces during a discrete position tracking task. We built a one degree of freedom flexion-extension active trunk support device that was tested on 10 healthy men. An experiment, based on the Fitts law, was conducted, whereby subjects were asked to steer a cursor representing the angle of the Trunk Drive into a target that was shown on a graphical user interface, using the above-mentioned control interfaces. The users could operate the Trunk Drive via each of the control interfaces. In general, the joystick and force on sternum were the fastest in movement time (more than 40%) without any significant difference between them, but there was a significant difference between force on sternum on the one hand, and EMG and force on feet on the other. All control interfaces proved to be feasible solutions for controlling an active trunk support, each of which had specific advantages.

Feasibility and safety of shared EEG/EOG and vision-guided autonomous whole-arm exoskeleton control to perform activities of daily living

Arm and finger paralysis, e.g. due to brain stem stroke, often results in the inability to perform activities of daily living (ADLs) such as eating and drinking. Recently, it was shown that a hybrid electroencephalography/electrooculography (EEG/EOG) brain/neural hand exoskeleton can restore hand function to quadriplegics, but it was unknown whether such control paradigm can be also used for fluent, reliable and safe operation of a semi-autonomous whole-arm exoskeleton restoring ADLs. To test this, seven abled-bodied participants (seven right-handed males, mean age 30 ± 8 years) were instructed to use an EEG/EOG-controlled whole-arm exoskeleton attached to their right arm to perform a drinking task comprising multiple sub-tasks (reaching, grasping, drinking, moving back and releasing a cup). Fluent and reliable control was defined as average 'time to initialize' (TTI) execution of each sub-task below 3 s with successful initializations of at least 75% of sub-tasks within 5 s. During use of the system, no undesired side effects were reported. All participants were able to fluently and reliably control the vision-guided autonomous whole-arm exoskeleton (average TTI 2.12 ± 0.78 s across modalities with 75% successful initializations reached at 1.9 s for EOG and 4.1 s for EEG control) paving the way for restoring ADLs in severe arm and hand paralysis.

Long-term use of the JACO robotic arm: a case series

PURPOSE

Past research with JACO has principally focused on the short-term impacts on new users. Therefore, this study aims to document the long-term impacts of this assistive device on users and their family caregivers following prolonged use.

METHODS

Users' characteristics, caregivers' characteristics and expenses related to JACO were documented with questionnaires designed for this study. Upper extremity performance was measured with an adaptation of an upper extremity performance test, the TEMPA, and accomplishment of life habits was documented in an interview based on the LIFE-H questionnaire. Satisfaction with JACO and psychosocial impacts of its use were measured with validated questionnaires, namely the QUEST and the PIADS-10. Impacts of JACO on family caregivers were documented with a validated questionnaire, the CATOM. Descriptive statistics were used to report the results.

RESULTS

Seven users and five caregivers were recruited. One user had expenses related to JACO in the past two months. Users had a better upper extremity performance with JACO than without it and they used their robotic arm to accomplish certain life habits. Most users were satisfied with JACO and the psychosocial impacts were positive. Impacts on family caregivers were slight.

CONCLUSIONS

JACO increased performance in manipulation and facilitated the accomplishment of certain life habits. Users' increased participation in their life habits may slightly decrease the amount of caregiver assistance required. Future studies are needed to clarify its economic potential, its impact on caregivers' burden, including paid caregivers, and the variability in the tasks performed using JACO. Implications for Rehabilitation The use of JACO may have positive impacts on its users in terms of upper extremity performance, accomplishment of life habits, satisfaction with the device and psychosocial impacts. More research is needed to quantify more accurately the economic potential of the long-term use of JACO, to explore the factors related to the variability in the tasks performed using JACO, and to clarify the impact of JACO on caregivers' burden, including paid caregivers.

Performance Evaluation of a Mobile Touchscreen Interface for Assistive Robotic Manipulators: A Pilot Study

Background: Assistive robotic manipulators (ARMs) have been developed to provide enhanced assistance and independence in performance of daily activities among people with spinal cord injury when a caregiver is not on site. However, the current commercial ARM user interfaces (UIs) may be difficult to learn and control. A touchscreen mobile UI was developed to overcome these challenges. Objective: The object of this study was to evaluate the performance between 2 ARM UIs, touchscreen and the original joystick, using an ARM evaluation tool (ARMET). Methods: This is a pilot study of people with upper extremity impairments (N = 8). Participants were trained on 2 UIs, and then they chose one to use when performing 3 tasks on the ARMET: flipping a toggle switch, pushing down a door handle, and turning a knob. Task completion time, mean velocity, and open interviews were the main outcome measurements. Results: Among 8 novice participants, 7 chose the touchscreen UI and 1 chose the joystick UI. All participants could complete the ARMET tasks independently. Use of the touchscreen UI resulted in enhanced ARMET performance (higher mean moving speed and faster task completion). Conclusions: Mobile ARM UIs demonstrated easier learning experience, less physical effort, and better ARMET performance. The improved performance, the accessibility, and lower physical effort suggested that the touchscreen UI might be an efficient tool for the ARM users.

Wireless intraoral tongue control of an assistive robotic arm for individuals with tetraplegia

BACKGROUND

For an individual with tetraplegia assistive robotic arms provide a potentially invaluable opportunity for rehabilitation. However, there is a lack of available control methods to allow these individuals to fully control the assistive arms.

METHODS

Here we show that it is possible for an individual with tetraplegia to use the tongue to fully control all 14 movements of an assistive robotic arm in a three dimensional space using a wireless intraoral control system, thus allowing for numerous activities of daily living. We developed a tongue-based robotic control method incorporating a multi-sensor inductive tongue interface. One abled-bodied individual and one individual with tetraplegia performed a proof of concept study by controlling the robot with their tongue using direct actuator control and endpoint control, respectively.

RESULTS

After 30 min of training, the able-bodied experimental participant tongue controlled the assistive robot to pick up a roll of tape in 80% of the attempts. Further, the individual with tetraplegia succeeded in fully tongue controlling the assistive robot to reach for and touch a roll of tape in 100% of the attempts and to pick up the roll in 50% of the attempts. Furthermore, she controlled the robot to grasp a bottle of water and pour its contents into a cup; her first functional action in 19 years.

CONCLUSION

To our knowledge, this is the first time that an individual with tetraplegia has been able to fully control an assistive robotic arm using a wireless intraoral tongue interface. The tongue interface used to control the robot is currently available for control of computers and of powered wheelchairs, and the robot employed in this study is also commercially available. Therefore, the presented results may translate into available solutions within reasonable time.

BRIDGE - Behavioural reaching interfaces during daily antigravity activities through upper limb exoskeleton: Preliminary results

People with neuromuscular diseases such as muscular dystrophy experience a distributed and evolutive weakness in the whole body. Recent technological developments have changed the daily life of disabled people strongly improving the perceived quality of life, mostly concentrating on powered wheelchairs, so to assure autonomous mobility and respiratory assistance, essential for survival. The key concept of the BRIDGE project is to contrast the everyday experience of losing functions by providing them of a system able to exploit the best their own residual capabilities in arm movements so to keep them functional and autonomous as much as possible. BRIDGE is composed by a light, wearable and powered five degrees of freedom upper limb exoskeleton under the direct control of the user through a joystick or gaze control. An inverse kinematic model allows to determine joints position so to track patient desired hand position. BRIDGE prototype has been successfully tested in simulation environment, and by a small group of healthy volunteers. Preliminary results show a good tracking performance of the implemented control scheme. The interaction procedure was easy to understand, and the interaction with the system was successful.

Design of a wearable interface for lightweight robotic arm for people with mobility impairments

Many common activities of daily living like open a door or fill a glass of water, which most of us take for granted, could be an insuperable problem for people who have limited mobility or impairments. For years the unique alternative to overcame this limitation was asking for human help. Nowadays thanks to recent studies and technology developments, having an assistive devices to compensate the loss of mobility is becoming a real opportunity. Off-the-shelf assistive robotic manipulators have the capability to improve the life of people with motor impairments. Robotic lightweight arms represent one of the most spread solution, in particular some of them are designed specifically to be mounted on wheelchairs to assist users in performing manipulation tasks. On the other hand, usually their control interface relies on joystick and buttons, making the use very challenging for people affected by impaired motor abilities. In this paper, we present a novel wearable control interface for users with limb mobility impairments. We make use of muscles residual motion capabilities, captured through a Body-Machine Interface based on a combination of head tilt estimation and electromyography signals. The proposed BMI is completely wearable, wireless and does not require frequently long calibrations. Preliminary experiments showed the effectiveness of the proposed system for subjects with motor impairments, allowing them to easily control a robotic arm for activities of daily living.

Comparison of tongue interface with keyboard for control of an assistive robotic arm

This paper demonstrates how an assistive 6 DoF robotic arm with a gripper can be controlled manually using a tongue interface. The proposed method suggests that it possible for a user to manipulate the surroundings with his or her tongue using the inductive tongue control system as deployed in this study. The sensors of an inductive tongue-computer interface were mapped to the Cartesian control of an assistive robotic arm. The resulting control system was tested manually in order to compare manual control of the robot using a standard keyboard and using the tongue interface. Two healthy subjects controlled the robotic arm to precisely move a bottle of water from one location to another. The results shows that the tongue interface was able to fully control the robotic arm in a similar manner as the standard keyboard resulting in the same number of successful manipulations and an average increase in task duration of up to 30% as compared with the standard keyboard.

Intuitive adaptive orientation control of assistive robots for people living with upper limb disabilities

Robotic assistive devices enhance the autonomy of individuals living with physical disabilities in their day-to-day life. Although the first priority for such devices is safety, they must also be intuitive and efficient from an engineering point of view in order to be adopted by a broad range of users. This is especially true for assistive robotic arms, as they are used for the complex control tasks of daily living. One challenge in the control of such assistive robots is the management of the end-effector orientation which is not always intuitive for the human operator, especially for neophytes. This paper presents a novel orientation control algorithm designed for robotic arms in the context of human-robot interaction. This work aims at making the control of the robot's orientation easier and more intuitive for the user, in particular, individuals living with upper limb disabilities. The performance and intuitiveness of the proposed orientation control algorithm is assessed through two experiments with 25 able-bodied subjects and shown to significantly improve on both aspects.

Comparative performance analysis of M-IMU/EMG and voice user interfaces for assistive robots

People with a high level of disability experience great difficulties to perform activities of daily living and resort to their residual motor functions in order to operate assistive devices. The commercially available interfaces used to control assistive manipulators are typically based on joysticks and can be used only by subjects with upper-limb residual mobilities. Many other solutions can be found in the literature, based on the use of multiple sensory systems for detecting the human motion intention and state. Some of them require a high cognitive workload for the user. Some others are more intuitive and easy to use but have not been widely investigated in terms of usability and user acceptance. The objective of this work is to propose an intuitive and robust user interface for assistive robots, not obtrusive for the user and easily adaptable for subjects with different levels of disability. The proposed user interface is based on the combination of M-IMU and EMG for the continuous control of an arm-hand robotic system by means of M-IMUs. The system has been experimentally validated and compared to a standard voice interface. Sixteen healthy subjects volunteered to participate in the study: 8 subjects used the combined M-IMU/EMG robot control, and 8 subjects used the voice control. The arm-hand robotic system made of the KUKA LWR 4+ and the IH2 Azzurra hand was controlled to accomplish the daily living task of drinking. Performance indices and evaluation scales were adopted to assess performance of the two interfaces.

Modeling and Simulation of Two Wheelchair Accessories for Pushing Doors

Independent mobility is vital to individuals of all ages, and wheelchairs have proven to be great personal mobility devices. The tasks of opening and navigating through a door are trivial for healthy people, while the same tasks could be difficult for some wheelchair users. A wide range of intelligent wheelchair controllers and systems, robotic arms, or manipulator attachments integrated with wheelchairs have been developed for various applications, including manipulating door knobs. Unfortunately, the intelligent wheelchairs and robotic attachments are not widely available as commercial products. Therefore, the current manuscript presents the modeling and simulation of a novel but simple technology in the form of a passive wheelchair accessory (straight, arm-like with a single wheel, and arc-shaped with multiple wheels) for pushing doors open from a wheelchair. From the simulations using different wheel shapes and sizes, it was found that the arc-shaped accessory could push open the doors faster and with almost half the required force as compared to the arm-like accessory. Also, smaller spherical wheels were found to be best in terms of reaction forces on the wheels. Prototypes based on the arc-shaped accessory design will be manufactured and evaluated for pushing doors open and dodging or gliding other obstacles.

Performance evaluation of 3D vision-based semi-autonomous control method for assistive robotic manipulator

We developed a 3D vision-based semi-autonomous control interface for assistive robotic manipulators. It was implemented based on one of the most popular commercially available assistive robotic manipulator combined with a low-cost depth-sensing camera mounted on the robot base. To perform a manipulation task with the 3D vision-based semi-autonomous control interface, a user starts operating with a manual control method available to him/her. When detecting objects within a set range, the control interface automatically stops the robot, and provides the user with possible manipulation options through audible text output, based on the detected object characteristics. Then, the system waits until the user states a voice command. Once the user command is given, the control interface drives the robot autonomously until the given command is completed. In the empirical evaluations conducted with human subjects from two different groups, it was shown that the semi-autonomous control can be used as an alternative control method to enable individuals with impaired motor control to more efficiently operate the robot arms by facilitating their fine motion control. The advantage of semi-autonomous control was not so obvious for the simple tasks. But, for the relatively complex real-life tasks, the 3D vision-based semi-autonomous control showed significantly faster performance. Implications for Rehabilitation A 3D vision-based semi-autonomous control interface will improve clinical practice by providing an alternative control method that is less demanding physically as well cognitively. A 3D vision-based semi-autonomous control provides the user with task specific intelligent semiautonomous manipulation assistances. A 3D vision-based semi-autonomous control gives the user the feeling that he or she is still in control at any moment. A 3D vision-based semi-autonomous control is compatible with different types of new and existing manual control methods for ARMs.

Performing Complex Tasks by Users With Upper-Extremity Disabilities Using a 6-DOF Robotic Arm: A Study

In this paper, we report on the results of a study that was conducted to examine how users suffering from severe upper-extremity disabilities can control a 6 degrees-of-freedom (DOF) robotics arm to complete complex activities of daily living. The focus of the study is not on assessing the robot arm but on examining the human-robot interaction patterns. Three participants were recruited. Each participant was asked to perform three tasks: eating three pieces of pre-cut bread from a plate, drinking three sips of soup from a bowl, and opening a right-handed door with lever handle. Each of these tasks was repeated three times. The arm was mounted on the participant's wheelchair, and the participants were free to move the arm as they wish to complete these tasks. Each task consisted of a sequence of modes where a mode is defined as arm movement in one DOF. Results show that participants used a total of 938 mode movements with an average of 75.5 (std 10.2) modes for the eating task, 70 (std 8.8) modes for the soup task, and 18.7 (std 4.5) modes for the door opening task. Tasks were then segmented into smaller subtasks. It was found that there are patterns of usage per participant and per subtask. These patterns can potentially allow a robot to learn from user's demonstration what is the task being executed and by whom and respond accordingly to reduce user effort.

Vibrotactile feedback for conveying object shape information as perceived by artificial sensing of robotic arm

This work is a preliminary study towards developing an alternative communication channel for conveying shape information to aid in recognition of items when tactile perception is hindered. Tactile data, acquired during object exploration by sensor fitted robot arm, are processed to recognize four basic geometric shapes. Patterns representing each shape, classified from tactile data, are generated using micro-controller-driven vibration motors which vibrotactually stimulate users to convey the particular shape information. These motors are attached on the subject's arm and their psychological (verbal) responses are recorded to assess the competence of the system to convey shape information to the user in form of vibrotactile stimulations. Object shapes are classified from tactile data with an average accuracy of 95.21 %. Three successive sessions of shape recognition from vibrotactile pattern depicted learning of the stimulus from subjects' psychological response which increased from 75 to 95 %. This observation substantiates the learning of vibrotactile stimulation in user over the sessions which in turn increase the system efficacy. The tactile sensing module and vibrotactile pattern generating module are integrated to complete the system whose operation is analysed in real-time. Thus, the work demonstrates a successful implementation of the complete schema of artificial tactile sensing system for object-shape recognition through vibrotactile stimulations.

Bypassing the Natural Visual-Motor Pathway to Execute Complex Movement Related Tasks Using Interval Type-2 Fuzzy Sets

In visual-motor coordination, the human brain processes visual stimuli representative of complex motion-related tasks at the occipital lobe to generate the necessary neuronal signals for the parietal and pre-frontal lobes, which in turn generates movement related plans to excite the motor cortex to execute the actual tasks. The paper introduces a novel approach to provide rehabilitative support to patients suffering from neurological damage in their pre-frontal, parietal and/or motor cortex regions. An attempt to bypass the natural visual-motor pathway is undertaken using interval type-2 fuzzy sets to generate the approximate EEG response of the damaged pre-frontal/parietal/motor cortex from the occipital EEG signals. The approximate EEG response is used to trigger a pre-trained joint coordinate generator to obtain the desired joint coordinates of the link end-points of a robot imitating the human subject. The robot arm is here employed as a rehabilitative aid in order to move each link end-points to the desired locations in the reference coordinate system by appropriately activating its links using the well-known inverse kinematics approach. The mean-square positional errors obtained for each link end-points is found within acceptable limits for all experimental subjects including subjects with partial parietal damage, indicating a possible impact of the proposed approach in rehabilitative robotics. Subjective variation in EEG features over different sessions of experimental trials is modeled here using interval type-2 fuzzy sets for its inherent power to handle uncertainty. Experiments undertaken confirm that interval type-2 fuzzy realization outperforms its classical type-1 counterpart and back-propagation neural approaches in all experimental cases, considering link positional error as a metric. The proposed research offers a new opening for the development of possible rehabilitative aids for people with partial impairment in visual-motor coordination.

Assistive Teleoperation of Robot Arms via Automatic Time-Optimal Mode Switching

Assistive robotic arms are increasingly enabling users with upper extremity disabilities to perform activities of daily living on their own. However, the increased capability and dexterity of the arms also makes them harder to control with simple, low-dimensional interfaces like joysticks and sip-and-puff interfaces. A common technique to control a high-dimensional system like an arm with a low-dimensional input like a joystick is through switching between multiple control modes. However, our interviews with daily users of the Kinova JACO arm identified mode switching as a key problem, both in terms of time and cognitive load. We further confirmed objectively that mode switching consumes about 17.4% of execution time even for able-bodied users controlling the JACO. Our key insight is that using even a simple model of mode switching, like time optimality, and a simple intervention, like automatically switching modes, significantly improves user satisfaction.

Texture- and deformability-based surface recognition by tactile image analysis

Deformability and texture are two unique object characteristics which are essential for appropriate surface recognition by tactile exploration. Tactile sensation is required to be incorporated in artificial arms for rehabilitative and other human-computer interface applications to achieve efficient and human-like manoeuvring. To accomplish the same, surface recognition by tactile data analysis is one of the prerequisites. The aim of this work is to develop effective technique for identification of various surfaces based on deformability and texture by analysing tactile images which are obtained during dynamic exploration of the item by artificial arms whose gripper is fitted with tactile sensors. Tactile data have been acquired, while human beings as well as a robot hand fitted with tactile sensors explored the objects. The tactile images are pre-processed, and relevant features are extracted from the tactile images. These features are provided as input to the variants of support vector machine (SVM), linear discriminant analysis and k-nearest neighbour (kNN) for classification. Based on deformability, six household surfaces are recognized from their corresponding tactile images. Moreover, based on texture five surfaces of daily use are classified. The method adopted in the former two cases has also been applied for deformability- and texture-based recognition of four biomembranes, i.e. membranes prepared from biomaterials which can be used for various applications such as drug delivery and implants. Linear SVM performed best for recognizing surface deformability with an accuracy of 83 % in 82.60 ms, whereas kNN classifier recognizes surfaces of daily use having different textures with an accuracy of 89 % in 54.25 ms and SVM with radial basis function kernel recognizes biomembranes with an accuracy of 78 % in 53.35 ms. The classifiers are observed to generalize well on the unseen test datasets with very high performance to achieve efficient material recognition based on its deformability and texture.