A Novel Master-Slave Interventional Surgery Robot with Force Feedback and Collaborative Operation

In recent years, master-slave vascular robots have been developed to address the problem of radiation exposure during vascular interventions for surgeons. However, the single visual feedback reduces surgeon immersion and transparency of the system. In this work, we have developed a haptic interface based on the magnetorheological fluid (MRF) on the master side. The haptic interface can provide passive feedback force with high force fidelity and low inertia. Additionally, the manipulation of the master device does not change the operating posture of traditional surgery, which allows the surgeon to better adapt to the robotic system. For the slave robot, the catheter and guidewire can be navigated simultaneously which allows the two degrees of action on the catheter and axial action of a guidewire. The resistance force of the catheter navigation is measured and reflected to the user through the master haptic interface. To verify the proposed master-slave robotic system, the evaluation experiments are carried out in vitro, and the effectiveness of the system was demonstrated experimentally.

PREDICTOR: A Physical emulatoR enabling safEty anD ergonomICs evaluation and Training of physical human-rObot collaboRation

Safety and ergonomics of Physical Human-Robot Collaboration (PHRC) are crucial to make human-robot collaborative systems trustworthy and make a significant impact in real-world applications. One big obstacle to the development of relevant research is the lack of a general platform for evaluating the safety and ergonomics of proposed PHRC systems. This paper aims to create a Physical emulatoR enabling safEty anD ergonomICs evaluation and Training of physical human-rObot collaboRation (PREDICTOR). PREDICTOR consists of a dual-arm robot system and a VR headset as its hardware and contains physical simulation, haptic rendering and visual rendering modules as its software. The dual-arm robot system is used as an integrated admittance-type haptic device, which senses the force/torque applied by a human operator as an input to drive the simulation of a PHRC system and constrains the handles' motion to match their virtual counterparts in the simulation. The motion of the PHRC system in the simulation is fed back to the operator through the VR headset. PREDICTOR combines haptics and VR to emulate PHRC tasks in a safe environment since the interactive forces are monitored to avoid any risky events. PREDICTOR also brings flexibility as different PHRC tasks can be easily set up by changing the PHRC system model and the robot controller in the simulation. The effectiveness and performance of PREDICTOR were evaluated by experiments.

Potential Haptic Perceptual Dimensionality of Rendered Compliance

Studies have proven that humans perceive haptic textures through different perceptual dimensions, such as rough/smooth and soft/hard, which provide useful guidance in the design of haptic devices. However, few of these have focused on the perception of compliance, which is another important perceptual property in haptic interfaces. This research was conducted to investigate the potential basic perceptual dimensions of the rendered compliance and quantify the effects of the simulation parameters. Two perceptual experiments were designed based on 27 stimuli samples generated by a 3-DOF haptic feedback device. Subjects were asked to describe these stimuli using adjectives, classify the samples, and rate them according to corresponding adjective labels. Multi-dimensional scaling (MDS) methods were then used to project adjective ratings into 2D and 3D perception spaces. According to the results, hardness and viscosity are considered two basic perceptual dimensions of the rendered compliance, while crispness can be regarded as a subsidiary perceptual dimension. Then, the relations between simulation parameters and perceptual feelings were analyzed by the regression analysis. This paper may provide a better understanding of the compliance perception mechanism and useful guidance for the improvement of rendering algorithms and devices for haptic human-computer interaction.

TactCube: An Intelligent Device to 'converse' with Smart Environments

Ambient Intelligence is a vision of daily life in which intelligent devices interact with humans to make their lives easier, and the technology is invisible. Artificial Intelligence (AI) governs this smart environment and must interact with humans to best meet their needs and demands. Although voice assistants are very popular and efficient as conversational AI, under some conditions they cannot be used. Therefore, this work proposed a complementary tactile and tangible interface to converse with AI, creating new Tactile Signs. A prototype of TactCube, a wireless cube-shaped device that can interact with AI using only the tactile sense, is presented. The hypothesis is that TactCube can be manipulated with one hand and generate a sequence of numbers that can be interpreted as a new tactile language by a neural network solution. The paper describes the initial research made to define how these sequences can be generated and demonstrates how TactCube is able to do it.

Perception by Palpation: Development and Testing of a Haptic Ferrogranular Jamming Surface

Tactile hands-only training is particularly important for medical palpation. Generally, equipment for palpation training is expensive, static, or provides too few study cases to practice on. We have therefore developed a novel haptic surface concept for palpation training, using ferrogranular jamming. The concept’s design consists of a tactile field spanning 260 x 160 mm, and uses ferromagnetic granules to alter shape, position, and hardness of palpable irregularities. Granules are enclosed in a compliant vacuum-sealed chamber connected to a pneumatic system. A variety of geometric shapes (output) can be obtained by manipulating and arranging granules with permanent magnets. The tactile hardness of the palpable output can be controlled by adjusting the chamber’s vacuum level. A psychophysical experiment (N = 28) investigated how people interact with the palpable surface and evaluated the proposed concept. Untrained participants characterized irregularities with different position, form, and hardness through palpation, and their performance was evaluated. A baseline (no irregularity) was compared to three irregularity conditions: two circular shapes with different hardness (Hard Lump and Soft Lump), and an Annulus shape. 100% of participants correctly identified an irregularity in the three irregularity conditions, whereas 78.6% correctly identified baseline. Overall agreement between participants was high (κ= 0.723). The Intersection over Union (IoU) for participants sketched outline over the actual shape was IoU Mdn = 79.3% for Soft Lump, IoU Mdn = 68.8% for Annulus, and IoU Mdn = 76.7% for Hard Lump. The distance from actual to drawn center was Mdn = 6.4 mm (Soft Lump), Mdn = 5.3 mm (Annulus), and Mdn = 7.4 mm (Hard Lump), which are small distances compared to the size of the field. The participants subjectively evaluated Soft Lump to be significantly softer than Hard Lump and Annulus. Moreover, 71% of participants thought they improved their palpation skills throughout the experiment. Together, these results show that the concept can render irregularities with different position, form, and hardness, and that users are able to locate and characterize these through palpation. Participants experienced an improvement in palpation skills throughout the experiment, which indicates the concepts feasibility as a palpation training device.

Evaluation of Physical Interaction during Walker-Assisted Gait with the AGoRA Walker: Strategies Based on Virtual Mechanical Stiffness

Smart walkers are commonly used as potential gait assistance devices, to provide physical and cognitive assistance within rehabilitation and clinical scenarios. To understand such rehabilitation processes, several biomechanical studies have been conducted to assess human gait with passive and active walkers. Several sessions were conducted with 11 healthy volunteers to assess three interaction strategies based on passive, low and high mechanical stiffness values on the AGoRA Smart Walker. The trials were carried out in a motion analysis laboratory. Kinematic data were also collected from the smart walker sensory interface. The interaction force between users and the device was recorded. The force required under passive and low stiffness modes was 56.66% and 67.48% smaller than the high stiffness mode, respectively. An increase of 17.03% for the hip range of motion, as well as the highest trunk’s inclination, were obtained under the resistive mode, suggesting a compensating motion to exert a higher impulse force on the device. Kinematic and physical interaction data suggested that the high stiffness mode significantly affected the users’ gait pattern. Results suggested that users compensated their kinematics, tilting their trunk and lower limbs to exert higher impulse forces on the device.

Haptic Glove Using Tendon-Driven Soft Robotic Mechanism

Recent advancements in virtual reality and augmented reality call for light-weight and compliant haptic interfaces to maximize the task-performance interactivity with the virtual or extended environment. Noting this, we propose a haptic glove using a tendon-driven compliant robotic mechanism. Our proposed interface can provide haptic feedback to two fingers of a user, an index finger and a thumb. It can provide both cutaneous and kinesthetic feedback to the fingers by using the tendon-driven system. Each actuator is paired with a force sensor to exert the desired tension accurately. In order to optimize the perception of the kinesthetic feedback, we propose a perception-based kinesthetic feedback distribution strategy. We experimentally measured the force perception weight for peripheral interphalangeal (PIP) and metacarpophalangeal (MCP) joints. We observed no significant difference in the force perception between the two joints. Then, based on the obtained weights, our proposed force distribution method calculates the force for each joint. We also evaluated the effect of additional cutaneous feedback to the kinesthetic feedback, on the force perception at the fingertip. The experimental result has shown that additional cutaneous feedback has significantly increased the sensitivity of the human perception. Finally, we evaluated our proposed system and force distribution algorithm by conducting a human subject test. The experimental result indicates that the availability of the cutaneous feedback significantly improved the perceived realism and acuity of the contact force.

Periodic Kinesthetic Guidance Cannot Expedite Learning Surgical Skills

Introduction. Connecting multiple haptic devices in a master-slave fashion enables us to deliver kinesthetic (haptic) feedback from 1 person to another. This study examined whether inter-user feedback delivered from an expert to a novice would facilitate skill acquisition of the novice in learning laparoscopic surgery and expedite it compared to traditional methods. Methods. We recruited fourteen novices and divided them into 1 of 2 training groups with 6 half-hour training sessions. The task was precision cutting adopted from one of the tasks listed in Fundamentals of Laparoscopic Surgery using laparoscopic instruments. In the haptic feedback group (haptic), 8 subjects had the chance to passively feel an expert's performance before they started to practice in each training session. In the self-learning group (control), 6 subjects watched a video before practicing. Each session was video recorded, and task performance was measured by task completion time, number of grasper adjustments, and instrument crossings. Cutting accuracy, defined as the percentage of deviation of the cutting line from the predefined line, was analyzed via computer analysis. Results. Results show no significant difference among performance measures between the 2 groups. Participants performed similarly when practicing alone or with periodic haptic feedback. Discussion. Further research will be needed for improving our way of integrating between-person haptic feedback with skills training protocol.

A Theoretical Framework of Haptic Processing in Automotive User Interfaces and Its Implications on Design and Engineering

Driving a car is a highly visual task. Despite the trend towards increased driver assistance and autonomous vehicles, drivers still need to interact with the car for both driving and non-driving relevant tasks, at times simultaneously. The often-resulting high cognitive load is a safety issue, which can be addressed by providing the driver with alternative feedback modalities, such as haptics. Recent trends in the automotive industry are moving towards the seamless integration of control elements through touch-sensitive surfaces. Psychological knowledge on optimally utilizing haptic technologies remains limited. The literature on automotive haptic feedback consists mainly of singular findings without putting them into a broader user context with respect to haptic design of interfaces. Moreover, haptic feedback has primarily been limited to the confirmation of control actions rather than the searching or finding of control elements, the latter of which becomes particularly important considering the current trends. This paper presents an integrated framework on haptic processing in automotive user interfaces and provides guidelines for haptic design of user interfaces in car interiors.

Magnetorheological Fluids Actuated Haptic-Based Teleoperated Catheter Operating System

During conventional catheter endovascular procedures, surgeons needs to adjust the catheter intervention moving direction and velocity according to the direct sensation. Moreover, in the conventional method, both the surgeon and the patient are inevitable exposed to a large amount of, and for a long period of time, X-ray radiation during the surgical procedure. The purpose of this paper is to ensure surgical safety and to protect the surgeon from X-ray radiation during the surgical procedure by adopting a novel haptic-based robot-assisted master-slave system mode. In this paper, a kind of magnetorheological fluids (MR fluids)-based haptic interface has been developed to generate a kind of controllable haptic sensation providing to the catheter operator, and the catheter intervention kinematics parameters measured the motion capture part to control the salve robotic catheter operating system following the master side kinematics. The slave catheter operating the mechanical system has also been designed and manufactured to manipulate the clinical catheter by mimicking the surgeon operating the catheter intervention surgical procedure, which has a 2-DOF (advance, retreat, and rotate) catheter motion characteristic; in addition, the interaction force between the catheter and inner wall of vasculature can be measured by its force sensing unit and the feedback to the master system. The catheter intervention synchronous evaluation experiments between the master and slave system are tested. Also, the advantages of integrating the controllable haptic sensation to the master-slave system experimental evaluations have been done in vitro. The experimental results demonstrated that the proposed haptic-based robot-assisted master-slave system mode can reduce the surgical time and protect the surgeon from X-ray radiation.

An Electroactive and Transparent Haptic Interface Utilizing Soft Elastomer Actuators with Silver Nanowire Electrodes

An electroactive and transparent haptic interface having a rectangular void pattern creates tunable surface textures by controlling the wavelength and amplitude of independent void-lines. To make an active tactile surface, the transparent haptic interface employs a silver nanowire (AgNW) electrode to be compliant with the deformed elastomer surface. Here, the dielectric elastomer is newly blended with polydimethylsiloxane and Ecoflex prepolymer to simultaneously control the mechanical stiffness and transparency. The relative resistance of the AgNW electrode on a single void line is nearly unchanged under bending test, confirming the high stretchability and conductivity of the nanowire-networked electrode. The optical transparencies are 92-85%, depending on the ratio of the Ecoflex solution. Transparency values decreas by 7 and 16% after coating with AgNWs at densities of 30 and 140 mg m^-2 , respectively. Using EP31, the void line is deformed by 90 µm under a field intensity of 13.0 V µm^-1 . The haptic surface is successfully controlled by applying voltage, which produces four different surface textures, from relatively smooth to rough feeling, depending on the distance between deformed void lines. This haptic interface can be applied to diverse display systems as an external add-on screen and will help to realize programmable surface textures in the future.

Haptics to improve task performance in people with disabilities: A review of previous studies and a guide to future research with children with disabilities

This review examines the studies most pertinent to the potential of haptics on the functionality of assistive robots in manipulation tasks for use by children with disabilities. Haptics is the fast-emerging science that studies the sense of touch concerning the interaction of a human and his/her environment; this paper particularly studies the human–machine interaction that happens through a haptic interface to enable touch feedback. Haptics-enabled user interfaces for assistive robots can potentially benefit children whose haptic exploration is impaired due to a disability in their infancy and throughout their childhood. A haptic interface can provide touch feedback and potentially contribute to an enhancement in perception of objects and overall ability to perform manipulation tasks. The intention of this paper is to review the research on the applications of haptics, exclusively focusing on attributes affecting task performance. A review of studies will give a retrospective insight into previous research with various disability populations, and inform potential limitations/challenges in research regarding haptic interfaces for assistive robots for use by children with disabilities.

Graphical message transmission using the monotonic vibration function of a smart phone

This study aimed at evaluating four vibrotactile capabilities for perceiving graphical information presented on a smart phone. Thirty-two blindfolded college students participated in four experiments to test their capabilities of two-point discrimination, relative and absolute judgments of line thickness, and recognition of basic shapes. All the information was received through the default vibration function of the phone, sensed by their scanning fingers. The results showed a good two-point discrimination accuracy rate, reaching 98.8% when the distance between two points was set at 3.2 mm; the relative judgment of line thickness reached the level of 78.3% accuracy when the two-line width difference ratio was set at 3%; the absolute judgment reached the level of 78.8% when the participants recognised line thickness from one of two. Overall, especially for the shapes judgment, the information transmitted by the various codes may be quite low. These findings should inspire advanced investigations and provide design guidelines.

PRACTITIONER SUMMARY

This study tested four vibrotactile capabilities for perception of graphical information when solely using the monotonic vibration function of a smart phone. The results show low information transmission. These findings encourage advanced investigations of new coding systems so that relevant mobile applications could be developed to help the visually impaired.

Role of combined tactile and kinesthetic feedback in minimally invasive surgery

BACKGROUND

Haptic feedback is of critical importance in surgical tasks. However, conventional surgical robots do not provide haptic feedback to surgeons during surgery. Thus, in this study, a combined tactile and kinesthetic feedback system was developed to provide haptic feedback to surgeons during robotic surgery.

METHODS

To assess haptic feasibility, the effects of two types of haptic feedback were examined empirically - kinesthetic and tactile feedback - to measure object-pulling force with a telesurgery robotics system at two desired pulling forces (1 N and 2 N). Participants answered a set of questionnaires after experiments.

RESULTS

The experimental results reveal reductions in force error (39.1% and 40.9%) when using haptic feedback during 1 N and 2 N pulling tasks. Moreover, survey analyses show the effectiveness of the haptic feedback during teleoperation.

CONCLUSIONS

The combined tactile and kinesthetic feedback of the master device in robotic surgery improves the surgeon's ability to control the interaction force applied to the tissue. Copyright © 2014 John Wiley & Sons, Ltd.