Force-feedback grasper helps restore sense of touch in minimally invasive surgery

Haptics in Teleoperated Medical Interventions: Force Measurement, Haptic Interfaces and Their Influence on User's Performance

OBJECTIVES

Haptics in teleoperated medical interventions enables measurement and transfer of force information to the operator during robot-environment interaction. This paper provides an overview of the current research in this domain and guidelines for future investigations.

METHODS

We review current technologies in force measurement and haptic devices as well as their experimental evaluation and influence on user's performance.

RESULTS

Force sensing is moving away from the conventional proximal measurement methods to distal sensing and contact-less methods. Wearable devices that deliver haptic feedback on different body parts are increasingly playing an important role. Performance and accuracy improvement are the widely reported benefits of haptic feedback, while there is a debate on its effect on task completion time and exerted force.

CONCLUSION

With the surge of new ideas, there is a need for better and more systematic validation of the new sensing and feedback technology, through better user studies and novel methods like validated benchmarks and new taxonomies.

SIGNIFICANCE

This review investigates haptics from sensing to interfaces within the context of user's performance and the validation procedures to highlight salient advances. It provides guidelines to future developments and highlights the shortcomings in the field.

Freeform fabrication of tissue-simulating phantom for potential use of surgical planning in conjoined twins separation surgery

Preoperative assessment of tissue anatomy and accurate surgical planning is crucial in conjoined twin separation surgery. We developed a new method that combines three-dimensional (3D) printing, assembling, and casting to produce anatomic models of high fidelity for surgical planning. The related anatomic features of the conjoined twins were captured by computed tomography (CT), classified as five organ groups, and reconstructed as five computer models. Among these organ groups, the skeleton was produced by fused deposition modeling (FDM) using acrylonitrile-butadiene-styrene. For the other four organ groups, shell molds were prepared by FDM and cast with silica gel to simulate soft tissues, with contrast enhancement pigments added to simulate different CT and visual contrasts. The produced models were assembled, positioned firmly within a 3D printed shell mold simulating the skin boundary, and cast with transparent silica gel. The produced phantom was subject to further CT scan in comparison with that of the patient data for fidelity evaluation. Further data analysis showed that the produced model reassembled the geometric features of the original CT data with an overall mean deviation of less than 2 mm, indicating the clinical potential to use this method for surgical planning in conjoined twin separation surgery.

Finite element analysis for evaluating liver tissue damage due to mechanical compression

The development of robotic-assisted minimally invasive surgery (RMIS) has resulted in increased research to improve surgeon training, proficiency and patient safety. Minimizing tissue damage is an essential consideration in RMIS. Various studies have reported the quantified tissue damage resulting from mechanical compression; however, most of them require bench work analysis, which limits their application in clinical conditions of RMIS. We present a new methodology based on nonlinear finite element (FE) analysis that can predict damage degree inside tissue. The effects of the boundary conditions and material property of the FE model on the simulated von Mises stress value and tissue damage were investigated. Four FE models were analyzed: two-dimensional (2D) plane strain model, 2D plane stress model, full three-dimensional (3D) model, and 3D thin membrane model. Nonlinear material properties of liver tissue used in the FEA were derived from previously reported in vivo and in vitro experiments. Our study showed that for integrated von Mises stress and tissue damage computations, the 3D thin membrane model yielded results closest to the full 3D analysis and required only 0.2% of the compute time. The results from 3D thin membrane and the full 3D models fell below plane-strain model and above the plane-stress model. Both stress and necrosis distributions were impacted by the material property of FE models. This study can guide engineers to design surgical instruments to improve patient safety. Additionally it is useful for improving the surgical simulator performance by reflecting more realistic tissue material property and displaying tissue damage severity.

Comparison of Force Matching Performance in Conventional and Laparoscopic Force-Based Task

Laparoscopic instruments have limited haptics feedback. Hence, novices tend to exert excessive force which leads to tissue trauma. In laparoscopic surgery, no external information is available on the magnitude of excessive force. Therefore, novices should be trained to accurately perceive their own force output. This study analyzed the force perception of 18 novices in the absence of external information, by comparing the isometric force matching performance of index finger (i.e. used in conventional procedures) in extended arm posture with that of laparoscopic instrument in a force-based probing task. The study also examined the effect of handedness on force perception. A contra-lateral force matching paradigm was employed to analyze the matching performance of the novice subjects. Interestingly, matching error was found to be lower for laparoscopic instrument. An effect of handedness was visible for laparoscopic instrument only. The dominant hand overestimated the forces of non-dominant hand. The results can be used as a performance metric to evaluate the force perception of novices in laparoscopic force skills-training tasks.

A psychophysical evaluation of haptic controllers: viscosity perception of soft environments

In this paper, human viscosity perception in haptic teleoperation systems is thoroughly analyzed. An accurate perception of viscoelastic environmental properties such as viscosity is a critical ability in several contexts, such as telesurgery, telerehabilitation, telemedicine, and soft-tissue interaction. We study and compare the ability to perceive viscosity from the standpoint of detection and discrimination using several relevant control methods for the teleoperator. The perception-based method, which was proposed by the authors to enhance the operator's kinesthetic perception, is compared with the conventional transparency-based control method for the teleoperation system. The fidelity-based method, which is a primary method among perception-centered control schemes in teleoperation, is also studied. We also examine the necessity and impact of the remote-site force information for each of the methods. The comparison is based on a series of psychophysical experiments measuring absolute threshold and just noticeable difference for all conditions. The results clearly show that the perception-based method enhances both detection and discrimination abilities compare with other control methods. The results further show that the fidelity-based method confers a better discrimination ability than the transparency-based method, although this is not true with respect to detection ability. In addition, we show that force information improves viscosity detection for all control methods, as predicted from previous theoretical analysis, but improves the discrimination threshold only for the perception-based method.

Auditory force feedback substitution improves surgical precision during simulated ophthalmic surgery

PURPOSE

To determine the extent that auditory force feedback (AFF) substitution improves performance during a simulated ophthalmic peeling procedure.

METHODS

A 25-gauge force-sensing microforceps was linked to two AFF modes. The "alarm" AFF mode sounded when the force reached 9 mN. The "warning" AFF mode made beeps with a frequency proportional to the generated force. Participants with different surgical experience levels were asked to peel a series of bandage strips off a platform as quickly as possible without exceeding 9 mN of force. In study arm A, participants peeled with alarm and warning AFF modes, the order randomized within the experience level. In study arm B, participants first peeled without AFF, then alarm or warning AFF (order randomized within the experience level), and finally without AFF.

RESULTS

Of the 28 "surgeon" participants, AFF improved membrane peeling performance, reducing average force generated (P < 0.01), SD of forces (P < 0.05), and force × time above 9 mN (P < 0.01). Short training periods with AFF improved subsequent peeling performance when AFF was turned off, with reductions in average force, SD of force, maximum force, time spent above 9 mN, and force × time above 9 mN (all P < 0.001). Except for maximum force, peeling with AFF reduced all force parameters (P < 0.05) more than peeling without AFF after completing a training session.

CONCLUSIONS

AFF enables the surgeon to reduce the forces generated with improved precision during phantom membrane peeling, regardless of surgical experience. New force-sensing surgical tools combined with AFF offer the potential to enhance surgical training and improve surgical performance.

A triple-jaw actuated and sensorized instrument for grasping large organs during minimally invasive robotic surgery

BACKGROUND

Secure grasping and effective manipulation of delicate large organs during robotic surgery operations needs especially designed instruments that can enclose a large amount of tissue and feed back the pinch forces.

METHODS

A large organ triple-jaw grasper was instrumented using practical force sensory and actuating systems. A force tracking scheme was proposed to facilitate auto-grasping of large organs during robotic teleoperation surgery. An on-site force commanding/reflecting mechanism was also implemented to use the device as an independent hand-held robotic instrument. The efficacy of the robotic grasper was examined in phantom tests.

RESULTS

The instrument grasped large soft objects effectively and safely with accurately measured and controlled pinch forces. Furthermore, it could characterize the overall mechanical behavior of the grasping objects.

CONCLUSIONS

The instrument designed provides a potential solution for the safe and effective grasping and manipulation of large abdominal organs, either as a hand-held device, or in a teleoperation framework.

The Effect of Augmented Feedback on Grasp Force in Laparoscopic Grasp Control

Little is known about the influence of augmented feedback, on laparoscopic grasp control. To gain more knowledge on the influence of this on the learning curve, two experiments were conducted. In the first experiment, four groups learned a single-handed laparoscopic lifting task. Three groups received augmented feedback (visual, haptic, or a combination of feedback modes) on slip and excessive pinch force. In the second experiment, a two-handed task had to be accomplished to investigate whether paying reduced attention would influence grasp-force control. The surgeons and novices either received tactile feedback or no augmented feedback on grasp forces. In both experiments, learning sessions and a retention test followed a pretest. In the two-handed task, novices who received tactile feedback could control their pinch force in order to remain within the required limits unlike participants who did not receive augmented feedback. Approximately, one-third of the participants who received augmented feedback became dependent on the signal. Regardless of their level of experience, participants benefited from augmented feedback. This research supports the claim that there is a need for augmented tactile feedback when learning laparoscopic grasp control. It enhances learning and goes beyond what could be achieved without.

Estimation of environmental force for the haptic interface of robotic surgery

BACKGROUND

The success of a telerobotic surgery system with haptic feedback requires accurate force-tracking and position-tracking capacity of the slave robot. The two-channel force-position control architecture is widely used in teleoperation systems with haptic feedback for its better force-tracking characteristics and superior position-tracking capacity for the maximum stability margin. This control architecture, however, requires force sensors at the end-effector of the slave robot to measure the environment force. However, it is difficult to attach force sensors to slave robots, mainly due to their large size, insulation issues and also large currents often flowing through the end-effector for incision or cautery of tissues.

METHODS

This paper provides a method to estimate the environment force, using a function parameter matrix and a recursive least-squares method. The estimated force is used to feed back the force information to the surgeon through the control architecture without involving the force sensors.

RESULTS

The simulation and experimental results verify the efficacy of the proposed method. The force estimation error is negligible and the slave device successfully tracks the position of the master device while the stability of the teleoperation system is maintained.

CONCLUSIONS

The developed method allows practical haptic feedback for telerobotic surgery systems in the two-channel force-position control scheme without the direct employment of force sensors at the end-effector of the slave robot.

The impact of inherent and environmental factors on surgical performance in laparoscopy: a review

Training and experience amongst laparoscopic surgeons remain varied. The demands associated with this form of surgery are thought to be greater than those for traditional open surgery. This article examines the surgeon-specific and environmental factors that contribute to performance in laparoscopic surgery.

Haptics in minimally invasive surgery--a review

This article gives an overview of research performed in the field of haptic information feedback during minimally invasive surgery (MIS). Literature has been consulted from 1985 to present. The studies show that currently, haptic information feedback is rare, but promising, in MIS. Surgeons benefit from additional feedback about force information. When it comes to grasping forces and perceiving slip, little is known about the advantages additional haptic information can give to prevent tissue trauma during manipulation. Improvement of haptic perception through augmented haptic information feedback in MIS might be promising.

Effects of experience on force perception threshold in minimally invasive surgery

BACKGROUND

Distorted haptic feedback by the surgical instrumentation is a major problem in minimally invasive surgery (MIS). Friction force generated by the rubber seal in the trocars masks the haptic information needed to perceive the properties and structure of the target tissue, resulting in an increased haptic perception threshold in naïve subjects. This can lead to over application of forces in surgery.

OBJECTIVE

This paper examines the effect of surgical experience on the psychophysics of force perception and force application efficiency in MIS.

METHOD

A controlled experiment was conducted using a mixed design, with friction and vision as independent within-subjects factors, experience as a between-subjects factor, and applied force and detection time as dependent measures. Fourteen subjects (eight novices and six experienced surgeons) performed a simulated tissue probing task. Performance data were recorded by a custom-built force-sensing system.

RESULTS

When friction was present, higher thresholds and longer detection times were observed for both experienced and inexperienced subjects. In all cases, experienced surgeons applied a greater force than novices, but were quicker to detect contact with tissue, resulting in higher force application efficiency.

CONCLUSION

Surgeons seem to have adapted to the higher threshold in haptic perception by reacting faster, even while applying more force to the tissue, keeping within the limits of safety.

Development of actuated and sensor integrated forceps for minimally invasive robotic surger

In minimally invasive surgery (MIS) the patient's skin forms a spatial barrier between the operation area and the surgeon. This prevents direct access to the operation site which causes a lack of dexterity and limits the sensation of tissue manipulation forces, therefore complicating MIS procedures significantly. A telepresence approach can overcome these limitations: Additional degrees of freedom (DoF) inside the patient provide full manipulability and force torque sensors at the distal end of the instrument allow precise measurement of interaction forces. Using a suitable man-machine interface and free cartesian motion kinaesthetic feedback can be achieved, thus providing a virtual open surgery environment to the surgeon. This article focuses on the development and first results of actuated and sensor integrated instruments as part of the DLR minimally invasive robotic surgery (MIRS) setup. The instruments as a front-end part of the MIRS setup form one base of a telepresence working environment and are crucial for semi-autonomous functions, e.g. motion compensation.

Natural orifice surgery with an endoluminal mobile robot

Natural orifice transgastric endoscopic surgery promises to eliminate skin incisions and reduce postoperative pain and discomfort. Such an approach provides a distinct benefit as compared with conventional laparoscopy, in which multiple entry incisions are required for tools and camera. Endoscopy currently is the only method for performing procedures through the gastrointestinal tract. However, this approach is limited by instrumentation and the need to pass the entire scope into the patient. In contrast, an untethered miniature robot inserted through the mouth would be able to enter the abdominal cavity through a gastrotomy for exploration of the entire peritoneal cavity. In this study, the authors developed an endoluminal robot capable of transgastric abdominal exploration under esophagogastroduodenoscopic (EGD) control. Under EGD control, a gastrotomy was created, and the miniature robot was deployed into the abdominal cavity under remote control. Ultimately, future procedures will include a family of robots working together inside the gastric and abdominal cavities after their insertion through the esophagus. Such technology will help to reduce patient trauma while providing surgical flexibility.

Recent in vivo surgical robot and mechanism developments

The surgical landscape is quickly changing because of the major driving force of robotics. Well-established technology that provides robotic assistance from outside the patient may soon give way to alternative approaches that place the robotic mechanisms inside the patient, whether through traditional laparoscopic ports or through other, natural orifices. While some of this technology is still being developed, other concepts are being evaluated through clinical trials. This article examines the state of the art in surgical robots and mechanisms by providing an overview of the ex vivo robotic systems that are commercially available to in vivo mechanisms, and robotic assistants that are being tested in animal models.

Employing bending beam transducer design and statistical algorithms to develop a clinical real time tissue compliance mapping system

In keyhole surgery, the use of long surgical instruments inserted through small ports in the body diminishes tactile feedback. Earlier methodologies to overcome this challenge never gained popularity in routine clinical practice due to either major modifications to the design of conventional surgical instruments, or relying on surgeons' subjective interpretation of compliance data that is often inaccurate with crossovers. In this paper we present a real time compliance mapping system which comprises of (i) bending beam transducer design to conventional surgical forceps, (ii) statistical analysis for real time objective interpretation of output signals, and (iii) novel human computer interaction techniques suitable for use in the operative theatre working environment. The system was calibrated and put into clinical practice in four routine human keyhole settings. In a research experiment involving 10 surgeons, the system's tissue discriminatory power was three times more sensitive, and 10% less specific than surgeon's hand.

The Role of Tactile Feedback in Laparoscopic Surgery

Two experiments aiming at comparing palpation with gloved fingers, conventional laparoscopic instruments, and a laparoscopic instrument with a sensor array attached to its end effector are described. The sensor array provides the surgeon with visually presented tactile information. Fifteen subjects were asked to discriminate hardness and size of objects (rubber balls hidden in pig's intestine) with the 3 palpation methods. The experiments showed that the gloved fingers are better at differentiating hardness and size compared with conventional laparoscopic instruments and the instrument with sensor. There was no significant difference between conventional instruments and the instrument with sensor, although the results showed a higher average score with the instrument with sensor. This indicates that visual presentation may not be an ideal way of presenting tactile information. It also indicates that the presence of the array does not make the task more difficult.

An In Vivo Mobile Robot for Surgical Vision and Task Assistance

Current laparoscopic surgical robots are expensive, bulky, and fundamentally constrained by the small entry incisions. A potential new approach to minimally invasive surgery is to place the robot completely within the patient. We have developed several such miniature mobile robots and conducted tests during animal surgeries. These robots can provide vision and task assistance to the surgeon without being constrained by the entry port. We used a mobile biopsy and camera robot to sample hepatic tissue from an anesthetized porcine animal model. This successful test demonstrated the capability of performing a single port laparoscopic biopsy procedure. In the future, a family of such robots could be remotely controlled and used to perform surgical procedures without the need for conventional laparoscopic tools.

Effects of vision and friction on haptic perception

OBJECTIVE

Two experiments were conducted to examine the effects of vision and masking friction on contact perception and compliance differentiation thresholds in a simulated tissue-probing task.

BACKGROUND

In minimally invasive surgery, the surgeon receives limited haptic feedback because of the current design of the instrumentation and relies on visual feedback to judge the amount of force applied to the tissues. It is suggested that friction forces inherent in the instruments contribute to errors in surgeons' haptic perception. This paper investigated the psychophysics of contact detection and cross-modal sensory processing in the context of minimally invasive surgery.

METHOD

A within-subjects repeated measures design was used, with friction, vision, tissue softness, and order of presentation as independent factors, and applied force, detection time, error, and confidence as dependent measures. Eight participants took part in each experiment, with data recorded by a custom force-sensing system.

RESULTS

In both detection and differentiation tasks, higher thresholds, longer detection times, and more errors were observed when vision was not available. The effect was more pronounced when haptic feedback was masked by friction forces in the surgical device (p < .05).

CONCLUSION

Visual and haptic feedback were equally important for tissue compliance differentiation.

APPLICATION

A frictionless endoscopic instrument can be designed to restore critical haptic information to surgeons without having to create haptic feedback artificially.

Surface exploration using laparoscopic surgical instruments: the perception of surface roughness

During laparoscopic surgery video images are used to guide the movements of the hand and instruments, and objects in the operating field often obscure these images. Thus, surgeons often rely heavily on tactile information (sense of touch) to help guide their movements. It is important to understand how tactile perception is affected when using laparoscopic instruments, since many surgical judgements are based on how a tissue 'feels' to the surgeon, particularly in situations where visual inputs are degraded. Twelve naïve participants used either their index finger or a laparoscopic instrument to explore sandpaper surfaces of various grits (60, 100, 150 and 220). These movements were generated with either vision or no vision. Participants were asked to estimate the roughness of the surfaces they explored. The normal and tangential forces of either the finger or instrument on the sandpaper surfaces were measured. Results showed that participants were able to judge the roughness of the sandpaper surfaces when using both the finger and the instrument. However, post hoc comparisons showed that perceptual judgements of surface texture were altered in the no vision condition compared to the vision condition. This was also the case when using the instrument, compared to the judgements provided when exploring with the finger. This highlights the importance of the completeness of the video images during laparoscopic surgery. More normal and tangential force was used when exploring the surfaces with the finger as opposed to the instrument. This was probably an attempt to increase the contact area of the fingertip to maximize tactile input. With the instrument, texture was probably sensed through vibrations of the instrument in the hand. Applications of the findings lie in the field of laparoscopic surgery simulation techniques and tactile perception.

In vivo measurement of surgical gestures

Virtual reality techniques are now more and more widely used in the field of surgical training. However, the realism of the simulation devices requires a good knowledge of the mechanical behavior of the living organs. To provide perioperative measurement of laparoscopic surgical operations, we equipped a conventional operating grasper with a force sensor and a position sensor. The entire apparatus was connected to a PC that controlled the real-time data acquisition. After calibrating the sensors, we conducted three series of in vivo measurements on animals under video control. A standardized protocol was set up to perform various surgical gestures in a reproducible manner. Under these conditions, we can assess an original tool for a quantitative approach of surgical gestures' mechanics. The preliminary results will be extended by measurements during other operations and with other surgical instruments. The in vivo quantification of the mechanical interactions between operating instruments and anatomical structures is of great interest for the introduction of the force feedback in virtual surgery, for the modeling of the mechanical behavior of living organs, and for the design of new surgical instruments. This quantification of manipulations opens new prospects in the evaluation of surgical practices.

Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills

The best method of training for laparoscopic surgical skills is controversial. Some advocate observation in the operating room, while others promote animal and simulated models or a combination of surgery-related tasks. A crucial process in surgical education is to evaluate the level of surgical skills. For laparoscopic surgery, skill evaluation is traditionally performed subjectively by experts grading a video of a procedure performed by a student. By its nature, this process uses fuzzy criteria. The objective of the current study was to develop and assess a skill scale using Markov models (MMs). Ten surgeons [five novice surgeons (NS); five expert surgeons (ES)] performed a cholecystectomy and Nissen fundoplication in a porcine model. An instrumented laparoscopic grasper equipped with a three-axis force/torque (F/T) sensor was used to measure the forces/torques at the hand/tool interface synchronized with a video of the tool operative maneuvers. A synthesis of frame-by-frame video analysis and a vector quantization algorithm, allowed to define F/T signatures associated with 14 different types of tool/tissue interactions. The magnitude of F/T applied by NS and ES were significantly different (p < 0.05) and varied based on the task being performed. High F/T magnitudes were applied by NS compared with ES while performing tissue manipulation and vise versa in tasks involved tissue dissection. From each step of the surgical procedures, two MMs were developed representing the performance of three surgeons out of the five in the ES and NS groups. The data obtained by the remaining two surgeons in each group were used for evaluating the performance scale. The final result was a surgical performance index which represented a ratio of statistical similarity between the examined surgeon's MM and the MM of NS and ES. The difference between the performance index value, for a surgeon under study, and the NS/ES boundary, indicated the level of expertise in the surgeon's own group. Using this index, 87.5% of the surgical procedures were correctly classified into the NS and ES groups. The 12.5% of the procedures that were misclassified were performed by the ES and classified as NS. However in these cases the performance index values were very close to the NS/ES boundary. Preliminary data suggest that a performance index based on MM and F/T signatures provides an objective means of distinguishing NS from ES. In addition, this methodology can be further applied to evaluate haptic virtual reality surgical simulators for improving realism in surgical education.