Active constraint control for the surgical robotic platform with concentric connector joints

Robotic minimally invasive surgery (MIS) has changed numerous surgical techniques in the past few years and enhanced their results. Haptic feedback is integrated into robotic surgical systems to restore the surgeon's perception of forces in response to interaction with objects in the surgical environment. The ideal exact emulation of the robot's interaction with its physical environment in free space is a very challenging problem to solve completely. Previously, we introduced the surgical robotic platform (SRP) with a novel concentric connector joint (CCJ). This study aims to develop a haptic control system that integrates an active constraint controller into a surgical robot platform. We have successfully established haptic feedback control for the surgical robot using constraint control and inverse kinematic relationships integrated into the overall positioning structure. A preliminary feasibility study, modelling, and simulation were presented.

Dynamic load response of human dura mater at different velocities

Despite of its assumed role to mitigate brain tissue response under dynamic loading conditions, the human dura mater is frequently neglected in computational and physical human head models. A reason for this is the lack of load-deformation data when the dura mater is loaded dynamically. To date, the biomechanical characterization of the human dura mater predominantly involved quasi-static testing setups. This study aimed to investigate the strain rate-dependent mechanical properties of the human dura mater comparing three different velocities of 0.3, 0.5 and 0.7 m/s. Samples were chosen in a perpendicular orientation to the visible main fiber direction on the samples' surface, which was mostly neglected in previous studies. The elastic modulus of dura mater significantly increased at higher velocities (5.16 [3.38; 7.27] MPa at 0.3 m/s versus 44.38 [35.30; 74.94] MPa at 0.7 m/s). Both the stretch at yield point λ_f (1.148 [1.137; 1.188] for 0.3 m/s, 1.062 [1.054; 1.066] for 0.5 m/s and 1.015 [1.012; 1.021] for 0.7 m/s) and stress at yield point σ_f of dura mater (519.14 [366.74; 707.99] kPa for 0.3 m/s versus 300.52 [245.31; 354.89] kPa at 0.7 m/s) significantly decreased with increasing velocities. Conclusively, increasing the load application velocity increases stiffness and decreases tensile strength as well as straining potential of human dura mater between 0.3 and 0.7 m/s. The elastic modulus of human dura mater should be adapted to the respective velocities in computational head impact simulations.

A Transparent Teleoperated Robotic Surgical System with Predictive Haptic Feedback and Force Modelling

In recent years, robotic minimally invasive surgery has transformed many types of surgical procedures and improved their outcomes. Implementing effective haptic feedback into a teleoperated robotic surgical system presents a significant challenge due to the trade-off between transparency and stability caused by system communication time delays. In this paper, these time delays are mitigated by implementing an environment estimation and force prediction methodology into an experimental robotic minimally invasive surgical system. At the slave, an exponentially weighted recursive least squares (EWRLS) algorithm estimates the respective parameters of the Kelvin-Voigt (KV) and Hunt-Crossley (HC) force models. The master then provides force feedback by interacting with a virtual environment via the estimated parameters. Palpation experiments were conducted with the slave in contact with polyurethane foam during human-in-the-loop teleoperation. The experimental results indicated that the prediction RMSE of error between predicted master force feedback and measured slave force was reduced to 0.076 N for the Hunt-Crossley virtual environment, compared to 0.356 N for the Kelvin-Voigt virtual environment and 0.560 N for the direct force feedback methodology. The results also demonstrated that the HC force model is well suited to provide accurate haptic feedback, particularly when there is a delay between the master and slave kinematics. Furthermore, a haptic feedback approach that incorporates environment estimation and force prediction improve transparency during teleoperation. In conclusion, the proposed bilateral master-slave robotic system has the potential to provide transparent and stable haptic feedback to the surgeon in surgical robotics procedures.

Directional touch sensing for stiffness singularity search in an object using microfinger with tactile sensor

Palpation is widely used as the initial medical diagnosis. Integration of micro tactile sensors and artificial muscles enables a soft microfinger for active touch sensing using its bending actuation. Active touch sensing by pushing-in motion of microfinger enables to evaluate stiffness distribution on an elastic object. Due to its compactness, the microfinger can enter a narrow space, such as gastrointestinal and abdominal spaces in a body. However, a microfinger can only touch and sense limited points. We aim at efficient method for searching a stiffness singular part in an elastic object by the directional touch sensing of a microfinger. This study presents a microfinger for active touch sensing using bending and push-in actuation and proposes an algorithm utilizing directivity in touch sensing by a microfinger for efficient localization of the stiffness singular part in an object. A gelatin block structure with a small rigid ball was prepared and touch sensed by the microfinger. Consequently, the position of the buried rigid ball could be efficiently identified based on the proposed algorithm. This result implies that the proposed method has potential applications in endoscopic medical diagnosis, particularly in identifying tumor positions.

A Force-Feedback Methodology for Teleoperated Suturing Task in Robotic-Assisted Minimally Invasive Surgery

With robotic-assisted minimally invasive surgery (RAMIS), patients and surgeons benefit from a reduced incision size and dexterous instruments. However, current robotic surgery platforms lack haptic feedback, which is an essential element of safe operation. Moreover, teleportation control challenges make complex surgical tasks like suturing more time-consuming than those that use manual tools. This paper presents a new force-sensing instrument that semi-automates the suturing task and facilitates teleoperated robotic manipulation. In order to generate the ideal needle insertion trajectory and pass the needle through its curvature, the end-effector mechanism has a rotating degree of freedom. Impedance control was used to provide sensory information about needle-tissue interaction forces to the operator using an indirect force estimation approach based on data-based models. The operator's motion commands were then regulated using a hyperplanar virtual fixture (VF) designed to maintain the desired distance between the end-effector and tissue surface while avoiding unwanted contact. To construct the geometry of the VF, an optoelectronic sensor-based approach was developed. Based on the experimental investigation of the hyperplane VF methodology, improved needle-tissue interaction force, manipulation accuracy, and task completion times were demonstrated. Finally, experimental validation of the trained force estimation models and the perceived interaction forces by the user was conducted using online data, demonstrating the potential of the developed approach in improving task performance.

Optical-Waveguide Based Tactile Sensing for Surgical Instruments of Minimally Invasive Surgery

In recent years, with the rapid development of minimally invasive surgery (MIS), the lack of force sensing associated with the surgical instrument used in MIS has been increasingly a desirable technology amongst clinicians. However, it is still an open technical challenge to date since most existing tactile sensing principles are not suitable to small 3-dimensional (3D) curved surfaces often seen in surgical instruments, and as a result multi-point force detection cannot be realized. In this paper, a novel optical waveguide-based sensor was proposed to deal with the above research gap. A sensor prototype for curved surfaces resembling the surface of dissection forceps was developed and experimentally evaluated. The static parameters and dynamic response characteristics of the sensor were measured. Results show that the static hysteresis error is less than 3%, the resolution is 0.026 N, and the repeatability is less than 1.5%. Under a frequency of 12.5 Hz, the sensor could quickly measure the variation of the force signal. We demonstrated that this small and high-precision sensitive sensor design is promising to be used for creating multiple-point tactile sensing for minimally invasive surgical instruments with 3D surfaces.

Force sensing in robot-assisted keyhole endoscopy: A systematic survey

Instrument–tissue interaction forces in minimally invasive surgery (MIS) provide valuable information that can be used to provide haptic perception, monitor tissue trauma, develop training guidelines, and evaluate the skill level of novice and expert surgeons. Force and tactile sensing is lost in many robot-assisted surgery (RAS) systems. Therefore, many researchers have focused on recovering this information through sensing systems and estimation algorithms. This article provides a comprehensive systematic review of the current force sensing research aimed at RAS and, more generally, keyhole endoscopy, in which instruments enter the body through small incisions. Articles published between January 2011 and May 2020 are considered, following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. The literature search resulted in 110 papers on different force estimation algorithms and sensing technologies, sensor design specifications, and fabrication techniques.

Realization of Force Detection and Feedback Control for Slave Manipulator of Master/Slave Surgical Robot

Robot-assisted minimally invasive surgery (MIS) has received increasing attention, both in the academic field and clinical operation. Master/slave control is the most widely adopted manipulation mode for surgical robots. Thus, sensing the force of the surgical instruments located at the end of the slave manipulator through the main manipulator is critical to the operation. This study mainly addressed the force detection of the surgical instrument and force feedback control of the serial surgical robotic arm. A measurement device was developed to record the tool end force from the slave manipulator. An elastic element with an orthogonal beam structure was designed to sense the strain induced by force interactions. The relationship between the acting force and the output voltage was obtained through experiment, and the three-dimensional force output was decomposed using an extreme learning machine algorithm while considering the nonlinearity. The control of the force from the slave manipulator end was achieved. An impedance control strategy was adopted to restrict the force interaction amplitude. Modeling, simulation, and experimental verification were completed on the serial robotic manipulator platform along with virtual control in the MATLAB/Simulink software environment. The experimental results show that the measured force from the slave manipulator can provide feedback for impedance control with a delay of 0.15 s.

An extended algorithm for autonomous grasping of soft tissues during robotic surgery

BACKGROUND

Autonomous grasping of soft tissues can facilitate the robotic surgery procedures. The previous attempts for implementing auto-grasping have been based on a simplistic representation of the actual surgery maneuvers.

METHOD

A generalized three-zone grasp model was introduced to consider the effect of the pull force angulation on the grasp mode, that is, damage, slip, or safe grasp. Also, an extended auto-grasping algorithm was proposed in which the trigger force is automatically controlled against the pull force magnitude and direction, to achieve a safe and secure grasp.

RESULTS

The autonomous grasping experiments against a varying pull force in a phantom study indicated a good agreement between the desired and actual pinch and trigger forces (root mean square errors lower than 0.168 N and 0.280 N, respectively) and no sign of tissue tear or slippage.

CONCLUSIONS

The proposed auto-grasping algorithm can help manipulating the soft tissues safely and effectively during robotic surgery procedures.

Surgeon-Centered Analysis of Robot-Assisted Needle Driving Under Different Force Feedback Conditions

Robotic assisted minimally invasive surgery (RAMIS) systems present many advantages to the surgeon and patient over open and standard laparoscopic surgery. However, haptic feedback, which is crucial for the success of many surgical procedures, is still an open challenge in RAMIS. Understanding the way that haptic feedback affects performance and learning can be useful in the development of haptic feedback algorithms and teleoperation control systems. In this study, we examined the performance and learning of inexperienced participants under different haptic feedback conditions in a task of surgical needle driving via a soft homogeneous deformable object-an artificial tissue. We designed an experimental setup to characterize their movement trajectories and the forces that they applied on the artificial tissue. Participants first performed the task in an open condition, with a standard surgical needle holder, followed by teleoperation in one of three feedback conditions: (1) no haptic feedback, (2) haptic feedback based on position exchange, and (3) haptic feedback based on direct recording from a force sensor, and then again with the open needle holder. To quantify the effect of different force feedback conditions on the quality of needle driving, we developed novel metrics that assess the kinematics of needle driving and the tissue interaction forces, and we combined our novel metrics with classical metrics. We analyzed the final teleoperated performance in each condition, the improvement during teleoperation, and the aftereffect of teleoperation on the performance when using the open needle driver. We found that there is no significant difference in the final performance and in the aftereffect between the 3 conditions. Only the two conditions with force feedback presented statistically significant improvement during teleoperation in several of the metrics, but when we compared directly between the improvements in the three different feedback conditions none of the effects reached statistical significance. We discuss possible explanations for the relative similarity in performance. We conclude that we developed several new metrics for the quality of surgical needle driving, but even with these detailed metrics, the advantage of state of the art force feedback methods to tasks that require interaction with homogeneous soft tissue is questionable.

Reference tool kinematics-kinetics and tissue surface strain data during fundamental surgical acts

Haptic based surgical simulations are popular training aids in medicine. Previously, surgical tool loads and motion were measured during cutting and needle insertion on non-human tissue and several haptic based simulations were developed to enhance surgical training. However, there was a lack of realistic foundational data regarding the mechanical responses of human tissue and tools during fundamental acts of surgery, i.e., cutting, suturing, retracting, pinching and indenting. This study used four recently developed surgical tools in a variety of procedures on a diverse set of cadaver leg specimens from human donors. The kinematics and kinetics of surgical tools were recorded along with topical three-dimensional strain during commonly performed surgical procedures. Full motion and load signatures of foundational surgical acts can also be used beyond the development of authentic visual and haptic simulations of surgery, i.e., they provide mechanical specifications for the development of autonomous surgical systems.

A sensorless force-feedback system for robot-assisted laparoscopic surgery

The existing surgical robots for laparoscopic surgery offer no or limited force feedback, and there are many problems for the traditional sensor-based solutions. This paper builds a teleoperation surgical system and validates the effectiveness of sensorless force feedback. The tool-tissue interaction force at the surgical grasper tip is estimated using the driving motor's current, and fed back to the master robot with position-force bilateral control algorithm. The stiffness differentiation experiment and tumor detection experiment were conducted. In the stiffness differentiation experiment, 43 out of 45 pairs of ranking relationships were identified correctly, yielding a success rate of 96%. In the tumor detection experiment, 4 out of 5 participants identified the correct tumor location with force feedback, yielding a success rate of 80%. The proposed sensorless force-feedback system for robot-assisted laparoscopic surgery can help surgeons regain tactile information and distinguish between the healthy and cancerous tissue.

Artificial palpation in robotic surgery using haptic feedback

BACKGROUND

The loss of tactile feedback in minimally invasive robotic surgery remains a major challenge to the expanding field. With visual cue compensation alone, tissue characterization via palpation proves to be immensely difficult. This work evaluates a bimodal vibrotactile system as a means of conveying applied forces to simulate haptic feedback in two sets of studies simulating an artificial palpation task using the da Vinci surgical robot.

METHODS

Subjects in the first study were tasked with localizing an embedded vessel in a soft tissue phantom using a single-sensor unit. In the second study, subjects localized tumor-like structures using a three-sensor array. In both sets of studies, subjects completed the task under three trial conditions: no feedback, normal force tactile feedback, and hybrid vibrotactile feedback. Recordings of correct localization, incorrect localization, and time-to-completion were used to evaluate performance outcomes.

RESULTS

With the addition of vibrotactile and pneumatic feedback, significant improvements in the percentage of correct localization attempts were detected (p = 0.0001 and p = 0.0459, respectively) during the first experiment with phantom vessels. Similarly, significant improvements in correct localization were found with the addition of vibrotactile (p = 2.57E-5) and pneumatic significance (p = 8.54E-5) were observed in the second experiment involving tumor phantoms.

CONCLUSIONS

This work demonstrates not only the superior benefits of a multi-modal feedback over traditional single-modality feedback, but also the effectiveness of vibration in providing haptic feedback to artificial palpation systems.

Three-dimensional nonlinear force-sensing method based on double microgrippers with E-type vertical elastomer for minimally invasive robotic surgery

This paper presents a new type of forceps that consist of two microgrippers with the capability of direct force sensing, which enables grasping and manipulating forces at the tip of surgical instrument for minimally invasive robotic surgery. For the prototype design of the forceps, a double E-type vertical elastomer with four strain beams is presented, whose force-sensing principle is expounded. Thus, the forceps with the elastomer can be considered a compliant component, which provides tiny displacements that allow large strain, and the overall diameter is 10 mm. The sizes of the elastomer and forceps are successively determined by analyzing the relationship of several parameters and strain. Then, the linearity analysis of strain beams determines the positions to apply gauges for sensing. The two-dimensional force decoupling models for a single microgripper are proposed based on piecewise analytical polynomials of the strain difference and employed to develop a new three-dimensional force nonlinear decoupling algorithm based on double microgrippers, which realizes single-axial grasping and three-axial pulling forces. Finally, the required force-sensing performance of the proposed method is successfully verified in theory using finite-element simulations.

Master-slave robotic system for needle indentation and insertion

Bilateral control of a master-slave robotic system is a challenging issue in robotic-assisted minimally invasive surgery. It requires the knowledge on contact interaction between a surgical (slave) robot and soft tissues. This paper presents a master-slave robotic system for needle indentation and insertion. This master-slave robotic system is able to characterize the contact interaction between the robotic needle and soft tissues. A bilateral controller is implemented using a linear motor for robotic needle indentation and insertion. A new nonlinear state observer is developed to online monitor the contact interaction with soft tissues. Experimental results demonstrate the efficacy of the proposed master-slave robotic system for robotic needle indentation and needle insertion.

Research of the master-slave robot surgical system with the function of force feedback

BACKGROUND

Surgical robots lack force feedback, which may lead to operation errors. In order to improve surgical outcomes, this research developed a new master-slave surgical robot, which was designed with an integrated force sensor.

METHODS

The new structure designed for the master-slave robot employs a force feedback mechanism. A six-dimensional force sensor was mounted on the tip of the slave robot's actuator. Sliding model control was adopted to control the slave robot. According to the movement of the master system manipulated by the surgeon, the slave's movement and the force feedback function were validated.

RESULTS

The motion was completed, the standard deviation was calculated, and the force data were detected. Hence, force feedback was realized in the experiment.

CONCLUSIONS

The surgical robot can help surgeons to complete trajectory motions with haptic sensation.

Development and testing of a grasper for NOTES powered by variable stiffness pneumatic actuation

BACKGROUND

In natural orifice transluminal endoscopic surgery (NOTES) the peritoneal cavity is reached through natural orifices (mouth, rectus and transvaginal duct), by means of little cuttings in the walls of hollow organs. Due to narrow spaces, NOTES needs robotic systems to assure operation/movement precision and patient safety. Variable stiffness actuation (VSA) assures both requirements.

METHODS

The authors developed a grasper for NOTES, provided with VSA, to use as an end-effector for snail robot devices. The present paper deals with basic concepts of VSA and describes the design and architecture of the grasper. Characterization and functional experiments were performed and results analysed.

RESULTS

A finite element model developed for the actuator design was validated, performance grasper characteristic curves were obtained, VSA was validated, and the gripping capability of several objects was assessed.

CONCLUSION

The grasper satisfies technical design specifications. On the basis of the results obtained, a control system can be developed to test grasper in a simulated surgery environment.

Real-Time Measurement of the Tool-Tissue Interaction in Minimally Invasive Abdominal Surgery: The First Step to Developing the Next Generation of Smart Laparoscopic Instruments

Introduction Analysis of force application in laparoscopic surgery is critical to understanding the nature of the tool-tissue interaction. The aim of this study is to provide real-time data about manipulations to abdominal organs. Methods An instrumented short fenestrated grasper was used in an in vivo porcine model, measuring force at the grasper handle. Grasping force and duration over 5 small bowel manipulation tasks were analyzed. Forces required to retract gallbladder, bladder, small bowel, large bowel, and rectum were measured over 30 seconds. Four parameters were calculated-T(hold), the grasp time; T(close), time taken for the jaws to close; F(max), maximum force reached; and F(rms), root mean square force (representing the average force across the grasp time). Results Mean F(max) to manipulate the small bowel was 20.5 N (±7.2) and F(rms) was 13.7 N (±5.4). Mean T(close) was 0.52 seconds (±0.26) and T(hold) was 3.87 seconds (±1.5). In individual organs, mean F(max) was 49 N (±15) to manipulate the rectum and 59 N (±13.4) for the colon. The mean F(max) for bladder and gallbladder retraction was 28.8 N (±7.4) and 50.7 N (±3.8), respectively. All organs exhibited force relaxation, the F(rms) reduced to below 25 N for all organs except the small bowel, with a mean F(rms) of less than 10 N. Conclusion This study has commenced the process of quantifying tool-tissue interaction. The static measurements discussed here should evolve to include dynamic measurements such as shear, torque, and retraction forces, and be correlated with evidence of histological damage to tissue.

A force-sensing surgical tool with a proximally located force/torque sensor

BACKGROUND

Robotic surgery has seen a rapid increase in popularity in the last few decades because advantages such as increased accuracy and dexterity can be realized. These systems still lack force-feedback, where such a capability is believed to be beneficial to the surgeon and can improve safety.

METHOD

In this paper a force-feedback enabled surgical robotic system is described in which the developed force-sensing surgical tool is discussed in detail. The developed surgical tool makes use of a proximally located force/torque sensor, which, in contrast to a distally located sensor, requires no miniaturization or sterilizability.

RESULTS

Experimental results are presented, and indicate high force sensing accuracies with errors <0.09 N.

CONCLUSIONS

It is shown that developing a force-sensing surgical tool utilizing a proximally located force/torque sensor is feasible, where a tool outer diameter of 12 mm can be achieved. For future work it is desired to decrease the current tool outer diameter to 10 mm. Copyright © 2016 John Wiley & Sons, Ltd.

A modular force-controlled robotic instrument for minimally invasive surgery - efficacy for being used in autonomous grasping against a variable pull force

BACKGROUND

Many deficiencies of minimally invasive robotic surgery systems can be eliminated by using automated laparoscopic tools with force measurement and control capability.

METHOD

A fully modular, automated laparoscopic instrument with a proximal force sensory system was designed and fabricated. The efficacy of the instrument was evaluated experimentally when functioning in an autonomous force-controlled grasping scheme.

RESULTS

The designed instrument was shown to work easily with standard laparoscopic tools, with the whole distal part detachable for autoclave sterilization. The root mean squared error (RMSE) of the actual pinch force from the target ramp was 0.318 N; it was 0.402 N for a sinusoidal pull force, which dropped by 21% using a static friction compensation. A secure grasping condition was achieved, in spite of this error, by applying a sufficiently large margin from the slip boundary.

CONCLUSIONS

With a simple and practical design, the instrument enjoys affordability, versatility and autoclave sterilizability for clinical usage, with an acceptable performance for being used in an auto-grasping control scheme. Copyright © 2016 John Wiley & Sons, Ltd.