Development of a human machine interface for robotically assisted surgery optimized for laparoscopic workflows

INTRODUCTION

In robotic-assisted surgery (RAS), the input device is the primary site for the flow of information between the user and the robot. Most RAS systems remove the surgeon's console from the sterile surgical site. Beneficial for performing lengthy procedures with complex systems, this ultimately lacks the flexibility that comes with the surgeon being able to remain at the sterile site.

METHODS

A prototype of an input device for RAS is constructed. The focus lies on intuitive control for surgeons and a seamless integration into the surgical workflow within the sterile environment. The kinematic design is translated from the kinematics of laparoscopic surgery. The input device uses three degrees of freedom from a flexible instrument as input. The prototype's performance is compared to that of a commercially available device in an evaluation. Metrics are used to evaluate the surgeons' performance with the respective input device in a virtual environment implemented for the evaluation.

RESULTS

The evaluation of the two input devices shows statistically significant differences in the performance metrics. With the proposed prototype, the surgeons perform the tasks faster, more precisely, and with fewer errors.

CONCLUSION

The prototype is an efficient and intuitive input device for surgeons with laparoscopic experience. The placement in the sterile working area allows for seamless integration into the surgical workflow and can potentially enable new robotic approaches.

A Parallel Robot With Remote Centre-of-Motion for Eye Surgery: Design, Kinematics, Prototype, and Experiments

BACKGROUND

Millions of patients suffering from eye disease cannot receive proper treatment due to the lack of qualified surgeons. Medical robots have the potential to solve this problem and have attracted significant attention in the research community.

METHOD

This paper proposes a novel parallel robot with a remote centre of motion for minimally invasive eye surgery. Kinematics models, singularity and workspace analyses, and dimension optimisation are conducted. A prototype was developed, and experiments were conducted to test its mobility, accuracy, precision and stiffness.

RESULTS

The prototype robot can successfully perform the required motions, and has a precision ranging from 7 ± 2 μm to 30 ± 8 μm, accuracy from 21 ± 10 μm to 568 ± 374 μm, and stiffness ranging from 1.22 ± 0.39 N/mm to 10.53 ± 5.18 N/mm.

CONCLUSION

The prototype robot has a great potential for performing the minimally invasive surgery. Its stiffness meets the design requirement, but its accuracy and precision need to be further improved.

A Lightweight and Affordable Wearable Haptic Controller for Robot-Assisted Microsurgery

In robot-assisted microsurgery (RAMS), surgeons often face the challenge of operating with minimal feedback, particularly lacking in haptic feedback. However, most traditional desktop haptic devices have restricted operational areas and limited dexterity. This report describes a novel, lightweight, and low-budget wearable haptic controller for teleoperated microsurgical robotic systems. We designed a wearable haptic interface entirely made using off-the-shelf material-PolyJet Photopolymer, fabricated using liquid and solid hybrid 3D co-printing technology. This interface was designed to resemble human soft tissues and can be wrapped around the fingertips, offering direct contact feedback to the operator. We also demonstrated that the device can be easily integrated with our motion tracking system for remote microsurgery. Two motion tracking methods, marker-based and marker-less, were compared in trajectory-tracking experiments at different depths to find the most effective motion tracking method for our RAMS system. The results indicate that within the 4 to 8 cm tracking range, the marker-based method achieved exceptional detection rates. Furthermore, the performance of three fusion algorithms was compared to establish the unscented Kalman filter as the most accurate and reliable. The effectiveness of the wearable haptic controller was evaluated through user studies focusing on the usefulness of haptic feedback. The results revealed that haptic feedback significantly enhances depth perception for operators during teleoperated RAMS.

A class of spatial remote center-of-motion mechanisms and its forward kinematics

Robot-assisted minimally invasive surgery (MIS) has shown tremendous advances over the traditional technique. A crucial challenge for developing a MIS robot is the kinematic design of the remote center-of-motion (RCM) mechanism. In this paper, a class of spatial RCM mechanism is analyzed. They are obtained by generating virtual parallelograms. The main process is to construct a line that passes through a fixed point under the mechanical constraint. The axis of the surgical tool is then constrained to parallel with that line. Hence, due to the geometrical feature of the parallel lines, the axis of the surgical tool will always pass through a fixed point, i.e., the RCM point. Due to the specially designed structure, the fixed point does not need to be physically belonging to the mechanism. The geometrical analysis method is employed to obtain the closed-form solution of the forward kinematics of the proposed mechanisms. Due to the high load capacity of parallel mechanism, the robots based on the proposed RCM mechanisms have promising applications as an external positioner to be used in robotic single-port surgeries.

Stereotactic technology for 3D bioprinting: from the perspective of robot mechanism

Three-dimensional (3D) bioprinting has been widely applied in the field of biomedical engineering because of its rapidly individualized fabrication and precisely geometric designability. The emerging demand for bioprinted tissues/organs with bio-inspired anisotropic property is stimulating new bioprinting strategies. Stereotactic bioprinting is regarded as a preferable strategy for this purpose, which can perform bioprinting at the target position from any desired orientation in 3D space. In this work, based on the motion characteristics analysis of the stacked bioprinting technologies, mechanism configurations and path planning methods for robotic stereotactic bioprinting were investigated and a prototype system based on the double parallelogram mechanism was introduced in detail. Moreover, the influence of the time dimension on stereotactic bioprinting was discussed. Finally, technical challenges and future trends of stereotactic bioprinting within the field of biomedical engineering were summarized.

Development status of telesurgery robotic system

As an emerging field, telesurgery robotic system is changing the traditional medical mode and can delivery remote surgical treatment anywhere in the world. Advances in telesurgery robotic technology achieve the remote control beyond the current limitation of distance and special medical environment. This review introduces the development history, the current status and the potential in future of the telesurgery robotic system. In addition, it presents the construction of control platform and the application, especially in trauma treatment, as well as the challenge in clinic.

A Confidence-Based Supervised-Autonomous Control Strategy for Robotic Vaginal Cuff Closure

Autonomous robotic suturing has the potential to improve surgery outcomes by leveraging accuracy, repeatability, and consistency compared to manual operations. However, achieving full autonomy in complex surgical environments is not practical and human supervision is required to guarantee safety. In this paper, we develop a confidence-based supervised autonomous suturing method to perform robotic suturing tasks via both Smart Tissue Autonomous Robot (STAR) and surgeon collaboratively with the highest possible degree of autonomy. Via the proposed method, STAR performs autonomous suturing when highly confident and otherwise asks the operator for possible assistance in suture positioning adjustments. We evaluate the accuracy of our proposed control method via robotic suturing tests on synthetic vaginal cuff tissues and compare them to the results of vaginal cuff closures performed by an experienced surgeon. Our test results indicate that by using the proposed confidence-based method, STAR can predict the success of pure autonomous suture placement with an accuracy of 94.74%. Moreover, via an additional 25% human intervention, STAR can achieve a 98.1% suture placement accuracy compared to an 85.4% accuracy of completely autonomous robotic suturing. Finally, our experiment results indicate that STAR using the proposed method achieves 1.6 times better consistency in suture spacing and 1.8 times better consistency in suture bite sizes than the manual results.

A Historical Review of Medical Robotic Platforms

This paper provides a brief history of medical robotic systems. Since the first use of robots in medical procedures, there have been countless companies competing to developed robotic systems in hopes to dominate a field. Many companies have succeeded, and many have failed. This review paper shows the timeline history of some of the old and most successful medical robots and new robotic systems. As the patents of the most successful system, i.e., Da Vinci® Surgical System, have expired or are expiring soon, this paper can provide some insights for new designers and manufacturers to explore new opportunities in this field.

A Practical Approach to the Design and Development of Tele-Operated Surgical Robots for Resource Constrained Environments—A Case Study

Tele-operation has brought about a paradigm shift in the way minimally invasive surgeries are carried out. Several tele-operated robotic systems have been developed in the last three decades. In this research work, we present a practical approach for the complete design of a six degree-of-freedom (DoF) master–slave tele-operated robotic system with limited resources. This research work elaborates on the methodology followed by us for the complete system design, methods for simplifying the surgical tool design with decoupled DoF wrist utilizing stationary wire guides instead of pulleys, and method of reducing the number of balancing masses required for gravity compensation of master manipulator arms. We also demonstrate the avenues for utilizing compliant mechanism for several mechanisms of the system to reduce complexity and to mitigate the issue of biofouling. The concepts and design methodology described in this paper would serve as a starting point and design guideline for future designers of such systems in resource-constrained environments.

MU-LapaRobot: A Corporative Surgical Robot for Laparoscopic Surgery

Objective:

Development of surgical instrument robot (MU-LapaRobot) for assisting in conventional laparoscopic surgery.

Methods:

A robot was designed based on instrument movement for a conventional procedure. The mechanism mimics and constrains manipulation movement through the wire-driven transmission. It is flexible for robot end-effector, which has a lightweight and small size. The usability of the robot is passive and active robot tasks with an interconnected driving system. Three main parts of the robot are robot end-effector, transmission, and a driving system.

Results:

On a robot platform, a parameter for setup is robot posture. The adjustment of robot position and projection of manipulation area is influenced by the accuracy of movement. To verify movement, command and exact movements are measured. Compensation with the control system is improved in order to improve the accuracy of the system.

Conclusion:

MU-LapaRobot provides surgical instrument manipulation by using wire-driven transmission with an effective system and requires less interference in the conventional operation.

The Future of Skull Base Surgery: A View Through Tinted Glasses

In the present report, we have broadly outlined the potential advances in the field of skull base surgery, which might occur within the next 20 years based on the many areas of current research in biology and technology. Many of these advances will also be broadly applicable to other areas of neurosurgery. We have grounded our predictions for future developments in an exploration of what patients and surgeons most desire as outcomes for care. We next examined the recent developments in the field and outlined several promising areas of future improvement in skull base surgery, per se, as well as identifying the new hospital support systems needed to accommodate these changes. These include, but are not limited to, advances in imaging, Raman spectroscopy and microscopy, 3-dimensional printing and rapid prototyping, master-slave and semiautonomous robots, artificial intelligence applications in all areas of medicine, telemedicine, and green technologies in hospitals. In addition, we have reviewed the therapeutic approaches using nanotechnology, genetic engineering, antitumor antibodies, and stem cell technologies to repair damage caused by traumatic injuries, tumors, and iatrogenic injuries to the brain and cranial nerves. Additionally, we have discussed the training requirements for future skull base surgeons and stressed the need for adaptability and change. However, the essential requirements for skull base surgeons will remain unchanged, including knowledge, attention to detail, technical skill, innovation, judgment, and compassion. We believe that active involvement in these rapidly evolving technologies will enable us to shape some of the future of our discipline to address the needs of both patients and our profession.

5G ultra-remote robot-assisted laparoscopic surgery in China

BACKGROUND

5G communication technology has been applied to several fields in telemedicine, but its effectiveness, safety, and stability in remote laparoscopic telesurgery have not been established. Here, we conducted four ultra-remote laparoscopic surgeries on a swine model under the 5G network. The aim of the study was to investigate the effectiveness, safety, and stability of the 5G network in remote laparoscopic telesurgery.

METHODS

Four ultra-remote laparoscopic surgeries (network communication distance of nearly 3000 km), including left nephrectomy, partial hepatectomy, cholecystectomy, and cystectomy, were performed on a swine model with a 5G wireless network connection using a domestically produced "MicroHand" surgical robot. The average network delay, operative time, blood loss, and intraoperative complications were recorded.

RESULTS

Four laparoscopic telesurgeries were safely performed through a 5G network, with an average network delay of 264 ms (including a mean round-trip transporting delay of 114 ms and a 1.20% data packet loss ratio). The total operation time was 2 h. The total blood loss was 25 ml, and no complications occurred during the procedures.

CONCLUSIONS

Ultra-remote laparoscopic surgery can be performed safely and smoothly with 5G wireless network connection using domestically produced equipment. More importantly, our model can provide insights for promoting the future development of telesurgery, especially in areas where Internet cables are difficult to lay or cannot be laid.

SmartArm: Integration and validation of a versatile surgical robotic system for constrained workspaces

BACKGROUND

With the increasing presence of surgical robots minimally invasive surgery, there is a growing necessity of a versatile surgical system for deep and narrow workspaces.

METHODS

We developed a versatile system for constrained workspaces called SmartArm. It has two industrial-type robotic arms with flexible tools attached to its distal tip, with a total of nine active degrees-of-freedom. The system has a control algorithm based on constrained optimization that allows the safe generation of task constraints and intuitive teleoperation.

RESULTS

The SmartArm system is evaluated in a master-slave experiment in which a medically untrained user operates the robot to suture the dura mater membrane at the skull base of a realistic head phantom. Our results show that the user could accomplish the task proficiently, with speed and accuracy comparable to manual suturing by surgeons. Conclusions We demonstrated the integration and validation of the SmartArm.

Robots and Tools for Remodeling Bone

The field of robotic surgery has progressed from small teams of researchers repurposing industrial robots, to a competitive and highly innovative subsection of the medical device industry. Surgical robots allow surgeons to perform tasks with greater ease, accuracy, or safety, and fall under one of four levels of autonomy; active, semi-active, passive, and remote manipulator. The increased accuracy afforded by surgical robots has allowed for cementless hip arthroplasty, improved postoperative alignment following knee arthroplasty, and reduced duration of intraoperative fluoroscopy among other benefits. Cutting of bone has historically used tools such as hand saws and drills, with other elaborate cutting tools now used routinely to remodel bone. Improvements in cutting accuracy and additional options for safety and monitoring during surgery give robotic surgeries some advantages over conventional techniques. This article aims to provide an overview of current robots and tools with a common target tissue of bone, proposes a new process for defining the level of autonomy for a surgical robot, and examines future directions in robotic surgery.

Surgical Robotic Technology for Developing an Endonasal Endoscopic Transsphenoidal Surgery (EETS) Robotic System

Purpose:

Neurosurgical robots are acknowledged for their advantages in assisting neurosurgeons in enhancing their precision and accuracy. Here, the aim of this study is to report the first use as a robot-assisted Endonasal Endoscopic Transsphenoidal (EET) approach, applied to reach sphenoid sinus in a cadaver. The introduction of the seven tenets for the Endonasal Endoscopic Transsphenoidal approach will propel the feasibility of neurosurgical techniques and instruments for intrasellar lesions.

Methods:

Endonasal endoscopic transsphenoidal approach in a cadaver was performed under robot assistance with simple navigation system. This preliminary study reveals the accuracy and precision of the robot to reach a target at sphenoid sinus safely and within the shortest duration .

Results:

This robotic technology provided the foundation to support neurosurgeons when they are working in narrow and complicated surgical corridors with accuracy and precision.

Conclusion:

This article reveals the first robot-assisted Endonasal Endoscopic Transsphenoidal approach. This demonstrates the feasibility of the evolution and will augment neurosurgeons toward their limits of minimally invasive surgical techniques, manual dexterity, and spatial orientation. These tenets will be as state of the art and overcome the future challenges of Endonasal Endoscopic Transsphenoidal approach shortly.

Autonomous Laparoscopic Robotic Suturing with a Novel Actuated Suturing Tool and 3D Endoscope

Compared to open surgical techniques, laparoscopic surgical methods aim to reduce the collateral tissue damage and hence decrease the patient recovery time. However, constraints imposed by the laparoscopic surgery, i.e. the operation of surgical tools in limited spaces, turn simple surgical tasks such as suturing into time-consuming and inconsistent tasks for surgeons. In this paper, we develop an autonomous laparoscopic robotic suturing system. More specific, we expand our smart tissue anastomosis robot (STAR) by developing i) a new 3D imaging endoscope, ii) a novel actuated laparoscopic suturing tool, and iii) a suture planning strategy for the autonomous suturing. We experimentally test the accuracy and consistency of our developed system and compare it to sutures performed manually by surgeons. Our test results on suture pads indicate that STAR can reach 2.9 times better consistency in suture spacing compared to manual method and also eliminate suture repositioning and adjustments. Moreover, the consistency of suture bite sizes obtained by STAR matches with those obtained by manual suturing.

Human–Robot Cooperative Control Based on Virtual Fixture in Robot-Assisted Endoscopic Sinus Surgery

In endoscopic sinus surgery, the robot assists the surgeon in holding the endoscope and acts as the surgeon’s third hand, which helps to reduce the surgeon’s operating burden and improve the quality of the operation. This paper proposes a human–robot cooperative control method based on virtual fixture to realize accurate and safe human–robot interaction in endoscopic sinus surgery. Firstly, through endoscopic trajectory analysis, the endoscopic motion constraint requirements of different surgical stages are obtained, and three typical virtual fixtures suitable for endoscopic sinus surgery are designed and implemented. Based on the typical virtual fixtures, a composite virtual fixture is constructed, and then the overall robot motion constraint model is obtained. Secondly, based on the obtained robot motion constraint model, a human–robot cooperative control method based on virtual fixture is proposed. The method adopts admittance control to realize efficient human–robot interaction between the surgeon and robot during the surgery; the virtual fixture is used to restrain and guide the motion of the robot, thereby ensuring motion safety of the robot. Finally, the proposed method is evaluated through a robot-assisted nasal endoscopy experiment, and the result shows that the proposed method can improve the accuracy and safety of operation during endoscopic sinus surgery.

A Confidence-Based Shared Control Strategy for the Smart Tissue Autonomous Robot (STAR)

Autonomous robotic assisted surgery (RAS) systems aim to reduce human errors and improve patient outcomes leveraging robotic accuracy and repeatability during surgical procedures. However, full automation of RAS in complex surgical environments is still not feasible and collaboration with the surgeon is required for safe and effective use. In this work, we utilize our Smart Tissue Autonomous Robot (STAR) to develop and evaluate a shared control strategy for the collaboration of the robot with a human operator in surgical scenarios. We consider 2D pattern cutting tasks with partial blood occlusion of the cutting pattern using a robotic electrocautery tool. For this surgical task and RAS system, we i) develop a confidence-based shared control strategy, ii) assess the pattern tracking performances of manual and autonomous controls and identify the confidence models for human and robot as well as a confidence-based control allocation function, and iii) experimentally evaluate the accuracy of our proposed shared control strategy. In our experiments on porcine fat samples, by combining the best elements of autonomous robot controller with complementary skills of a human operator, our proposed control strategy improved the cutting accuracy by 6.4%, while reducing the operator work time to 44 % compared to a pure manual control.

Master-slave motion alignment for an open surgical console

BACKGROUND

Open surgical consoles widely employed in minimally invasive surgery have better ergonomics than closed consoles. To enhance surgical robots' ergonomics, operational efficiency, and safety, an effective master-slave motion alignment model should be established.

METHODS

The kinematic model of the robot system based on laparoscopic camera coordinate system is built in the first place. Then, the relative pose between the operator's eyes and the display is measured by Tobii Eye Tracking Sensor and is subsequently used to improve the master-slave motion alignment model.

RESULTS

Robot threading experiments are conducted by two doctors and three testers to verify the kinematic model. As a result, in contrast to the original model, the improved model reduces both operation time and the number of collisions.

CONCLUSIONS

The improved master-slave motion alignment model, in which the transformation matrix between the operator's eyes and the display is employed, raises the ergonomics, operational efficiency, and safety.

Dimensional synthesis and concept design of a novel minimally invasive surgical robot

A new minimally invasive surgical (MIS) robot consisting of a spherical remote center motion (RCM) mechanism with modular design is proposed. A multi-objective dimensional synthesis model is presented to obtain the excellent performance indices. There are four objectives: a global kinematic index, a compactness index, a global comprehensive stiffness index, and a global dynamic index. Other indices characterizing the design requirement, such as workspace, mechanical parameter, and mass, are chosen as constraints. A new decoupled mechanism is raised to solve the coupled motion between the linear platform and the four degrees of freedom (DoF) of surgical instrument as a result of post-driving motors. Another new mechanical decoupled method is proposed to eliminate the coupled motion between the wrist and the forceps, enhance the dexterity of surgical instrument, and improve the independence of each motor. Then, a 7-DoF MIS robotic prototype based on optimization results has been built up. Experiment results validate the effectiveness of the two mechanical decoupled methods. The position change of the RCM point, accuracy, and repeatability of the MIS robot meet the requirements of MIS. Successful animal experiments validate the effectiveness of the novel MIS robot.

Compact teleoperated laparoendoscopic single-site robotic surgical system: Kinematics, control, and operation

BACKGROUND

To date a variety of teleoperated surgical robotic systems have been developed to improve a surgeon's ability to perform demanding single-port procedures. However typical large systems are bulky, expensive, and afford limited angular motion, while smaller designs suffer complications arising from limited motion range, speed, and force generation. This work was to develop and validate a simple, compact, low cost single site teleoperated laparoendoscopic surgical robotic system, with demonstrated capability to carry out basic surgical procedures.

METHODS

This system builds upon previous work done at the University of Hawaii at Manoa and includes instrument and endoscope manipulators as well as compact articulated instruments designed to overcome single incision geometry complications. A robotic endoscope holder was used for the base, with an added support frame for teleoperated manipulators and instruments fabricated mostly from 3D printed parts. Kinematics and control methods were formulated for the novel manipulator configuration.

RESULTS

Trajectory following results from an optical motion tracker and sample task performance results are presented.

CONCLUSIONS

Results indicate that the system has successfully met the goal of basic surgical functionality while minimizing physical size, complexity, and cost.

Kinematic Design of a Generalized Double Parallelogram Based Remote Center-of-Motion Mechanism for Minimally Invasive Surgical Robot

Robot-assisted minimally invasive surgery (MIS) has shown tremendous advances over the traditional techniques. To improve dexterity and back-drivability of the existing planar remote center-of-motion (RCM) mechanism, on which an active prismatic joint is required to drive the surgical tool move in–out of the patient's body, a two degrees-of-freedom (DOFs) planar RCM mechanism is proposed by constructing virtual parallelograms in this paper. The mechanism can be considered as a generalized double parallelogram; both of the actuated joints are revolute joints. This feature enhances the intrinsic back-drivability of the mechanism. The mathematical framework is introduced first to prove that the mechanism could execute RCM. Then, the inverse kinematics of the planar mechanism is solved, and the Jacobian matrix is derived in this paper. Further, the singularity and the kinematic performance based on the kinematic equations are investigated, and the workspace of the mechanism is verified. Finally, a prototype was built to test the function of the proposed RCM mechanism. The results show that the mechanism can fulfill the constraint of MIS, and it can be used as the basic element of the active manipulator in an MIS robot.

Medical telerobotic systems: current status and future trends

Teleoperated medical robotic systems allow procedures such as surgeries, treatments, and diagnoses to be conducted across short or long distances while utilizing wired and/or wireless communication networks. This study presents a systematic review of the relevant literature between the years 2004 and 2015, focusing on medical teleoperated robotic systems which have witnessed tremendous growth over the examined period. A thorough insight of telerobotics systems discussing design concepts, enabling technologies (namely robotic manipulation, telecommunications, and vision systems), and potential applications in clinical practice is provided, while existing limitations and future trends are also highlighted. A representative paradigm of the short-distance case is the da Vinci Surgical System which is described in order to highlight relevant issues. The long-distance telerobotics concept is exemplified through a case study on diagnostic ultrasound scanning. Moreover, the present review provides a classification into short- and long-distance telerobotic systems, depending on the distance from which they are operated. Telerobotic systems are further categorized with respect to their application field. For the reviewed systems are also examined their engineering characteristics and the employed robotics technology. The current status of the field, its significance, the potential, as well as the challenges that lie ahead are thoroughly discussed.

Adverse Events in Robotic Surgery: A Retrospective Study of 14 Years of FDA Data

BACKGROUND

Use of robotic systems for minimally invasive surgery has rapidly increased during the last decade. Understanding the causes of adverse events and their impact on patients in robot-assisted surgery will help improve systems and operational practices to avoid incidents in the future.

METHODS

By developing an automated natural language processing tool, we performed a comprehensive analysis of the adverse events reported to the publicly available MAUDE database (maintained by the U.S. Food and Drug Administration) from 2000 to 2013. We determined the number of events reported per procedure and per surgical specialty, the most common types of device malfunctions and their impact on patients, and the potential causes for catastrophic events such as patient injuries and deaths.

RESULTS

During the study period, 144 deaths (1.4% of the 10,624 reports), 1,391 patient injuries (13.1%), and 8,061 device malfunctions (75.9%) were reported. The numbers of injury and death events per procedure have stayed relatively constant (mean = 83.4, 95% confidence interval (CI), 74.2-92.7 per 100,000 procedures) over the years. Surgical specialties for which robots are extensively used, such as gynecology and urology, had lower numbers of injuries, deaths, and conversions per procedure than more complex surgeries, such as cardiothoracic and head and neck (106.3 vs. 232.9 per 100,000 procedures, Risk Ratio = 2.2, 95% CI, 1.9-2.6). Device and instrument malfunctions, such as falling of burnt/broken pieces of instruments into the patient (14.7%), electrical arcing of instruments (10.5%), unintended operation of instruments (8.6%), system errors (5%), and video/imaging problems (2.6%), constituted a major part of the reports. Device malfunctions impacted patients in terms of injuries or procedure interruptions. In 1,104 (10.4%) of all the events, the procedure was interrupted to restart the system (3.1%), to convert the procedure to non-robotic techniques (7.3%), or to reschedule it (2.5%).

CONCLUSIONS

Despite widespread adoption of robotic systems for minimally invasive surgery in the U.S., a non-negligible number of technical difficulties and complications are still being experienced during procedures. Adoption of advanced techniques in design and operation of robotic surgical systems and enhanced mechanisms for adverse event reporting may reduce these preventable incidents in the future.

A new forecasting kinematic algorithm of automatic navigation for a laparoscopic minimally invasive surgical robotic system

This paper presents a novel forecasting kinematic algorithm for autonomously navigating the 3D visual window of laparoscopic minimally invasive surgical robotic system (LMISRS). By the application of the proposed technique, a constant distribution area ratio of the micro devices can be guaranteed in the visual window; real-time concurrency motion of the visual window of the laparoscope and the mark points of the instruments is realized, i.e. the visual window can keep tracking the movement of the marks automatically, so that the user does not have to switch between the master-slave controlling targets. The implementation of the new technique is summarized as follows: the robotic kinematics and space analytic geometry are thoroughly analyzed and modeled, and a “following kinematic algorithm” is proposed for the visual window of the laparoscope, which tracks the mark points of the instrument arms; a “forecasting kinematic algorithm” is established by using a combination of the “following kinematic algorithm”, the basic visual parameters of 3D visual field, the Verhulst Grey Model and the filtered amendment method. The proposed technique is verified by a series of simulations by using two groups of marks' motion trails with different sampling times, indicating that the technique is accurate, feasible and robust.

Miniature in vivo MEMS-based line-scanned dual-axis confocal microscope for point-of-care pathology

There is a need for miniature optical-sectioning microscopes to enable in vivo interrogation of tissues as a real-time and noninvasive alternative to gold-standard histopathology. Such devices could have a transformative impact for the early detection of cancer as well as for guiding tumor-resection procedures. Miniature confocal microscopes have been developed by various researchers and corporations to enable optical sectioning of highly scattering tissues, all of which have necessitated various trade-offs in size, speed, depth selectivity, field of view, resolution, image contrast, and sensitivity. In this study, a miniature line-scanned (LS) dual-axis confocal (DAC) microscope, with a 12-mm diameter distal tip, has been developed for clinical point-of-care pathology. The dual-axis architecture has demonstrated an advantage over the conventional single-axis confocal configuration for reducing background noise from out-of-focus and multiply scattered light. The use of line scanning enables fast frame rates (16 frames/sec is demonstrated here, but faster rates are possible), which mitigates motion artifacts of a hand-held device during clinical use. We have developed a method to actively align the illumination and collection beams in a DAC microscope through the use of a pair of rotatable alignment mirrors. Incorporation of a custom objective lens, with a small form factor for in vivo clinical use, enables our device to achieve an optical-sectioning thickness and lateral resolution of 2.0 and 1.1 microns respectively. Validation measurements with reflective targets, as well as in vivo and ex vivo images of tissues, demonstrate the clinical potential of this high-speed optical-sectioning microscopy device.

Floating autostereoscopic 3D display with multidimensional images for telesurgical visualization

PURPOSE

We propose a combined floating autostereoscopic three-dimensional (3D) display approach for telesurgical visualization, which could reproduce live surgical scene in a realistic and intuitive manner.

METHODS

A polyhedron-shaped 3D display device is developed for spatially floating autostereoscopic 3D image. Integral videography (IV) technique is adopted to generate real-time 3D images. Combined two-dimensional (2D) and 3D displays are presented floatingly around the center of the display device through reflection of semitransparent mirrors. Intra-operative surgery information is fused and updated in the 3D display, so that telesurgical visualization could be enhanced remotely.

RESULTS

The experimental results showed that our approach can achieve a combined floating autostereoscopic display that presents 2D and 3D fusion images. The glasses-free IV 3D display has full parallax and can be observed by multiple persons from surrounding areas at the same time. Furthermore, real-time surgical scene could be presented and updated in a realistic and intuitive visualization platform. It is shown that the proposed method is feasible for facilitating telesurgical visualization.

CONCLUSION

The proposed floating autostereoscopic display device presents surgical information in an efficient form, so as to enhance operative cooperation and efficiency during operation. Combined presentation of imaging information is promising for medical applications.

Telerobotic Haptic Exploration in Art Galleries and Museums for Individuals with Visual Impairments

This paper presents a haptic telepresence system that enables visually impaired users to explore locations with rich visual observation such as art galleries and museums by using a telepresence robot, a RGB-D sensor (color and depth camera), and a haptic interface. The recent improvement on RGB-D sensors has enabled real-time access to 3D spatial information in the form of point clouds. However, the real-time representation of this data in the form of tangible haptic experience has not been challenged enough, especially in the case of telepresence for individuals with visual impairments. Thus, the proposed system addresses the real-time haptic exploration of remote 3D information through video encoding and real-time 3D haptic rendering of the remote real-world environment. This paper investigates two scenarios in haptic telepresence, i.e., mobile navigation and object exploration in a remote environment. Participants with and without visual impairments participated in our experiments based on the two scenarios, and the system performance was validated. In conclusion, the proposed framework provides a new methodology of haptic telepresence for individuals with visual impairments by providing an enhanced interactive experience where they can remotely access public places (art galleries and museums) with the aid of haptic modality and robotic telepresence.

A Dual-Arm 7-Degrees-of-Freedom Haptics-Enabled Teleoperation Test Bed for Minimally Invasive Surgery

This paper describes a dual-arm teleoperation (master-slave) system which has been developed to explore the effect of haptics in robotics-assisted minimally invasive surgery (RAMIS). This setup is capable of measuring forces in 7 degrees of freedom (DOF) and fully reflecting them to the operator through two 7-DOF haptic interfaces. An application of the test bed is in enabling the evaluation of the effect of replacing haptic feedback by other sensory cues such as visual representation of haptic information (sensory substitution). This paper discusses the design rationale, kinematic analysis and dynamic modeling of the robot manipulators, and the control system developed for the setup. Using the accurate model developed in this paper, a highly transparent haptics-enabled system can be achieved and used in robot-assisted telesurgery. Validation results obtained through experiments are presented and demonstrate the correctness and effectiveness of the developed models. The application of the setup for two RAMIS surgical tasks, a suture manipulation task and a tumor localization task, is described with different haptics modalities available through the developed haptics-enabled system for each application.

Surgical robot for single-incision laparoscopic surgery

This paper introduces a novel surgical robot for single-incision laparoscopic surgeries. The robot system includes the cone-type remote center-of-motion (RCM) mechanism and two articulated instruments having a flexible linkage-driven elbow. The RCM mechanism, which has two revolute joints and one prismatic joint, is designed to maintain a stationary point at the apex of the cone shape. By placing the stationary point on the incision area, the mechanism allows a surgical instrument to explore the abdominal area through a small incision point. The instruments have six articulated joints, including an elbow pitch joint, which make the triangulation position for the surgery possible inside of the abdominal area. The presented elbow pitch structure is similar to the slider-crank mechanism but the connecting rod is composed of a flexible leaf spring for high payload and small looseness error. We verified the payload of the robot is more than 10 N and described preliminary experiments on peg transfer and suture motion by using the proposed surgical robot.

[Robotic colorectal surgery: current status and future developments]

Robotic assistance has the potential to compensate for the limitations inherent in standard laparoscopic surgery. The daVinci® surgical system remains the only currently available commercial robotic system. It has found popularity in rectal cancer surgery where its application has consistently been shown to reduce the need to convert to open surgery. With this exception, the technological advances of the robotic system have not so far translated into any reproducible patient benefit. The first part of this manuscript presents an overview of the current daVinci® platform, its applications, the evidence base and future developments in colorectal surgery. The second part of the manuscript looks at other robot systems in development and the different innovations and strategies taken to advance minimally invasive surgery.The English full-text version of this article is available at SpringerLink (under supplemental).

Determination of the latency effects on surgical performance and the acceptable latency levels in telesurgery using the dV-Trainer(®) simulator

BACKGROUND

The primary limitation of telesurgery is the communication latency. Accurate and detailed data are lacking to reveal the latency effects on surgical performance; furthermore, the maximum acceptable latency in telesurgery remains unclear.

METHODS

Sixteen medical students performed an energy dissection exercise and a needle-driving exercise on the robotic simulator dV-Trainer(®), and latencies varying between 0 and 1,000 ms with a 100-ms interval were randomly and blindly presented. Task completion time, instrument motion, and errors were automatically recorded. The difficulty, security, precision, and fluidity of manipulation were self-scored by subjects between 0 and 4 (0 the best, 2 moderate, and 4 the worst).

RESULTS

Task completion time, motion, and errors increased gradually as latency increased. An exponential regression was fit to the mean times and motions (R (2) > 0.98). Subjective scorings of the four items were similar. The mean scores were less than 1 at delays ≤200 ms, then increased from 1 to 2 at 300-700 ms, and finally approached 3 at delays above. In both exercises, latencies ≤300 ms were judged to be safe by all and 400-500 ms were accepted by 66-75 % of subjects. Less than 20 % of subjects accepted delays ≥800 ms.

CONCLUSIONS

The surgical performance deteriorates in an exponential way as the latency increases. The delay impact on instrument manipulation is mild at 0-200 ms, then increases from small to large at 300-700 ms, and finally becomes very large at 800-1,000 ms. Latencies ≤200 ms are ideal for telesurgery; 300 ms is also suitable; 400-500 ms may be acceptable but are already tiring; and 600-700 ms are difficult to deal with and only acceptable for low risk and simple procedures. Surgery is quite difficult at 800-1,000 ms, telementoring would be a better choice in this case.