Review of robotic surgery platforms and end effectors

In the last 50 years, the number of companies producing automated devices for surgical operations has grown extensively. The population started to be more confident about the technology capabilities. The first patents related to surgical robotics are expiring and this knowledge is becoming a common base for the development of future surgical robotics. The review describes some of the most popular companies manufacturing surgical robots. The list of the company does not pretend to be exhaustive but wishes to give an overview of the sector. Due to space constraints, only a limited selction of companies is reported. Most of the companies described are born in America or Europe. Advantages and limitations of each product firm are described. A special focus is given to the end effectors; their shape and dexterity are crucial for the positive outcome of the surgical operations. New robots are developed every year, and existing robots are allowed to perform a wider range of procedures. Robotic technologies improve the abilities of surgeons in the domains of urology, gynecology, neurology, spine surgery, orthopedic reconstruction (knee, shoulder), hair restoration, oral surgery, thoracic surgery, laparoscopic surgery, and endoscopy.

Clinical applications of robotic surgery platforms: a comprehensive review

Robotic surgery has expanded globally across various medical specialties since its inception more than 20 years ago. Accompanying this expansion were significant technological improvements, providing tremendous benefits to patients and allowing the surgeon to perform with more precision and accuracy. This review lists some of the different types of platforms available for use in various clinical applications. We performed a literature review of PubMed and Web of Science databases in May 2023, searching for all available articles describing surgical robotic platforms from January 2000 (the year of the first approved surgical robot, da Vinci® System, by Intuitive Surgical) until May 1st, 2023. All retrieved robotic platforms were then divided according to their clinical application into four distinct groups: soft tissue robotic platforms, orthopedic robotic platforms, neurosurgery and spine platforms, and endoluminal robotic platforms. Robotic surgical technology has undergone a rapid expansion over the last few years. Currently, multiple robotic platforms with specialty-specific applications are entering the market. Many of the fields of surgery are now embracing robotic surgical technology. We review some of the most important systems in clinical practice at this time.

Robot-assisted implantation of additively manufactured patient-specific orthopaedic implants: evaluation in a sheep model

PURPOSE

Bone tumours must be surgically excised in one piece with a margin of healthy tissue. The unique nature of each bone tumour case is well suited to the use of patient-specific implants, with additive manufacturing allowing production of highly complex geometries. This work represents the first assessment of the combination of surgical robotics and patient-specific additively manufactured implants.

METHODS

The development and evaluation of a robotic system for bone tumour excision, capable of milling complex osteotomy paths, is described. The developed system was evaluated as part of an animal trial on 24 adult male sheep, in which robotic bone excision of the distal femur was followed by placement of patient-specific implants with operative time evaluated. Assessment of implant placement accuracy was completed based on post-operative CT scans.

RESULTS

A mean overall implant position error of 1.05 ± 0.53 mm was achieved, in combination with a mean orientation error of 2.38 ± 0.98°. A mean procedure time (from access to implantation, excluding opening and closing) of 89.3 ± 25.25 min was observed, with recorded surgical time between 58 and 133 min, with this approximately evenly divided between robotic (43.9 ± 15.32) and implant-based (45.4 ± 18.97) tasks.

CONCLUSIONS

This work demonstrates the ability for robotics to achieve repeatable and precise removal of complex bone volumes of the type that would allow en bloc removal of a bone tumour. These robotically created volumes can be precisely filled with additively manufactured patient-specific implants, with minimal gap between cut surface and implant interface.

Versatile end effector for laparoscopic robotic scrub nurse

PURPOSE

Integrating robotic scrub nurses in the operating room has the potential to help overcome staff shortages and limited use of available operating capacities in hospitals. Existing approaches of robotic scrub nurses are mainly focused on open surgical procedures, neglecting laparoscopic procedures. Laparoscopic interventions offer great potential for the context-sensitive integration of robotic systems due to possible standardization. However, the first step is to ensure the safe manipulation of laparoscopic instruments.

METHODS

A robotic platform with a universal gripper system was designed to pick up and place laparoscopic as well as da Vinci[Formula: see text] instruments in an efficient workflow. The robustness of the gripper system was studied using a test protocol, which included a force absorption test to determine the operational safety limits of the design and a grip test to determine the system performance.

RESULTS

The test protocol shows results regarding force and torque absorption capabilities of the end effector, which are essential when transferring an instrument to the surgeon to enable a robust handover. The grip tests show that the laparoscopic instruments can be safely picked up, manipulated and returned independent of unexpected positional deviations. The gripper system also enables the manipulation of da Vinci[Formula: see text] instruments, opening the door for robot-robot interaction.

CONCLUSION

Our evaluation tests have shown that our robotic scrub nurse with the universal gripper system can safely and robustly manipulate laparoscopic and da Vinci[Formula: see text] instruments. The system design will continue with the integration of context-sensitive capabilities.

Integration of a surgical robotic arm to the connected operating room via ISO IEEE 11073 SDC

PURPOSE

Since 2019, intraoperative networking with ISO IEEE 11073 SDC has, for the first time, enabled standardized multi-vendor data exchange between medical devices. For seamless plug-and-play integration of devices without previous configuration, further specifications for device profiles ("device specializations") on top of the existing core standards must be developed. These generic interfaces are then incorporated into the standardization process.

METHODS

An existing classification scheme of robotic assistance functions is being adopted and used as a baseline to derive functional requirements for a universal interface for modular robot arms. Additionally, the robot system requires machine-machine interfaces (MMI) to a surgical navigation system and a surgical planning software in order to carry out its function. Further technical requirements are derived from these MMI. The functional and technical requirements motivate the design of an SDC-compatible device profile. The device profile is then assessed for feasibility.

RESULTS

We present a new modeling of a device profile for surgical robotic arms intended for neurosurgery and orthopedic surgery. The modeling in SDC succeeds for the most part. However, some details of the proposed model cannot yet be realized within the framework of the existing SDC standards. Some aspects can already be realized, but could be better supported in the future by the nomenclature system. These improvements are being presented as well.

CONCLUSION

The proposed device profile presents a first step toward a uniform technical description model for modular surgical robot systems. The current SDC core standards lack some functionality to fully support the proposed device profile. These could be defined in future work and then included in standardization efforts.

A force-sensing surgical drill for real-time force feedback in robotic mastoidectomy

PURPOSE

Robotic assistance in otologic surgery can reduce the task load of operating surgeons during the removal of bone around the critical structures in the lateral skull base. However, safe deployment into the anatomical passageways necessitates the development of advanced sensing capabilities to actively limit the interaction forces between the surgical tools and critical anatomy.

METHODS

We introduce a surgical drill equipped with a force sensor that is capable of measuring accurate tool-tissue interaction forces to enable force control and feedback to surgeons. The design, calibration and validation of the force-sensing surgical drill mounted on a cooperatively controlled surgical robot are described in this work.

RESULTS

The force measurements on the tip of the surgical drill are validated with raw-egg drilling experiments, where a force sensor mounted below the egg serves as ground truth. The average root mean square error for points and path drilling experiments is 41.7 (± 12.2) mN and 48.3 (± 13.7) mN, respectively.

CONCLUSION

The force-sensing prototype measures forces with sub-millinewton resolution and the results demonstrate that the calibrated force-sensing drill generates accurate force measurements with minimal error compared to the measured drill forces. The development of such sensing capabilities is crucial for the safe use of robotic systems in a clinical context.

Deep reinforcement learning in continuous action space for autonomous robotic surgery

PURPOSE

Reinforcement learning methods have shown promising results for the automation of sub-tasks in robotic surgery systems. With the development of these methods, surgical robots have been able to achieve good performances, so that they can be used in complex and high-risk environments such as surgical pattern cutting to reduce stress and pressure on the surgeon and increase surgical accuracy. This study has aimed at providing a deep reinforcement learning-based approach to control the gripper arm when cutting soft tissue in a continuous action space.

METHODS

Surgical soft tissue cutting in this study is performed by controlling the gripper arm in a continuous action space and a grid observation space. In the proposed method using deep reinforcement learning, we find an optimal tensioning policy in the continuous action space that increases the cutting accuracy of the predetermined pattern.

RESULTS

The simulation results demonstrated that in the cutting of many complex patterns, the proposed method works better than the methods in which the tensioning was performed in a discrete action space and the observation space was modeled as a partial and random representation.

CONCLUSION

We introduced a deep reinforcement learning-based method for obtaining the optimal tensioning policy in a continuous action space when cutting a predetermined pattern. We showed that the proposed approach outperforms the state-of-the-art method in the soft pattern cutting task with respect to accuracy.

Quantifying the Impact of Signal-to-background Ratios on Surgical Discrimination of Fluorescent Lesions

PURPOSE

Surgical fluorescence guidance has gained popularity in various settings, e.g., minimally invasive robot-assisted laparoscopic surgery. In pursuit of novel receptor-targeted tracers, the field of fluorescence-guided surgery is currently moving toward increasingly lower signal intensities. This highlights the importance of understanding the impact of low fluorescence intensities on clinical decision making. This study uses kinematics to investigate the impact of signal-to-background ratios (SBR) on surgical performance.

METHODS

Using a custom grid exercise containing hidden fluorescent targets, a da Vinci Xi robot with Firefly fluorescence endoscope and ProGrasp and Maryland forceps instruments, we studied how the participants' (N = 16) actions were influenced by the fluorescent SBR. To monitor the surgeon's actions, the surgical instrument tip was tracked using a custom video-based tracking framework. The digitized instrument tracks were then subjected to multi-parametric kinematic analysis, allowing for the isolation of various metrics (e.g., velocity, jerkiness, tortuosity). These were incorporated in scores for dexterity (Dx), decision making (DM), overall performance (PS) and proficiency. All were related to the SBR values.

RESULTS

Multi-parametric analysis showed that task completion time, time spent in fluorescence-imaging mode and total pathlength are metrics that are directly related to the SBR. Below SBR 1.5, these values substantially increased, and handling errors became more frequent. The difference in Dx and DM between the targets that gave SBR < 1.50 and SBR > 1.50, indicates that the latter group generally yields a 2.5-fold higher Dx value and a threefold higher DM value. As these values provide the basis for the PS score, proficiency could only be achieved at SBR > 1.55.

CONCLUSION

By tracking the surgical instruments we were able to, for the first time, quantitatively and objectively assess how the instrument positioning is impacted by fluorescent SBR. Our findings suggest that in ideal situations a minimum SBR of 1.5 is required to discriminate fluorescent lesions, a substantially lower value than the SBR 2 often reported in literature.

Deep neural network architecture for automated soft surgical skills evaluation using objective structured assessment of technical skills criteria

PURPOSE

Classic methods of surgery skills evaluation tend to classify the surgeon performance in multi-categorical discrete classes. If this classification scheme has proven to be effective, it does not provide in-between evaluation levels. If these intermediate scoring levels were available, they would provide more accurate evaluation of the surgeon trainee.

METHODS

We propose a novel approach to assess surgery skills on a continuous scale ranging from 1 to 5. We show that the proposed approach is flexible enough to be used either for scores of global performance or several sub-scores based on a surgical criteria set called Objective Structured Assessment of Technical Skills (OSATS). We established a combined CNN+BiLSTM architecture to take advantage of both temporal and spatial features of kinematic data. Our experimental validation relies on real-world data obtained from JIGSAWS database. The surgeons are evaluated on three tasks: Knot-Tying, Needle-Passing and Suturing. The proposed framework of neural networks takes as inputs a sequence of 76 kinematic variables and produces an output float score ranging from 1 to 5, reflecting the quality of the performed surgical task.

RESULTS

Our proposed model achieves high-quality OSATS scores predictions with means of Spearman correlation coefficients between the predicted outputs and the ground-truth outputs of 0.82, 0.60 and 0.65 for Knot-Tying, Needle-Passing and Suturing, respectively. To our knowledge, we are the first to achieve this regression performance using the OSATS criteria and the JIGSAWS kinematic data.

CONCLUSION

An effective deep learning tool was created for the purpose of surgical skills assessment. It was shown that our method could be a promising surgical skills evaluation tool for surgical training programs.

[Robotics Helps Usher in a New Era of Neurosurgery]

The application of surgical robots in neurosurgery has formed a rapidly developing and fascinating new field that is revolutionizing the way neurosurgeries are performed. Herein, we discussed the prospects of the future development of neurosurgery robots. We found that, at present, surgical robots are most widely used in stereotactic surgeries in the field of neurosurgery. The use of surgical robots has greatly improved puncturing precision, but it cannot be used in other types of neurosurgeries.With the highly integrated development of imaging technology, mechanical technology, computer control technology, and artificial intelligence, surgical robotics will inevitably witness a surge of rapid development in line with the trend of contemporary needs. Surgical robotics will be applied to more fields of neurosurgery in the future, enhancing surgical safety and efficiency.

Transfer of learned dynamics between different surgical robots and operative configurations

PURPOSE

Using the da Vinci Research Kit (dVRK), we propose and experimentally demonstrate transfer learning (Xfer) of dynamics between different configurations and robots distributed around the world. This can extend recent research using neural networks to estimate the dynamics of the patient side manipulator (PSM) to provide accurate external end-effector force estimation, by adapting it to different robots and instruments, and in different configurations, with additional forces applied on the instruments as they pass through the trocar.

METHODS

The goal of the learned models is to predict internal joint torques during robot motion. First, exhaustive training is performed during free-space (FS) motion, using several configurations to include gravity effects. Second, to adapt to different setups, a limited amount of training data is collected and then the neural network is updated through Xfer.

RESULTS

Xfer can adapt a FS network trained on one robot, in one configuration, with a particular instrument, to provide comparable joint torque estimation for a different robot, in a different configuration, using a different instrument, and inserted through a trocar. The robustness of this approach is demonstrated with multiple PSMs (sampled from the dVRK community), instruments, configurations and trocar ports.

CONCLUSION

Xfer provides significant improvements in prediction errors without the need for complete training from scratch and is robust over a wide range of robots, kinematic configurations, surgical instruments, and patient-specific setups.

Modeling and Operator Control of a Robotic Tool for Bidirectional Manipulation in Targeted Prostate Biopsy

This work introduces design, manipulation, and operator control of a bidirectional robotic tool for minimally invasive targeted prostate biopsy. The robotic tool is purposed to be used as a compliant flexure section of active biopsy needles. The design of the robotic tool comprises of a flexure section fabricated on a nitinol tube that enables bidirectional bending via actuation of two internal tendons. The statics of the flexure section is presented and validated with experimental data. Finally, the capability of the robotic tool to reach targeted positions inside prostate gland is evaluated.

Endoscopic surgery suturing techniques: a randomized study on learning

BACKGROUND

Surgeons have widely adopted endoscopic suturing techniques using conventional laparoscopic instruments and the more advanced robotic systems. The FlexDex is a novel articulating laparoscopic needle driver providing enhanced dexterity in laparoscopic surgery. This study evaluates and compares the learning curve of endoscopic suturing with conventional laparoscopy, the FlexDex and robotic suturing in novices.

METHODS

Participants performed a minimal invasive suturing task in three different ways in a randomized order: with a conventional laparoscopic needle driver, using the FlexDex needle driver and third, using the Da Vinci Si surgical system. Primary outcome was suturing task time. Secondary outcome parameters were assessment of suturing quality and workload perception.

RESULTS

A total of 10 novice participants were included and completed a total of 300 sessions. Median (IQR) suturing time of the first 5 sessions was 231 s (188-291) in the laparoscopic group versus 378 s (282-471) in the FlexDex group versus 189 s (160-247) in the DaVinci Si group. The last 5 sessions showed significant reduction of median suturing time of 143 s (120-190), 232 s (180-265) and 172 s (134-199) respectively. Analysis identified that the learning curve for the laparoscopic needle driver and Da Vinci Si was reached in 5 sessions, compared to 8 sessions for the Flexdex. The laparoscopic needle driver and Da Vinci Si showed a significant shorter median suturing time compared to the FlexDex (p = 0.00). The FlexDex quality assessment scores were significantly lower compared to the laparoscopic (p = 0.00) and robotic (p = 0.00) scores and perceived workload remains high for the FlexDex users.

CONCLUSIONS

Ex vivo endoscopic suturing with the FlexDex demonstrated a prolonged learning curve compared to laparoscopic and robotic suturing. The learning curve of the FlexDex is fundamentally different from conventional laparoscopic and robotic instruments. This study provides further insights in the implementation and training of endoscopic suturing techniques.

Development and experiments of a continuum robotic system for transoral laryngeal surgery

PURPOSE

Currently, self-retaining laryngoscopic surgery is not suitable for some patients, and there are dead zones relating to surgical field exposure and operation. The quality of the surgery can also be affected by the long periods of time required to complete it. Teleoperated continuum robots with flexible joints are expected to solve these issues. However, at the current stage of developing transoral robotic surgery systems, their large size affects the precision of surgical operative actions and there are high development and treatment costs.

METHODS

We fabricated a flexible joint based on selective laser melting technology and designed a shallow neural network-based kinematic modeling approach for a continuum surgical robot. Then, human model and animal experiments were completed by master-slave teleoperation to verify the force capability and dexterity of the robot, respectively.

RESULTS

As verified by human model and animal experiments, the designed continuum robot was demonstrated to achieve transoral laryngeal surgical field exposure without laryngoscope assistance, with sufficient load capability to finish the biopsy of vocal fold tissue in living animals.

CONCLUSION

The designed continuum robotic system allows the biopsy of vocal fold tissue without laryngoscope assistance. Its stiffness and dexterity indicate the system's potential for applications in the diagnosis and treatment of vocal fold nodules and polyps. The limitations of this robotic system as shown in the experiments were also analyzed.

A variable baseline stereoscopic camera with fast deployable structure for natural orifice transluminal endoscopic surgery

PURPOSE

Stereo vision can provide surgeons with 3D images and reduce the difficulty of operation in robot-assisted surgery. In natural orifice transluminal endoscopic surgery, distortions of the stereoscopic images could be induced at different observation depths. This would increase the risk of surgery. We proposed a novel camera to solve this problem.

METHODS

This study integrated the camera calibration matrix and the geometric model of stereoscopic system to find the cause of distortion. It was found that image distortions were caused by inappropriate disparity, and this could be avoided by changing the camera baseline. We found the relationship between camera baseline and observation depth with the model. A variable baseline stereoscopic camera with deployable structure was designed to achieve this requirement. The baseline could be adjusted to provide appropriate disparity.

RESULTS

Three controlled experiments were conducted to verify the stereo vision of the proposed camera at different observation depths. No significant difference was observed in the completion time. At the observation depths of 30 mm and 90 mm, the number of errors apparently decreased by 62.90% and 51.06%, respectively.

CONCLUSIONS

The significant decrease in number of errors shows that the proposed camera has a better stereo vision than a regular camera at both small and large observation depths. It can produce more accurate stereoscopic images at any depth. This will further improve the safety of robot-assisted surgery.

Investigation of Micro-motion Kinematics of Continuum Robots for Volumetric OCT and OCT-guided Visual Servoing

Continuum robots (CR) have been recently shown capable of micron-scale motion resolutions. Such motions are achieved through equilibrium modulation using indirect actuation for altering either internal preload forces or changing the cross-sectional stiffness along the length of a continuum robot. Previously reported, but unexplained, turning point behavior is modeled using two approaches. An energy minimization approach is first used to explain the source of this behavior. Subsequently, a kinematic model using internal constraints in multi-backbone CRs is used to replicate this turning point behavior. An approach for modeling the micro-motion differential kinematics is presented using experimental data based on the solution of a system of linear matrix equations. This approach provides a closed-form approximation of the empirical micro-motion kinematics and could be easily used for real-time control. A motivating application of image-based biopsy using 3D optical coherence tomography (OCT) is envisioned and demonstrated in this paper. A system integration for generating OCT volumes by sweeping a custom B-mode OCT probe is presented. Results showing high accuracy in obtaining 3D OCT measurements are shown using a commercial OCT probe. Qualitative results using a miniature probe integrated within the robot are also shown. Finally, closed-loop visual servoing using OCT data is demonstrated for guiding a needle into an agar channel. Results of this paper present what we believe is the first embodiment of a continuum robot capable of micro and macro motion control for 3D OCT imaging. This approach can support the development of new technologies for CRs capable of surgical intervention and micro-motion for ultra-precision tasks.

Future Directions in Robotic Neurosurgery

In this paper, we highlight promising technologies in each phase of a robotic neurosurgery operation, and identify key factors affecting how quickly these technologies will mature into products in the operating room. We focus on specific technology trends in image-guided cranial and spinal procedures, including advances in imaging, machine learning, robotics, and novel interfaces. For each technology, we discuss the required effort to overcome safety or implementation challenges, as well as identifying example regulatory approved products in related fields for comparison. The goal is to provide a roadmap for clinicians as to which robotic and automation technologies are in the developmental pipeline, and which ones are likely to impact their practice sooner, rather than later.

The Beginning of a New Era: Artificial Intelligence in Healthcare

The healthcare sector is considered to be one of the largest and fast-growing industries in the world. Innovations and novel approaches have always remained the prime aims in order to bring massive development. Before the emergence of technology, the healthcare sector was dependent on manpower, which was time-consuming and less accurate with lack of efficiency. With the recent advancements in machine learning, the condition has been steadily revolutionizing. Artificial intelligence (AI) lies in the computer science department, which stresses on the intelligent machines’ creation, that work and react just like human beings. Currently, the applications of AI have been expanding into those fields, which was once thought to be the only domain of human expertise such as healthcare sector. In this review, we have shed light on the present usage of AI in the healthcare sector, such as its working, and the way this system is being implemented in different domains, such as drug discovery, diagnosis of diseases, clinical trials, remote patient monitoring, and nanotechnology. We have also briefly touched upon its applications in other sectors as well. The public opinions have also been analyzed and discussed along with the future prospects. We have discussed the merits, and the other side of AI, i.e. the disadvantages in the last part of the manuscript.

Robotic and Endoscopic Approaches to Head and Neck Surgery

Technological developments have disrupted the practice of medicine throughout history. Endoscopic and robotic techniques in head and neck surgery have emerged over the past half-century and have been incrementally adapted to expanding indications within otolaryngology. Robotic and endoscopic surgery have an established role in treatment of oropharyngeal and laryngeal cancers, reducing surgical morbidity and improving survival relative to traditional open approaches. Surgical treatment of human papillomavirus-mediated oropharyngeal cancer via transoral robotic surgery offers equivalent oncologic and functional outcomes relative to radiotherapy. Newer iterations of single-port robotic systems continue to expand the scope of robotics in head and neck surgery.

Mask then classify: multi-instance segmentation for surgical instruments

PURPOSE

The detection and segmentation of surgical instruments has been a vital step for many applications in minimally invasive surgical robotics. Previously, the problem was tackled from a semantic segmentation perspective, yet these methods fail to provide good segmentation maps of instrument types and do not contain any information on the instance affiliation of each pixel. We propose to overcome this limitation by using a novel instance segmentation method which first masks instruments and then classifies them into their respective type.

METHODS

We introduce a novel method for instance segmentation where a pixel-wise mask of each instance is found prior to classification. An encoder-decoder network is used to extract instrument instances, which are then separately classified using the features of the previous stages. Furthermore, we present a method to incorporate instrument priors from surgical robots.

RESULTS

Experiments are performed on the robotic instrument segmentation dataset of the 2017 endoscopic vision challenge. We perform a fourfold cross-validation and show an improvement of over 18% to the previous state-of-the-art. Furthermore, we perform an ablation study which highlights the importance of certain design choices and observe an increase of 10% over semantic segmentation methods.

CONCLUSIONS

We have presented a novel instance segmentation method for surgical instruments which outperforms previous semantic segmentation-based methods. Our method further provides a more informative output of instance level information, while retaining a precise segmentation mask. Finally, we have shown that robotic instrument priors can be used to further increase the performance.

Cross-modal self-supervised representation learning for gesture and skill recognition in robotic surgery

PURPOSE

Multi- and cross-modal learning consolidates information from multiple data sources which may offer a holistic representation of complex scenarios. Cross-modal learning is particularly interesting, because synchronized data streams are immediately useful as self-supervisory signals. The prospect of achieving self-supervised continual learning in surgical robotics is exciting as it may enable lifelong learning that adapts to different surgeons and cases, ultimately leading to a more general machine understanding of surgical processes.

METHODS

We present a learning paradigm using synchronous video and kinematics from robot-mediated surgery. Our approach relies on an encoder-decoder network that maps optical flow to the corresponding kinematics sequence. Clustering on the latent representations reveals meaningful groupings for surgeon gesture and skill level. We demonstrate the generalizability of the representations on the JIGSAWS dataset by classifying skill and gestures on tasks not used for training.

RESULTS

For tasks seen in training, we report a 59 to 70% accuracy in surgical gestures classification. On tasks beyond the training setup, we note a 45 to 65% accuracy. Qualitatively, we find that unseen gestures form clusters in the latent space of novice actions, which may enable the automatic identification of novel interactions in a lifelong learning scenario.

CONCLUSION

From predicting the synchronous kinematics sequence, optical flow representations of surgical scenes emerge that separate well even for new tasks that the model had not seen before. While the representations are useful immediately for a variety of tasks, the self-supervised learning paradigm may enable research in lifelong and user-specific learning.

MINARO HD: control and evaluation of a handheld, highly dynamic surgical robot

Purpose

Current surgical robotic systems are either large serial arms, resulting in higher risks due to their high inertia and no inherent limitations of the working space, or they are bone-mounted, adding substantial additional task steps to the surgical workflow. The robot presented in this paper has a handy and lightweight design and can be easily held by the surgeon. No rigid fixation to the bone or a cart is necessary. A high-speed tracking camera together with a fast control system ensures the accurate positioning of a burring tool.

Methods

The capabilities of the robotic system to dynamically compensate for unintended motion, either of the robot itself or the patient, was evaluated. Therefore, the step response was analyzed as well as the capability to follow a moving target.

Results

The step response show that the robot can compensate for undesired motions up to 12 Hz in any direction. While following a moving target, a maximum positioning error of 0.5 mm can be obtained with a target motion of up to 18 mm/s.

Conclusion

The requirements regarding dynamic motion compensation, accuracy, and machining speed of unicompartmental knee arthroplasties, for which the robot was optimized, are achieved with the presented robotic system. In particular, the step response results show that the robot is able to compensate for human tremor.

Usability of cooperative surgical telemanipulation for bone milling tasks

Purpose

Cooperative surgical systems enable humans and machines to combine their individual strengths and collaborate to improve the surgical outcome. Cooperative telemanipulated systems offer the widest spectrum of cooperative functionalities, because motion scaling is possible. Haptic guidance can be used to assist surgeons and haptic feedback makes acting forces at the slave side transparent to the operator, however, overlapping and masking of forces needs to be avoided. This study evaluates the usability of a cooperative surgical telemanipulator in a laboratory setting.

Methods

Three experiments were designed and conducted for characteristic surgical task scenarios derived from field studies in orthopedics and neurosurgery to address bone tissue differentiation, guided milling and depth sensitive milling. Interaction modes were designed to ensure that no overlapping or masking of haptic guidance and haptic feedback occurs when allocating information to the haptic channel. Twenty participants were recruited to compare teleoperated modes, direct manual execution and an exemplary automated milling with respect to usability.

Results

Participants were able to differentiate compact and cancellous bone, both directly manually and teleoperatively. Both telemanipulated modes increased effectiveness measured by the mean absolute depth and contour error for guided and depth sensitive millings. Efficiency is decreased if solely a boundary constraint is used in hard material, while a trajectory guidance and manual milling perform similarly. With respect to subjective user satisfaction trajectory guidance is rated best for guided millings followed by boundary constraints and the direct manual interaction. Haptic feedback only improved subjective user satisfaction.

Conclusion

A cooperative surgical telemanipulator can improve effectiveness and efficiency close to an automated execution and enhance user satisfaction compared to direct manual interaction. At the same time, the surgeon remains part of the control loop and is able to adjust the surgical plan according to the intraoperative situation and his/her expertise at any time.

Evaluation of long-term stability of monolithic 3D-printed robotic manipulator structures for minimally invasive surgery

Purpose

In the era of patient-centered medicine, clinical procedures, tools and instruments should be individually adapted to the patient. In this context, the presented 3D-printed Single-Port Overtube Manipulator System follows the aims to provide patient- and task-specific disposable manipulators for minimally invasive surgery. In a first experiment, the robustness of the monolithic flexure hinge structures in use as robotic manipulators will be investigated.

Methods

Customizable monolithic manipulator structures designed by means of an automated design process and manufactured with selective laser sintering were investigated with regard to long-term stability in an endurance test. Therefore, a bare manipulator arm, an arm equipped with a standard instrument and finally loaded with an additional load of 0.5 N were evaluated by continuously following a trajectory within the workspace of the manipulator arms over a period of 90 min.

Results

The unloaded manipulator as well as the manipulator arm equipped with a standard instrument showed a sufficient reproducibility (deviation of 1.5 mm and 2.5 mm, respectively, on average) with regard to an application as telemanipulated master–slave surgical robotic system. The 3D-printed manipulators showed no damage and maintained integrity after the experiment.

Conclusion

It has been shown that 3D-printed manipulators in principle are suitable for use as disposable surgical manipulator systems and offer a long-term stability over at least 90 min. The developed manipulator design shows great potential for the production of patient-, task- and user-specific robot systems. However, the manipulator geometries as well as the control strategies still show room for improvements.

Cadaveric feasibility study of a teleoperated parallel continuum robot with variable stiffness for transoral surgery

Robot-assisted technologies are overcoming the limitations of the current approaches for transoral surgeries, which are suffering from limited vision and workspace. As a result, we develop a novel teleoperated parallel continuum robot with variable stiffness for collision avoidance. This paper focuses on the feasibility study on a cadaveric model for the robotic system as a first trial. We introduce the configuration of the robotic system, the description of the processes of the trial, including the setting of the robotic system, the test of stiffness, and the action of the manipulation. The contact force between the manipulators with different stiffness and the surrounding tissues and a series of surgical operations of the manipulator, including grasping, cutting, pushing, and pulling tissues under the master-slave control mode, were recorded and analyzed. Experimental results suggest that the typical surgical procedure on a cadaveric model was successfully performed. Moreover, the efficacy and feasibility of the developed robotic system are verified to satisfy the requirements of transoral robotic surgery (TORS). Graphical abstract.

Optic Nerve Sheath Fenestration With a Multi-Arm Continuum Robot

This article presents a medical robotic system for deep orbital interventions, with a focus on Optic Nerve Sheath Fenestration (ONSF). ONSF is a currently invasive ophthalmic surgical approach that can reduce potentially blinding elevated hydrostatic intracranial pressure on the optic disc via an incision on the optic nerve. The prototype is a multi-arm system capable of dexterous manipulation and visualization of the optic nerve area, allowing for a minimally invasive approach. Each arm is an independently controlled concentric tube robot collimated by a bespoke guide that is secured on the eye sclera via sutures. In this article, we consider the robot's end-effector design in order to reach/navigate the optic nerve according to the clinical requirements of ONSF. A prototype of the robot was engineered, and its ability to penetrate the optic nerve was analysed by conducting ex vivo experiments on porcine optic nerves and comparing their stiffness to human ones. The robot was successfully deployed in a custom-made realistic eye phantom. Our simulation studies and experimental results demonstrate that the robot can successfully navigate to the operation site and carry out the intervention.

Investigating exploration for deep reinforcement learning of concentric tube robot control

Purpose

Concentric tube robots are composed of multiple concentric, pre-curved, super-elastic, telescopic tubes that are compliant and have a small diameter suitable for interventions that must be minimally invasive like fetal surgery. Combinations of rotation and extension of the tubes can alter the robot’s shape but the inverse kinematics are complex to model due to the challenge of incorporating friction and other tube interactions or manufacturing imperfections. We propose a model-free reinforcement learning approach to form the inverse kinematics solution and directly obtain a control policy.

Method

Three exploration strategies are shown for deep deterministic policy gradient with hindsight experience replay for concentric tube robots in simulation environments. The aim is to overcome the joint to Cartesian sampling bias and be scalable with the number of robotic tubes. To compare strategies, evaluation of the trained policy network to selected Cartesian goals and associated errors are analyzed. The learned control policy is demonstrated with trajectory following tasks.

Results

Separation of extension and rotation joints for Gaussian exploration is required to overcome Cartesian sampling bias. Parameter noise and Ornstein–Uhlenbeck were found to be optimal strategies with less than 1 mm error in all simulation environments. Various trajectories can be followed with the optimal exploration strategy learned policy at high joint extension values. Our inverse kinematics solver in evaluation has 0.44 mm extension and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.3^{\circ }$$\end{document}0.3∘ rotation error.

Conclusion

We demonstrate the feasibility of effective model-free control for concentric tube robots. Directly using the control policy, arbitrary trajectories can be followed and this is an important step towards overcoming the challenge of concentric tube robot control for clinical use in minimally invasive interventions.

A multi-camera, multi-view system for training and skill assessment for robot-assisted surgery

PURPOSE

This paper introduces the concept of using an additional intracorporeal camera for the specific goal of training and skill assessment and explores the benefits of such an approach. This additional camera can provide an additional view of the surgical scene, and we hypothesize that this additional view would improve surgical training and skill assessment in robot-assisted surgery.

METHODS

We developed a multi-camera, multi-view system, and we conducted two user studies ([Formula: see text]) to evaluate its effectiveness for training and skill assessment. In the training user study, subjects were divided into two groups: a single-view group and a dual-view group. The skill assessment study was a within-subject study, in which every subject was shown single- and dual view recorded videos of a surgical training task, and the goal was to count the number of errors committed in each video.

RESULTS

The results show the effectiveness of using an additional intracorporeal camera view for training and skill assessment. The benefits of this view are modest for skill assessment as it improves the assessment accuracy by approximately 9%. For training, the additional camera view is clearly more effective. Indeed, the dual-view group is 57% more accurate than the single-view group in a retention test. In addition, the dual-view group is 35% more accurate and 25% faster than the single-view group in a transfer test.

CONCLUSION

A multi-camera, multi-view system has the potential to significantly improve training and moderately improve skill assessment in robot-assisted surgery. One application of our work is to include an additional camera view in existing virtual reality surgical training simulators to realize its benefits in training. The views from the additional intracorporeal camera can also be used to improve on existing surgical skill assessment criteria used in training systems for robot-assisted surgery.

Ultrasound 3D reconstruction of malignant masses in robotic-assisted partial nephrectomy using the PAF rail system: a comparison study

Purpose

In robotic-assisted partial nephrectomy (RAPN), the use of intraoperative ultrasound (IOUS) helps to localise and outline the tumours as well as the blood vessels within the kidney. The aim of this work is to evaluate the use of the pneumatically attachable flexible (PAF) rail system for US 3D reconstruction of malignant masses in RAPN. The PAF rail system is a novel device developed and previously presented by the authors to enable track-guided US scanning.

Methods

We present a comparison study between US 3D reconstruction of masses based on: the da Vinci Surgical System kinematics, single- and stereo-camera tracking of visual markers embedded on the probe. An US-realistic kidney phantom embedding a mass is used for testing. A new design for the US probe attachment to enhance the performance of the kinematic approach is presented. A feature extraction algorithm is proposed to detect the margins of the targeted mass in US images.

Results

To evaluate the performance of the investigated approaches the resulting 3D reconstructions have been compared to a CT scan of the phantom. The data collected indicates that single camera reconstruction outperformed the other approaches, reconstructing with a sub-millimetre accuracy the targeted mass.

Conclusions

This work demonstrates that the PAF rail system provides a reliable platform to enable accurate US 3D reconstruction of masses in RAPN procedures. The proposed system has also the potential to be employed in other surgical procedures such as hepatectomy or laparoscopic liver resection.

Augmentation of haptic feedback for teleoperated robotic surgery

PURPOSE

A frequently mentioned lack of teleoperated surgical robots is the lack of haptic feedback. Haptics are not only able to mirror force information from the situs, but also to provide spatial guidance according to a surgical plan. However, superposition of the two haptic information can lead to overlapping and masking of the feedback and guidance forces. This study investigates different approaches toward a combination of both information and investigates effects on system usability.

METHODS

Preliminary studies are conducted to define parameters for two main experiments. The two main experiments constitute simulated surgical interventions where haptic guidance as well as haptic feedback provide information for the surgeon. The first main experiment considers drilling for pedicle screw placements, while the second main experiment refers to three-dimensional milling tasks such as during partial knee replacements or craniectomies. For both experiments, different guidance modes in combination with haptic feedback are evaluated regarding effectiveness (e.g., distance to target depth), efficiency and user satisfaction (e.g., detectability of discrepancies in case of technical guidance error).

RESULTS

Regarding pedicle screw placements a combination of a peripheral visual signal and a vibration constitutes a good compromise regarding distance to target depth and detectability of discrepancies. For milling tasks, trajectory guidance is able to improve efficiency and user satisfaction (e.g., perceived workload), while boundary constraints improve effectiveness. If, assistance cannot be offered in all degrees of freedom (e.g., craniectomies), a visual substitution of the haptic force feedback shows the best results, though participants prefer using haptic force feedback.

CONCLUSION

Our results suggest that in case haptic feedback and haptic assistance are combined appropriately, benefits of both haptic modalities can be exploited. Thereby, capabilities of the human-machine system are improved compared to usage of exclusively one of the haptic information.

A "eye-in-body" integrated surgery robot system for stereotactic surgery

PURPOSE

Current stereotactic surgical robots system relies on cumbersome operations such as calibration, tracking and registration to establish the accurate intraoperative coordinate transformation chain, which makes the system not easy to use. To overcome this problem, a novel stereotactic surgical robot system has been proposed and validated.

METHODS

First, a hand-eye integrated scheme is proposed to avoid the intraoperative calibration between robot arm and motion tracking system. Second, a special reference-tool-based patient registration and tracking method is developed to avoid intraoperative registration. Third, a model-free visual servo method is used to reduce the accuracy requirement of hand-eye relationship and robot kinematic model. Finally, a prototype of the system is constructed and performance tests and a pedicle screw drilling experiment are performed.

RESULTS

The results show that the proposed system has acceptable accuracy. The target positioning error in the plane was - 0.68 ± 0.52 mm and 0.06 ± 0.41 mm. The orientation error was 0.43 ± 0.25°. The pedicle screw drilling experiment shows that the system can complete accurate stereotactic surgery.

CONCLUSIONS

The stereotactic surgical robot system described in this paper can perform stereotactic surgery without the intraoperative hand-eye calibration and nor manual registration and can achieve an acceptable position and orientation accuracy while tolerating the errors in the hand-eye coordinate transformation error and the robot kinematics model error.

A "pickup" stereoscopic camera with visual-motor aligned control for the da Vinci surgical system: a preliminary study

PURPOSE

The current state-of-the-art surgical robotic systems use only a single endoscope to view the surgical field. Research has been conducted to introduce additional cameras to the surgical system, giving rise to new camera angles that cannot be achieved using the endoscope alone. While this additional visualization certainly aids in surgical performance, current systems lack visual-motor compatibility with respect to the additional camera views. We propose a new system that overcomes this limitation.

METHODS

In this paper, we introduce a novel design of an additional "pickup" camera that can be integrated into the da Vinci Surgical System. We also introduce a solution to work comfortably in the various arbitrary views this camera provides by eliminating visual-motor misalignment. This is done by changing the working frame of the surgical instruments to work with respect to the coordinate system at the "pickup" camera instead of the endoscope.

RESULTS

Human user trials ([Formula: see text]) were conducted to evaluate the effect of visual-motor alignment with respect to the "pickup" camera on surgical performance. An inanimate surgical peg transfer task from the validated Fundamentals of Laparoscopic Surgery (FLS) Training Curriculum was used, and an improvement of 73% in task completion time and 80% in accuracy was observed with the visual-motor alignment over the case without it.

CONCLUSION

Our study shows that there is a requirement to achieve visual-motor alignment when utilizing views from external cameras in current clinical surgical robotics setups. We introduce a complete system that provides additional camera views with visual-motor aligned control. Such a system would be useful in existing surgical procedures and could also impact surgical planning and navigation.

On the feasibility of transperineal 3D ultrasound image guidance for robotic radical prostatectomy

PURPOSE

Prostate cancer is the most prevalent form of male-specific cancers. Robot-assisted laparoscopic radical prostatectomy (RALRP) using the da Vinci surgical robot has become the gold-standard treatment for organ-confined prostate cancer. To improve intraoperative visualization of anatomical structures, many groups have developed techniques integrating transrectal ultrasound (TRUS) into the surgical workflow. TRUS, however, is intrusive and does not provide real-time volumetric imaging.

METHODS

We propose a proof-of-concept system offering an alternative noninvasive transperineal view of the prostate and surrounding structures using 3D ultrasound (US), allowing for full-volume imaging in any anatomical plane desired. The system aims to automatically track da Vinci surgical instruments and display a real-time US image registered to preoperative MRI. We evaluate the approach using a custom prostate phantom, an iU22 (Philips Healthcare, Bothell, WA) US machine with an xMATRIX X6-1 transducer, and a custom probe fixture. A novel registration method between the da Vinci kinematic frame and 3D US is presented. To evaluate the entire registration pipeline, we use a previously developed MRI to US deformable registration algorithm.

RESULTS

Our US calibration technique yielded a registration error of 0.84 mm, compared to 1.76 mm with existing methods. We evaluated overall system error with a prostate phantom, achieving a target registration error of 2.55 mm.

CONCLUSION

Transperineal imaging using 3D US is a promising approach for image guidance during RALRP. Preliminary results suggest this system is comparable to existing guidance systems using TRUS. With further development and testing, we believe our system has the potential to improve patient outcomes by imaging anatomical structures and prostate cancer in real time.

EyeSAM: graph-based localization and mapping of retinal vasculature during intraocular microsurgery

PURPOSE

Robot-assisted intraocular microsurgery can improve performance by aiding the surgeon in operating on delicate micron-scale anatomical structures of the eye. In order to account for the eyeball motion that is typical in intraocular surgery, there is a need for fast and accurate algorithms that map the retinal vasculature and localize the retina with respect to the microscope.

METHODS

This work extends our previous work by a graph-based SLAM formulation using a sparse incremental smoothing and mapping (iSAM) algorithm.

RESULTS

The resulting technique, "EyeSAM," performs SLAM for intraoperative vitreoretinal surgical use while avoiding spurious duplication of structures as with the previous simpler technique. The technique also yields reduction in average pixel error in the camera motion estimation.

CONCLUSIONS

This work provides techniques to improve intraoperative tracking of retinal vasculature by handling loop closures and achieving increased robustness to quick shaky motions and drift due to uncertainties in the motion estimation.

Conditions for reliable grip force and jaw angle estimation of da Vinci surgical tools

PURPOSE

This work presents an estimation technique as well as corresponding conditions which are necessary to produce an accurate estimate of grip force and jaw angle on a da Vinci surgical tool using back-end sensors alone.

METHODS

This work utilizes an artificial neural network as the regression estimator on a dataset acquired from custom hardware on the proximal and distal ends. Through a series of experiments, we test the effect of estimation accuracy due to change in operating frequency, using the opposite jaw, and using different tools. A case study is then presented comparing our estimation technique with direct measurements of material response curves on two synthetic tissue surrogates.

RESULTS

We establish the following criteria as necessary to produce an accurate estimate: operate within training frequency bounds, use the same side jaw, and use the same tool. Under these criteria, an average root mean square error of 1.04 mN m in grip force and 0.17 degrees in jaw angle is achieved. Additionally, applying these criteria in the case study resulted in direct measurements which fell within the 95% confidence bands of our estimation technique.

CONCLUSION

Our estimation technique, along with important training criteria, is presented herein to further improve the literature pertaining to grip force estimation. We propose the training criteria to begin establishing bounds on the applicability of estimation techniques used for grip force estimation for eventual translation into clinical practice.

Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery

PURPOSE

With the advent of robot-assisted surgery, the role of data-driven approaches to integrate statistics and machine learning is growing rapidly with prominent interests in objective surgical skill assessment. However, most existing work requires translating robot motion kinematics into intermediate features or gesture segments that are expensive to extract, lack efficiency, and require significant domain-specific knowledge.

METHODS

We propose an analytical deep learning framework for skill assessment in surgical training. A deep convolutional neural network is implemented to map multivariate time series data of the motion kinematics to individual skill levels.

RESULTS

We perform experiments on the public minimally invasive surgical robotic dataset, JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS). Our proposed learning model achieved competitive accuracies of 92.5%, 95.4%, and 91.3%, in the standard training tasks: Suturing, Needle-passing, and Knot-tying, respectively. Without the need of engineered features or carefully tuned gesture segmentation, our model can successfully decode skill information from raw motion profiles via end-to-end learning. Meanwhile, the proposed model is able to reliably interpret skills within a 1-3 second window, without needing an observation of entire training trial.

CONCLUSION

This study highlights the potential of deep architectures for efficient online skill assessment in modern surgical training.

Combined OCT distance and FBG force sensing cannulation needle for retinal vein cannulation: in vivo animal validation

PURPOSE

Retinal vein cannulation is an experimental procedure during which a clot-dissolving drug is injected into an obstructed retinal vein. However, due to the fragility and minute size of retinal veins, such procedure is considered too risky to perform manually. With the aid of surgical robots, key limiting factors such as: unwanted eye rotations, hand tremor and instrument immobilization can be tackled. However, local instrument anatomy distance and force estimation remain unresolved issues. A reliable, real-time local interaction estimation between instrument tip and the retina could be a solution. This paper reports on the development of a combined force and distance sensing cannulation needle, and its experimental validation during in vivo animal trials.

METHODS

Two prototypes are reported, relying on force and distance measurements based on FBG and OCT A-scan fibres, respectively. Both instruments provide an 80 [Formula: see text] needle tip and have outer shaft diameters of 0.6 and 2.3 mm, respectively.

RESULTS

Both prototypes were characterized and experimentally validated ex vivo. Then, paired with a previously developed surgical robot, in vivo experimental validation was performed. The first prototype successfully demonstrated the feasibility of using a combined force and distance sensing instrument in an in vivo setting.

CONCLUSION

The results demonstrate the feasibility of deploying a combined sensing instrument in an in vivo setting. The performed study provides a foundation for further work on real-time local modelling of the surgical scene. This paper provides initial insights; however, additional processing remains necessary.

Design, Modelling and Teleoperation of a 2 mm Diameter Compliant Instrument for the da Vinci Platform

This work explores the feasibility of creating and accurately controlling an instrument for robotic surgery with a 2 mm diameter and a three degree-of-freedom (DoF) wrist which is compatible with the da Vinci platform. The instrument's wrist is composed of a two DoF bending notched-nitinol tube pattern, for which a kinematic model has been developed. A base mechanism for controlling the wrist is designed for integration with the da Vinci Research Kit. A basic teleoperation task is successfully performed using two of the miniature instruments. The performance and accuracy of the instrument suggest that creating and accurately controlling a 2 mm diameter instrument is feasible and the design and modelling proposed in this work provide a basis for future miniature instrument development.

Varying ultrasound power level to distinguish surgical instruments and tissue

We investigate a new framework of surgical instrument detection based on power-varying ultrasound images with simple and efficient pixel-wise intensity processing. Without using complicated feature extraction methods, we identified the instrument with an estimated optimal power level and by comparing pixel values of varying transducer power level images. The proposed framework exploits the physics of ultrasound imaging system by varying the transducer power level to effectively distinguish metallic surgical instruments from tissue. This power-varying image-guidance is motivated from our observations that ultrasound imaging at different power levels exhibit different contrast enhancement capabilities between tissue and instruments in ultrasound-guided robotic beating-heart surgery. Using lower transducer power levels (ranging from 40 to 75% of the rated lowest ultrasound power levels of the two tested ultrasound scanners) can effectively suppress the strong imaging artifacts from metallic instruments and thus, can be utilized together with the images from normal transducer power levels to enhance the separability between instrument and tissue, improving intraoperative instrument tracking accuracy from the acquired noisy ultrasound volumetric images. We performed experiments in phantoms and ex vivo hearts in water tank environments. The proposed multi-level power-varying ultrasound imaging approach can identify robotic instruments of high acoustic impedance from low-signal-to-noise-ratio ultrasound images by power adjustments.

Precisely positioning the tip of an instrument inserted through an orifice with a free wrist robot: application to prostate biopsies

PURPOSE

Robots with a spherical unactuated wrist can be used for minimally invasive surgery. With such a robot, positioning the wrist center controls the instrument tip position when assuming that the insertion site behaves like a lever with a fixed and known fulcrum. In practice, this assumption is not always respected. In this paper we first study the practical consequences of this problem in terms of tip precision positioning. We then propose a robotic control scheme that improves the precision compared to the fixed point assumption approach.

METHODS

In the first part of the paper, data recorded during robot-assisted transrectal needle positioning for prostate biopsies (nine patients) are exploited to quantify the positioning error induced by the use of a fixed point hypothesis in the positioning process. In the second part of the paper advanced control techniques allow for the online identification of a locally linear system that describes a model characterized by anisotropy and center displacement. A laboratory apparatus is used to demonstrate the resulting improvement on tip positioning precision.

RESULTS

Errors obtained by processing the clinical data reach 7.5 mm at the tip in average. Errors obtained with the laboratory apparatus drop from 2.4 mm in average to 0.8 mm when using real-time model update.

Review of emerging surgical robotic technology

BACKGROUND

The use of laparoscopic and robotic procedures has increased in general surgery. Minimally invasive robotic surgery has made tremendous progress in a relatively short period of time, realizing improvements for both the patient and surgeon. This has led to an increase in the use and development of robotic devices and platforms for general surgery. The purpose of this review is to explore current and emerging surgical robotic technologies in a growing and dynamic environment of research and development.

METHODS

This review explores medical and surgical robotic endoscopic surgery and peripheral technologies currently available or in development. The devices discussed here are specific to general surgery, including laparoscopy, colonoscopy, esophagogastroduodenoscopy, and thoracoscopy. Benefits and limitations of each technology were identified and applicable future directions were described.

RESULTS

A number of FDA-approved devices and platforms for robotic surgery were reviewed, including the da Vinci Surgical System, Sensei X Robotic Catheter System, FreeHand 1.2, invendoscopy E200 system, Flex® Robotic System, Senhance, ARES, the Single-Port Instrument Delivery Extended Research (SPIDER), and the NeoGuide Colonoscope. Additionally, platforms were reviewed which have not yet obtained FDA approval including MiroSurge, ViaCath System, SPORT™ Surgical System, SurgiBot, Versius Robotic System, Master and Slave Transluminal Endoscopic Robot, Verb Surgical, Miniature In Vivo Robot, and the Einstein Surgical Robot.

CONCLUSIONS

The use and demand for robotic medical and surgical platforms is increasing and new technologies are continually being developed. New technologies are increasingly implemented to improve on the capabilities of previously established systems. Future studies are needed to further evaluate the strengths and weaknesses of each robotic surgical device and platform in the operating suite.

A Continuum Robot and Control Interface for Surgical Assist in Fetoscopic Interventions

Twin–twin transfusion syndrome requires interventional treatment using a fetoscopically introduced laser to sever the shared blood supply between the fetuses. This is a delicate procedure relying on small instrumentation with limited articulation to guide the laser tip and a narrow field of view to visualize all relevant vascular connections. In this letter, we report on a mechatronic design for a comanipulated instrument that combines concentric tube actuation to a larger manipulator constrained by a remote centre of motion. A stereoscopic camera is mounted at the distal tip and used for imaging. Our mechanism provides enhanced dexterity and stability of the imaging device. We demonstrate that the imaging system can be used for computing geometry and enhancing the view at the operating site. Results using electromagnetic sensors for verification and comparison to visual odometry from the distal sensor show that our system is promising and can be developed further for multiple clinical needs in fetoscopic procedures.

Algorithms for Automatically Pointing Ultrasound Imaging Catheters

A system for automatically pointing ultrasound (US) imaging catheters will enable clinicians to monitor anatomical structures and track instruments during interventional procedures. Off-the-shelf US catheters provide high quality US images from within the patient. While this method of imaging has been proven to be effective for guiding many interventional treatments, significant training is required to overcome the difficulty in manually steering the imager to point at desired structures. Our system uses closed-form four degree of freedom (DOF) kinematic solutions to automatically position the US catheter and point the imager. Algorithms for steering and imager pointing were developed for a range of useful diagnostic and interventional motions. The system was validated on a robotic test bed by steering the catheter within a water environment containing phantom objects. While the system described here was designed for pointing ultrasound catheters, these algorithms are applicable to accurate 4-DOF steering and orientation control of any long thin tendon-driven tool with single or bi-directional bending.

Medical telerobotic systems: current status and future trends

Teleoperated medical robotic systems allow procedures such as surgeries, treatments, and diagnoses to be conducted across short or long distances while utilizing wired and/or wireless communication networks. This study presents a systematic review of the relevant literature between the years 2004 and 2015, focusing on medical teleoperated robotic systems which have witnessed tremendous growth over the examined period. A thorough insight of telerobotics systems discussing design concepts, enabling technologies (namely robotic manipulation, telecommunications, and vision systems), and potential applications in clinical practice is provided, while existing limitations and future trends are also highlighted. A representative paradigm of the short-distance case is the da Vinci Surgical System which is described in order to highlight relevant issues. The long-distance telerobotics concept is exemplified through a case study on diagnostic ultrasound scanning. Moreover, the present review provides a classification into short- and long-distance telerobotic systems, depending on the distance from which they are operated. Telerobotic systems are further categorized with respect to their application field. For the reviewed systems are also examined their engineering characteristics and the employed robotics technology. The current status of the field, its significance, the potential, as well as the challenges that lie ahead are thoroughly discussed.

Robot-like dexterity without computers and motors: a review of hand-held laparoscopic instruments with wrist-like tip articulation

INTRODUCTION

Conventional manual laparoscopic instruments for minimally invasive surgery have limited dexterity within the patient, making procedures challenging. Surgical robotic systems offer enhanced articulation, but at substantial financial costs. This has motivated the development of high-dexterity, low-cost laparoscopic instruments.

AREAS COVERED

This article reviews both commercial and academic results on creating fully mechanical (i.e. non-robotic) laparoscopic instruments that provide wrists or wrist-like dexterity within the patient. We review the state of the art in the development of these mechanical instruments, focusing on the surgeon interface, wrist mechanism, and the kinematic mapping between the two. Expert commentary: Current articulated mechanical laparoscopic instruments exhibit a wide range of designs, with no clear consensus on what makes such devices easy to use. As these technologies mature, user studies are needed to determine surgeon preferences. Articulated, low-cost instruments have the potential to impact the minimally invasive surgery market if they provide compelling benefits to surgeons.

Magnetic resonance imaging compatible remote catheter navigation system with 3 degrees of freedom

PURPOSE

To facilitate MRI-guided catheterization procedures, we present an MRI-compatible remote catheter navigation system that allows remote navigation of steerable catheters with 3 degrees of freedom.

METHODS

The system consists of a user interface (master), a robot (slave), and an ultrasonic motor control servomechanism. The interventionalist applies conventional motions (axial, radial and plunger manipulations) on an input catheter in the master unit; this user input is measured and used by the servomechanism to control a compact catheter manipulating robot, such that it replicates the interventionalist's input motion on the patient catheter. The performance of the system was evaluated in terms of MRI compatibility (SNR and artifact), feasibility of remote navigation under real-time MRI guidance, and motion replication accuracy.

RESULTS

Real-time MRI experiments demonstrated that catheter was successfully navigated remotely to desired target references in all 3 degrees of freedom. The system had an absolute value error of [Formula: see text]1 mm in axial catheter motion replication over 30 mm of travel and [Formula: see text] for radial catheter motion replication over [Formula: see text]. The worst case SNR drop was observed to be [Formula: see text]3 %; the robot did not introduce any artifacts in the MR images.

CONCLUSION

An MRI-compatible compact remote catheter navigation system has been developed that allows remote navigation of steerable catheters with 3 degrees of freedom. The proposed system allows for safe and accurate remote catheter navigation, within conventional closed-bore scanners, without degrading MR image quality.

Time-of-flight-assisted Kinect camera-based people detection for intuitive human robot cooperation in the surgical operating room

BACKGROUND

Scene supervision is a major tool to make medical robots safer and more intuitive. The paper shows an approach to efficiently use 3D cameras within the surgical operating room to enable for safe human robot interaction and action perception. Additionally the presented approach aims to make 3D camera-based scene supervision more reliable and accurate.

METHODS

A camera system composed of multiple Kinect and time-of-flight cameras has been designed, implemented and calibrated. Calibration and object detection as well as people tracking methods have been designed and evaluated.

RESULTS

The camera system shows a good registration accuracy of 0.05 m. The tracking of humans is reliable and accurate and has been evaluated in an experimental setup using operating clothing. The robot detection shows an error of around 0.04 m.

CONCLUSIONS

The robustness and accuracy of the approach allow for an integration into modern operating room. The data output can be used directly for situation and workflow detection as well as collision avoidance.

Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions

PURPOSE

Advances in technology and computing play an increasingly important role in the evolution of modern surgical techniques and paradigms. This article reviews the current role of machine learning (ML) techniques in the context of surgery with a focus on surgical robotics (SR). Also, we provide a perspective on the future possibilities for enhancing the effectiveness of procedures by integrating ML in the operating room.

METHODS

The review is focused on ML techniques directly applied to surgery, surgical robotics, surgical training and assessment. The widespread use of ML methods in diagnosis and medical image computing is beyond the scope of the review. Searches were performed on PubMed and IEEE Explore using combinations of keywords: ML, surgery, robotics, surgical and medical robotics, skill learning, skill analysis and learning to perceive.

RESULTS

Studies making use of ML methods in the context of surgery are increasingly being reported. In particular, there is an increasing interest in using ML for developing tools to understand and model surgical skill and competence or to extract surgical workflow. Many researchers begin to integrate this understanding into the control of recent surgical robots and devices.

CONCLUSION

ML is an expanding field. It is popular as it allows efficient processing of vast amounts of data for interpreting and real-time decision making. Already widely used in imaging and diagnosis, it is believed that ML will also play an important role in surgery and interventional treatments. In particular, ML could become a game changer into the conception of cognitive surgical robots. Such robots endowed with cognitive skills would assist the surgical team also on a cognitive level, such as possibly lowering the mental load of the team. For example, ML could help extracting surgical skill, learned through demonstration by human experts, and could transfer this to robotic skills. Such intelligent surgical assistance would significantly surpass the state of the art in surgical robotics. Current devices possess no intelligence whatsoever and are merely advanced and expensive instruments.

Configuration optimization and experimental accuracy evaluation of a bone-attached, parallel robot for skull surgery

PURPOSE

Minimally invasive cochlear implantation is a novel surgical technique which requires highly accurate guidance of a drilling tool along a trajectory from the mastoid surface toward the basal turn of the cochlea. The authors propose a passive, reconfigurable, parallel robot which can be directly attached to bone anchors implanted in a patient's skull, avoiding the need for surgical tracking systems. Prior to clinical trials, methods are necessary to patient specifically optimize the configuration of the mechanism with respect to accuracy and stability. Furthermore, the achievable accuracy has to be determined experimentally.

METHODS

A comprehensive error model of the proposed mechanism is established, taking into account all relevant error sources identified in previous studies. Two optimization criteria to exploit the given task redundancy and reconfigurability of the passive robot are derived from the model. The achievable accuracy of the optimized robot configurations is first estimated with the help of a Monte Carlo simulation approach and finally evaluated in drilling experiments using synthetic temporal bone specimen.

RESULTS

Experimental results demonstrate that the bone-attached mechanism exhibits a mean targeting accuracy of [Formula: see text] mm under realistic conditions. A systematic targeting error is observed, which indicates that accurate identification of the passive robot's kinematic parameters could further reduce deviations from planned drill trajectories.

CONCLUSION

The accuracy of the proposed mechanism demonstrates its suitability for minimally invasive cochlear implantation. Future work will focus on further evaluation experiments on temporal bone specimen.

Hand-held transendoscopic robotic manipulators: A transurethral laser prostate surgery case study

Natural orifice endoscopic surgery can enable incisionless approaches, but a major challenge is the lack of small and dexterous instrumentation. Surgical robots have the potential to meet this need yet often disrupt the clinical workflow. Hand-held robots that combine thin manipulators and endoscopes have the potential to address this by integrating seamlessly into the clinical workflow and enhancing dexterity. As a case study illustrating the potential of this approach, we describe a hand-held robotic system that passes two concentric tube manipulators through a 5 mm port in a rigid endoscope for transurethral laser prostate surgery. This system is intended to catalyze the use of a clinically superior, yet rarely attempted, procedure for benign prostatic hyperplasia. This paper describes system design and experiments to evaluate the surgeon's functional workspace and accuracy using the robot. Phantom and cadaver experiments demonstrate successful completion of the target procedure via prostate lobe resection.