Dynamic 3-D virtual fixtures for minimally invasive beating heart procedures

Real-time active constraint generation and enforcement for surgical tools using 3D detection and localisation network

Introduction: Collaborative robots, designed to work alongside humans for manipulating end-effectors, greatly benefit from the implementation of active constraints. This process comprises the definition of a boundary, followed by the enforcement of some control algorithm when the robot tooltip interacts with the generated boundary. Contact with the constraint boundary is communicated to the human operator through various potential forms of feedback. In fields like surgical robotics, where patient safety is paramount, implementing active constraints can prevent the robot from interacting with portions of the patient anatomy that shouldn't be operated on. Despite improvements in orthopaedic surgical robots, however, there exists a gap between bulky systems with haptic feedback capabilities and miniaturised systems that only allow for boundary control, where interaction with the active constraint boundary interrupts robot functions. Generally, active constraint generation relies on optical tracking systems and preoperative imaging techniques. Methods: This paper presents a refined version of the Signature Robot, a three degrees-of-freedom, hands-on collaborative system for orthopaedic surgery. Additionally, it presents a method for generating and enforcing active constraints "on-the-fly" using our previously introduced monocular, RGB, camera-based network, SimPS-Net. The network was deployed in real-time for the purpose of boundary definition. This boundary was subsequently used for constraint enforcement testing. The robot was utilised to test two different active constraints: a safe region and a restricted region. Results: The network success rate, defined as the ratio of correct over total object localisation results, was calculated to be 54.7% ± 5.2%. In the safe region case, haptic feedback resisted tooltip manipulation beyond the active constraint boundary, with a mean distance from the boundary of 2.70 mm ± 0.37 mm and a mean exit duration of 0.76 s ± 0.11 s. For the restricted-zone constraint, the operator was successfully prevented from penetrating the boundary in 100% of attempts. Discussion: This paper showcases the viability of the proposed robotic platform and presents promising results of a versatile constraint generation and enforcement pipeline.

Dynamic Virtual Fixture Generation Based on Intra-Operative 3D Image Feedback in Robot-Assisted Minimally Invasive Thoracic Surgery

This paper proposes a method for generating dynamic virtual fixtures with real-time 3D image feedback to facilitate human-robot collaboration in medical robotics. Seamless shared control in a dynamic environment, like that of a surgical field, remains challenging despite extensive research on collaborative control and planning. To address this problem, our method dynamically creates virtual fixtures to guide the manipulation of a trocar-placing robot arm using the force field generated by point cloud data from an RGB-D camera. Additionally, the "view scope" concept selectively determines the region for computational points, thereby reducing computational load. In a phantom experiment for robot-assisted port incision in minimally invasive thoracic surgery, our method demonstrates substantially improved accuracy for port placement, reducing error and completion time by 50% (p=1.06×10-2) and 35% (p=3.23×10-2), respectively. These results suggest that our proposed approach is promising in improving surgical human-robot collaboration.

Preoperative Planning Framework for Robot-Assisted Dental Implant Surgery: Finite-Parameter Surrogate Model and Optimization of Instrument Placement

For robot-assisted dental implant surgery, it is necessary to feed the instrument into a specified position to perform surgery. To improve safety and efficiency, a preoperative planning framework, including a finite-parameter surrogate model (FPSM) and an automatic instrument-placement method, is proposed in this paper. This framework is implemented via two-stage optimization. In the first stage, a group of closed curves in polar coordinates is used to represent the oral cavity. By optimizing a finite number of parameters for these curves, the oral structure is simplified to form the FPSM. In the second stage, the FPSM serves as a fast safety estimator with which the target position/orientation of the instrument for the feeding motion is automatically determined through particle swarm optimization (PSO). The optimized feeding target can be used to generate a virtual fixture (VF) to avoid undesired operations and to lower the risk of collision. This proposed framework has the advantages of being safe, fast, and accurate, overcoming the computational burden and insufficient real-time performance of complex 3D models. The framework has been developed and tested, preliminarily verifying its feasibility, efficiency, and effectiveness.

Evaluating Human-Robot Interaction Algorithms in Shared Autonomy via Quality Diversity Scenario Generation

The growth of scale and complexity of interactions between humans and robots highlights the need for new computational methods to automatically evaluate novel algorithms and applications. Exploring diverse scenarios of humans and robots interacting in simulation can improve understanding of the robotic system and avoid potentially costly failures in real-world settings. We formulate this problem as a quality diversity (QD) problem, where the goal is to discover diverse failure scenarios by simultaneously exploring both environments and human actions. We focus on the shared autonomy domain, where the robot attempts to infer the goal of a human operator, and adopt the QD algorithms CMA-ME and MAP-Elites to generate scenarios for two published algorithms in this domain: shared autonomy via hindsight optimization and linear policy blending. Some of the generated scenarios confirm previous theoretical findings, while others are surprising and bring about a new understanding of state-of-the-art implementations. Our experiments show that the QD algorithms CMA-ME and MAP-Elites outperform Monte-Carlo simulation and optimization based methods in effectively searching the scenario space, highlighting their promise for automatic evaluation of algorithms in human-robot interaction.

Cardiac X-ray image-based haptic guidance for robot-assisted coronary intervention: a feasibility study

PURPOSE

Effective and efficient haptic guidance is desirable for tele-operated robotic surgery because it has a potential to enhance surgeon's skills, especially in coronary interventions where surgeon loses both an eye-hand coordination and a direct sight to the organ. This paper proposes a novel haptic guidance procedure-both kinesthetic and cutaneous, which solely depends upon X-ray images, for tele-robotic system that assists an efficient navigation of the guidewire towards the target location during a coronary intervention.

METHODS

Proposed methodology requires cardiologists to draw virtual fixtures (VFs) on angiograms as a preoperative procedure. During an operation, these VFs direct the guidewire to the desired coronary vessel. For this, the position and orientation of guidewire tip are calculated with respect to VFs' anatomy, using image processing on the real-time 2D fluoroscopic images. The haptic feedbacks are then rendered on to the master device depending on the interaction with attractive and repulsive, guidance and forbidden region VFs.

RESULTS

A feasibility study in the laboratory environment is performed by using a webcam as an image acquisition device and a phantom-based coronary vessel model. The subsequent statistical analysis shows that, on an average, a decrease of approx. 37% in task completion time is observed with haptic feedback. Moreover, haptic guidance is found effective for most difficult branch, whereas there is a minimal significance of such haptics for the easiest branch.

CONCLUSIONS

The proposed haptic guidance procedure may assist cardiologists for an efficient and effective guidewire navigation during a surgical procedure. The cutaneous haptics (vibration feedback) is found more helpful in coronary interventions compared with kinesthetic haptics (force feedback).

Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR-Guided prostate biopsies

BACKGROUND

User interfaces play a vital role in the planning and execution of an interventional procedure. The objective of this study is to investigate the effect of using different user interfaces for planning transrectal robot-assisted MR-guided prostate biopsy (MRgPBx) in an augmented reality (AR) environment.

METHOD

End-user studies were conducted by simulating an MRgPBx system with end- and side-firing modes. The information from the system to the operator was rendered on HoloLens as an output interface. Joystick, mouse/keyboard, and holographic menus were used as input interfaces to the system.

RESULTS

The studies indicated that using a joystick improved the interactive capacity and enabled operator to plan MRgPBx in less time. It efficiently captures the operator's commands to manipulate the augmented environment representing the state of MRgPBx system.

CONCLUSIONS

The study demonstrates an alternative to conventional input interfaces to interact and manipulate an AR environment within the context of MRgPBx planning.

Research on spatial motion safety constraints and cooperative control of robot-assisted craniotomy: Beagle model experiment verification

BACKGROUND

Traditional craniotomy depends primarily on the experience of the surgeon. However, the accuracy of manual operation is limited and carries certain surgical risks. The interaction method of current robot-assisted craniotomy is unnatural and inadaptive to the operating style of the surgeon. In this research, we built a hands-on synergistic robotics craniotomy system with human-machine collaboration. Safe isometric surfaces and virtual restraint methods are combined to achieve highly accurate, efficient, minimally invasive and safe craniotomy.

MATERIALS AND METHODS

Fifteen three-dimensional (3D)-printed beagle skull models were used to evaluate the system accuracy and the related image guidance process. It mainly includes the design of the surgical plan, the adopted strategy based on motion constraint and safe isometric surface, and the impedance control method based on the position inner loop via the human-machine collaboration method. The trajectory tracking experiment was performed by applying human-machine collaboration, and completed an experiment on the 3D-printed beagle skull models involving drilling and milling of the skull performed by the robot, and evaluation of accuracy via computed tomographic (CT) scanning verification after the operation.

RESULTS

The 3D-printed beagle skull model experiment shows that the average errors for the top surface and the bottom surface, and the angle error were 0.81 ± 0.15 mm, 0.89 ± 0.12 mm, and 1.74° ± 0.16°, respectively. The average milling position errors for the top and bottom surfaces were 0.87 ± 0.19 and 0.93 ± 0.22 mm, respectively.

CONCLUSION

The performance of the robot system was evaluated and verified using a 3D-printed beagle model experiment. The proposed collaborative surgical robot system is feasible and can complete a craniotomy, with improved accuracy and surgical safety.

Review of Advanced Medical Telerobots

The advent of telerobotic systems has revolutionized various aspects of the industry and human life. This technology is designed to augment human sensorimotor capabilities to extend them beyond natural competence. Classic examples are space and underwater applications when distance and access are the two major physical barriers to be combated with this technology. In modern examples, telerobotic systems have been used in several clinical applications, including teleoperated surgery and telerehabilitation. In this regard, there has been a significant amount of research and development due to the major benefits in terms of medical outcomes. Recently telerobotic systems are combined with advanced artificial intelligence modules to better share the agency with the operator and open new doors of medical automation. In this review paper, we have provided a comprehensive analysis of the literature considering various topologies of telerobotic systems in the medical domain while shedding light on different levels of autonomy for this technology, starting from direct control, going up to command-tracking autonomous telerobots. Existing challenges, including instrumentation, transparency, autonomy, stochastic communication delays, and stability, in addition to the current direction of research related to benefit in telemedicine and medical automation, and future vision of this technology, are discussed in this review paper.

Force quantification and simulation of pedicle screw tract palpation using direct visuo-haptic volume rendering

Purpose

We present a feasibility study for the visuo-haptic simulation of pedicle screw tract palpation in virtual reality, using an approach that requires no manual processing or segmentation of the volumetric medical data set.

Methods

In a first experiment, we quantified the forces and torques present during the palpation of a pedicle screw tract in a real boar vertebra. We equipped a ball-tipped pedicle probe with a 6-axis force/torque sensor and a motion capture marker cluster. We simultaneously recorded the pose of the probe relative to the vertebra and measured the generated forces and torques during palpation. This allowed us replaying the recorded palpation movements in our simulator and to fine-tune the haptic rendering to approximate the measured forces and torques. In a second experiment, we asked two neurosurgeons to palpate a virtual version of the same vertebra in our simulator, while we logged the forces and torques sent to the haptic device.

Results

In the experiments with the real vertebra, the maximum measured force along the longitudinal axis of the probe was 7.78 N and the maximum measured bending torque was 0.13 Nm. In an offline simulation of the motion of the pedicle probe recorded during the palpation of a real pedicle screw tract, our approach generated forces and torques that were similar in magnitude and progression to the measured ones. When surgeons tested our simulator, the distributions of the computed forces and torques were similar to the measured ones; however, higher forces and torques occurred more frequently.

Conclusions

We demonstrated the suitability of direct visual and haptic volume rendering to simulate a specific surgical procedure. Our approach of fine-tuning the simulation by measuring the forces and torques that are prevalent while palpating a real vertebra produced promising results.

Electronic supplementary material

The online version of this article (10.1007/s11548-020-02258-0) contains supplementary material, which is available to authorized users.

Multimodal Physiological Signals for Workload Prediction in Robot-assisted Surgery

Monitoring surgeon workload during robot-assisted surgery can guide allocation of task demands, adapt system interfaces, and assess the robotic system's usability. Current practices for measuring cognitive load primarily rely on questionnaires that are subjective and disrupt surgical workflow. To address this limitation, a computational framework is demonstrated to predict user workload during telerobotic surgery. This framework leverages wireless sensors to monitor surgeons’ cognitive load and predict their cognitive states. Continuous data across multiple physiological modalities (e.g., heart rate variability, electrodermal, and electroencephalogram activity) were simultaneously recorded for twelve surgeons performing surgical skills tasks on the validated da Vinci Skills Simulator. These surgical tasks varied in difficulty levels, e.g., requiring varying visual processing demand and degree of fine motor control. Collected multimodal physiological signals were fused using independent component analysis, and the predicted results were compared to the ground-truth workload level. Results compared performance of different classifiers, sensor fusion schemes, and physiological modality (i.e., prediction with single vs. multiple modalities). It was found that our multisensor approach outperformed individual signals and can correctly predict cognitive workload levels 83.2% of the time during basic and complex surgical skills tasks.

Human–Robot Cooperative Control Based on Virtual Fixture in Robot-Assisted Endoscopic Sinus Surgery

In endoscopic sinus surgery, the robot assists the surgeon in holding the endoscope and acts as the surgeon’s third hand, which helps to reduce the surgeon’s operating burden and improve the quality of the operation. This paper proposes a human–robot cooperative control method based on virtual fixture to realize accurate and safe human–robot interaction in endoscopic sinus surgery. Firstly, through endoscopic trajectory analysis, the endoscopic motion constraint requirements of different surgical stages are obtained, and three typical virtual fixtures suitable for endoscopic sinus surgery are designed and implemented. Based on the typical virtual fixtures, a composite virtual fixture is constructed, and then the overall robot motion constraint model is obtained. Secondly, based on the obtained robot motion constraint model, a human–robot cooperative control method based on virtual fixture is proposed. The method adopts admittance control to realize efficient human–robot interaction between the surgeon and robot during the surgery; the virtual fixture is used to restrain and guide the motion of the robot, thereby ensuring motion safety of the robot. Finally, the proposed method is evaluated through a robot-assisted nasal endoscopy experiment, and the result shows that the proposed method can improve the accuracy and safety of operation during endoscopic sinus surgery.

Haptic Guidance Based on All-Optical Ultrasound Distance Sensing for Safer Minimally Invasive Fetal Surgery

By intervening during the early stage of gestation, fetal surgeons aim to correct or minimize the effects of congenital disorders. As compared to postnatal treatment of these disorders, such early interventions can often actually save the life of the fetus and also improve the quality of life of the newborn. However, fetal surgery is considered one of the most challenging disciplines within Minimally Invasive Surgery (MIS), owing to factors such as the fragility of the anatomic features, poor visibility, limited manoeuvrability, and extreme requirements in terms of instrument handling with precise positioning. This work is centered on a fetal laser surgery procedure treating placental disorders. It proposes the use of haptic guidance to enhance the overall safety of this procedure and to simplify instrument handling. A method is described that provides effective guidance by installing a forbidden region virtual fixture over the placenta, thereby safeguarding adequate clearance between the instrument tip and the placenta. With a novel application of all-optical ultrasound distance sensing in which transmission and reception are performed with fibre optics, this method can be used with a sole reliance on intraoperatively acquired data. The added value of the guidance approach, in terms of safety and performance, is demonstrated in a series of experiments with a robotic platform.

Virtual wall-based haptic-guided teleoperated surgical robotic system for single-port brain tumor removal surgery

This article presents a haptic-guided teleoperation for a tumor removal surgical robotic system, so-called a SIROMAN system. The system was developed in our previous work to make it possible to access tumor tissue, even those that seat deeply inside the brain, and to remove the tissue with full maneuverability. For a safe and accurate operation to remove only tumor tissue completely while minimizing damage to the normal tissue, a virtual wall-based haptic guidance together with a medical image-guided control is proposed and developed. The virtual wall is extracted from preoperative medical images, and the robot is controlled to restrict its motion within the virtual wall using haptic feedback. Coordinate transformation between sub-systems, a collision detection algorithm, and a haptic-guided teleoperation using a virtual wall are described in the context of using SIROMAN. A series of experiments using a simplified virtual wall are performed to evaluate the performance of virtual wall-based haptic-guided teleoperation. With haptic guidance, the accuracy of the robotic manipulator's trajectory is improved by 57% compared to one without. The tissue removal performance is also improved by 21% ( p < 0.05). The experiments show that virtual wall-based haptic guidance provides safer and more accurate tissue removal for single-port brain surgery.

Combined 2D and 3D tracking of surgical instruments for minimally invasive and robotic-assisted surgery

PURPOSE

Computer-assisted interventions for enhanced minimally invasive surgery (MIS) require tracking of the surgical instruments. Instrument tracking is a challenging problem in both conventional and robotic-assisted MIS, but vision-based approaches are a promising solution with minimal hardware integration requirements. However, vision-based methods suffer from drift, and in the case of occlusions, shadows and fast motion, they can be subject to complete tracking failure.

METHODS

In this paper, we develop a 2D tracker based on a Generalized Hough Transform using SIFT features which can both handle complex environmental changes and recover from tracking failure. We use this to initialize a 3D tracker at each frame which enables us to recover 3D instrument pose over long sequences and even during occlusions.

RESULTS

We quantitatively validate our method in 2D and 3D with ex vivo data collected from a DVRK controller as well as providing qualitative validation on robotic-assisted in vivo data.

CONCLUSIONS

We demonstrate from our extended sequences that our method provides drift-free robust and accurate tracking. Our occlusion-based sequences additionally demonstrate that our method can recover from occlusion-based failure. In both cases, we show an improvement over using 3D tracking alone suggesting that combining 2D and 3D tracking is a promising solution to challenges in surgical instrument tracking.

Experimental evaluation of co-manipulated ultrasound-guided flexible needle steering

BACKGROUND

A teleoperation system for bevel-tipped flexible needle steering has been evaluated. Robotic systems have been exploited as the main tool to achieve high accuracy and reliability. However, for reasons of safety and acceptance by the surgical community, keeping the physician tightly in the loop is preferable.

METHODS

The system uses ultrasound imaging, path planning, and control to compute the desired needle orientation during the insertion and intuitively passes this information to the operator, who teleoperates the motion of the needle's tip. Navigation cues about the computed orientation are provided through haptic and visual feedback to the operator to steer the needle.

RESULTS

The targeting accuracy of several co-manipulation strategies were studied in four sets of experiments involving human subjects with clinical backgrounds.

CONCLUSIONS

Experimental results show that receiving feedback regarding the desired needle orientation improves the targeting accuracy by a factor of 9 with respect to manual insertions. Copyright © 2015 John Wiley & Sons, Ltd.

Dynamic virtual fixture on the Euclidean group for admittance-type manipulator in deforming environments

BACKGROUND

In a deforming anatomic environment, the motion of an instrument suffers from complex geometrical and dynamic constraints, robot assisted minimally invasive surgery therefore requires more sophisticated skills for surgeons. This paper proposes a novel dynamic virtual fixture (DVF) to enhance the surgical operation accuracy of admittance-type medical robotics in the deforming environment.

METHODS

A framework for DVF on the Euclidean Group SE(3) is presented, which unites rotation and translation in a compact form. First, we constructed the holonomic/non-holonomic constraints, and then searched for the corresponded reference to make a distinction between preferred and non-preferred directions. Second, different control strategies are employed to deal with the tasks along the distinguished directions. The desired spatial compliance matrix is synthesized from an allowable motion screw set to filter out the task unrelated components from manual input, the operator has complete control over the preferred directions; while the relative motion between the surgical instrument and the anatomy structures is actively tracked and cancelled, the deviation relative to the reference is compensated jointly by the operator and DVF controllers. The operator, haptic device, admittance-type proxy and virtual deforming environment are involved in a hardware-in-the-loop experiment, human-robot cooperation with the assistance of DVF controller is carried out on a deforming sphere to simulate beating heart surgery, performance of the proposed DVF on admittance-type proxy is evaluated, and both human factors and control parameters are analyzed.

RESULTS

The DVF can improve the dynamic properties of human-robot cooperation in a low-frequency (0 ~ 40 rad/sec) deforming environment, and maintain synergy of orientation and translation during the operation. Statistical analysis reveals that the operator has intuitive control over the preferred directions, human and the DVF controller jointly control the motion along the non-preferred directions, the target deformation is tracked actively.

CONCLUSIONS

The proposed DVF for an admittance-type manipulator is capable of assisting the operator to deal with skilled operations in a deforming environment.

A novel flexible virtual fixtures for teleoperation

This paper proposed a novel spatial-motion-constraints virtual fixtures (VFs) method for the human-machine interface collaborative technique. In our method, two 3D flexible VFs have been presented: warning pipe and safe pipe. And a potential-collision-detection method based on two flexible VFs has been proposed. The safe pipe constructs the safe workspace dynamically for the robot, which makes it possible to detect the potential collision between the robot and the obstacles. By calculating the speed and the acceleration of the robot end-effecter (EE), the warning pipe can adjust its radius to detect the deviation from the EE to the reference path. These spatial constraints serve as constraint conditions for constrained robot control. The approach enables multiobstacle manipulation task of telerobot in precise interactive teleoperation environment. We illustrate our approach on a teleoperative manipulation task and analyze the performance results. The performance-comparison experimental results demonstrate that the control mode employing our method can assist the operator more precisely in teleoperative tasks. Due to the properties such as collision avoidance and safety, operators can complete the tasks more efficiently along with reduction in operating tension.

Deformation invariant bounding spheres for dynamic active constraints in surgery

Active constraints are collaborative robot control strategies, which can be used to guide a surgeon or protect delicate tissue structures during robot-assisted surgery. Tissue structures of interest often move and deform throughout a surgical intervention, and therefore, dynamic active constraints, which adapt and conform to these changes, are required. A fundamental element of an active constraint controller is the computation of the geometric relationship between the constraint geometry and the surgical instrument. For a static active constraint, there are a variety of computationally efficient methods for computing this relative configuration; however, for a dynamic active constraint, it becomes significantly more challenging. Deformation invariant bounding spheres are a novel bounding volume formulation, which can be used within a hierarchy to allow efficient proximity queries within dynamic active constraints. These bounding spheres are constructed in such a way that as the surface deforms, they do not require time-consuming rebuilds or updates, rather they are implicitly updated and continue to represent the underlying geometry as it changes. Experimental results show that performing proximity queries with deformation invariant bounding sphere hierarchies is faster than common methods from the literature when the deformation rate is within the range expected from conventional imaging systems.

Vision-Based Control of a Handheld Surgical Micromanipulator with Virtual Fixtures

Performing micromanipulation and delicate operations in submillimeter workspaces is difficult because of destabilizing tremor and imprecise targeting. Accurate micromanipulation is especially important for microsurgical procedures, such as vitreoretinal surgery, to maximize successful outcomes and minimize collateral damage. Robotic aid combined with filtering techniques that suppress tremor frequency bands increases performance; however, if knowledge of the operator's goals is available, virtual fixtures have been shown to further improve performance. In this paper, we derive a virtual fixture framework for active handheld micromanipulators that is based on high-bandwidth position measurements rather than forces applied to a robot handle. For applicability in surgical environments, the fixtures are generated in real-time from microscope video during the procedure. Additionally, we develop motion scaling behavior around virtual fixtures as a simple and direct extension to the proposed framework. We demonstrate that virtual fixtures significantly outperform tremor cancellation algorithms on a set of synthetic tracing tasks (p < 0.05). In more medically relevant experiments of vein tracing and membrane peeling in eye phantoms, virtual fixtures can significantly reduce both positioning error and forces applied to tissue (p < 0.05).

Dimensionality Reduction in Controlling Articulated Snake Robot for Endoscopy Under Dynamic Active Constraints

This paper presents a real-time control framework for a snake robot with hyper-kinematic redundancy under dynamic active constraints for minimally invasive surgery. A proximity query (PQ) formulation is proposed to compute the deviation of the robot motion from predefined anatomical constraints. The proposed method is generic and can be applied to any snake robot represented as a set of control vertices. The proposed PQ formulation is implemented on a graphic processing unit, allowing for fast updates over 1 kHz. We also demonstrate that the robot joint space can be characterized into lower dimensional space for smooth articulation. A novel motion parameterization scheme in polar coordinates is proposed to describe the transition of motion, thus allowing for direct manual control of the robot using standard interface devices with limited degrees of freedom. Under the proposed framework, the correct alignment between the visual and motor axes is ensured, and haptic guidance is provided to prevent excessive force applied to the tissue by the robot body. A resistance force is further incorporated to enhance smooth pursuit movement matched to the dynamic response and actuation limit of the robot. To demonstrate the practical value of the proposed platform with enhanced ergonomic control, detailed quantitative performance evaluation was conducted on a group of subjects performing simulated intraluminal and intracavity endoscopic tasks.

Emerging robotic platforms for minimally invasive surgery

Recent technological advances in surgery have resulted in the development of a range of new techniques that have reduced patient trauma, shortened hospitalization, and improved diagnostic accuracy and therapeutic outcome. Despite the many appreciated benefits of minimally invasive surgery (MIS) compared to traditional approaches, there are still significant drawbacks associated with conventional MIS including poor instrument control and ergonomics caused by rigid instrumentation and its associated fulcrum effect. The use of robot assistance has helped to realize the full potential of MIS with improved consistency, safety and accuracy. The development of articulated, precision tools to enhance the surgeon's dexterity has evolved in parallel with advances in imaging and human-robot interaction. This has improved hand-eye coordination and manual precision down to micron scales, with the capability of navigating through complex anatomical pathways. In this review paper, clinical requirements and technical challenges related to the design of robotic platforms for flexible access surgery are discussed. Allied technical approaches and engineering challenges related to instrument design, intraoperative guidance, and intelligent human-robot interaction are reviewed. We also highlight emerging designs and research opportunities in the field by assessing the current limitations and open technical challenges for the wider clinical uptake of robotic platforms in MIS.

Generation of 4D access corridors from real-time multislice MRI for guiding transapical aortic valvuloplasties

Real-time image-guided cardiac procedures (manual or robot-assisted) are emerging due to potential improvement in patient management and reduction in the overall cost. These minimally invasive procedures require both real-time visualization and guidance for maneuvering an interventional tool safely inside the dynamic environment of a heart. In this work, we propose an approach to generate dynamic 4D access corridors from the apex to the aortic annulus for performing real-time MRI guided transapical valvuloplasties. Ultrafast MR images (collected every 49.3 ms) are processed on-the-fly using projections to extract a conservative dynamic trace in form of a three-dimensional access corridor. Our experimental results show that the reconstructed corridors can be refreshed with a delay of less than 0.5ms to reflect the changes inside the left ventricle caused by breathing motion and the heartbeat.

Gaze-Contingent Motor Channelling, haptic constraints and associated cognitive demand for robotic MIS

The success of MIS is coupled with an increasing demand on surgeons' manual dexterity and visuomotor coordination due to the complexity of instrument manipulations. The use of master-slave surgical robots has avoided many of the drawbacks of MIS, but at the same time, has increased the physical separation between the surgeon and the patient. Tissue deformation combined with restricted workspace and visibility of an already cluttered environment can raise critical issues related to surgical precision and safety. Reconnecting the essential visuomotor sensory feedback is important for the safe practice of robot-assisted MIS procedures. This paper introduces a novel gaze-contingent framework for real-time haptic feedback and virtual fixtures by transforming visual sensory information into physical constraints that can interact with the motor sensory channel. We demonstrate how motor tracking of deforming tissue can be made more effective and accurate through the concept of Gaze-Contingent Motor Channelling. The method is also extended to 3D by introducing the concept of Gaze-Contingent Haptic Constraints where eye gaze is used to dynamically prescribe and update safety boundaries during robot-assisted MIS without prior knowledge of the soft-tissue morphology. Initial validation results on both simulated and robot assisted phantom procedures demonstrate the potential clinical value of the technique. In order to assess the associated cognitive demand of the proposed concepts, functional Near-Infrared Spectroscopy is used and preliminary results are discussed.

Off-Pump Positioning of a Conventional Aortic Valve Prosthesis through the Left Ventricular Apex with the Universal Cardiac Introducer under Sole Ultrasound Guidance, in the Pig

Objective

To test an alternative to catheter and open-heart techniques, by documenting the feasibility of implanting an unmodified mechanical aortic valve (AoV) in the off pump, beating heart using the universal cardiac introducer (UCI) attached to the left ventricular (LV) apex.

Methods

In six pigs, the LV apex was exposed by a median sternotomy. The UCI was attached to the apex. A 12-mm punching tool (punch), introduced through the UCI, was used to create a cylindrical opening through the apex. Then, the AoV, secured to a holder, was introduced into the LV, using transesophageal echocardiographic, guided through the apical LV opening, navigated into the LV outflow tract, and positioned within the aortic annulus. Trans-esophageal echocardiographic guidance was useful for navigation and positioning by superimposing the aortic annulus and prosthetic ring while Doppler imaging verified preserved prosthetic function and absence of perivalvular leaks. The valve function and hemodynamics were observed before termination for macroscopic evaluation.

Results

The punch produced a clean opening without fragmentation or myocardial embolization. During advancement of the mechanical AoV, there were no arrhythmias, mitral valve dysfunctions, evidence of myocardial ischemia, or hemodynamic instability. The AoVs were well seated over the annulus, without obstructing the coronaries or contact with the conduction system. The ring of AoVs was well circumscribed by the aortic annulus.

Conclusions

This study documented the feasibility of positioning a mechanical AoV on the closed, beating heart. These results should encourage the development of adjunct technologies to deliver current tissue or mechanical AoV with minimal side effects.