Autonomous control of an ultrasound probe for intra-operative ultrasonography using vision-based shape sensing of pneumatically attachable flexible rails

PURPOSE

In robotic-assisted minimally invasive surgery, surgeons often use intra-operative ultrasound to visualise endophytic structures and localise resection margins. This must be performed by a highly skilled surgeon. Automating this subtask may reduce the cognitive load for the surgeon and improve patient outcomes.

METHODS

We demonstrate vision-based shape sensing of the pneumatically attachable flexible (PAF) rail by using colour-dependent image segmentation. The shape-sensing framework is evaluated on known curves ranging from r = 30 to r = 110 mm, replicating curvatures in a human kidney. The shape sensing is then used to inform path planning of a collaborative robot arm paired with an intra-operative ultrasound probe. We execute 15 autonomous ultrasound scans of a tumour-embedded kidney phantom and retrieve viable ultrasound images, as well as seven freehand ultrasound scans for comparison.

RESULTS

The vision-based sensor is shown to have comparable sensing accuracy with FBGS-based systems. We find the RMSE of the vision-based shape sensing of the PAF rail compared with ground truth to be 0.4975 ± 0.4169 mm. The ultrasound images acquired by the robot and by the human were evaluated by two independent clinicians. The median score across all criteria for both readers was '3-good' for human and '4-very good' for robot.

CONCLUSION

We have proposed a framework for autonomous intra-operative US scanning using vision-based shape sensing to inform path planning. Ultrasound images were evaluated by clinicians for sharpness of image, clarity of structures visible, and contrast of solid and fluid areas. Clinicians evaluated that robot-acquired images were superior to human-acquired images in all metrics. Future work will translate the framework to a da Vinci surgical robot.

Ex Vivo Perfusion of Porcine Pancreas and Liver Sourced from Commercial Abattoirs after Circulatory Death as a Research Resource: A Methodological Study

BACKGROUND

Machine perfusion (MP) is increasingly used for human transplant organ preservation. The use of MP for research purposes is another opportunity for this technology. The porcine pancreas and liver are similar in anatomical size and function to their human counterparts, making them an excellent resource for research, but they have some important differences from human organs which can influence their research use. In this paper, we describe a technique developed and tested for the retrieval of porcine organs for use in research on perfused viable organs.

METHODS

Whole-organ porcine pancreata and livers were harvested at a commercial abattoir, following standard slaughterhouse processes. The standard slaughterhouse process involved a thoracotomy and mid-line laparotomy, and all the thoracoabdominal organs were removed. The pancreas, fixed in the retroperitoneum, was carefully dissected from its attachments to the surrounding structures, and tissue planes between the pancreas, spleen, duodenum, and colon were meticulously identified and dissected. Vessel exposure and division: The aorta, portal vein (PV), hepatic vein (HV), and hepatic artery (HA) were dissected and isolated, preserving the input and output channels for the liver and pancreas. A distal 3 cm of the aorta was preserved and divided and served as the input for the pancreas perfusions. The liver, PV, HV, and HA were preserved and divided to preserve the physiological channels of the input (PV and HA) and output (HV) for the liver perfusions. The porcine hepatic and pancreas anatomy shares significant resemblance with the gross anatomy found in humans, and this was taken into consideration when designing the perfusion circuitry. The porcine pancreas and spleen shared a common blood supply, with branches arising from the splenic artery. The organs were flushed with cold, heparinised normal saline and transported in a temperature-regulated receptacle maintained at a core temperature between 4 and 8 °C, in line with the standards of static cold storage (SCS), to a dedicated perfusion lab and perfused using our novel perfusion machine with autologous, heparinised porcine blood, also collected at the abattoir.

The Application of Machine Perfusion as an Enhanced ex vivo Model for Optical Imaging

Optical imaging techniques such as spectral imaging show promise for the assessment of tissue health during surgery; however, the validation and translation of such techniques into clinical practise is limited by the lack of representative tissue models. In this paper, we demonstrate the application of an organ perfusion machine as an ex vivo tissue model for optical imaging. Three porcine livers are perfused at stepped blood oxygen saturations. Over the duration of each perfusion, spectral data of the tissue are captured via diffuse optical spectroscopy and multispectral imaging. These data are synchronised with blood oxygen saturation measurements recorded by the perfusion machine. Shifts in the optical properties of the tissue are demonstrated over the duration of the each perfusion, as the tissue becomes reperfused and as the oxygen saturation is varied.

An adaptable research platform for ex vivo normothermic machine perfusion of the liver

PURPOSE

This paper presents an assessment of a low-cost organ perfusion machine designed for use in research settings. The machine is modular and versatile in nature, built on a robotic operating system (ROS2) pipeline allowing for the addition of specific sensors for different research applications. Here we present the system and the development stages to achieve viability of the perfused organ.

METHODS

The machine's perfusion efficacy was assessed by monitoring the distribution of perfusate in livers using methylene blue dye. Functionality was evaluated by measuring bile production after 90 min of normothermic perfusion, while viability was examined using aspartate transaminase assays to monitor cell damage throughout the perfusion. Additionally, the output of the pressure, flow, temperature, and oxygen sensors was monitored and recorded to track the health of the organ during perfusion and assess the system's capability of maintaining the quality of data over time.

RESULTS

The results show the system is capable of successfully perfusing porcine livers for up to three hours. Functionality and viability assessments show no deterioration of liver cells once normothermic perfusion had occurred and bile production was within normal limits of approximately 26 ml in 90 min showing viability.

CONCLUSION

The developed low-cost perfusion system presented here has been shown to keep porcine livers viable and functional ex vivo. Additionally, the system is capable of easily incorporating several sensors into its framework and simultaneously monitor and record them during perfusion. The work promotes further exploration of the system in different research domains.

Organ curvature sensing using pneumatically attachable flexible rails in robotic-assisted laparoscopic surgery

In robotic-assisted partial nephrectomy, surgeons remove a part of a kidney often due to the presence of a mass. A drop-in ultrasound probe paired to a surgical robot is deployed to execute multiple swipes over the kidney surface to localise the mass and define the margins of resection. This sub-task is challenging and must be performed by a highly-skilled surgeon. Automating this sub-task may reduce cognitive load for the surgeon and improve patient outcomes. The eventual goal of this work is to autonomously move the ultrasound probe on the surface of the kidney taking advantage of the use of the Pneumatically Attachable Flexible (PAF) rail system, a soft robotic device used for organ scanning and repositioning. First, we integrate a shape-sensing optical fibre into the PAF rail system to evaluate the curvature of target organs in robotic-assisted laparoscopic surgery. Then, we investigate the impact of the PAF rail's material stiffness on the curvature sensing accuracy, considering that soft targets are present in the surgical field. We found overall curvature sensing accuracy to be between 1.44% and 7.27% over the range of curvatures present in adult kidneys. Finally, we use shape sensing to plan the trajectory of the da Vinci surgical robot paired with a drop-in ultrasound probe and autonomously generate an Ultrasound scan of a kidney phantom.

Evaluation of A Novel Organ Perfusion Research Platform

This paper presents a novel, low cost, organ perfusion machine designed for use in research. The modular and versatile nature of the system allows for additional sensing equipment to be added or adapted for specific use. Here we introduce the system and present its preliminary evaluation by assessing its ability to maintain a predetermined input pressure. A proportional-integral-derivative (PID) controller was implemented and tested on a porcine liver to maintain input pressure to the hepatic artery and compared to bench tests. The results confirmed the effectiveness of the controller for maintaining input through the hepatic artery (HA) in a timely manner. Clinical Relevance-Machine Perfusion (MP) is proving to be an invaluable adjunct in clinical practice. With its ongoing success in the transplant arena, we propose MP for use in research. A cost-effective, versatile system that can be modified for specific research use to test new pharmacological therapies, imaging techniques or develop simulation training would be beneficial.

Autonomous pick-and-place using the dVRK

Purpose

Robotic-assisted partial nephrectomy (RAPN) is a tissue-preserving approach to treating renal cancer, where ultrasound (US) imaging is used for intra-operative identification of tumour margins and localisation of blood vessels. With the da Vinci Surgical System (Sunnyvale, CA), the US probe is inserted through an auxiliary access port, grasped by the robotic tool and moved over the surface of the kidney. Images from US probe are displayed separately to the surgical site video within the surgical console leaving the surgeon to interpret and co-registers information which is challenging and complicates the procedural workflow.

Methods

We introduce a novel software architecture to support a hardware soft robotic rail designed to automate intra-operative US acquisition. As a preliminary step towards complete task automation, we automatically grasp the rail and position it on the tissue surface so that the surgeon is then able to manipulate manually the US probe along it.

Results

A preliminary clinical study, involving five surgeons, was carried out to evaluate the potential performance of the system. Results indicate that the proposed semi-autonomous approach reduced the time needed to complete a US scan compared to manual tele-operation.

Conclusion

Procedural automation can be an important workflow enhancement functionality in future robotic surgery systems. We have shown a preliminary study on semi-autonomous US imaging, and this could support more efficient data acquisition.

Ultrasound 3D reconstruction of malignant masses in robotic-assisted partial nephrectomy using the PAF rail system: a comparison study

Purpose

In robotic-assisted partial nephrectomy (RAPN), the use of intraoperative ultrasound (IOUS) helps to localise and outline the tumours as well as the blood vessels within the kidney. The aim of this work is to evaluate the use of the pneumatically attachable flexible (PAF) rail system for US 3D reconstruction of malignant masses in RAPN. The PAF rail system is a novel device developed and previously presented by the authors to enable track-guided US scanning.

Methods

We present a comparison study between US 3D reconstruction of masses based on: the da Vinci Surgical System kinematics, single- and stereo-camera tracking of visual markers embedded on the probe. An US-realistic kidney phantom embedding a mass is used for testing. A new design for the US probe attachment to enhance the performance of the kinematic approach is presented. A feature extraction algorithm is proposed to detect the margins of the targeted mass in US images.

Results

To evaluate the performance of the investigated approaches the resulting 3D reconstructions have been compared to a CT scan of the phantom. The data collected indicates that single camera reconstruction outperformed the other approaches, reconstructing with a sub-millimetre accuracy the targeted mass.

Conclusions

This work demonstrates that the PAF rail system provides a reliable platform to enable accurate US 3D reconstruction of masses in RAPN procedures. The proposed system has also the potential to be employed in other surgical procedures such as hepatectomy or laparoscopic liver resection.

Real-Time Vision-Based Stiffness Mapping †

This paper presents new findings concerning a hand-held stiffness probe for the medical diagnosis of abnormalities during palpation of soft-tissue. Palpation is recognized by the medical community as an essential and low-cost method to detect and diagnose disease in soft-tissue. However, differences are often subtle and clinicians need to train for many years before they can conduct a reliable diagnosis. The probe presented here fills this gap providing a means to easily obtain stiffness values of soft tissue during a palpation procedure. Our stiffness sensor is equipped with a multi degree of freedom (DoF) Aurora magnetic tracker, allowing us to track and record the 3D position of the probe whilst examining a tissue area, and generate a 3D stiffness map in real-time. The stiffness probe was integrated in a robotic arm and tested in an artificial environment representing a good model of soft tissue organs; the results show that the sensor can accurately measure and map the stiffness of a silicon phantom embedded with areas of varying stiffness.

Fingertip Fiber Optical Tactile Array with Two-Level Spring Structure

Tactile perception is a feature benefiting reliable grasping and manipulation. This paper presents the design of an integrated fingertip force sensor employing an optical fiber based approach where applied forces modulate light intensity. The proposed sensor system is developed to support grasping of a broad range of objects, including those that are hard as well those that are soft. The sensor system is comprised of four sensing elements forming a tactile array integrated with the tip of a finger. We investigate the design configuration of a separate force sensing element with the aim to improve its measurement range. The force measurement of a single tactile element is based on a two-level displacement that is achieved thanks to a hybrid sensing structure made up of a stiff linear and flexible ortho-planar spring. An important outcome of this paper is a miniature tactile fingertip sensor that is capable of perceiving light contact, typically occurring during the initial stages of a grasp, as well as measuring higher forces, commonly present during tight grasps.

Endoscopic add-on stiffness probe for real-time soft surface characterisation in MIS

This paper explores a novel stiffness sensor which is mounted on the tip of a laparoscopic camera. The proposed device is able to compute stiffness when interacting with soft surfaces. The sensor can be used in Minimally Invasive Surgery, for instance, to localise tumor tissue which commonly has a higher stiffness when compared to healthy tissue. The purely mechanical sensor structure utilizes the functionality of an endoscopic camera to the maximum by visually analyzing the behavior of trackers within the field of view. Two pairs of spheres (used as easily identifiable features in the camera images) are connected to two springs with known but different spring constants. Four individual indenters attached to the spheres are used to palpate the surface. During palpation, the spheres move linearly towards the objective lens (i.e. the distance between lens and spheres is changing) resulting in variations of their diameters in the camera images. Relating the measured diameters to the different spring constants, a developed mathematical model is able to determine the surface stiffness in real-time. Tests were performed using a surgical endoscope to palpate silicon phantoms presenting different stiffness. Results show that the accuracy of the sensing system developed increases with the softness of the examined tissue.