Needle path planning and steering in a three-dimensional non-static environment using two-dimensional ultrasound images

Needle Tip Tracking through Photoluminescence for Minimally Invasive Surgery

Minimally invasive surgery continues to prioritize patient safety by improving imaging techniques and tumor detection methods. In this work, an all-optical alternative to the current image based techniques for in vitro minimally invasive procedures has been explored. The technique uses a highly fluorescent marker for the surgical needle to be tracked inside simulated tissues. A series of markers were explored including inorganic (Perovskite and PbS) and organic (carbon dots) nanoparticles and organic dye (Rhodamine 6G) to identify layers of different stiffnesses within a tissue. Rhodamine 6G was chosen based on its high fluorescence signal to track 3D position of a surgical needle in a tissue. The needle was tracked inside homogeneous and inhomogeneous gelatin tissues successfully. This exploratory study of tissue characterization and needle tip tracking using fluorescent markers or photoluminescence technique show potential for real-time application of robot-assisted needle insertions during in vivo procedures.

A survey on puncture models and path planning algorithms of bevel-tipped flexible needles

Percutaneous needle insertion is a minimally invasive surgery with broad medical application prospects, such as biopsy and brachytherapy. However, the currently adopted rigid needles have limitations, as they cannot bypass obstacles or correct puncture deviations and can only travel along a straight path. Bevel-tip flexible needles are increasingly being adopted to address these issues, owing to their needle body's ease of deformation and bending. Successful puncture of flexible needles relies on accurate models and path planning, ensuring the needle reaches the target while avoiding vital tissues. This review investigates puncture models and path-planning algorithms by reviewing recent literature, focusing on the path-planning part. According to the literature, puncture models can be divided into three types: mechanical, finite element method (FEM), and kinematic models, while path-planning algorithms are categorized and discussed following the division used for mobile robots, which differs from the conventional approach for flexible needles-an innovation in this review. This review systematically summarizes the following categories: graph theory search, sampling-based, intelligent search, local obstacle avoidance, and other algorithms, including their implementation, advantages, and disadvantages, to further explore the potential to overcome obstacles in path planning for minimally invasive puncture needles. Finally, this study proposes future development trends in path-planning algorithms, providing possible directions for subsequent research for bevel-tipped flexible needles. This research aims to provide a resource for researchers to quickly learn about common path-planning algorithms, their backgrounds, and puncture models.

A Model to Predict Deflection of an Active Tendon-Driven Notched Needle Inside Soft Tissue

The last decade has witnessed major progress in the field of minimally invasive and robotic-assisted surgeries. Needle insertion, a minimally invasive technique, has proven its efficacy in procedures such as brachytherapy, ablation, drug delivery, and biopsy. Manual needle steering inside tissue is a challenging task due to complex needle-tissue interactions, needle and tissue movement, lack of actuation and control, as well as poor sensing and visualization. Recently, active tendon-driven notched needles, and robotic manipulation systems have been proposed to assist surgeons to guide the needles in desired trajectories toward target positions. This work introduces a new deflection model for the active tendon-driven notched needle steering inside soft tissue for intention to use in model-based robotic control. The model is developed to predict needle deflection in a single-layer tissue. To validate the proposed deflection model, five sets of needle insertion experiments with a bevel-tipped active needle into single-layer phantom tissues were performed. A real-time robot-assisted ultrasound tracking method was used to track the needle tip during needle insertion. It was shown that the model predicts needle deflection with an average error of 0.58 ± 0.14 mm for the bevel-tipped active needle insertion into a single-layer phantom tissue.

Model-free control for autonomous prevention of adverse events in robotics

Introduction: Preventive control is a critical feature in autonomous technology to ensure safe system operations. One application where safety is most important is robot-assisted needle interventions. During incisions into a tissue, adverse events such as mechanical buckling of the needle shaft and tissue displacements can occur on encounter with stiff membranes causing potential damage to the organ. Methods: To prevent these events before they occur, we propose a new control subroutine that autonomously chooses a) a reactive mechanism to stop the insertion procedure when a needle buckling or a severe tissue displacement event is predicted and b) an adaptive mechanism to continue the insertion procedure through needle steering control when a mild tissue displacement is detected. The subroutine is developed using a model-free control technique due to the nonlinearities of the unknown needle-tissue dynamics. First, an improved version of the model-free adaptive control (IMFAC) is developed by computing a fast time-varying partial pseudo derivative analytically from the dynamic linearization equation to enhance output convergence and robustness against external disturbances. Results and Discussion: Comparing IMFAC and MFAC algorithms on simulated nonlinear systems in MATLAB, IMFAC shows 20% faster output convergence against arbitrary disturbances. Next, IMFAC is integrated with event prediction algorithms from prior work to prevent adverse events during needle insertions in real time. Needle insertions in gelatin tissues with known environments show successful prevention of needle buckling and tissue displacement events. Needle insertions in biological tissues with unknown environments are performed using live fluoroscopic imaging as ground truth to verify timely prevention of adverse events. Finally, statistical ANOVA analysis on all insertion data shows the robustness of the prevention algorithm to various needles and tissue environments. Overall, the success rate of preventing adverse events in needle insertions through adaptive and reactive control was 95%, which is important toward achieving safety in robotic needle interventions.

A novel computational fracture toughness model for soft tissue in needle insertion

During the process of percutaneous puncture vascular intervention operation in endoscopic liver surgery, high precision needle manipulation requires the accurate needle tissue interaction model where the tissue fracture toughness is an important parameter to describe the tissue crack propagation, as well as to estimate tissue deformation and target displacement. However, the existing studies on fracture toughness estimation did not consider Young's modulus and the organ capsule structure. In this paper, a novel computational fracture toughness model is proposed considering insertion velocity, needle diameter and Young's modulus in insertion process, where the fracture toughness is determined by the tissue surface deformation, which was estimated through energy modeling using integrated shell element and three-dimensional solid element. The testbed is built to study the effect of different insertion velocities, needle diameters and Young's modulus on fracture toughness. The experiment result shows that the estimated result of computational fracture toughness model agrees well with the physical experimental data. In addition, the sensitivity analysis of different factors is conducted. Meanwhile, the model robustness analysis is investigated with different observation noises of Young's modulus and puncture displacement.

Learning-based needle tip tracking in 2D ultrasound by fusing visual tracking and motion prediction

Visual trackers are the most commonly adopted approach for needle tip tracking in ultrasound (US)-based procedures. However, they often perform unsatisfactorily in biological tissues due to the significant background noise and anatomical occlusion. This paper presents a learning-based needle tip tracking system, which consists of not only a visual tracking module, but also a motion prediction module. In the visual tracking module, two sets of masks are designed to improve the tracker's discriminability, and a template update submodule is used to keep up to date with the needle tip's current appearance. In the motion prediction module, a Transformer network-based prediction architecture estimates the target's current position according to its historical position data to tackle the problem of target's temporary disappearance. A data fusion module then integrates the results from the visual tracking and motion prediction modules to provide robust and accurate tracking results. Our proposed tracking system showed distinct improvement against other state-of-the-art trackers during the motorized needle insertion experiments in both gelatin phantom and biological tissue environments (e.g. 78% against <60% in terms of the tracking success rate in the most challenging scenario of "In-plane-static" during the tissue experiments). Its robustness was also verified in manual needle insertion experiments under varying needle velocities and directions, and occasional temporary needle tip disappearance, with its tracking success rate being >18% higher than the second best performing tracking system. The proposed tracking system, with its computational efficiency, tracking robustness, and tracking accuracy, will lead to safer targeting during existing clinical practice of US-guided needle operations and potentially be integrated in a tissue biopsy robotic system.

Flexible Needle Steering with Tethered and Untethered Actuation: Current States, Targeting Errors, Challenges and Opportunities

Accurate needle targeting is critical for many clinical procedures, such as transcutaneous biopsy or radiofrequency ablation of tumors. However, targeting errors may arise, limiting the widespread adoption of these procedures. Advances in flexible needle (FN) steering are emerging to mitigate these errors. This review summarizes the state-of-the-art developments of FNs and addresses possible targeting errors that can be overcome with steering actuation techniques. FN steering techniques can be classified as passive and active. Passive steering directly results from the needle-tissue interaction forces, whereas active steering requires additional forces to be applied at the needle tip, which enhances needle steerability. Therefore, the corresponding targeting errors of most passive FNs and active FNs are between 1 and 2 mm, and less than 1 mm, respectively. However, the diameters of active FNs range from 1.42 to 12 mm, which is larger than the passive steering needle varying from 0.5 to 1.4 mm. Therefore, the development of active FNs is an area of active research. These active FNs can be steered using tethered internal direct actuation or untethered external actuation. Examples of tethered internal direct actuation include tendon-driven, longitudinal segment transmission and concentric tube transmission. Tendon-driven FNs have various structures, and longitudinal segment transmission needles could be adapted to reduce tissue damage. Additionally, concentric tube needles have immediate advantages and clinical applications in natural orifice surgery. Magnetic actuation enables active FN steering with untethered external actuation and facilitates miniaturization. The challenges faced in the fabrication, sensing, and actuation methods of FN are analyzed. Finally, bio-inspired FNs may offer solutions to address the challenges faced in FN active steering mechanisms.

Review of research on path planning and control methods of flexible steerable needle puncture robot

In the field of minimally invasive interventional therapy, the related research on the soft tissue puncture robot and its technology based on the flexible steerable needle as a research hot topic at present, and it has been developed rapidly in the past ten years. In order to better understand the development status of the flexible steerable needle puncture (FSNP) robot and provide reference for its design and improvement in subsequent research, it is necessary to introduce in two aspects of FSNP robot: the puncture path planning and the control methods. First, this article introduced the concept of the FSNP technology, and the necessity of the application of FSNP soft tissue robot in minimally invasive interventional surgery. Second, this article mainly introduced the principle of FSNP, the path planning of FSNP, the navigation and positioning control of the needle tip of the flexible steerable needle, the control method of FSNP system, and the controllable flexible needle. Finally, combined with the above analysis and introduction, it was pointed out that FSNP soft tissue robot and its related technology would be an important development direction in the field of minimally invasive interventional therapy in the future, and the current existing problems were pointed out. Meanwhile, the development trend of FSNP robot control technology was summarized and prospected.

Integrating robot-assisted ultrasound tracking and 3D needle shape prediction for real-time tracking of the needle tip in needle steering procedures

BACKGROUND

Needle insertions have been used in several minimally invasive procedures for diagnostic and therapeutic purposes. Real-time position of the needle tip is an important information in needle steering systems.

METHODS

This work introduces a robot-assisted ultrasound tracking (R-AUST) system integrated with a needle shape prediction method to provide 3D position of the needle tip. The tracking system is evaluated in phantom and ex vivo beef liver tissues.

RESULTS

An average error of 0.60 mm was found for needle insertion tests inside the phantom tissue. The R-AUST integrated with shape prediction in the beef liver tissue was able to track the needle tip with an average and maximum error of 0.37 and 0.67 mm, respectively. The average error reported in this work is within the mean allowable needle placement error (<2.7 mm) in targeted procedures.

CONCLUSIONS

Integration of R-AUST tracking method with needle shape prediction results in a reasonably accurate real-time tracking suitable for ultrasound-guided needle insertions.

Intraoperative optimization of seed implantation plan in breast brachytherapy

PURPOSE

Low-dose-rate permanent-seed (LDR-PS) brachytherapy has shown a great potential for treating breast cancer. An implantation scheme indicating the template pose and needle trajectories is determined before the operation. However, when performing the pre-planned scheme intraoperatively, a change of the patient's posture will cause seed placements away from the desired locations. Hence, the implantation scheme should update based on the current patient's posture.

METHODS

A numerical method of optimizing the implantation scheme for the LDR-PS breast brachytherapy is presented here. The proposed algorithm determines the fewest needle trajectories and template poses for delivering the seeds to the intraoperative desired locations. The clinical demand, such as the minimum distance between the chest wall and the needle, is considered in the optimization process.

RESULTS

The method was simulated for a given LDR-PS brachytherapy procedure to evaluate the optimal scheme as the number of the template poses changing. The optimization parameters of the needles' number and the implantation errors are used to adjust the algorithm outcome. The results show that the implantation schemes obtained by our method have a satisfactory accuracy in the cases of 2 or 3 template poses. The computation time is about 76s to 150s according to the number of the template poses from 1 to 3.

CONCLUSION

The proposed method can find the optimal implantation scheme corresponding to the current desired seed locations immediately once there is a change of patient's posture. This work can be applied to the robot-assisted LDR-PS breast brachytherapy for improving the operation accuracy and efficiency.

Dynamic path planning for percutaneous procedures in the abdomen during free breathing

PURPOSE

Percutaneous procedures are increasingly used for the treatment of tumors in abdominal structures. Most of the time, these procedures are planned based on static preoperative images and do not take into account any motions, while breathing control is not always applicable. In this paper, we present a method to automatically adjust the planned path in real time according to the breathing.

METHODS

First, an estimation of the organs motions during breathing is performed during an observation phase. Then we propose an approach named Real Time Intelligent Trajectory (RTIT) that consists in finding the most appropriate moments to push the needle along the initially planned path, based on the motions and the distance to surrounding organs. We also propose a second approach called Real Time Straight Trajectory (RTST) that examines sixteen scenarios of needle insertion at constant speed, starting at eight different moments of the breathing cycle with two different speeds.

RESULTS

We evaluated our methods on six 3D models of abdominal structures built using image datasets and a real-time simulation of breathing movements. We measured the deviation from the initial path, the target positioning error, and the distance of the actual path to risky structures. The path proposed by RTIT approach is compared to the best path proposed by RTST.

CONCLUSIONS

We show that the RTIT approach is relevant and adapted to breathing movements. The modification of the path remains minimal while collisions with obstacles are avoided. This study on simulations constitutes a first step towards intelligent robotic insertion under real-time image guidance.

Intelligent magnetic manipulation for gastrointestinal ultrasound

Diagnostic endoscopy in the gastrointestinal tract has remained largely unchanged for decades and is limited to the visualization of the tissue surface, the collection of biopsy samples for diagnoses, and minor interventions such as clipping or tissue removal. In this work, we present the autonomous servoing of a magnetic capsule robot for in-situ, subsurface diagnostics of microanatomy. We investigated and showed the feasibility of closed-loop magnetic control using digitized microultrasound (μUS) feedback; this is crucial for obtaining robust imaging in an unknown and unconstrained environment. We demonstrated the functionality of an autonomous servoing algorithm that uses μUS feedback, both on benchtop trials as well as in-vivo in a porcine model. We have validated this magnetic-μUS servoing in instances of autonomous linear probe motion and were able to locate markers in an agar phantom with 1.0 ± 0.9 mm position accuracy using a fusion of robot localization and μUS image information. This work demonstrates the feasibility of closed-loop robotic μUS imaging in the bowel without the need for either a rigid physical link between the transducer and extracorporeal tools or complex manual manipulation.

Event-Triggered 3D Needle Control Using a Reduced-Order Computationally Efficient Bicycle Model in a Constrained Optimization Framework

Long flexible needles used in percutaneous procedures such as biopsy and brachytherapy deflect during insertion, thus reducing needle tip placement accuracy. This paper presents a surgeon-in-the-loop system to automatically steer the needle during manual insertion and compensate for needle deflection using an event-triggered controller. A reduced-order kinematic bicycle model incorporating needle tip measurement data from ultrasound images is used to determine steering actions required to minimize needle deflection. To this end, an analytic solution to the reduced-order bicycle model, which is shown to be more computationally efficient than a discrete-step implementation of the same model, is derived and utilized for needle tip trajectory prediction. These needle tip trajectory predictions are used online to optimize the insertion depths (event-trigger points) for steering actions such that needle deflection is minimized. The use of the analytic model and the event-triggered controller also allows for limiting the number and extent of needle rotations (to reduce tissue trauma) in a constrained optimization framework. The system was tested experimentally in three different ex-vivo tissue phantoms with a surgeon-in-the-loop needle insertion device. The proposed needle steering controller was shown to keep the average needle deflection within 0.47 [Formula: see text] 0.21[Formula: see text]mm at the final insertion depth of 120[Formula: see text]mm.

DUTY-CYCLED SPINNING BASED 3D MOTION CONTROL APPROACH FOR BEVEL-TIPPED FLEXIBLE NEEDLE INSERTION

Flexible needle has been widely used in the therapy delivery because it can advance along the curved lines to avoid the obstacles like important organs and bones. However, most control algorithms for the flexible needle are still limited to address its motion along a set of arcs in the two-dimensional (2D) plane. To resolve this problem, this paper has proposed an improved duty-cycled spinning based three-dimensional (3D) motion control approach to ensure that the beveled-tip flexible needle can track a desired trajectory to reach the target within the tissue. Compared with the existing open-loop duty-cycled spinning method which is limited to tracking 2D trajectory comprised of few arcs, the proposed closed-loop control method can be used for tracking any 3D trajectory comprised of numerous arcs. Distinctively, the proposed method is independent of the tissue parameters and robust to such disturbances as tissue deformation. In the trajectory tracking simulation, the designed controller is tested on the helical trajectory, the trajectory generated by rapidly-exploring random tree (RRT) algorithm and the helical trajectory. The simulation results show that the mean tracking error and the target error are less than 0.02[Formula: see text]mm for the former two kinds of trajectories. In the case of tracking the helical trajectory, the mean tracking error target error is less than 0.5[Formula: see text]mm and 1.5[Formula: see text]mm, respectively. The simulation results prove the effectiveness of the proposed method.

Path optimization algorithm and its robustness for bevel tip flexible needle

Bevel tip flexible needle is a novel application in minimally invasive surgery, for it can avoid obstacles by performing a curved trajectory to reach the target. In clinical surgeries, path optimization is a basis for a robot-assisted surgery, and robustness is a crucial issue for an algorithm. However, to the best of our knowledge, none of the researches has an intensive study on the robustness of an algorithm for a bevel tip needle’s path optimization. In this article, a path optimization algorithm for a bevel tip flexible needle is proposed based on a mathematical calculation method by establishing an optimization objective function, and the robustness of the algorithm is analyzed regarding to each weighting coefficient of the objective function. Simulation results show that on the one hand, the algorithm can obtain the optimal path effectively in the presence of obstacles; and on the other hand, the optimization function has little sensitivity to any of the weighting coefficients, verifying strong robustness of the algorithm. Experiments for three typical paths are performed, and the accuracy is within 2 mm which fulfills the surgical requirements. The experimental results not only prove the feasibility of the paths obtained by the algorithm but also verify the validity of the proposed path optimization algorithm.

Closed-loop control of bevel-tip needles based on path planning

Bevel-tip needles have the potential to improve paracentetic precision and decrease paracentetic traumas. In order to drive bevel-tip needles precisely with the constrains of path length and path dangerousness, we propose a closed-loop control method that only requires the position of the needle tip and can be easily applied in a clinical setting. The control method is based on the path planning method proposed in this paper. To establish the closed-loop control method, a kinematic model of bevel-tip needles is first presented, and the relationship between the puncture path and controlled variables is established. Second, we transform the path planning method into a multi-objective optimization problem, which takes the path error, path length and path dangerousness into account. Multi-objective particle swarm optimization is employed to solve the optimization problem. Then, a control method based on path planning is presented. The current needle tip attitude is essential to plan an insertion path. We analyze two methods to obtain the tip attitude and compare their effects using both simulations and experiments. In the end, simulations and experiments in phantom tissue are executed and analyzed, the results show that our methods have high accuracy and have the ability to deal with the model parameter uncertainty.

Pattern generation and motion control of a vortex-like paramagnetic nanoparticle swarm

Controlling a swarm of microrobots with external fields is one of the major challenges for untethered microrobots. In this work, we present a new method to generate a vortex-like paramagnetic nanoparticle swarm (VPNS) from dispersed nanoparticles with a diameter of 500 nm, using rotating magnetic fields. The VPNS exhibits a dynamic-equilibrium structure, in which the nanoparticles perform synchronized motions. The mechanisms of the pattern-generation process are analyzed, simulated, and validated by experiments. By tuning the rotating frequency of the input magnetic field, the pattern of a VPNS changes accordingly. Analytical models for estimating the areal change of the pattern are proposed, and they have good agreement with the experimental data. Moreover, reversible merging and splitting of vortex-like swarms are demonstrated and investigated. Serving as a mobile robotic end-effector, a VPNS is capable of making locomotion by tuning the pitch angle of the actuating rotating field. With a small pitch angle, e.g. 2°, the whole swarm moves as an entity, and the shape of the pattern remains intact. In addition, the trapping forces of VPNSs are verified, showing the critical input parameters of the magnetic field that affect the morphology of the swarm. Finally, we demonstrate that VPNSs pass through curved and branched channels with high positioning precision, and the access rates for targeted delivery are over 90%, which are significantly higher than those in the cases of particle swarms moving with tumbling motions.

Closed-Loop Active Compensation for Needle Deflection and Target Shift During Cooperatively Controlled Robotic Needle Insertion

Intra-operative imaging is sometimes available to assist needle biopsy, but typical open-loop insertion does not account for unmodeled needle deflection or target shift. Closed-loop image-guided compensation for deviation from an initial straight-line trajectory through rotational control of an asymmetric tip can reduce targeting error. Incorporating robotic closed-loop control often reduces physician interaction with the patient, but by pairing closed-loop trajectory compensation with hands-on cooperatively controlled insertion, a physician's control of the procedure can be maintained while incorporating benefits of robotic accuracy. A series of needle insertions were performed with a typical 18G needle using closed-loop active compensation under both fully autonomous and user-directed cooperative control. We demonstrated equivalent improvement in accuracy while maintaining physician-in-the-loop control with no statistically significant difference (p > 0.05) in the targeting accuracy between any pair of autonomous or individual cooperative sets, with average targeting accuracy of 3.56 mm_rms. With cooperatively controlled insertions and target shift between 1 and 10 mm introduced upon needle contact, the system was able to effectively compensate up to the point where error approached a maximum curvature governed by bending mechanics. These results show closed-loop active compensation can enhance targeting accuracy, and that the improvement can be maintained under user directed cooperative insertion.

Path planning for minimally-invasive knee surgery using a hybrid optimization procedure

The aim of this study was to develop a procedure for medical tool path planning in minimally-invasive knee surgery. The collision-free paths for the tool were obtained using the control locations method with a hybrid optimization strategy. The tool and knee elements were described with surface meshes. The knee model allowed for bones displacement and variable incision size and location. The proposed procedure was proven to be effective in path planning for minimally-invasive surgery. It can serve as a valuable aid in surgery planning and may also be used in systems for autonomous or semi-autonomous knee surgery.

A novel planning solution for semi-autonomous aspiration of Baker's cysts

BACKGROUND

A Baker's cyst is a pathological structure located near a kneepit, which causes discomfort and reduces mobility of the knee. It is commonly treated with aspiration, which often requires MRI scanning and US guidance. The aim of this study was to propose a novel planning solution for semi-autonomous aspiration of the Baker's cyst using only MRI imaging.

METHODS

The proposed method requires minimal user input and offers automatic cyst segmentation with collision-free path planning for the assumed robotic structure with four degrees of freedom.

RESULTS

The prepared software was tested on four image sets obtained from patients eligible for cyst aspiration. It was possible to accurately segment the cyst in the considered cases. The collision-free path planning method was investigated in numerical scenarios.

CONCLUSIONS

The simulations verified the proposed software solution. Future work will be devoted to experimental verification of the path planning procedure.

Closed-loop asymmetric-tip needle steering under continuous intraoperative MRI guidance

Magnetic resonance imaging (MRI) provides excellent image contrast for various types of tissues, making it a suitable choice over other imaging modalities for various image-guided needle interventions. Furthermore, robot-assistance is maturing for surgical procedures such as percutaneous prostate and brain interventions. Although MRI-guided, robot-assisted needle interventions are approaching clinical usage, they are still typically open-loop in nature due to the lack of continuous intraoperative needle tracking. Closed-loop needle-based procedures can improve the accuracy of needle tip placement by correcting the needle trajectory during insertion. This paper proposes a system for robot-assisted, flexible asymmetric-tipped needle interventions under continuous intraoperative MRI guidance. A flexible needle's insertion depth and rotation angle are manipulated by an MRI-compatible robot in the bore of the MRI scanner during continuous multi-planar image acquisition to reach a desired target location. Experiments are performed on gelatin phantoms to assess the accuracy of needle placement into the target location. The system was able to successfully utilize live MR imaging to guide the path of the needle, and results show an average total targeting error of 2.5±0.47mm, with an average in-plane error of 2.09±0.33mm.

Histological evaluation of tissue damage caused by rotational needle insertion

Needles used in percutaneous insertion must be as thin as possible to minimize invasiveness. However, using extra-thin needles with a diameter less than 25G (0.53 mm diameter) can cause needle deflection. Needle deflection can be minimized by insertion with axial rotation along the needle shaft; this rotation is also useful for steering the insertion direction of the needle tip. However, although high rotation speeds may decrease needle deflection, this may increase tissue damage. Therefore, the purpose of this study was to histologically evaluate tissue damage caused by the rotational needle-insertion method, and to verify the needle-tip deflection caused by tissue damage. In this paper, we evaluated tissue damage and needle deflection caused by needle insertion with no rotation, unidirectional rotation, and bidirectional rotation. The results suggest that percutaneous needle insertion under unidirectional rotation is potentially risky in humans, as this causes wound-up tissue and expansion of the area of the hole created by the needle path. In contrast, needle insertion under bidirectional rotation appeared to minimize deflection, and prevented winding of tissue and expansion of the hole created by the needle path.

Active Localization and Tracking of Needle and Target in Robotic Image-Guided Intervention Systems

This paper describes a framework of algorithms for the active localization and tracking of flexible needles and targets during image-guided percutaneous interventions. The needle and target configurations are tracked by Bayesian filters employing models of the needle and target motions and measurements of the current system state obtained from an intra-operative imaging system which is controlled by an entropy-minimizing active localization algorithm. Versions of the system were built using particle and unscented Kalman filters and their performance was measured using both simulations and hardware experiments with real magnetic resonance imaging data of needle insertions into gel phantoms. Performance of the localization algorithms is given in terms of accuracy of the predictions and computational efficiency is discussed.

Towards synergistic control of hands-on needle insertion with automated needle steering for MRI-guided prostate interventions

A significant hurdle of accurate needle tip placement in percutaneous needle-based prostate interventions is unmodeled needle deflection and tissue deformation during insertion. This paper introduces a robotic platform for developing synergistic, cooperatively controlled needle insertion algorithms decoupled from closed-loop image-guided needle steering. Shared control of the surgical workspace through human-robot synergy creates a balance between the accuracy of robotic autonomy while still providing ultimate control of the procedure to the physician. Validation tests were performed using camera-based image-guided feedback control of needle steering with cooperative hands-on needle insertion. Locations were targeted inside a transparent gelatin phantom with an average total error of 2.68 ± 0.34mm and in-plane error of 2.59 ± 0.30mm.

Autonomous planning and control of soft untethered grippers in unstructured environments

The use of small, maneuverable, untethered and reconfigurable robots could provide numerous advantages in various micromanipulation tasks. Examples include microassembly, pick-and-place of fragile micro-objects for lab-on-a-chip applications, assisted hatching for in-vitro fertilization and minimally invasive surgery. This study assesses the potential of soft untethered magnetic grippers as alternatives or complements to conventional tethered or rigid micromanipulators. We demonstrate closed-loop control of untethered grippers and automated pick-and-place of biological material on porcine tissue in an unstructured environment. We also demonstrate the ability of the soft grippers to recognize and sort non-biological micro-scale objects. The fully autonomous nature of the experiments is made possible by the integration of planning and decision-making algorithms, as well as by closed-loop temperature and electromagnetic motion control. The grippers are capable of completing pick-and-place tasks of biological material at an average velocity of 1.8 ±0.71 mm/s and a drop-off error of 0.62 ±0.22 mm. Color-sensitive sorting of three micro-scale objects is completed at a velocity of 1.21 ±0.68 mm/s and a drop-off error of 0.85 ±0.41 mm. Our findings suggest that improved autonomous untethered grippers could augment the capabilities of current soft-robotic instruments especially in advancedtasks involving manipulation.

Human-robot collaborated path planning for bevel-tip needle steering in simulated human environment

Clinical Application of linear percutaneous needle insertion is restricted due to issues such as limited path and deflection. Thus steering of flexible needle is critical demanded in the clinic. Previous studies tended to use autonomous methods to conduct path planning for needle steering. However, these methods had very limited adaptabilities, and they also decreased the human operator's domination of the operation, as clinically required. In this case, teleoperation has been an option, while in complicated environments sole teleoperation is not sufficient for a human operator to generate multi-curved insertion path. Therefore, in this paper, we propose a semiautonomous human-robot collaborated path planning method for teleoperated bevel-tip needle steering. The key module of this method is a human-robot collaboration mechanism which consists of the operator input, environment constraints, and path constraints. The proposed method were tested semi-physically in a simulated human environment and the results validated that the proposed method were able to efficiently assist the operator to generate multi-curved paths under human operator's domination.

3D Dynamic Motion Planning for Robot-Assisted Cannula Flexible Needle Insertion into Soft Tissue

In robot-assisted needle-based medical procedures, insertion motion planning is a crucial aspect. 3D dynamic motion planning for a cannula flexible needle is challenging with regard to the nonholonomic motion of the needle tip, the presence of anatomic obstacles or sensitive organs in the needle path, as well as uncertainties due to the dynamic environment caused by the movements and deformations of the organs. The kinematics of the cannula flexible needle is calculated in this paper. Based on a rapid and robust static motion planning algorithm, referred to as greedy heuristic and reachability-guided rapidly-exploring random trees, a 3D dynamic motion planner is developed by using replanning. Aiming at the large detour problem, the convergence problem and the accuracy problem that replanning encounters, three novel strategies are proposed and integrated into the conventional replanning algorithm. Comparisons are made between algorithms with and without the strategies to verify their validity. Simulations showed that the proposed algorithm can overcome the above-noted problems to realize real-time replanning in a 3D dynamic environment, which is appropriate for intraoperative planning.

Ultrasound-Guided Model Predictive Control of Needle Steering in Biological Tissue

In needle-based medical procedures, beveled tip flexible needles are steered inside soft tissue to reach the desired target locations. In this paper, we have developed an autonomous image-guided needle steering system that enhances targeting accuracy in needle insertion while minimizing tissue trauma. The system has three main components. First is a novel mechanics-based needle steering model that predicts needle deflection and accepts needle tip rotation as an input for needle steering. The second is a needle tip tracking system that determines needle deflection from the ultrasound images. The needle steering model employs the estimated needle deflection at the present time to predict needle tip trajectory in the future steps. The third component is a nonlinear model predictive controller (NMPC) that steers the needle inside the tissue by rotating the needle beveled tip. The MPC controller calculates control decisions based on iterative optimization of the predictions of the needle steering model. To validate the proposed ultrasound-guided needle steering system, needle insertion experiments in biological tissue phantoms are performed in two cases–with and without obstacle. The results demonstrate that our needle steering strategy guides the needle to the desired targets with the maximum error of 2.85[Formula: see text]mm.

The First Interlaced Continuum Robot, Devised to Intrinsically Follow the Leader

Flexible probes that are safely deployed to hard-to-reach targets while avoiding critical structures are strategic in several high-impact application fields, including the biomedical sector and the sector of inspections at large. A critical problem for these tools is the best approach for deploying an entire tool body, not only its tip, on a sought trajectory. A probe that achieves this deployment is considered to follow the leader (or to achieve follow-the-leader deployment) because its body sections follow the track traced by its tip. Follow-the-leader deployment through cavities is complicated due to a lack of external supports. Currently, no definitive implementation for a probe that is intrinsically able to follow the leader, i.e., without relying on external supports, has been achieved. In this paper, we present a completely new device, namely the first interlaced continuum robot, devised to intrinsically follow the leader. We developed the interlaced configuration by pursuing a conceptual approach irrespective of application-specific constraints and assuming two flexible tools with controllable stiffness. We questioned the possibility of solving the previously mentioned deployment problem by harnessing probe symmetry during the design process. This study examines the entire development of the novel interlaced probe: model-based conceptual design, detailed design and prototyping, and preliminary experimental assessment. Our probe can build a track with a radius of curvature that is as small as twice the probe diameter, which enables it to outperform state-of-the-art tools that are aimed at follow-the-leader deployment. Despite the limitations that are inherently associated with its original character, this study provides a prototypical approach to the design of interlaced continuum systems and demonstrates the first interlaced continuum probe, which is intrinsically able to follow the leader.

Estimating needle tip deflection in biological tissue from a single transverse ultrasound image: application to brachytherapy

PURPOSE

This paper proposes a method to predict the deflection of a flexible needle inserted into soft tissue based on the observation of deflection at a single point along the needle shaft.

METHODS

We model the needle-tissue as a discretized structure composed of several virtual, weightless, rigid links connected by virtual helical springs whose stiffness coefficient is found using a pattern search algorithm that only requires the force applied at the needle tip during insertion and the needle deflection measured at an arbitrary insertion depth. Needle tip deflections can then be predicted for different insertion depths.

RESULTS

Verification of the proposed method in synthetic and biological tissue shows a deflection estimation error of [Formula: see text]2 mm for images acquired at 35 % or more of the maximum insertion depth, and decreases to 1 mm for images acquired closer to the final insertion depth. We also demonstrate the utility of the model for prostate brachytherapy, where in vivo needle deflection measurements obtained during early stages of insertion are used to predict the needle deflection further along the insertion process.

CONCLUSION

The method can predict needle deflection based on the observation of deflection at a single point. The ultrasound probe can be maintained at the same position during insertion of the needle, which avoids complications of tissue deformation caused by the motion of the ultrasound probe.

Three-Dimensional Needle Shape Estimation in TRUS-Guided Prostate Brachytherapy Using 2-D Ultrasound Images

In this paper, we propose an automated method to reconstruct the three-dimensional (3-D) needle shape during needle insertion procedures using only 2-D transverse ultrasound (US) images. Using a set of transverse US images, image processing and random sample consensus are used to locate the needle within each image and estimate the needle shape. The method is validated with an in vitro needle insertion setup and a transparent tissue phantom, where two orthogonal cameras are used to capture the true 3-D needle shape for verification. Results showed that the use of at least three images obtained at 75% of the maximum insertion depth or greater allows for maximum needle shape estimation errors of less than 2 mm. In addition, the needle shape can be calculated consistently as long as the needle can be identified in 30% of the transverse US images obtained. Application to permanent prostate brachytherapy is also presented, where the estimated needle shape is compared to manual segmentation and sagittal US images. Our method is intended to help to assess needle placement during manual or robot-assisted needle insertion procedures after the needle has been inserted.