Real-Time Estimation of 3-D Needle Shape and Deflection for MRI-Guided Interventions

Simulation-based Flexible Needle Control with Single-core FBG Feedback for Spinal Injections

Objective

We present a general framework of simultaneous needle shape reconstruction and control input generation for robot-assisted spinal injection procedures, without continuous imaging feedback.

Methods

System input-output mapping is generated with a real-time needle-tissue interaction simulation, and single-core FBG sensor readings are used as local needle shape feedback within the same simulation framework. FBG wavelength shifts due to temperature variation is removed by exploiting redundancy in fiber arrangement.

Results

Targeting experiments performed on both plastisol lumbar phantoms as well as an ex vivo porcine lumbar section achieved in-plane tip errors of 0.6 ± 0.3 mm and 1.6 ± 0.9 mm , and total tip errors of 0.9 ± 0.7 mm and 2.1 ± 0.8 mm for the two testing environments.

Significance

Our clinically inspired control strategy and workflow is self-contained and not dependent on the modality of imaging guidance. The generalizability of the proposed approach can be applied to other needle-based interventions where medical imaging cannot be reliably utilized as part of a closed-loop control system for needle guidance.

Unified Shape and External Load State Estimation for Continuum Robots

Continuum robots navigate narrow, winding passageways while safely and compliantly interacting with their environments. Sensing the robot's shape under these conditions is often done indirectly, using a few coarsely distributed (e.g. strain or position) sensors combined with the robot's mechanics-based model. More recently, given high-fidelity shape data, external interaction loads along the robot have been estimated by solving an inverse problem on the mechanics model of the robot. In this paper, we argue that since shape and force are fundamentally coupled, they should be estimated simultaneously in a statistically principled approach. We accomplish this by applying continuous-time batch estimation directly to the arclength domain. A general continuum robot model serves as a statistical prior which is fused with discrete, noisy measurements taken along the robot's backbone. The result is a continuous posterior containing both shape and load functions of arclength, as well as their uncertainties. We first test the approach with a Cosserat rod, i.e. the underlying modeling framework that is the basis for a variety of continuum robots. We verify our approach numerically using distributed loads with various sensor combinations. Next, we experimentally validate shape and external load errors for highly concentrated force distributions (point loads). Finally, we apply the approach to a tendon-actuated continuum robot demonstrating applicability to more complex actuated robots.

Location deviation correction method based on cross-correlation spectrum in OFDR

A location deviation compensation algorithm based on cross-correlation spectrum is proposed for optical frequency domain reflection (OFDR) measurement. The strain and the imperfect repeatability of the laser source may cause a location deviation of the OFDR measurement system, resulting in a demodulation error. This paper proposes to use the peak-to-average ratio (PAR) to measure the quality of the correlation results and achieve compensation, which achieves efficient compensation for location deviation. Compared to other compensation methods, this algorithm can deal with the complex location deviation with better demodulation accuracy and stability. Through a strain measurement experiment, we demonstrate that the demodulation error caused by location deviation is largely eliminated. As a result, the spatial resolution of the system is improved from 8 cm to 2 cm, and the maximum measurement range is expanded to 5000µε.

Optical Fiber-Based Needle Shape Sensing in Real Tissue: Single Core vs. Multicore Approaches

Flexible needle insertion procedures are common for minimally-invasive surgeries for diagnosing and treating prostate cancer. Bevel-tip needles provide physicians the capability to steer the needle during long insertions to avoid vital anatomical structures in the patient and reduce post-operative patient discomfort. To provide needle placement feedback to the physician, sensors are embedded into needles for determining the real-time 3D shape of the needle during operation without needing to visualize the needle intra-operatively. Through expansive research in fiber optics, a plethora of bio-compatible, MRI-compatible, optical shape-sensors have been developed to provide real-time shape feedback, such as single-core and multicore fiber Bragg gratings. In this paper, we directly compare single-core fiber-based and multicore fiber-based needle shape-sensing through identically constructed, four-active area sensorized bevel-tip needles inserted into phantom and ex-vivo tissue on the same experimental platform. In this work, we found that for shape-sensing in phantom tissue, the two needles performed identically with a p -value of 0.164 > 0.05, but in ex-vivo real tissue, the single-core fiber sensorized needle significantly outperformed the multicore fiber configuration with a p -value of 0.0005 < 0.05. This paper also presents the experimental platform and method for directly comparing these optical shape sensors for the needle shape-sensing task, as well as provides direction, insight and required considerations for future work in constructively optimizing sensorized needles.

Design and Fabrication of a Fiber Bragg Grating Shape Sensor for Shape Reconstruction of a Continuum Manipulator

Continuum dexterous manipulators (CDMs) are suitable for performing tasks in a constrained environment due to their high dexterity and maneuverability. Despite the inherent advantages of CDMs in minimally invasive surgery, real-time control of CDMs' shape during nonconstant curvature bending is still challenging. This study presents a novel approach for the design and fabrication of a large deflection fiber Bragg grating (FBG) shape sensor embedded within the lumens inside the walls of a CDM with a large instrument channel. The shape sensor consisted of two fibers, each with three FBG nodes. A shape-sensing model was introduced to reconstruct the centerline of the CDM based on FBG wavelengths. Different experiments, including shape sensor tests and CDM shape reconstruction tests, were conducted to assess the overall accuracy of the shape-sensing. The FBG sensor evaluation results revealed the linear curvature-wavelength relationship with the large curvature detection of 0.045 mm and a high wavelength shift of up to 5.50 nm at a 90° bending angle in both the bending directions. The CDM's shape reconstruction experiments in a free environment demonstrated the shape-tracking accuracy of 0.216 ± 0.126 mm for positive/negative deflections. Also, the CDM shape reconstruction error for three cases of bending with obstacles was observed to be 0.436 ± 0.370 mm for the proximal case, 0.485 ± 0.418 mm for the middle case, and 0.312 ± 0.261 mm for the distal case. This study indicates the adequate performance of the FBG sensor and the effectiveness of the model for tracking the shape of the large-deflection CDM with nonconstant-curvature bending for minimally invasive orthopedic applications.

Lie-Group Theoretic Approach to Shape-Sensing Using FBG-Sensorized Needles Including Double-Layer Tissue and S-Shape Insertions

Flexible bevel-tipped needles are often used for needle insertion in minimally-invasive surgical techniques due to their ability to be steered in cluttered environments. Shapesensing enables physicians to determine the location of needles intra-operatively without requiring radiation of the patient, enabling accurate needle placement. In this paper, we validate a theoretical method for flexible needle shape-sensing that allows for complex curvatures, extending upon a previous sensor-based model. This model combines curvature measurements from fiber Bragg grating (FBG) sensors and the mechanics of an inextensible elastic rod to determine and predict the 3D needle shape during insertion. We evaluate the model's shape sensing capabilities in C- and S-shape insertions in single-layer isotropic tissue, and C-shape insertions in two-layer isotropic tissue. Experiments on a four-active area, FBG-sensorized needle were performed in varying tissue stiffnesses and insertion scenarios under stereo vision to provide the 3D ground truth needle shape. The results validate a viable 3D needle shape-sensing model accounting for complex curvatures in flexible needles with mean needle shape sensing root-mean-square errors of 0.160 ± 0.055 mm over 650 needle insertions.

Soft joint shape measurement device based on FBG with a simple demodulating system

Soft joint shape measurement is challenging because, in most cases, it relies solely on internal sensors. Existing shape estimation methods commonly take measurements at discrete points and utilize curve-fitting schemes, which are inefficient for complex joint shapes that require continuous measurements. Therefore joint shape measurement sensors rely on the fiber Bragg grating (FBG) due to its sensitivity, immunity to electromagnetic interference, and flexibility. Nevertheless, FBG demodulation is still an open research case. Hence, we propose a shape measurement device appropriate for FBG-based continuous measurements that employs a sensor with only three FBGs thrusting inside the soft joint to measure its 3D shape. Moreover, we develop a simple demodulating system exploiting the FBG's filter overlapping properties and design a calibrating process for FBG signals. Soft joint shape measurement experiments highlight our method's effectiveness, providing a relative error within 0.7%. Further tests involving continuum robot measurement reveal that the achieved precision is of the same level as a motion-capturing system.

State of the Art and Future Opportunities in MRI-Guided Robot-Assisted Surgery and Interventions

Magnetic resonance imaging (MRI) can provide high-quality 3-D visualization of target anatomy, surrounding tissue, and instrumentation, but there are significant challenges in harnessing it for effectively guiding interventional procedures. Challenges include the strong static magnetic field, rapidly switching magnetic field gradients, high-power radio frequency pulses, sensitivity to electrical noise, and constrained space to operate within the bore of the scanner. MRI has a number of advantages over other medical imaging modalities, including no ionizing radiation, excellent soft-tissue contrast that allows for visualization of tumors and other features that are not readily visible by other modalities, true 3-D imaging capabilities, including the ability to image arbitrary scan plane geometry or perform volumetric imaging, and capability for multimodality sensing, including diffusion, dynamic contrast, blood flow, blood oxygenation, temperature, and tracking of biomarkers. The use of robotic assistants within the MRI bore, alongside the patient during imaging, enables intraoperative MR imaging (iMRI) to guide a surgical intervention in a closed-loop fashion that can include tracking of tissue deformation and target motion, localization of instrumentation, and monitoring of therapy delivery. With the ever-expanding clinical use of MRI, MRI-compatible robotic systems have been heralded as a new approach to assist interventional procedures to allow physicians to treat patients more accurately and effectively. Deploying robotic systems inside the bore synergizes the visual capability of MRI and the manipulation capability of robotic assistance, resulting in a closed-loop surgery architecture. This article details the challenges and history of robotic systems intended to operate in an MRI environment and outlines promising clinical applications and associated state-of-the-art MRI-compatible robotic systems and technology for making this possible.

Fabrication and Characterization of a Flexible FBG-based Shape Sensor using Single-mode Fibers

Minimally invasive surgical procedures have become the preferable option, as the recovery period and the risk of infections are significantly lower than traditional surgeries. However, the main challenge in using flexible tools for minimal surgical interventions is the lack of precise feedback on their shape and tip position inside the patient's body. Shape sensors based on fiber Bragg gratings (FBGs) can provide accurate shape information depending on their design. One of the most common configurations in FBG-based shape sensors is to attach three single-mode optical fibers with arrays of FBGs in a triangular fashion around a substrate. Usually, the selected substrates dominate the bending stiffness of the sensor probe, as they have a larger diameter and show less flexibility compared to the optical fibers. Although sensors with this configuration can accurately estimate the shape, they cannot be implemented in flexible endoscopes where large deflections are expected. This paper investigates the shape sensor's performance when using a superelastic substrate with a small diameter instead of a substrate with dominating bending stiffness. A generalized model is also designed for characterizing this type of flexible FBG-based shape sensor. Moreover, we evaluated the sensor in single and multi-bend deformations using two shape reconstruction methods.

Shape Reconstruction Processes for Interventional Application Devices: State of the Art, Progress, and Future Directions

Soft and continuum robots are transforming medical interventions thanks to their flexibility, miniaturization, and multidirectional movement abilities. Although flexibility enables reaching targets in unstructured and dynamic environments, it also creates challenges for control, especially due to interactions with the anatomy. Thus, in recent years lots of efforts have been devoted for the development of shape reconstruction methods, with the advancement of different kinematic models, sensors, and imaging techniques. These methods can increase the performance of the control action as well as provide the tip position of robotic manipulators relative to the anatomy. Each method, however, has its advantages and disadvantages and can be worthwhile in different situations. For example, electromagnetic (EM) and Fiber Bragg Grating (FBG) sensor-based shape reconstruction methods can be used in small-scale robots due to their advantages thanks to miniaturization, fast response, and high sensitivity. Yet, the problem of electromagnetic interference in the case of EM sensors, and poor response to high strains in the case of FBG sensors need to be considered. To help the reader make a suitable choice, this paper presents a review of recent progress on shape reconstruction methods, based on a systematic literature search, excluding pure kinematic models. Methods are classified into two categories. First, sensor-based techniques are presented that discuss the use of various sensors such as FBG, EM, and passive stretchable sensors for reconstructing the shape of the robots. Second, imaging-based methods are discussed that utilize images from different imaging systems such as fluoroscopy, endoscopy cameras, and ultrasound for the shape reconstruction process. The applicability, benefits, and limitations of each method are discussed. Finally, the paper draws some future promising directions for the enhancement of the shape reconstruction methods by discussing open questions and alternative methods.

Towards Automatic Robotic Calibration System for Flexible Needles with FBG Sensors

There has been much research exploring the use of fiber Bragg grating (FBG)-sensorized needles in the prostate biopsy procedure, but all FBG needles used in the research need to be calibrated, which is time consuming and prone to human errors. In this work, a semi-automatic robotic system was developed to perform FBG needle calibration. Compared to manual calibration results, the robotic system is able to calibrate FBG needles with the similar level of accuracy as achieved by an experienced manual operator, thus reducing the time cost during the needle calibration process.

Real-Time Shape Estimation for Concentric Tube Continuum Robots with a Single Force/Torque Sensor

Shape-sensing in real-time is a key requirement for the development of advanced algorithms for concentric tube continuum robots when safe interaction with the environment is important e.g., for path planning, advanced control, and human-machine interaction. We propose a real-time shape-estimation algorithm for concentric tube continuum robots based on the force-torque information measured at the tubes' basis. It extends a shape estimation algorithm for elastic rods based on discrete Kirchhoff rod theory. For simplicity and efficiency of calculation, we combine it with a model under piece-wise constant curvature assumption, in which we model a concentric tube continuum robot as a combination of segments of planar constant curvatures lying on different equilibrium planes. We evaluate our approach for a single and two combined additively manufactured tubes and achieve an estimation frequency of 333 Hz for two combined tubes with a mean deviation along the backbone of the tubes of 1.91-5.22 mm.

Recent Advances in Biomedical Photonic Sensors: A Focus on Optical-Fibre-Based Sensing

In this invited review, we provide an overview of the recent advances in biomedical photonic sensors within the last five years. This review is focused on works using optical-fibre technology, employing diverse optical fibres, sensing techniques, and configurations applied in several medical fields. We identified technical innovations and advancements with increased implementations of optical-fibre sensors, multiparameter sensors, and control systems in real applications. Examples of outstanding optical-fibre sensor performances for physical and biochemical parameters are covered, including diverse sensing strategies and fibre-optical probes for integration into medical instruments such as catheters, needles, or endoscopes.

Development of a variable-stiffness and shape-detection manipulator based on low-melting-point-alloy for minimally invasive surgery

There is an increasingly popularity for the continuum robot in minimally invasive surgery(MIS), because of the compliance and dexterity. In the first place, a variable stiffness manipulator can resolve the two contradictions of the demands for predominant flexibility and strong payload capacity. In the second place, to control the continuum robot more precisely and avoid the collision between robot and human body, real-time tracking of the shape of the continuum robot is of great significance. A new type of flexible manipulator with variable stiffness is proposed which can track the bending shape timely. The low-melting-point-alloy (LMPA) is used to realize the variable stiffness and shape detection for the flexible manipulator. The concept design for a single module is put forward. Then the stiffness control method and finite element simulation, the method of shape detection are presented. Moreover, the presented method of shape detection is evaluated by experiments.

Modeling of active shimming of metallic needles for interventional MRI

PURPOSE

Artifacts caused by large magnetic susceptibility differences between metallic needles and tissue are a persistent problem in many interventional MRI applications. The signal void caused by the needle can hide procedure targets and prevent accurate image-based monitoring. In this paper, a solution to this problem is presented in the form of an active shim insert inspired from degaussing coils used in naval vessels, that is designed to correct the field disturbance (ΔB₀ ) caused by the needle.

METHODS

The ΔB₀ induced by a 10 gauge hollow single-beveled titanium needle at 3T is modeled in different orientations. A set of 63 orthogonal coil pairs with unique tip paths are evaluated for shimming performance, and an optimal coil pair is chosen. Shimming performance and current demands are evaluated over a range of needle orientations.

RESULTS

Robust correction of the titanium needle induced ΔB₀ is predicted using a flat no-loop coil combined with an orthogonal 1½ turn loop coil angled at the bevel angle for most orientations, with currents well below 1 amp per coil. Reductions in ΔB₀ standard deviations with shimming ranged from ~49% to ~10% depending on needle orientation, with performance worsening as the needle is aligned more along B₀ .

CONCLUSION

Simulations predict that it is possible to minimize metallic probe induced ΔB₀ and signal losses using externally supplied direct current shim coil inserts in arbitrary orientations for potential benefits in many interventional MRI applications.

Soft Miniaturized Actuation and Sensing Units for Dynamic Force Control of Cardiac Ablation Catheters

Recently, there has been active research in finding robotized solutions for the treatment of atrial fibrillation (AF) by augmenting catheter systems through the integration of force sensors at the tip. However, limited research has been aimed at providing automatic force control by also integrating actuation of the catheter tip, which can significantly enhance safety in such procedures. This article solves the demanding challenge of miniaturizing both actuation and sensing for integration into flexible catheters. Fabrication strategies are presented for a series of novel soft thick-walled cylindrical actuators, with embedded sensing using eutectic gallium-indium. The functional catheter tips have a diameter in the range of 2.6-3.6 mm and can both generate and detect forces in the range of < 0.4 N, with a bandwidth of 1-2 Hz. The deformation modeling of thick-walled cylinders with fiber reinforcement is presented in the article. An experimental setup developed for static and dynamic characterization of these units is presented. The prototyped units were validated with respect to the design specifications. The preliminary force control results indicate that these units can be used in tracking and control of contact force, which has the potential to make AF procedures much safer and more accurate.

Separation method of bending and torsion in shape sensing based on FBG sensors array

This paper presents a theoretical method for separating bending and torsion of shape sensing sensor to improve sensing accuracy during its deformation. We design a kind of shape sensing sensor by encapsulating three fibers on the surface of a flexible rod and forming a triangular FBG sensors array. According to the configuration of FBG sensors array, we derive the relationship between bending curvature and bending strain, and set up a function about the packaging angle of FBG sensor and strain induced by torsion under different twist angles. Combined with the influence of bending and torsion on strain, we establish a nonlinear matrix equation resolving three unknown parameters including maximum strain, bending direction and wavelength shift induced by torsion and temperature. The three parameters are sufficient to separate bending and torsion, and acquire two scalar functions including curvature and torsion, which could describe 3D shape of rod according to Frenet-Serret formulas. Experimental results show that the relative average error of measurement about maximum strain, bending direction is respectively 2.65% and 0.86% when shape-sensing sensor is bent into an arc with a radius of 260 mm. The separating method also applied to 2D shape and 3D shape of reconstruction, and the absolute spatial position maximum error is respectively 3.79mm and 11.10mm when shape-sensing sensor with length 500mm is bent into arc shape with a radius 260mm and helical curve. The experiment results verify the feasibility of separating method, which would provide effective parameters for precise 3D reconstruction model of shape sensing sensor.

Sensitivity and phase response of FBG based acousto-optic sensors for real-time MRI applications

Fiber Bragg grating (FBG) based sensors have recently been introduced to the field of magnetic resonance imaging (MRI). Real-time MRI applications demand highly amplitude and phase sensitive MRI compatible sensors. Thus, a model and detailed analysis of FBG based ultrasound detection are required for designing better performing sensors. A hybrid FBG model incorporating numerical and FEA methods was developed and used for sensitivity and linearity analysis. The transfer matrix method was used for the modeling of optical modulation whereas FEA was used for pressure field calculations within the grating. The model was verified through reflection spectrum and acoustic pressure sensitivity testing of two π-phase shifted FBGs in a side slope read-out configuration. The sensitivity curves with respect to the operation point on the side slope was characterized in terms of amplitude and phase, and nonlinearity of the phase response has been quantified. Lastly, the impact of phase linearity of the FBG based acousto-optic sensor was tested under MRI when the sensor was used as a position marker and an analog phase shifter based solution was demonstrated.

Directional bending sensor based on triangular shaped fiber Bragg gratings

We propose a directional bending sensor by mechanically bonded FBGs in a radially placed triangular shape to measure the bending moment and its applied direction. Bending measurement and the determination of its direction is essential to get the maximum bending stress and to figure out the deflection shape by the curvature information in a wide range of engineering applications. The proposed sensor configuration is original in terms of a pointwise sensing scheme based on the mechanically bonded scheme of three optical fibers and its simple fabrication process only using optical fibers. This set-up made of three FBG sensors provides a simple relationship of the bending moment to the independent strain measurement of each FBG. We demonstrate two sensor configurations such as the self-bending measurement and the structural bending measurement in a surface-mount (or embedded) measurement. Through the verification experiments, the proposed bending sensor is successfully shown to measure the applied bending moment and its direction.

Fiber Optic Shape Sensing for Soft Robotics

While soft material actuators can undergo large deformations to execute very complex motions, what is critically lacking in soft material robotic systems is the ability to collect high-resolution shape information for sophisticated functions such as environmental mapping, collision detection, and full state feedback control. This work explores the potential of a nearly commercial fiber optic shape sensor (FOSS) and presents the first demonstrations of a monolithic, multicore FOSS integrated into the structure of a fiber-reinforced soft actuator. In this pilot study, we report an open loop sensorized soft actuator capable of submillimeter position feedback that can detect the soft actuator's shape, environmental shapes, collision locations, and material stiffness properties.

Fiber optical shape sensing of flexible instruments for endovascular navigation

Purpose

Endovascular aortic repair procedures are currently conducted with 2D fluoroscopy imaging. Tracking systems based on fiber Bragg gratings are an emerging technology for the navigation of minimally invasive instruments which can reduce the X-ray exposure and the used contrast agent. Shape sensing of flexible structures is challenging and includes many calculations steps which are prone to different errors. To reduce this errors, we present an optimized shape sensing model.

Methods

We analyzed for every step of the shape sensing process, which errors can occur, how the error affects the shape and how it can be compensated or minimized. Experiments were done with one multicore fiber system with 38 cm sensing length, and the effects of different methods and parameters were analyzed. Furthermore, we compared 3D shape reconstructions with the segmented shape of the corresponding CT scans of the fiber to evaluate the accuracy of our optimized shape sensing model. Finally, we tested our model in a realistic endovascular scenario by using a 3D printed vessel system created from patient data.

Results

Depending on the complexity of the shape, we reached an average error of 0.35–1.15 mm and maximal error of 0.75–7.53 mm over the whole 38 cm sensing length. In the endovascular scenario, we obtained an average and maximal error of 1.13 mm and 2.11 mm, respectively.

Conclusion

The accuracies of the 3D shape sensing model are promising, and we plan to combine the shape sensing based on fiber Bragg gratings with the position and orientation of an electromagnetic tracking to obtain the located catheter shape.

Electronic supplementary material

The online version of this article (10.1007/s11548-019-02059-0) contains supplementary material, which is available to authorized users.

Modular FBG Bending Sensor for Continuum Neurosurgical Robot

We present a modular sensing system to measure the deflection of a minimally invasive neurosurgical intracranial robot: MINIR-II. The MINIR-II robot is a tendon-driven continuum robot comprised of multiple spring backbone segments, which has been developed in our prior work. Due to the flexibility of the spring backbone and unique tendon routing configuration, each segment of MINIR-II can bend up to a large curvature (≥100 m^-1) in multiple directions. However, the shape measurement of the robot based on tendon displacement is not precise due to friction and unknown external load/disturbance. In this regard, we propose a bending sensor module comprised of a fiber Bragg grating (FBG) fiber, a Polydimethylsiloxane (PDMS) cylinder, and a superelastic spring. The grating segment of the FBG fiber is enclosed inside a PDMS cylinder (1 mm in diameter), and the PDMS cylinder is bonded with the superelastic spring in series. The deflection or bending of the robot backbone segment is translated into an axial loading in the superelastic spring, which applies tension to the FBG; therefore, by measuring the peak wavelength shift of the FBG, the bending angle can be estimated. This paper describes the design, fabrication, and kinematic aspects of the sensor module in detail. To evaluate the proposed concept, one such sensor module has been tested and evaluated on the MINIR-II robot.

Optimized needle shape reconstruction using experimentally based strain sensors positioning

Needles are tools that are used daily during minimally invasive procedures. During the insertions, needles may be affected by deformations which may threaten the success of the procedure. To tackle this problem, needles with embedded strain sensors have been developed and associated with navigation systems. The localization of the needle in the tissues is then obtained in real time by reconstruction from the strain measurements, allowing the physician to optimize its gesture. As the number of strain sensors embedded is limited in number, their positions on the needle have a great impact on the accuracy of the shape reconstruction. The main contribution of this paper is a novel strain sensor positioning method to improve the reconstruction accuracy. A notable feature of our method is the use of experimental needle insertion data, which increases the relevancy of the resulting sensor optimal locations. To the best of the author's knowledge, no experimentally based needle sensor positioning method has been presented yet. Reconstruction validations from clinical data show that the localization accuracy of the needle tip is improved by almost 40% with optimal locations compared with equidistant locations when reconstructing with two sensor triplets or more. Graphical Abstract Improvement of the reconstruction accuracy of a deformed needle shape by using experimental data to position strain sensors.

Optical fiber technology enables smart needles for epidurals: an in-vivo swine study

Nowadays, epidural space identification is made by using subjective and manual techniques characterized by failure rates up to 7%. In this work, we propose a fiber optic sensor technology based needle guidance system, that is directly inspired by the most common technique currently used for epidurals; through real-time strain measurements, the fiber Bragg grating integrated inside the needle lumen is able to effectively perceive the typical force drop occurring when the needle enters the epidural space. An in vivo swine study demonstrates the validity of our approach, paving the way for the development of lab-in-a-needle systems.

Real-Time Mechanical-Encoding of Needle Shape for Image-Guided Medical and Surgical Interventions

Error and uncertainty in needle placement can drastically impact the clinical outcome of both diagnostic and therapeutic needle-based procedures. In this work, we aim to estimate the shape of a bent needle during insertion and provide a prototype design of a needle whose deflection is tracked in real time. We calculate slope along a needle by measuring the movement of fixed wires running along its length with a compact image-based sensor. Through the use of the Euler–Bernoulli beam theory, we calculate shape and trajectory of a needle. We constructed a prototype needle with two wires fixed along its length and measured wire-movement using a vertical-cavity surface-emitting laser (VCSEL) mouse sensor. This method was able to estimate needle tip deflection within 1 mm in a variety of deflection scenarios in real time. We then provide a design of a needle with real-time deflection tracking in 3D, providing the user with a simple display to convey needle deflection in tissue. This method could be applied to needle-based biopsy or therapy procedures to improve the diagnostic accuracy or treatment delivery quality.

Real-Time Curvature Detection of a Flexible Needle with a Bevel Tip

As one of the major methods for the diagnosis and treatment of cancers in their early stages, the percutaneous puncture technique has bright prospect in biopsy, ablation, proximity radiotherapy, and drug delivery. Recent years, researchers found the flexible needle cannot realize feedback control during the puncture surgeries only by path planning. To solve this problem, the flexible needle is tried to achieve real-time detection in this paper. Compared with previous methods, the strain gauges glued on the needle surface rather than the medical imaging techniques is used to collect the information to reconstruct the needle curve, which is benefit to integrate the whole system and obtain a more simple and accurate closed-loop control. This paper presented the math model of curve fitting and analyzed the causes of curve fitting errors. To verify the feasibility of this method, an experiment setup was built. Results from the experiments validated the solution in this paper to be effective.

Miniaturized Robotic End-Effector with Piezoelectric Actuation and Fiber Optic Sensing for Minimally Invasive Cardiac Procedures

Each year 35,000 cardiac ablation procedures are performed to treat atrial fibrillation through the use of catheter systems. The success rate of this treatment is highly dependent on the force which the catheter applies on the heart wall. If the magnitude of the applied force is much higher than a certain threshold the tissue perforates, whereas if the force is lower than this threshold the lesion size may be too large and is inconsistent. Furthermore, studies have shown large variability in the applied force from trained physicians during treatment, suggesting that physicians are unable to manually regulate the levels of the force at the site of treatment. Current catheter systems do not provide the physicians with active means for contact force control and are only at most aided by visual feedback of the forces measured in situ. This paper discusses a novel design of a robotic end-effector that integrates mechanisms of sensing and actively controlling of the applied forces into a miniaturized compact form. The required specifications for design and integration were derived from the current application under investigation. An off-the-shelf miniature piezoelectric motor was chosen for actuation, and a force sensing solution was developed to meet the specifications. Experimental characterization of the actuator and the force sensor within the integrated setup show compliance with the specifications and pave the way for future experimentation where closed-loop control of the system can be implemented according to the contact force control strategies for the application.

Kalman filter-based EM-optical sensor fusion for needle deflection estimation

PURPOSE

In many clinical procedures such as cryoablation that involves needle insertion, accurate placement of the needle's tip at the desired target is the major issue for optimizing the treatment and minimizing damage to the neighboring anatomy. However, due to the interaction force between the needle and tissue, considerable error in intraoperative tracking of the needle tip can be observed as needle deflects.

METHODS

In this paper, measurements data from an optical sensor at the needle base and a magnetic resonance (MR) gradient field-driven electromagnetic (EM) sensor placed 10 cm from the needle tip are used within a model-integrated Kalman filter-based sensor fusion scheme. Bending model-based estimations and EM-based direct estimation are used as the measurement vectors in the Kalman filter, thus establishing an online estimation approach.

RESULTS

Static tip bending experiments show that the fusion method can reduce the mean error of the tip position estimation from 29.23 mm of the optical sensor-based approach to 3.15 mm of the fusion-based approach and from 39.96 to 6.90 mm, at the MRI isocenter and the MRI entrance, respectively.

CONCLUSION

This work established a novel sensor fusion scheme that incorporates model information, which enables real-time tracking of needle deflection with MRI compatibility, in a free-hand operating setup.

Evaluation of fiber Bragg grating sensor interrogation using InGaAs linear detector arrays and Gaussian approximation on embedded hardware

Fiber Bragg Grating (FBG) sensors have become popular for applications related to structural health monitoring, biomedical engineering, and robotics. However, for successful large scale adoption, FBG interrogation systems are as important as sensor characteristics. Apart from accuracy, the required number of FBG sensors per fiber and the distance between the device in which the sensors are used and the interrogation system also influence the selection of the interrogation technique. For several measurement devices developed for applications in biomedical engineering and robotics, only a few sensors per fiber are required and the device is close to the interrogation system. For these applications, interrogation systems based on InGaAs linear detector arrays provide a good choice. However, their resolution is dependent on the algorithms used for curve fitting. In this work, a detailed analysis of the choice of algorithm using the Gaussian approximation for the FBG spectrum and the number of pixels used for curve fitting on the errors is provided. The points where the maximum errors occur have been identified. All comparisons for wavelength shift detection have been made against another interrogation system based on the tunable swept laser. It has been shown that maximum errors occur when the wavelength shift is such that one new pixel is included for curve fitting. It has also been shown that an algorithm with lower computation cost compared to the more popular methods using iterative non-linear least squares estimation can be used without leading to the loss of accuracy. The algorithm has been implemented on embedded hardware, and a speed-up of approximately six times has been observed.

Preliminary study on magnetic tracking-based planar shape sensing and navigation for flexible surgical robots in transoral surgery: methods and phantom experiments

PURPOSE

Flexible surgical robot can work in confined and complex environments, which makes it a good option for minimally invasive surgery. In order to utilize flexible manipulators in complicated and constrained surgical environments, it is of great significance to monitor the position and shape of the curvilinear manipulator in real time during the procedures. In this paper, we propose a magnetic tracking-based planar shape sensing and navigation system for flexible surgical robots in the transoral surgery. The system can provide the real-time tip position and shape information of the robot during the operation.

METHODS

We use wire-driven flexible robot to serve as the manipulator. It has three degrees of freedom. A permanent magnet is mounted at the distal end of the robot. Its magnetic field can be sensed with a magnetic sensor array. Therefore, position and orientation of the tip can be estimated utilizing a tracking method. A shape sensing algorithm is then carried out to estimate the real-time shape based on the tip pose. With the tip pose and shape display in the 3D reconstructed CT model, navigation can be achieved.

RESULTS

Using the proposed system, we carried out planar navigation experiments on a skull phantom to touch three different target positions under the navigation of the skull display interface. During the experiments, the real-time shape has been well monitored and distance errors between the robot tip and the targets in the skull have been recorded. The mean navigation error is [Formula: see text] mm, while the maximum error is 3.2 mm.

CONCLUSION

The proposed method provides the advantages that no sensors are needed to mount on the robot and no line-of-sight problem. Experimental results verified the feasibility of the proposed method.

Assessment of the Accuracy of Optical Shape Sensing for Needle Tracking Interventions

Accurate needle guidance is essential for a number of magnetic resonance imaging (MRI)-guided percutaneous procedures, such as radiofrequency ablation (RFA) of metastatic liver tumors. A promising technology to obtain real-time tracking of the shape and tip of a needle is by using high-frequency (up to 20 kHz) fiber Bragg grating (FBG) sensors embedded in optical fibers, which are insensitive to external magnetic fields. We fabricated an MRI-compatible needle designed for percutaneous procedures with a series of FBG sensors which would be tracked in an image-guidance system, allowing to display the needle shape within a navigation image. A series of phantom experiments demonstrated needle tip tracking errors of 1.05 ± 0.08 mm for a needle deflection up to 16.82 mm on a ground-truth model and showed nearly similar accuracy to electromagnetic (EM) tracking (i.e., 0.89 ± 0.09 mm). We demonstrated feasibility of the FBG-based tracking system for MRI-guided interventions with differences under 1 mm between tracking systems. This study establishes the needle tracking accuracy of FBG needle tracking for image-guided procedures.

Enhancement of accuracy in shape sensing of surgical needles using optical frequency domain reflectometry in optical fibers

We demonstrate a novel approach to enhance the precision of surgical needle shape tracking based on distributed strain sensing using optical frequency domain reflectometry (OFDR). The precision enhancement is provided by using optical fibers with high scattering properties. Shape tracking of surgical tools using strain sensing properties of optical fibers has seen increased attention in recent years. Most of the investigations made in this field use fiber Bragg gratings (FBG), which can be used as discrete or quasi-distributed strain sensors. By using a truly distributed sensing approach (OFDR), preliminary results show that the attainable accuracy is comparable to accuracies reported in the literature using FBG sensors for tracking applications (~1mm). We propose a technique that enhanced our accuracy by 47% using UV exposed fibers, which have higher light scattering compared to un-exposed standard single mode fibers. Improving the experimental setup will enhance the accuracy provided by shape tracking using OFDR and will contribute significantly to clinical applications.

Fiber Optic Force Sensors for MRI-Guided Interventions and Rehabilitation: A Review

Magnetic Resonance Imaging (MRI) provides both anatomical imaging with excellent soft tissue contrast and functional MRI imaging (fMRI) of physiological parameters. The last two decades have witnessed the manifestation of increased interest in MRI-guided minimally invasive intervention procedures and fMRI for rehabilitation and neuroscience research. Accompanying the aspiration to utilize MRI to provide imaging feedback during interventions and brain activity for neuroscience study, there is an accumulated effort to utilize force sensors compatible with the MRI environment to meet the growing demand of these procedures, with the goal of enhanced interventional safety and accuracy, improved efficacy and rehabilitation outcome. This paper summarizes the fundamental principles, the state of the art development and challenges of fiber optic force sensors for MRI-guided interventions and rehabilitation. It provides an overview of MRI-compatible fiber optic force sensors based on different sensing principles, including light intensity modulation, wavelength modulation, and phase modulation. Extensive design prototypes are reviewed to illustrate the detailed implementation of these principles. Advantages and disadvantages of the sensor designs are compared and analyzed. A perspective on the future development of fiber optic sensors is also presented which may have additional broad clinical applications. Future surgical interventions or rehabilitation will rely on intelligent force sensors to provide situational awareness to augment or complement human perception in these procedures.

Advanced sensors placement for accurate 3D needle shape reconstruction

Needles are tools widely used in minimally invasive surgery. During such procedures the localization of the needle and its tip is a challenging situation because of the needle deformations due to its interactions with tissues. To tackle this problem, instrumented needles with sensors have been currently developed to allow needle reconstruction and tip localization. In conventional surgery this difficulty is overcome by medical imaging. The interest in using an instrumented needle resides in the possible dispense of medical imaging. This papers develops new methods to reconstruct needles in three dimensions and to find the locations of sensors which minimizes the error of reconstruction of the needle. A notable feature of our method is that input data are based on real needle data, that should assure a better representation of reality. Reconstructions simulated with 22 gauge 200 mm long needles show that the localization of the needle tip is more accurate by 18% to 52% with optimal sensors positions compared to equidistant sensors positions.

Detecting stages of needle penetration into tissues through force estimation at needle tip using fiber Bragg grating sensors

Several medical procedures involve the use of needles. The advent of robotic and robot assisted procedures requires dynamic estimation of the needle tip location during insertion for use in both assistive systems as well as for automatic control. Most prior studies have focused on the maneuvering of solid flexible needles using external force measurements at the base of the needle holder. However, hollow needles are used in several procedures and measurements of forces in proximity of such needles can eliminate the need for estimating frictional forces that have high variations. These measurements are also significant for endoscopic procedures in which measurement of forces at the needle holder base is difficult. Fiber Bragg grating sensors, due to their small size, inert nature, and multiplexing capability, provide a good option for this purpose. Force measurements have been undertaken during needle insertion into tissue mimicking phantoms made of polydimethylsiloxane as well as chicken tissue using an 18-G needle instrumented with FBG sensors. The results obtained show that it is possible to estimate the different stages of needle penetration including partial rupture, which is significant for procedures in which precise estimation of needle tip position inside the organ or tissue is required.

Vessel-based registration of an optical shape sensing catheter for MR navigation

PURPOSE

Magnetic resonance navigation (MRN), achieved with an upgraded MRI scanner, aims to guide therapeutic nanoparticles from their release in the hepatic vascular network to embolize highly vascularized liver tumors. Visualizing the catheter in real-time within the arterial network is important for selective embolization within the MR gantry. To achieve this, a new MR-compatible catheter tracking technology based on optical shape sensing is used.

METHODS

This paper proposes a vessel-based registration pipeline to co-align this novel catheter tracking technology to the patient's diagnostic MR angiography (MRA) with 3D roadmapping. The method first extracts the 3D hepatic arteries from a diagnostic MRA based on concurrent deformable models, creating a detailed representation of the patient's internal anatomy. Once the optical shape sensing fibers, inserted in a double-lumen catheter, is guided into the hepatic arteries, the 3D centerline of the catheter is inferred and updated in real-time using strain measurements derived from fiber Bragg gratings sensors. Using both centerlines, a diffeomorphic registration based on a spectral representation of the high-level geometrical primitives is applied.

RESULTS

Results show promise in registration accuracy in five phantom models created from stereolithography of patient-specific vascular anatomies, with maximum target registration errors below 2 mm. Furthermore, registration accuracy with the shape sensing tracking technology remains insensitive to the magnetic field of the MR magnet.

CONCLUSIONS

This study demonstrates that an accurate registration procedure of a shape sensing catheter with diagnostic imaging is feasible.

Shape Tracking of a Dexterous Continuum Manipulator Utilizing Two Large Deflection Shape Sensors

Dexterous continuum manipulators (DCMs) can largely increase the reachable region and steerability for minimally and less invasive surgery. Many such procedures require the DCM to be capable of producing large deflections. The real-time control of the DCM shape requires sensors that accurately detect and report large deflections. We propose a novel, large deflection, shape sensor to track the shape of a 35 mm DCM designed for a less invasive treatment of osteolysis. Two shape sensors, each with three fiber Bragg grating sensing nodes is embedded within the DCM, and the sensors' distal ends fixed to the DCM. The DCM centerline is computed using the centerlines of each sensor curve. An experimental platform was built and different groups of experiments were carried out, including free bending and three cases of bending with obstacles. For each experiment, the DCM drive cable was pulled with a precise linear slide stage, the DCM centerline was calculated, and a 2D camera image was captured for verification. The reconstructed shape created with the shape sensors is compared with the ground truth generated by executing a 2D-3D registration between the camera image and 3D DCM model. Results show that the distal tip tracking accuracy is 0.40 ± 0.30 mm for the free bending and 0.61 ± 0.15 mm, 0.93 ± 0.05 mm and 0.23 ± 0.10 mm for three cases of bending with obstacles. The data suggest FBG arrays can accurately characterize the shape of large-deflection DCMs.