Disturbance observer based adaptive model predictive control for uncalibrated visual servoing in constrained environments

This paper presents an adaptive model predictive control (MPC) method based on disturbance observer (DOB) to improve the disturbance rejection performance of the image-based visual servoing (IBVS) system. The proposed control method is developed based on the depth-independent interaction matrix, which can simultaneously handle unknown camera intrinsic and extrinsic parameters, unknown depth parameters, system constraints, as well as external disturbances. The proposed control scheme includes two parts which are the feedback regulation part based on the adaptive MPC and the feedforward compensation part based on the modified DOB. Unlike the traditional DOB that is based on the fixed nominal plant model, the modified DOB here is based on the estimated plant model. The adaptive MPC controller consists of an iterative identification algorithm, which not only can provide the model parameters for both the controller and the modified DOB, but also can be used to control plant dynamics and to minimize the effects of DOB. Simulations for both the eye-in-hand and eye-to-hand camera configurations are conducted to illustrate the effectiveness of the proposed method.

Intelligent magnetic manipulation for gastrointestinal ultrasound

Diagnostic endoscopy in the gastrointestinal tract has remained largely unchanged for decades and is limited to the visualization of the tissue surface, the collection of biopsy samples for diagnoses, and minor interventions such as clipping or tissue removal. In this work, we present the autonomous servoing of a magnetic capsule robot for in-situ, subsurface diagnostics of microanatomy. We investigated and showed the feasibility of closed-loop magnetic control using digitized microultrasound (μUS) feedback; this is crucial for obtaining robust imaging in an unknown and unconstrained environment. We demonstrated the functionality of an autonomous servoing algorithm that uses μUS feedback, both on benchtop trials as well as in-vivo in a porcine model. We have validated this magnetic-μUS servoing in instances of autonomous linear probe motion and were able to locate markers in an agar phantom with 1.0 ± 0.9 mm position accuracy using a fusion of robot localization and μUS image information. This work demonstrates the feasibility of closed-loop robotic μUS imaging in the bowel without the need for either a rigid physical link between the transducer and extracorporeal tools or complex manual manipulation.

Photoacoustic-based visual servoing of a needle tip

In intraoperative settings, the presence of acoustic clutter and reflection artifacts from metallic surgical tools often reduces the effectiveness of ultrasound imaging and complicates the localization of surgical tool tips. We propose an alternative approach for tool tracking and navigation in these challenging acoustic environments by augmenting ultrasound systems with a light source (to perform photoacoustic imaging) and a robot (to autonomously and robustly follow a surgical tool regardless of the tissue medium). The robotically controlled ultrasound probe continuously visualizes the location of the tool tip by segmenting and tracking photoacoustic signals generated from an optical fiber inside the tool. System validation in the presence of fat, muscle, brain, skull, and liver tissue with and without the presence of an additional clutter layer resulted in mean signal tracking errors <2 mm, mean probe centering errors <1 mm, and successful recovery from ultrasound perturbations, representing either patient motion or switching from photoacoustic images to ultrasound images to search for a target of interest. A detailed analysis of channel SNR in controlled experiments with and without significant acoustic clutter revealed that the detection of a needle tip is possible with photoacoustic imaging, particularly in cases where ultrasound imaging traditionally fails. Results show promise for guiding surgeries and procedures in acoustically challenging environments with this novel robotic and photoacoustic system combination.

Robotic ultrasound systems in medicine

Robots ultrasound (RUS) can be defined as the combination of ultrasound imaging with a robotic system in medical interventions. With their potential for high precision, dexterity, and repeatability, robots are often uniquely suited for ultrasound integration. Although the field is relatively young, it has already generated a multitude of robotic systems for application in dozens of medical procedures. This paper reviews the robotic ultrasound systems that have been developed over the past two decades and describes their potential impact on modern medicine. The RUS projects reviewed include extracorporeal devices, needle guidance systems, and intraoperative systems.