Enhanced Real-Time Pose Estimation for Closed-Loop Robotic Manipulation of Magnetically Actuated Capsule Endoscopes

Exact and Computationally Robust Solutions for Cylindrical Magnets Systems with Programmable Magnetization

Magnetic systems based on permanent magnets are receiving growing attention, in particular for micro/millirobotics and biomedical applications. Their design landscape is expanded by the possibility to program magnetization, yet enabling analytical results, crucial for containing computational costs, are lacking. The dipole approximation is systematically used (and often strained), because exact and computationally robust solutions are to be unveiled even for common geometries such as cylindrical magnets, which are ubiquitously used in fundamental research and applications. In this study, exact solutions are disclosed for magnetic field and gradient of a cylindrical magnet with generic uniform magnetization, which can be robustly computed everywhere within and outside the magnet, and directly extend to magnets systems of arbitrary complexity. Based on them, exact and computationally robust solutions are unveiled for force and torque between coaxial magnets. The obtained analytical solutions overstep the dipole approximation, thus filling a long-standing gap, and offer strong computational gains versus numerical simulations (up to 106 , for the considered test-cases). Moreover, they bridge to a variety of applications, as illustrated through a compact magnets array that could be used to advance state-of-the-art biomedical tools, by creating, based on programmable magnetization patterns, circumferential and helical force traps for magnetoresponsive diagnostic/therapeutic agents.

Calibrated analytical model for magnetic localization of wireless capsule endoscope based on onboard sensing

Wireless capsule endoscopes (WCEs) are pill-sized camera-embedded devices that can provide visualization of the gastrointestinal (GI) tract by capturing and transmitting images to an external receiver. Determination of the exact location of the WCE is crucial for the accurate navigation of the WCE through external guidance, tracking of the GI abnormality, and the treatment of the detected disease. Despite the enormous progress in the real-time tracking of the WCE, a well-calibrated analytical model is still missing for the accurate localization of WCEs by the measurements from different onboard sensing units. In this paper, a well-calibrated analytical model for the magnetic localization of the WCE was established by optimizing the magnetic moment in the magnetic dipole model. The Jacobian-based iterative method was employed to solve the position of the WCE. An error model was established and experimentally verified for the analysis and prediction of the localization errors caused by inaccurate measurements from the magnetic field sensor. The assessment of the real-time localization of the WCE was performed via experimental trials using an external permanent magnet (EPM) mounted on a robotic manipulator and a WCE equipped with a 3-axis magnetic field sensor and an inertial measurement unit (IMU). The localization errors were measured under different translational and rotational motion modes and working spaces. The results showed that the selection of workspace (distance relative to the EPM) could lead to different positioning errors. The proposed magnetic localization method holds great potential for the real-time localization of WCEs when performing complex motions during GI diagnosis.

Robotic, self-propelled, self-steerable, and disposable colonoscopes: Reality or pipe dream? A state of the art review

Robotic colonoscopes could potentially provide a comfortable, less painful and safer alternative to standard colonoscopy. Recent exciting developments in this field are pushing the boundaries to what is possible in the future. This article provides a comprehensive review of the current work in robotic colonoscopes including self-propelled, steerable and disposable endoscopes that could be alternatives to standard colonoscopy. We discuss the advantages and disadvantages of these systems currently in development and highlight the technical readiness of each system to help the reader understand where and when such systems may be available for routine clinical use and get an idea of where and in which situation they can best be deployed.

Dynamic tracking effect of a magnetic navigated dual hemisphere capsule robot

For diagnostic and therapeutic applications in spacious spots of the gastrointestinal (GI) tract, the single rigid body capsule clinically applied is difficult to realize the fix-point posture adjustment function manipulated by the external permanent magnet system using the static balance control because the posture alignment and the locomotion interfere with each other. To realize this function easily, the dual hemisphere capsule robot (DHCR) is proposed, based on tracking effect—the axis of DHCR keeps tracking the normal orientation of the spatial universal rotating magnetic vector (SURMV). Since tracking effect employs dynamic balance control, dynamic stability of the DHCR system affects posture alignment performance. This paper focuses on posture alignment dynamic modeling and the influence of the magnetic flux density and the angular velocity of the SURMV, along with the damping coefficient of the GI tract surface on stability, obtaining the stability domains of parameters. Furthermore, to reduce error due to the uncertainties in complex GI tract environment, the sliding mode controller based on nominal model is proposed to achieve more accurate dynamic tracking, and Lyapunov theorem is employed to assess stability of controller. Finally, the tracking effect is verified through simulations and experiments, indicating that the fix-point posture adjustment can be realized with higher accuracy and efficiency.

Robotic Autonomy for Magnetic Endoscope Biopsy

Magnetically actuated endoscopes are currently transitioning in to clinical use for procedures such as colonoscopy, presenting numerous benefits over their conventional counterparts. Intelligent and easy-to-use control strategies are an essential part of their clinical effectiveness due to the un-intuitive nature of magnetic field interaction. However, work on developing intelligent control for these devices has mainly been focused on general purpose endoscope navigation. In this work, we investigate the use of autonomous robotic control for magnetic colonoscope intervention via biopsy, another major component of clinical viability. We have developed control strategies with varying levels of robotic autonomy, including semi-autonomous routines for identifying and performing targeted biopsy, as well as random quadrant biopsy. We present and compare the performance of these approaches to magnetic endoscope biopsy against the use of a standard flexible endoscope on bench-top using a colonoscopy training simulator and silicone colon model. The semi-autonomous routines for targeted and random quadrant biopsy were shown to reduce user workload with comparable times to using a standard flexible endoscope.

Active Stabilization of Interventional Tasks Utilizing a Magnetically Manipulated Endoscope

Magnetically actuated robots have become increasingly popular in medical endoscopy over the past decade. Despite the significant improvements in autonomy and control methods, progress within the field of medical magnetic endoscopes has mainly been in the domain of enhanced navigation. Interventional tasks such as biopsy, polyp removal, and clip placement are a major procedural component of endoscopy. Little advancement has been done in this area due to the problem of adequately controlling and stabilizing magnetically actuated endoscopes for interventional tasks. In the present paper we discuss a novel model-based Linear Parameter Varying (LPV) control approach to provide stability during interventional maneuvers. This method linearizes the non-linear dynamic interaction between the external actuation system and the endoscope in a set of equilibria, associated to different distances between the magnetic source and the endoscope, and computes different controllers for each equilibrium. This approach provides the global stability of the overall system and robustness against external disturbances. The performance of the LPV approach is compared to an intelligent teleoperation control method (based on a Proportional Integral Derivative (PID) controller), on the Magnetic Flexible Endoscope (MFE) platform. Four biopsies in different regions of the colon and at two different system equilibria are performed. Both controllers are asked to stabilize the endoscope in the presence of external disturbances (i.e. the introduction of the biopsy forceps through the working channel of the endoscope). The experiments, performed in a benchtop colon simulator, show a maximum reduction of the mean orientation error of the endoscope of 45.8% with the LPV control compared to the PID controller.

Telerobotic neurovascular interventions with magnetic manipulation

Advances in robotic technology have been adopted in various subspecialties of both open and minimally invasive surgery, offering benefits such as enhanced surgical precision and accuracy with reduced fatigue of the surgeon. Despite the advantages, robotic applications to endovascular neurosurgery have remained largely unexplored because of technical challenges such as the miniaturization of robotic devices that can reach the complex and tortuous vasculature of the brain. Although some commercial systems enable robotic manipulation of conventional guidewires for coronary and peripheral vascular interventions, they remain unsuited for neurovascular applications because of the considerably smaller and more tortuous anatomy of cerebral arteries. Here, we present a teleoperated robotic neurointerventional platform based on magnetic manipulation. Our system consists of a magnetically controlled guidewire, a robot arm with an actuating magnet to steer the guidewire, a set of motorized linear drives to advance or retract the guidewire and a microcatheter, and a remote-control console to operate the system under real-time fluoroscopy. We demonstrate our system's capability to navigate narrow and winding pathways both in vitro with realistic neurovascular phantoms representing the human anatomy and in vivo in the porcine brachial artery with accentuated tortuosity for preclinical evaluation. We further demonstrate telerobotically assisted therapeutic procedures including coil embolization and clot retrieval thrombectomy for treating cerebral aneurysms and ischemic stroke, respectively. Our system could enable safer and quicker access to hard-to-reach lesions while minimizing the radiation exposure to physicians and open the possibility of remote procedural services to address challenges in current stroke systems of care.

Magnetic Soft Materials and Robots

In conventional classification, soft robots feature mechanical compliance as the main distinguishing factor from traditional robots made of rigid materials. Recent advances in functional soft materials have facilitated the emergence of a new class of soft robots capable of tether-free actuation in response to external stimuli such as heat, light, solvent, or electric or magnetic field. Among the various types of stimuli-responsive materials, magnetic soft materials have shown remarkable progress in their design and fabrication, leading to the development of magnetic soft robots with unique advantages and potential for many important applications. However, the field of magnetic soft robots is still in its infancy and requires further advancements in terms of design principles, fabrication methods, control mechanisms, and sensing modalities. Successful future development of magnetic soft robots would require a comprehensive understanding of the fundamental principle of magnetic actuation, as well as the physical properties and behavior of magnetic soft materials. In this review, we discuss recent progress in the design and fabrication, modeling and simulation, and actuation and control of magnetic soft materials and robots. We then give a set of design guidelines for optimal actuation performance of magnetic soft materials. Lastly, we summarize potential biomedical applications of magnetic soft robots and provide our perspectives on next-generation magnetic soft robots.

Pose tracking method using magnetic excitations with frequency division for robotic endoscopic capsules

The poses of robotic endoscopic capsules are indispensable for further follow-up examinations, potential targeted drug delivery, and closed-loop controlling of active locomotion. A novel tracking method using the multiple magnetic excitations with frequency division has been investigated. The multiple excitation coils can simultaneously work at different frequency to improve real-time tracking. A novel model between the magnetic flux density and the capsule's pose has been derived, which shows a nonlinear equation group with multiple local extremum. Then, a Back-Propagation (BP) neural network algorithm combined with the mother wavelet is investigated to solve the pose. To reduce the volume and power consumption, the wireless magnetic sensing module uses digital signal processing as the core framework, which is beneficial to be miniaturized to integrate with the capsule. The functional prototype of the tracking system has been developed, which consists of a wireless magnetic sensing module mounted in the capsule, a magnetic excitation module with frequency division, a wireless receiver and data interface, an excitation coil array and a platform for pose solving. The experimental results show that the mean errors are 0.0098 m in x-component, 0.0122 m in y-component, 0.0077 m in z-component, 0.187 rad in α-component and 0.161 rad in β-component, respectively. The real-time performance of the tracking system is improved.

Autonomous navigation of a magnetic colonoscope using force sensing and a heuristic search algorithm

This paper presents an autonomous navigation system for cost-effective magnetic-assisted colonoscopy, employing force-based sensors, an actuator, a proportional-integrator controller and a real-time heuristic searching method. The force sensing system uses load cells installed between the robotic arm and external permanent magnets to derive attractive force data as the basis for real-time surgical safety monitoring and tracking information to navigate the disposable magnetic colonoscope. The average tracking accuracy on magnetic field navigator (MFN) platform in x-axis and y-axis are 1.14 ± 0.59 mm and 1.61 ± 0.45 mm, respectively, presented in mean error ± standard deviation. The average detectable radius of the tracking system is 15 cm. Three simulations of path planning algorithms are presented and the learning real-time A* (LRTA*) algorithm with our proposed directional heuristic evaluation design has the best performance. It takes 75 steps to complete the traveling in unknown synthetic colon map. By integrating the force-based sensing technology and LRTA* path planning algorithm, the average time required to complete autonomous navigation of a highly realistic colonoscopy training model on the MFN platform is 15 min 38 s and the intubation rate is 83.33%. All autonomous navigation experiments are completed without intervention by the operator.

Endorobots for Colonoscopy: Design Challenges and Available Technologies

Colorectal cancer (CRC) is the second most common cause of cancer death worldwide, after lung cancer (Sung et al., 2021). Early stage detection is key to increase the survival rate. Colonoscopy remains to be the gold standard procedure due to its dual capability to optically inspect the entire colonic mucosa and to perform interventional procedures at the same time. However, this causes pain and discomfort, whereby it requires sedation or anaesthesia of the patient. It is a difficult procedure to perform that can cause damage to the colonic wall in some cases. Development of new technologies aims to overcome the current limitations on colonoscopy by using advancements in endorobotics research. The design of these advanced medical devices is challenging because of the limited space of the lumen, the contorted shape, and the long tract of the large bowel. The force applied to the colonic wall needs to be controlled to avoid collateral effects such as injuries to the colonic mucosa and pain during the procedure. This article discusses the current challenges in the colonoscopy procedure, the available locomotion technologies for endorobots used in colonoscopy at a prototype level and the commercial products available.

A localization method for wireless capsule endoscopy using side wall cameras and IMU sensor

Localizing the endoscopy capsule inside gastrointestinal (GI) system provides key information which leads to GI abnormality tracking and precision medical delivery. In this paper, we have proposed a new method to localize the capsule inside human GI track. We propose to equip the capsule with four side wall cameras and an Inertial Measurement Unit (IMU), that consists of 9 Degree-Of-Freedom (DOF) including a gyroscope, an accelerometer and a magnetometer to monitor the capsule’s orientation and direction of travel. The low resolution mono-chromatic cameras, installed along the wide wall, are responsible to measure the actual capsule movement, not the involuntary motion of the small intestine. Finally, a fusion algorithm is used to combine all data to derive the traveled path and plot the trajectory. Compared to other methods, the presented system is resistive to surrounding conditions, such as GI nonhomogeneous structure and involuntary small bowel movements. In addition, it does not require external antenna or arrays. Therefore, GI tracking can be achieved without disturbing patients’ daily activities.

Next-generation ingestible devices: sensing, locomotion and navigation

There is significant interest in exploring the human body's internal activities and measuring important parameters to understand, treat and diagnose the digestive system environment and related diseases. Wireless capsule endoscopy (WCE) is widely used for gastrointestinal (GI) tract exploration due to its effectiveness as it provides no pain and is totally tolerated by the patient. Current ingestible sensing technology provides a valuable diagnostic tool to establish a platform for monitoring the physiological and biological activities inside the human body. It is also used for visualizing the GI tract to observe abnormalities by recording the internal cavity while moving. However, the capsule endoscopy is still passive, and there is no successful locomotion method to control its mobility through the whole GI tract. Drug delivery, localization of abnormalities, cost reduction and time consumption are improvements that can be gained from having active ingestible WCEs. In this article, the current technological developments of ingestible devices including sensing, locomotion and navigation are discussed and compared. The main features required to implement next-generation active WCEs are explored. The methods are evaluated in terms of the most important features such as safety, velocity, complexity of design, control, and power consumption.

Endoluminal Motion Recognition of a Magnetically-Guided Capsule Endoscope Based on Capsule-Tissue Interaction Force

A magnetically-guided capsule endoscope, embedding flexible force sensors, is designed to measure the capsule-tissue interaction force. The flexible force sensor is composed of eight force-sensitive elements surrounding the internal permanent magnet (IPM). The control of interaction force acting on the intestinal wall can reduce patient's discomfort and maintain the magnetic coupling between the external permanent magnet (EPM) and the IPM during capsule navigation. A flexible force sensor can achieve this control. In particular, by analyzing the signals of the force sensitive elements, we propose a method to recognize the status of the motion of the magnetic capsule, and provide corresponding formulas to evaluate whether the magnetic capsule follows the motion of the external driving magnet. Accuracy of the motion recognition in Ex Vivo tests reached 94% when the EPM was translated along the longitudinal axis. In addition, a method is proposed to realign the EPM and the IPM before the loss of their magnetic coupling. Its translational error, rotational error, and runtime are 7.04 ± 0.71 mm, 3.13 ± 0.47∘, and 11.4 ± 0.39 s, respectively. Finally, a control strategy is proposed to prevent the magnetic capsule endoscope from losing control during the magnetically-guided capsule colonoscopy.

Posture Dynamic Modeling and Stability Analysis of a Magnetic Driven Dual-Spin Spherical Capsule Robot

In order to realize the intervention operation in the unstructured and ample environments such as stomach and colon, a dual-spin spherical capsule robot (DSCR) driven by pure magnetic torque generated by the universal rotating magnetic field (URMF) is proposed. The coupled magnetic torque, the viscoelastic friction torque, and the gravity torque were analyzed. Furthermore, the posture dynamic model describing the electric-magnetic-mechanical-liquid coupling dynamic behavior of the DSCR in the gastrointestinal (GI) tract was established. This model is a second-order periodic variable coefficient dynamics equation, which should be regarded as an extension of the Lagrange case for the dual-spin body system under the fixed-point motion, since the external torques were applied. Based on the Floquet-Lyapunov theory, the stability domain of the DSCR for the asymptotically stable motion and periodic motion were obtained by investigating the influence of the angular velocity of the URMF, the magnetic induction intensity, and the centroid deviation. Research results show that the DSCR can realize three kinds of motion, which are asymptotically stable motion, periodic motion, and chaotic motion, according to the distribution of the system characteristic multipliers. Moreover, the posture stability of the DSCR can be improved by increasing the angular velocity of the URMF and reducing the magnetic induction intensity.

Magnetic flexible endoscope for colonoscopy: an initial learning curve analysis

Background and study aims  Colonoscopy is a technically challenging procedure that requires extensive training to minimize discomfort and avoid trauma due to its drive mechanism. Our academic team developed a magnetic flexible endoscope (MFE) actuated by magnetic coupling under supervisory robotic control to enable a front-pull maneuvering mechanism, with a motion controller user interface, to minimize colon wall stress and potentially reduce the learning curve. We aimed to evaluate this learning curve and understand the user experience. Methods  Five novices (no endoscopy experience), five experienced endoscopists, and five experienced MFE users each performed 40 trials on a model colon using 1:1 block randomization between a pediatric colonoscope (PCF) and the MFE. Cecal intubation (CI) success, time to cecum, and user experience (NASA task load index) were measured. Learning curves were determined by the number of trials needed to reach minimum and average proficiency-defined as the slowest average CI time by an experienced user and the average CI time by all experienced users, respectively. Results  MFE minimum proficiency was achieved by all five novices (median 3.92 trials) and five experienced endoscopists (median 2.65 trials). MFE average proficiency was achieved by four novices (median 14.21 trials) and four experienced endoscopists (median 7.00 trials). PCF minimum and average proficiency levels were achieved by only one novice. Novices' perceived workload with the MFE significantly improved after obtaining minimum proficiency. Conclusions  The MFE has a short learning curve for users with no prior experience-requiring relatively few attempts to reach proficiency and at a reduced perceived workload.

Robotic Localization Based on Planar Cable Robot and Hall Sensor Array Applied to Magnetic Capsule Endoscope

Recently an active locomotive capsule endoscope (CE) for diagnosis and treatment in the digestive system has been widely studied. However, real-time localization to achieve precise feedback control and record suspicious positioning in the intestine is still challenging owing to the limitation of capsule size, relatively large diagnostic volume, and compatibility of other devices in clinical site. To address this issue, we present a novel robotic localization sensing methodology based on the kinematics of a planar cable driven parallel robot (CDPR) and measurements of the quasistatic magnetic field of a Hall effect sensor (HES) array. The arrangement of HES and the Levenberg-Marquardt (LM) algorithm are applied to estimate the position of the permanent magnet (PM) in the CE, and the planar CDPR is incorporated to follow the PM in the CE. By tracking control of the planar CDPR, the position of PM in any arbitrary position can be obtained through robot forward kinematics with respect to the global coordinates at the bedside. The experimental results show that the root mean square error (RMSE) for the estimated position value of PM was less than 1.13 mm in the X, Y, and Z directions and less than 1.14° in the θ and φ orientation, where the sensing space could be extended to ±70 mm for the given 34 × 34 mm² HES array and the average moving distance in the Z-direction is 40 ± 2.42 mm. The proposed method of the robotic sensing with HES and CDPR may advance the sensing space expansion technology by utilizing the provided single sensor module of limited sensible volume.

A Novel Approach for Automatic State Detection of A Magnetically Actuated Capsule

In recent years, the Simultaneous Magnetic Actuation and Localization (SMAL) technology has been developed to accelerate and locate the wireless capsule endoscopy (WCE) in the intestine. In this paper, we propose a novel approach to detect the state of the capsule for improving the localization results. By creating a function to fit the relationship between the theoretical values of the actuating magnetic field and the measurement results, we present an algorithm for automatic estimation of the capsule state according to the fitting parameters. Experiment results on phantoms demonstrate the feasibility of the proposed method for detecting different states of the capsule during magnetic actuation.

Enabling the future of colonoscopy with intelligent and autonomous magnetic manipulation

Early diagnosis of colorectal cancer significantly improves survival. However, over half of cases are diagnosed late due to demand exceeding the capacity for colonoscopy - the "gold standard" for screening. Colonoscopy is limited by the outdated design of conventional endoscopes, associated with high complexity of use, cost and pain. Magnetic endoscopes represent a promising alternative, overcoming drawbacks of pain and cost, but struggle to reach the translational stage as magnetic manipulation is complex and unintuitive. In this work, we use machine vision to develop intelligent and autonomous control of a magnetic endoscope, for the first time enabling non-expert users to effectively perform magnetic colonoscopy in-vivo. We combine the use of robotics, computer vision and advanced control to offer an intuitive and effective endoscopic system. Moreover, we define the characteristics required to achieve autonomy in robotic endoscopy. The paradigm described here can be adopted in a variety of applications where navigation in unstructured environments is required, such as catheters, pancreatic endoscopy, bronchoscopy, and gastroscopy. This work brings alternative endoscopic technologies closer to the translational stage, increasing availability of early-stage cancer treatments.

The Modelling, Analysis, and Experimental Validation of a Novel Micro-Robot for Diagnosis of Intestinal Diseases

Intestinal-related diseases all around the world are increasing nowadays, and gradually become stubborn diseases threatening human health, and even lives. Diagnosis methods have attracted more and more attention. This article concerns a non-invasive way, a novel micro-robot, to diagnose intestinal diseases. This proposed micro-robot is a swallowable device, 14 mm in diameter, like a capsule. In order to make it possible for the micro-robot to move forward, backward, or anchor itself at a suspicious lesion point in the intestine with different lumen diameter sections, two key mechanisms have been proposed. One is an expanding mechanism with two-layer folded legs for anchoring. The designed expanding mechanism could realize a large variable diameter ratio, upwards of 3.43. In addition, a pair of specific annular gears instead of a traditional pinion drive is devised not only saving limited space, but also reducing energy loss. The other mechanism is a telescoping mechanism, possessing a self-locking lead screw nut system, which is used to obtain axial motion of the micro-robot. Then, the kinematics and dynamics of the micro-robot are analyzed. After that, the following experiments, including force tests and locomotion tests, are constructed. A good match is found between the theoretical results and the experimental data. Finally, in vitro experiments are performed with a prototype to verify the safety and reliability of the proposed micro-robot in porcine intestine.

EXPERIMENTAL ASSESSMENT OF INTACT COLON DEFORMATION UNDER LOCAL FORCES APPLIED BY MAGNETIC CAPSULE ENDOSCOPES

Magnetically guided capsule endoscopy is a promising technology for clinical application. A platform that simulates the magnetic capsule endoscope system is built to study the deformation process of the colon when its lumen suffers local forces. Force-displacement curves of the porcine large intestine under various experiment conditions, including different loading positions (haustra or taeniae coli), loading directions, colon inner pressures and specimen lengths, were measured to analyze the mechanical behavior of the intact large intestine during interactions with magnetic capsule endoscopes. In the practical application of the magnetic capsule endoscope, these data are imperative to optimize the control scheme and reduce operation risks. Based on our experiments, the taeniae coli of the intact large intestine show higher linear stiffness than the haustra, and inflation reduces the linear stiffness of the colon. Magnetic capsule with small edge radii can more easily damage or even perforate the colon. Based on our test results, we suggest that the force applied to the colon should be limited to below 17[Formula: see text]N when the capsule is actuated forward along the colon and limited to below 10[Formula: see text]N when the capsule is vertical to the colon during lesion screening.

Frontiers of Robotic Colonoscopy: A Comprehensive Review of Robotic Colonoscopes and Technologies

Flexible colonoscopy remains the prime mean of screening for colorectal cancer (CRC) and the gold standard of all population-based screening pathways around the world. Almost 60% of CRC deaths could be prevented with screening. However, colonoscopy attendance rates are affected by discomfort, fear of pain and embarrassment or loss of control during the procedure. Moreover, the emergence and global thread of new communicable diseases might seriously affect the functioning of contemporary centres performing gastrointestinal endoscopy. Innovative solutions are needed: artificial intelligence (AI) and physical robotics will drastically contribute for the future of the healthcare services. The translation of robotic technologies from traditional surgery to minimally invasive endoscopic interventions is an emerging field, mainly challenged by the tough requirements for miniaturization. Pioneering approaches for robotic colonoscopy have been reported in the nineties, with the appearance of inchworm-like devices. Since then, robotic colonoscopes with assistive functionalities have become commercially available. Research prototypes promise enhanced accessibility and flexibility for future therapeutic interventions, even via autonomous or robotic-assisted agents, such as robotic capsules. Furthermore, the pairing of such endoscopic systems with AI-enabled image analysis and recognition methods promises enhanced diagnostic yield. By assembling a multidisciplinary team of engineers and endoscopists, the paper aims to provide a contemporary and highly-pictorial critical review for robotic colonoscopes, hence providing clinicians and researchers with a glimpse of the major changes and challenges that lie ahead.

Simultaneous Independent Translational and Rotational Feedback Motion Control System for a Cylindrical Magnet using Planar Arrays of Magnetic Sensors and Cylindrical Coils

This letter describes an electromagnetic feedback control system for rigid-body motion control of a magnet. Its novel features are that sensing and actuation using magnetometer sensors and actuator coils operate simultaneously, and magnetic field models from the controlled magnet and each of the actuator coil currents are used together to calculate the 3D position and orientation of the magnet to control motion simultaneously and independently in multiple degrees of freedom including planar translation and two in rotation, leaving rotation about the cylindrical axis of magnetization uncontrolled. The system configuration and the localization and actuation methods are presented with experimental results of magnet localization with constant and varying coil currents, and during feedback control of trajectory following motion of the magnet in multiple directions on a planar surface and with controlled changes in orientation. The intended application of the system is for motion control of magnetic endoscope capsules and other miniature medical devices inside the human body.

Adaptive Dynamic Control for Magnetically Actuated Medical Robots

In the present work we discuss a novel dynamic control approach for magnetically actuated robots, by proposing an adaptive control technique, robust towards parametric uncertainties and unknown bounded disturbances. The former generally arise due to partial knowledge of the robots' dynamic parameters, such as inertial factors, the latter are the outcome of unpredictable interaction with unstructured environments. In order to show the application of the proposed approach, we consider controlling the Magnetic Flexible Endoscope (MFE) which is composed of a soft-tethered Internal Permanent Magnet (IPM), actuated with a single External Permanent Magnet (EPM). We provide with experimental analysis to show the possibility of levitating the MFE - one of the most difficult tasks with this platform - in case of partial knowledge of the IPM's dynamics and no knowledge of the tether's behaviour. Experiments in an acrylic tube show a reduction of contact of the 32% compared to non-levitating techniques and 1.75 times faster task completion with respect to previously proposed levitating techniques. More realistic experiments, performed in a colon phantom, show that levitating the capsule achieves faster and smoother exploration and that the minimum time for completing the task is attained by the proposed approach.

Miniature Pump for Treatment of Refractory Ascites Based on Local Magnetic Actuation

This paper presents the design, fabrication, and experimental validation of a novel low-cost implantable pump for the treatment of refractory ascites (RA) based on local magnetic actuation (LMA). A reciprocating positive displacement pump displaces liquid unidirectionally through magnetic coupling with a magnetic controller placed on the outside of the patient's body. The proposed solution is intuitive to use given an alignment algorithm that exploits externally placed magnetic field sensors (MFS). The implantable device has a catheter-like shape, is electronic free (no on-board battery), has low fabrication cost (<8 USD), and is able to generate a flow-rate of 3.65 L/h while effectively pumping fluids with various viscosity (1–5.5 cP). RA is commonly treated via costly paracentesis or invasive surgical placement of a transjugular portosystemic shunt (TIPS). The proposed solution can be implanted with minimally invasive techniques and can be used on a daily basis to drain a set amount of liquid, without requiring recurrent hospital visits.

Sensorless Estimation of the Planar Distal Shape of a Tip-Actuated Endoscope

Traditional endoscopes consist of a flexible body and a steerable tip with therapeutic capability. Although prior endoscopes have relied on operator pushing for actuation, recent robotic concepts have relied on the application of a tip force for guidance. In such case, the body of the endoscope can be passive and compliant; however, the body can have significant effect on mechanics of motion and may require modeling. As the endoscope body's shape is often unknown, we have developed an estimation method to recover the approximate distal shape, local to the endoscope's tip, where the tip position and orientation are the only sensed parameters in the system. We leverage a planar dynamic model and extended Kalman filter to obtain a constant-curvature shape estimate of a magnetically guided endoscope. We validated this estimator in both dynamic simulations and on a physical platform. We then used this estimate in a feed-forward control scheme and demonstrated improved trajectory following. This methodology can enable the use of inverse-dynamic control for the tip-based actuation of an endoscope, without the need for shape sensing.

Gastrointestinal diagnosis using non-white light imaging capsule endoscopy

Capsule endoscopy (CE) has proved to be a powerful tool in the diagnosis and management of small bowel disorders since its introduction in 2001. However, white light imaging (WLI) is the principal technology used in clinical CE at present, and therefore, CE is limited to mucosal inspection, with diagnosis remaining reliant on visible manifestations of disease. The introduction of WLI CE has motivated a wide range of research to improve its diagnostic capabilities through integration with other sensing modalities. These developments have the potential to overcome the limitations of WLI through enhanced detection of subtle mucosal microlesions and submucosal and/or transmural pathology, providing novel diagnostic avenues. Other research aims to utilize a range of sensors to measure physiological parameters or to discover new biomarkers to improve the sensitivity, specificity and thus the clinical utility of CE. This multidisciplinary Review summarizes research into non-WLI CE devices by organizing them into a taxonomic structure on the basis of their sensing modality. The potential of these capsules to realize clinically useful virtual biopsy and computer-aided diagnosis (CADx) is also reported.

Intelligent magnetic manipulation for gastrointestinal ultrasound

Diagnostic endoscopy in the gastrointestinal tract has remained largely unchanged for decades and is limited to the visualization of the tissue surface, the collection of biopsy samples for diagnoses, and minor interventions such as clipping or tissue removal. In this work, we present the autonomous servoing of a magnetic capsule robot for in-situ, subsurface diagnostics of microanatomy. We investigated and showed the feasibility of closed-loop magnetic control using digitized microultrasound (μUS) feedback; this is crucial for obtaining robust imaging in an unknown and unconstrained environment. We demonstrated the functionality of an autonomous servoing algorithm that uses μUS feedback, both on benchtop trials as well as in-vivo in a porcine model. We have validated this magnetic-μUS servoing in instances of autonomous linear probe motion and were able to locate markers in an agar phantom with 1.0 ± 0.9 mm position accuracy using a fusion of robot localization and μUS image information. This work demonstrates the feasibility of closed-loop robotic μUS imaging in the bowel without the need for either a rigid physical link between the transducer and extracorporeal tools or complex manual manipulation.

Sensitivity Ellipsoids for Force Control of Magnetic Robots with Localization Uncertainty

The navigation of magnetic medical robots typically relies on localizing an actuated, intracorporeal, ferromagnetic body and back-computing a necessary field and gradient that would result in a desired wrench on the device. Uncertainty in this localization degrades the precision of force transmission. Reducing applied force uncertainty may enhance tasks such as in-vivo navigation of miniature robots, actuation of magnetically guided catheters, tissue palpation, as well as simply ensuring a bound on forces applied on sensitive tissue. In this paper, we analyzed the effects of localization noise on force uncertainty by using sensitivity ellipsoids of the magnetic force Jacobian and introduced an algorithm for uncertainty reduction. We validated the algorithm in both a simulation study and in a physical experiment. In simulation, we observed reductions in estimated force uncertainty by factors of up to 2.8 and 3.1 when using one and two actuating magnets, respectively. On a physical platform, we demonstrated a force uncertainty reduction by a factor of up to 2.5 as measured using an external sensor. Being the first consideration of force uncertainty resulting from noisy localization, this work provides a strategy for investigators to minimize uncertainty in magnetic force transmission.

Explicit Model Predictive Control of a Magnetic Flexible Endoscope

In this paper, explicit model predictive control is applied in conjunction with nonlinear optimisation to a magnetically actuated flexible endoscope for the first time. The approach is aimed at computing the motion of the external permanent magnet, given the desired forces and torques. The strategy described here takes advantage of the nonlinear nature of the magnetic actuation and explicitly considers the workspace boundaries, as well as the actuation constraints. Initially, a simplified dynamic model of the tethered capsule, based on the Euler-Lagrange equations is developed. Subsequently, the explicit model predictive control is described and a novel approach for the external magnet positioning, based on a single step, nonlinear optimisation routine, is proposed. Finally, the strategy is implemented on the experimental platform, where bench-top trials are performed on a realistic colon phantom, showing the effectiveness of the technique. The work presented here constitutes an initial exploration for model-based control techniques applied to magnetically manipulated payloads, the techniques described here may be applied to a wide range of devices, including flexible endoscopes and wireless capsules. To our knowledge, this is the first example of advanced closed loop control of magnetic capsules.

Magnetic Levitation for Soft-Tethered Capsule Colonoscopy Actuated With a Single Permanent Magnet: A Dynamic Control Approach

The present letter investigates a novel control approach for magnetically driven soft-tethered capsules for colonoscopy-a potentially painless approach for colon inspection. The focus of this work is on a class of devices composed of a magnetic capsule endoscope actuated by a single external permanent magnet. Actuation is achieved by manipulating the external magnet with a serial manipulator, which in turn produces forces and torques on the internal magnetic capsule. We propose a control strategy which, counteracting gravity, achieves levitation of the capsule. This technique, based on a nonlinear backstepping approach, is able to limit contact with the colon walls, reducing friction, avoiding contact with internal folds, and facilitating the inspection of nonplanar cavities. The approach is validated on an experimental setup, which embodies a general scenario faced in colonoscopy. The experiments show that we can attain 19.5% of contact with the colon wall, compared to the almost 100% of previously proposed approaches. Moreover, we show that the control can be used to navigate the capsule through a more realistic environment-a colon phantom-with reasonable completion time.