Intuitive control of self-propelled microjets with haptic feedback

Investigating Haptic Feedback in Vision-Deficient Millirobot Telemanipulation

The evolution of magnetically actuated millirobots gives rise to unique teleoperation challenges due to their non-traditional kinematic and dynamic architectures, as well as their frequent use of suboptimal imaging modalities. Recent investigations into haptic interfaces for millirobots have shown promise but lack the clinically motivated task scenarios necessary to justify future development. In this work, we investigate the utility of haptic feedback on bilateral teleoperation of a magnetically actuated millirobot in visually deficient conditions. We conducted an N=23 user study in an aneurysm coiling inspired procedure, which required participants to navigate the robot through a maze in near total darkness to manipulate beads to a target under simulated fluoroscopy. We hypothesized that users will be better able to complete the telemanipulation task with haptic feedback while reducing excess forces on their surroundings compared to the no feedback conditions. Our results showed an over 40% improvement in participants' bead scoring, a nearly 10% reduction in mean force, and 13% reduction in maximum force with haptic feedback, as well as significant improvements in other metrics. Results highlight that benefits of haptic feedback are retained when haptic feedback is removed. These findings suggest that haptic feedback has the potential to significantly improve millirobot telemanipulation and control in traditionally vision deficient tasks.

Engineering microrobots for targeted cancer therapies from a medical perspective

Systemic chemotherapy remains the backbone of many cancer treatments. Due to its untargeted nature and the severe side effects it can cause, numerous nanomedicine approaches have been developed to overcome these issues. However, targeted delivery of therapeutics remains challenging. Engineering microrobots is increasingly receiving attention in this regard. Their functionalities, particularly their motility, allow microrobots to penetrate tissues and reach cancers more efficiently. Here, we highlight how different microrobots, ranging from tailor-made motile bacteria and tiny bubble-propelled microengines to hybrid spermbots, can be engineered to integrate sophisticated features optimised for precision-targeting of a wide range of cancers. Towards this, we highlight the importance of integrating clinicians, the public and cancer patients early on in the development of these novel technologies.

A GPU-Accelerated Model-Based Tracker for Untethered Submillimeter Grippers

Miniaturized grippers that possess an untethered structure are suitable for a wide range of tasks, ranging from micromanipulation and microassembly to minimally invasive surgical interventions. In order to robustly perform such tasks, it is critical to properly estimate their overall configuration. Previous studies on tracking and control of miniaturized agents estimated mainly their 2D pixel position, mostly using cameras and optical images as a feedback modality. This paper presents a novel solution to the problem of estimating and tracking the 3D position, orientation and configuration of the tips of submillimeter grippers from marker-less visual observations. We consider this as an optimization problem, which is solved using a variant of the Particle Swarm Optimization algorithm. The proposed approach has been implemented in a Graphics Processing Unit (GPU) which allows a user to track the submillimeter agents online. The proposed approach has been evaluated on several image sequences obtained from a camera and on B-mode ultrasound images obtained from an ultrasound probe. The sequences show the grippers moving, rotating, opening/closing and grasping biological material. Qualitative results obtained using both hydrogel (soft) and metallic (hard) grippers with different shapes and sizes ranging from 750 microns to 4 mm (tip to tip), demonstrate the capability of the proposed method to track the agent in all the video sequences. Quantitative results obtained by processing synthetic data reveal a tracking position error of 25 ± 7 μm and orientation error of 1.7 ± 1.3 degrees. We believe that the proposed technique can be applied to different stimuli responsive miniaturized agents, allowing the user to estimate the full configuration of complex agents from visual marker-less observations.

Haptic-Based Manipulation Scheme of Magnetic Nanoparticles in a Multi-Branch Blood Vessel for Targeted Drug Delivery

Magnetic drug targeting is a promising technique that can deliver drugs to the diseased region, while keeping the drug away from healthy parts of body. Introducing a human in the control loop of a targeted drug delivery system and using inherent bilateralism of a haptic device at the same time can considerably improve the performance of targeted drug delivery systems. In this paper, we suggest a novel intelligent haptic guidance scheme for steering a number of magnetic nanoparticles (MNPs) using forbidden region virtual fixtures and a haptic rendering scheme with multi particles. Forbidden region virtual fixtures are a general class of guidance modes implemented in software, which help a human-machine collaborative system accomplish a specific task by constraining a movement into limited regions. To examine the effectiveness of our proposed scheme, we implemented a magnetic guided drug delivery system in a virtual environment using a physics-based model of targeted drug delivery including a multi-branch blood vessel and realistic blood dynamics. We performed user studies with different guidance modes: unguided, semi virtual fixture and full virtual fixture modes. We found out that the efficiency of targeting was significantly improved using the forbidden region virtual fixture and the proposed haptic rendering of MNPs. We can expect that using intelligent haptic feedback in real targeted drug delivery systems can improve the targeting efficiency of MNPs in multi-branch vessels.

Steering and control of miniaturized untethered soft magnetic grippers with haptic assistance

Untethered miniature robotics have recently shown promising results in several scenarios at the microscale, such as targeted drug delivery, microassembly, and biopsy procedures. However, the vast majority of these small-scale robots have very limited manipulation capabilities, and none of the steering systems currently available enable humans to intuitively and effectively control dexterous miniaturized robots in a remote environment. In this paper, we present an innovative micro teleoperation system with haptic assistance for the intuitive steering and control of miniaturized self-folding soft magnetic grippers in 2-D space. The soft grippers can be wirelessly positioned using weak magnetic fields and opened/closed by changing their temperature. An image-guided algorithm tracks the position of the controlled miniaturized gripper in the remote environment. A haptic interface provides the human operator with compelling haptic sensations about the interaction between the gripper and the environment, as well as enables the operator to intuitively control the target position and grasping configuration of the gripper. Finally, magnetic and thermal control systems regulate the position and grasping configuration of the gripper. The viability of the proposed approach is demonstrated through two experiments involving 26 human subjects. Providing haptic stimuli elicited statistically significant improvements in the performance of the considered navigation and micromanipulation tasks. Note to Practitioners-The ability to accurately and intuitively control the motion of miniaturized grippers in remote environments can open new exciting possibilities in the fields of minimally-invasive surgery, micromanipulation, biopsy, and drug delivery. This paper presents a micro teleoperation system with haptic assistance through which a clinician can easily control the motion and open/close capability of miniaturized wireless soft grippers. It introduces the underlying autonomous magnetic and thermal control systems, their interconnection with the master haptic interface, and an extensive evaluation in two real-world scenarios: following of a predetermined trajectory, and pick-and-place of a microscopic object.

Experimental evaluation of magnified haptic feedback for robot-assisted needle insertion and palpation

BACKGROUND

Haptic feedback has been proven to play a key role in enhancing the performance of teleoperated medical procedures. However, due to safety issues, commercially-available medical robots do not currently provide the clinician with haptic feedback.

METHODS

This work presents the experimental evaluation of a teleoperation system for robot-assisted medical procedures able to provide magnified haptic feedback to the clinician. Forces registered at the operating table are magnified and provided to the clinician through a 7-DoF haptic interface. The same interface is also used to control the motion of a 6-DoF slave robotic manipulator. The safety of the system is guaranteed by a time-domain passivity-based control algorithm.

RESULTS

Two experiments were carried out on stiffness discrimination (during palpation and needle insertion) and one experiment on needle guidance.

CONCLUSIONS

Our haptic-enabled teleoperation system improved the performance with respect to direct hand interaction of 80%, 306%, and 27% in stiffness discrimination through palpation, stiffness discrimination during needle insertion, and guidance, respectively.

Magnetically Patterned Rolled-Up Exchange Bias Tubes: A Paternoster for Superparamagnetic Beads

We realized a deterministic transport system for superparamagnetic microbeads through micrometer-sized tubes acting as channels. Beads are moved stepwise in a paternoster-like manner through the tube and back on top of it by weak magnetic field pulses without changing the field pulse polarity and taking advantage of the magnetic stray field emerging from the tubular structures. The microtubes are engineered by rolling up exchange bias layer systems, magnetically patterned into parallel stripe magnetic domains. In this way, the tubes possess distinct azimuthally aligned magnetic domain patterns. This transport mechanism features high step velocities and remote control of not only the direction and trajectory but also the velocity of the transport without the need of fuel or catalytic material. Therefore, this approach has the potential to impact several fields of 3D applications in biotechnology, including particle transport related phenomena in lab-on-a-chip and lab-in-a-tube devices.