Performance of interventions with manipulator-driven real-time MR guidance: implementation and initial in vitro tests

Subject-specific optimization of background suppression for arterial spin labeling magnetic resonance imaging using a feedback loop on the scanner

Background suppression (BGS) in arterial spin labeling (ASL) magnetic resonance imaging leads to a higher temporal signal-to-noise ratio (tSNR) of the perfusion images compared with ASL without BGS. The performance of the BGS, however, depends on the tissue relaxation times and on inhomogeneities of the scanner's magnetic fields, which differ between subjects and are unknown at the moment of scanning. Therefore, we developed a feedback loop (FBL) mechanism that optimizes the BGS for each subject in the scanner during acquisition. We implemented the FBL for 2D pseudo-continuous ASL scans with an echo-planar imaging readout. After each dynamic scan, the acquired ASL images were automatically sent to an external computer and processed with a Python processing tool. Inversion times were optimized on the fly using 80 iterations of the Nelder-Mead method, by minimizing the signal intensity in the label image while maximizing the signal intensity in the perfusion image. The performance of this method was first tested in a four-component phantom. The regularization parameter was then tuned in six healthy subjects (three males, three females, age 24-62 years) and set as λ = 4 for all other experiments. The resulting ASL images, perfusion images, and tSNR maps obtained from the last 20 iterations of the FBL scan were compared with those obtained without BGS and with standard BGS in 12 healthy volunteers (five males, seven females, age 24-62 years) (including the six volunteers used for tuning of λ). The FBL resulted in perfusion images with a statistically significantly higher tSNR (2.20) compared with standard BGS (1.96) ( p < 5 x 10 - 3 , two-sided paired t-test). Minimizing signal in the label image furthermore resulted in control images, from which approximate changes in perfusion signal can directly be appreciated. This could be relevant to ASL applications that require a high temporal resolution. Future work is needed to minimize the number of initial acquisitions during which the performance of BGS is reduced compared with standard BGS, and to extend the technique to 3D ASL.

A Holographic Augmented Reality Interface for Visualizing of MRI Data and Planning of Neurosurgical Procedures

The recent introduction of wireless head-mounted displays (HMD) promises to enhance 3D image visualization by immersing the user into 3D morphology. This work introduces a prototype holographic augmented reality (HAR) interface for the 3D visualization of magnetic resonance imaging (MRI) data for the purpose of planning neurosurgical procedures. The computational platform generates a HAR scene that fuses pre-operative MRI sets, segmented anatomical structures, and a tubular tool for planning an access path to the targeted pathology. The operator can manipulate the presented images and segmented structures and perform path-planning using voice and gestures. On-the-fly, the software uses defined forbidden-regions to prevent the operator from harming vital structures. In silico studies using the platform with a HoloLens HMD assessed its functionality and the computational load and memory for different tasks. A preliminary qualitative evaluation revealed that holographic visualization of high-resolution 3D MRI data offers an intuitive and interactive perspective of the complex brain vasculature and anatomical structures. This initial work suggests that immersive experiences may be an unparalleled tool for planning neurosurgical procedures.

Needle artifact characteristics and insertion accuracy using a 1.2T open MRI scanner: A phantom study

PURPOSE

To evaluate the characteristics of needle artifacts and the accuracy of needle insertion using a 1.2 Tesla open magnetic resonance imaging (MRI) system in a phantom.

MATERIALS AND METHODS

First, the apparent width of the needle on the MRI and the needle tip position error of 16- and 18-gauge MRI-compatible introducer needles and a 17-gauge cryoneedle were examined with different needle angles (0°, 30°, 45°, 60°, and 90°) to the main magnetic field (B0), sequence types (balanced steady-state acquisition with rewound gradient echo [BASG] and T2-weighted fast spin echo [FSE] sequence), and frequency encoding directions. Second, the accuracy of needle insertion was evaluated after 10 MRI fluoroscopy-guided insertions in a phantom.

RESULTS

The apparent needle widths was larger when the angle of the needle axis relative to B0 was larger. The needles appeared larger on BASG than on T2-weighted FSE images, with the largest apparent widths of 16-, 17-, and 18-gauge needles of 14.3, 11.6, and 11.0mm, respectively. The apparent needle tip position was always more distal than the actual position on BASG images, with the largest longitudinal error of 4.0mm. Meanwhile, the 16- and 18-gauge needle tips appeared more proximal on T2-weighted FSE images with right-to-left frequency encoding direction. The mean accuracy of MRI fluoroscopy-guided needle insertion was 3.1mm.

CONCLUSION

These experiments clarify the characteristics of needle artifacts in a 1.2 Tesla open MRI. With this system, the MRI fluoroscopy-guided needle insertion demonstrated an acceptable accuracy for clinical use.

A modular and scalable computational framework for interactive immersion into imaging data with a holographic augmented reality interface

BACKGROUND AND OBJECTIVE

Modern imaging scanners produce an ever-growing body of 3D/4D multimodal data requiring image analytics and visualization of fused images, segmentations, and information. For the latter, augmented reality (AR) with head-mounted displays (HMDs) has shown potential. This work describes a framework (FI3D) for interactive immersion with data, integration of image processing and analytics, and rendering and fusion with an AR interface.

METHODS

The FI3D was designed and endowed with modules to communicate with peripherals, including imaging scanners and HMDs, and to provide computational power for data acquisition and processing. The core of FI3D is deployed to a dedicated computational unit that performs the computationally demanding processes in real-time, and the HMD is used as a display output peripheral and an input peripheral through gestures and voice commands. FI3D offers user-made processing and analysis dedicated modules. Users can customize and optimize these for a particular workflow while incorporating current or future libraries.

RESULTS

The FI3D framework was used to develop a workflow for processing, rendering, and visualization of CINE MRI cardiac sets. In this version, the data were loaded from a remote database, and the endocardium and epicardium of the left ventricle (LV) were segmented using a machine learning model and transmitted to a HoloLens HMD to be visualized in 4D. Performance results show that the system is capable of maintaining an image stream of one image per second with a resolution of 512 × 512. Also, it can modify visual properties of the holograms at 1 update per 16 milliseconds (62.5 Hz) while providing enough resources for the segmentation and surface reconstruction tasks without hindering the HMD.

CONCLUSIONS

We provide a system design and framework to be used as a foundation for medical applications that benefit from AR visualization, removing several technical challenges from the developmental pipeline.

A Platform Integrating Acquisition, Reconstruction, Visualization, and Manipulator Control Modules for MRI-Guided Interventions

This work presents a platform that integrates a customized MRI data acquisition scheme with reconstruction and three-dimensional (3D) visualization modules along with a module for controlling an MRI-compatible robotic device to facilitate the performance of robot-assisted, MRI-guided interventional procedures. Using dynamically-acquired MRI data, the computational framework of the platform generates and updates a 3D model representing the area of the procedure (AoP). To image structures of interest in the AoP that do not reside inside the same or parallel slices, the MRI acquisition scheme was modified to collect a multi-slice set of intraoblique to each other slices; which are termed composing slices. Moreover, this approach interleaves the collection of the composing slices so the same k-space segments of all slices are collected during similar time instances. This time matching of the k-space segments results in spatial matching of the imaged objects in the individual composing slices. The composing slices were used to generate and update the 3D model of the AoP. The MRI acquisition scheme was evaluated with computer simulations and experimental studies. Computer simulations demonstrated that k-space segmentation and time-matched interleaved acquisition of these segments provide spatial matching of the structures imaged with composing slices. Experimental studies used the platform to image the maneuvering of an MRI-compatible manipulator that carried tubing filled with MRI contrast agent. In vivo experimental studies to image the abdomen and contrast enhanced heart on free-breathing subjects without cardiac triggering demonstrated spatial matching of imaged anatomies in the composing planes. The described interventional MRI framework could assist in performing real-time MRI-guided interventions.

Touchless scanner control to support MRI-guided interventions

PURPOSE

MRI-guided interventions allow minimally invasive, radiation-free treatment but rely on real-time image data and free slice positioning. Interventional interaction with the data and the MRI scanner is cumbersome due to the diagnostic focus of current systems, confined space and sterile conditions.

METHODS

We present a touchless, hand-gesture-based interaction concept to control functions of the MRI scanner typically used during MRI-guided interventions. The system consists of a hand gesture sensor customised for MRI compatibility and a specialised UI that was developed based on clinical needs. A user study with 10 radiologists was performed to compare the gesture interaction concept and its components to task delegation-the prevalent method in clinical practice.

RESULTS

Both methods performed comparably in terms of task duration and subjective workload. Subjective performance with gesture input was perceived as worse compared to task delegation, but was rated acceptable in terms of usability while task delegation was not.

CONCLUSION

This work contributes by (1) providing access to relevant functions on an MRI scanner during percutaneous interventions in a (2) suitable way for sterile human-computer interaction. The introduced concept removes indirect interaction with the scanner via an assistant, which leads to comparable subjective workload and task completion times while showing higher perceived usability.

Percutaneous MR-guided interventions using an optical Moiré Phase tracking system: Initial results

The aim of this study was the development and evaluation of a real-time guidance support using optical Moiré Phase Tracking (MPT) for magnetic resonance (MR) guided percutaneous interventions. A gradient echo sequence, capable of real-time position updates by the MPT system, was modified to enable needle guidance based on four rigidly attached MPT markers at the back of a needle. Two perpendicular imaging planes were automatically aligned along the calibrated needle and centered at its tip. For user guidance, additional information about the needle trajectory and the tip to target distance were added as image overlay. Both, images and guiding information were displayed on the in-room monitor to facilitate MR guided interventions. The guidance support was evaluated by four experienced interventional radiologists and four novices targeting rubber O-rings embedded in a custom-made phantom on a 3T wide-bore MRI system (80 punctures). The skin to target time, user error, system error and total error were analyzed. The mean skin to target time was 146s±68s with no statistically significant difference between experts and novices. A low mean user error (0.91mm±0.43mm), system error (0.53mm±0.27mm) and total error (0.99mm±0.47mm) was reached in all directions. No statistically significant difference in user error, system error and total error could be found between experts and novices. The presented tracking and image guidance system combined with the user interface offers continuous and interactive control of the imaging plane while puncturing in the magnet enabling accurate real-time feedback for both, experienced and non-experienced users.

Motion compensation for MRI-compatible patient-mounted needle guide device: estimation of targeting accuracy in MRI-guided kidney cryoablations

Patient-mounted needle guide devices for percutaneous ablation are vulnerable to patient motion. The objective of this study is to develop and evaluate a software system for an MRI-compatible patient-mounted needle guide device that can adaptively compensate for displacement of the device due to patient motion using a novel image-based automatic device-to-image registration technique. We have developed a software system for an MRI-compatible patient-mounted needle guide device for percutaneous ablation. It features fully-automated image-based device-to-image registration to track the device position, and a device controller to adjust the needle trajectory to compensate for the displacement of the device. We performed: (a) a phantom study using a clinical MR scanner to evaluate registration performance; (b) simulations using intraoperative time-series MR data acquired in 20 clinical cases of MRI-guided renal cryoablations to assess its impact on motion compensation; and (c) a pilot clinical study in three patients to test its feasibility during the clinical procedure. FRE, TRE, and success rate of device-to-image registration were 2.71 ± 2.29 mm, 1.74 ± 1.13 mm, and 98.3% for the phantom images. The simulation study showed that the motion compensation reduced the targeting error for needle placement from 8.2 mm to 5.4 mm (p  <  0.0005) in patients under general anesthesia (GA), and from 14.4 mm to 10.0 mm (p < 1.0 × 10(−5)) in patients under monitored anesthesia care (MAC). The pilot study showed that the software registered the device successfully in a clinical setting. Our simulation study demonstrated that the software system could significantly improve targeting accuracy in patients treated under both MAC and GA. Intraprocedural image-based device-to-image registration was feasible.

Manipulator-driven selection of semi-active MR-visible markers

BACKGROUND

A method for the identification of semi-active fiducial magnetic resonance (MR) markers is presented based on selectively optically tuning and detuning them.

METHODS

Four inductively coupled solenoid coils with photoresistors were connected to light sources. A microcontroller timed the optical tuning/detuning of coils and image collection. The markers were tested on an MR manipulator linking the microcontroller to the manipulator control to visibly select the marker subset according to the actuated joint.

RESULTS

In closed-loop control, the average and maximum were 0.76° ± 0.41° and 1.18° errors for a rotational joint, and 0.87 mm ± 0.26 mm and 1.13 mm for the prismatic joint.

CONCLUSIONS

This technique is suitable for MR-compatible actuated devices that use semi-active MR-compatible markers.

A system for endoscopic mechanically scanned localized proton MR and light-induced fluorescence emission spectroscopies

Molecular and near-cellular modalities offer new opportunities in assessing living tissue in situ, and multimodality approaches, which offer complementary information, may lead to improved characterization of tissue pathophysiology benefiting diagnosis and focal therapy. However, many such modalities are limited by their low penetration through tissue, which has led to minimally invasive trans-cannula approaches to place the corresponding sensors locally at the area of interest. This work presents a system for performing localized fluorescence emission and proton magnetic resonance (MR) spectroscopies via endoscopic access. The in-house developed side-firing 1.9-mm wide dual-sensor integrates a three-fiber optical sensor for fluorescence emission optical spectroscopy and a 1-mm circular radiofrequency (RF) coil for localized MR proton spectroscopy. An MR-compatible manipulator was developed for carrying and mechanically translating the dual-sensor along a linear access channel. The hardware and software control of the system allows reconfigurable synchronization of the manipulator-assisted translation of the sensor, and MR and optical data collection. The manipulator serves as the mechanical link for the three modalities and MR images, MR spectra and optical spectra are inherently co-registered to the MR scanner coordinate system. These spectra were then used to generate spatio-spectral maps of the fluorophores and proton MR-signal sources in three-compartment phantoms with optically- and MR-visible, and distinguishable, materials. These data demonstrate a good spatial match between MR images, MR spectra and optical spectra along the scanned path. In addition to basic research, such a system may have clinical applications for assessing and characterizing cancer in situ, as well as guiding focal therapies.

Augmented reality visualization with use of image overlay technology for MR imaging-guided interventions: assessment of performance in cadaveric shoulder and hip arthrography at 1.5 T

PURPOSE

To prospectively assess overlay technology in providing accurate and efficient targeting for magnetic resonance (MR) imaging-guided shoulder and hip joint arthrography.

MATERIALS AND METHODS

A prototype augmented reality image overlay system was used in conjunction with a clinical 1.5-T MR imager. A total of 24 shoulder joint and 24 hip joint injections were planned in 12 human cadavers. Two operators (A and B) participated, each performing procedures on different cadavers using image overlay guidance. MR imaging was used to confirm needle positions, monitor injections, and perform MR arthrography. Accuracy was assessed according to the rate of needle adjustment, target error, and whether the injection was intraarticular. Efficiency was assessed according to arthrography procedural time. Operator differences were assessed with comparison of accuracy and procedure times between the operators. Mann-Whitney U test and Fisher exact test were used to assess group differences.

RESULTS

Forty-five arthrography procedures (23 shoulders, 22 hips) were performed. Three joints had prostheses and were excluded. Operator A performed 12 shoulder and 12 hip injections. Operator B performed 11 shoulder and 10 hip injections. Needle adjustment rate was 13% (six of 45; one for operator A and five for operator B). Target error was 3.1 mm±1.2 (standard deviation) (operator A, 2.9 mm±1.4; operator B, 3.5 mm±0.9). Intraarticular injection rate was 100% (45 of 45). The average arthrography time was 14 minutes (range, 6-27 minutes; 12 minutes [range, 6-25 minutes] for operator A and 16 minutes [range, 6-27 min] for operator B). Operator differences were not significant with regard to needle adjustment rate (P=.08), target error (P=.07), intraarticular injection rate (P>.99), and arthrography time (P=.22).

CONCLUSION

Image overlay technology provides accurate and efficient MR guidance for successful shoulder and hip arthrography in human cadavers.

MR-based real time path planning for cardiac operations with transapical access

Minimally invasive surgeries (MIS) have been perpetually evolving due to their potential high impact on improving patient management and overall cost effectiveness. Currently, MIS are further strengthened by the incorporation of magnetic resonance imaging (MRI) for amended visualization and high precision. Motivated by the fact that real-time MRI is emerging as a feasible modality especially for guiding interventions and surgeries in the beating heart; in this paper we introduce a real-time path planning algorithm for intracardiac procedures. Our approach creates a volumetric safety zone inside a beating heart and updates it on-the-fly using real-time MRI during the deployment of a robotic device. In order to prove the concept and assess the feasibility of the introduced method, a realistic operational scenario of transapical aortic valve replacement in a beating heart is chosen as the virtual case study.

MRI-guided robotics at the U of Houston: evolving methodologies for interventions and surgeries

Currently, we witness the rapid evolution of minimally invasive surgeries (MIS) and image guided interventions (IGI) for offering improved patient management and cost effectiveness. It is well recognized that sustaining and expand this paradigm shift would require new computational methodology that integrates sensing with multimodal imaging, actively controlled robotic manipulators, the patient and the operator. Such approach would include (1) assessing in real-time tissue deformation secondary to the procedure and physiologic motion, (2) monitoring the tool(s) in 3D, and (3) on-the-fly update information about the pathophysiology of the targeted tissue. With those capabilities, real time image guidance may facilitate a paradigm shift and methodological leap from "keyhole" visualization (i.e. endoscopy or laparoscopy) to one that uses a volumetric and informational rich perception of the Area of Operation (AoO). This capability may eventually enable a wider range and level of complexity IGI and MIS.

Towards a teleoperated needle driver robot with haptic feedback for RFA of breast tumors under continuous MRI

OBJECTIVE

The purpose of this paper is to explore the feasibility of developing a MRI-compatible needle driver system for radiofrequency ablation (RFA) of breast tumors under continuous MRI imaging while being teleoperated by a haptic feedback device from outside the scanning room. The developed needle driver prototype was designed and tested for both tumor targeting capability as well as RFA.

METHODS

The single degree-of-freedom (DOF) prototype was interfaced with a PHANToM haptic device controlled from outside the scanning room. Experiments were performed to demonstrate MRI-compatibility and position control accuracy with hydraulic actuation, along with an experiment to determine the PHANToM's ability to guide the RFA tool to a tumor nodule within a phantom breast tissue model while continuously imaging within the MRI and receiving force feedback from the RFA tool.

RESULTS

Hydraulic actuation is shown to be a feasible actuation technique for operation in an MRI environment. The design is MRI-compatible in all aspects except for force sensing in the directions perpendicular to the direction of motion. Experiments confirm that the user is able to detect healthy vs. cancerous tissue in a phantom model when provided with both visual (imaging) feedback and haptic feedback.

CONCLUSION

The teleoperated 1-DOF needle driver system presented in this paper demonstrates the feasibility of implementing a MRI-compatible robot for RFA of breast tumors with haptic feedback capability.

The Interconnection of MRI Scanner and MR-Compatible Robotic Device: Synergistic Graphical User Interface to Form a Mechatronic System

MRI scanner and magnetic resonance (MR)-compatible robotic devices are mechatronic systems. Without an interconnecting component, these two devices cannot be operated synergetically for medical interventions. In this paper, the design and properties of a graphical user interface (GUI) that accomplishes the task is presented. The GUI interconnects the two devices to obtain a larger mechatronic system by providing command and control of the robotic device based on the visual information obtained from the MRI scanner. Ideally, the GUI should also control imaging parameters of the MRI scanner. Its main goal is to facilitate image-guided interventions by acting as the synergistic component between the physician, the robotic device, the scanner, and the patient.

Magnetic resonance-compatible robotic and mechatronics systems for image-guided interventions and rehabilitation: a review study

The continuous technological progress of magnetic resonance imaging (MRI), as well as its widespread clinical use as a highly sensitive tool in diagnostics and advanced brain research, has brought a high demand for the development of magnetic resonance (MR)-compatible robotic/mechatronic systems. Revolutionary robots guided by real-time three-dimensional (3-D)-MRI allow reliable and precise minimally invasive interventions with relatively short recovery times. Dedicated robotic interfaces used in conjunction with fMRI allow neuroscientists to investigate the brain mechanisms of manipulation and motor learning, as well as to improve rehabilitation therapies. This paper gives an overview of the motivation, advantages, technical challenges, and existing prototypes for MR-compatible robotic/mechatronic devices.