Augmented Reality Visualization for CT-guided Interventions: System Description, Feasibility, and Initial Evaluation in an Abdominal Phantom

Assessment of resectability of pancreatic cancer using novel immersive high-performance virtual reality rendering of abdominal computed tomography and magnetic resonance imaging

PURPOSE

Virtual reality (VR) allows for an immersive and interactive analysis of imaging data such as computed tomography (CT) and magnetic resonance imaging (MRI). The aim of this study is to assess the comprehensibility of VR anatomy and its value in assessing resectability of pancreatic ductal adenocarcinoma (PDAC).

METHODS

This study assesses exposure to VR anatomy and evaluates the potential role of VR in assessing resectability of PDAC. Firstly, volumetric abdominal CT and MRI data were displayed in an immersive VR environment. Volunteering physicians were asked to identify anatomical landmarks in VR. In the second stage, experienced clinicians were asked to identify vascular involvement in a total of 12 CT and MRI scans displaying PDAC (2 resectable, 2 borderline resectable, and 2 locally advanced tumours per modality). Results were compared to 2D standard PACS viewing.

RESULTS

In VR visualisation of CT and MRI, the abdominal anatomical landmarks were recognised by all participants except the pancreas (30/34) in VR CT and the splenic (31/34) and common hepatic artery (18/34) in VR MRI, respectively. In VR CT, resectable, borderline resectable, and locally advanced PDAC were correctly identified in 22/24, 20/24 and 19/24 scans, respectively. Whereas, in VR MRI, resectable, borderline resectable, and locally advanced PDAC were correctly identified in 19/24, 19/24 and 21/24 scans, respectively. Interobserver agreement as measured by Fleiss κ was 0.7 for CT and 0.4 for MRI, respectively (p < 0.001). Scans were significantly assessed more accurately in VR CT than standard 2D PACS CT, with a median of 5.5 (IQR 4.75-6) and a median of 3 (IQR 2-3) correctly assessed out of 6 scans (p < 0.001).

CONCLUSION

VR enhanced visualisation of abdominal CT and MRI scan data provides intuitive handling and understanding of anatomy and might allow for more accurate staging of PDAC and could thus become a valuable adjunct in PDAC resectability assessment in the future.

Learning how to perform ultrasound-guided interventions with and without augmented reality visualization: a randomized study

OBJECTIVES

Augmented reality (AR), which entails overlay of in situ images onto the anatomy, may be a promising technique for assisting image-guided interventions. The purpose of this study was to investigate and compare the learning experience and performance of untrained operators in puncture of soft tissue lesions, when using AR ultrasound (AR US) compared with standard US (sUS).

METHODS

Forty-four medical students (28 women, 16 men) who had completed a basic US course, but had no experience with AR US, were asked to perform US-guided biopsies with both sUS and AR US, with a randomized selection of the initial modality. The experimental setup aimed to simulate biopsies of superficial soft tissue lesions, such as for example breast masses in clinical practice, by use of a turkey breast containing olives. Time to puncture(s) and success (yes/no) of the biopsies was documented. All participants completed questionnaires about their coordinative skills and their experience during the training.

RESULTS

Despite having no experience with the AR technique, time to puncture did not differ significantly between AR US and sUS (median [range]: 17.0 s [6-60] and 14.5 s [5-41], p = 0.16), nor were there any gender-related differences (p = 0.22 and p = 0.50). AR US was considered by 79.5% of the operators to be the more enjoyable means of learning and performing US-guided biopsies. Further, a more favorable learning curve was achieved using AR US.

CONCLUSIONS

Students considered AR US to be the preferable and more enjoyable modality for learning how to obtain soft tissue biopsies; however, they did not perform the biopsies faster than when using sUS.

KEY POINTS

• Performance of standard and augmented reality US-guided biopsies was comparable • A more favorable learning curve was achieved using augmented reality US. • Augmented reality US was the preferred technique and was considered more enjoyable.

Development and assessment of an educational application for the proper use of ceiling-suspended radiation shielding screens in angiography rooms using augmented reality technology

PURPOSE

An augmented reality (AR) application to help medical staff involved in interventional radiology (IR) learn how to properly use ceiling-suspended radiation shielding screens was created, and its utility was tested from the perspective of learner motivation.

METHOD

The distribution of scattered radiation in an angiography room was visualized with an AR application in three settings: when a ceiling-suspended radiation shielding screen is not used (incorrect); when there is a gap between the bottom edge of the shielding screen and the patient's torso (incorrect); and when there is no gap between the bottom edge of the shielding screen and the patient's torso (correct). This AR application was used by 33 medical staff, after which an Instructional Materials Motivation Survey (IMMS) based on the John Keller's ARCS (four categories of Attention, Relevance, Confidence, and Satisfaction) Motivation Model, consisting of 36-items with responses on a 5-point (1-5) Likert scale, was conducted.

RESULTS

The overall score was a high 4.67 ± 0.30 (mean ± standard deviation). Physician's scores tended to be lower than those of other medical staff in the categories of Attention, Relevance, and Satisfaction (not statistically significant).

CONCLUSIONS

The AR application to learn how to properly use ceiling-suspended radiation shielding screens was highly rated from the perspective of learner motivation.

Comparison of Augmented Reality Display Techniques to Support Medical Needle Insertion

Augmented reality (AR) may be a useful technique to overcome issues of conventionally used navigation systems supporting medical needle insertions, like increased mental workload and complicated hand-eye coordination. Previous research primarily focused on the development of AR navigation systems designed for specific displaying devices, but differences between employed methods have not been investigated before. To this end, a user study involving a needle insertion task was conducted comparing different AR display techniques with a monitor-based approach as baseline condition for the visualization of navigation information. A video see-through stationary display, an optical see-through head-mounted display and a spatial AR projector-camera-system were investigated in this comparison. Results suggest advantages of using projected navigation information in terms of lower task completion time, lower angular deviation and affirmative subjective participant feedback. Techniques requiring the intermediate view on screens, i.e. the stationary display and the baseline condition, showed less favorable results. Thus, benefits of providing AR navigation information compared to a conventionally used method could be identified. Significant objective measures results, as well as an identification of advantages and disadvantages of individual display techniques contribute to the development and design of improved needle navigation systems.

Augmented reality improves procedural efficiency and reduces radiation dose for CT-guided lesion targeting: a phantom study using HoloLens 2

Out-of-plane lesions pose challenges for CT-guided interventions. Augmented reality (AR) headsets are capable to provide holographic 3D guidance to assist CT-guided targeting. A prospective trial was performed assessing CT-guided lesion targeting on an abdominal phantom with and without AR guidance using HoloLens 2. Eight operators performed a cumulative total of 86 needle passes. Total needle redirections, radiation dose, procedure time, and puncture rates of nontargeted lesions were compared with and without AR. Mean number of needle passes to reach the target reduced from 7.4 passes without AR to 3.4 passes with AR (p = 0.011). Mean CT dose index decreased from 28.7 mGy without AR to 16.9 mGy with AR (p = 0.009). Mean procedure time reduced from 8.93 min without AR to 4.42 min with AR (p = 0.027). Puncture rate of a nontargeted lesion decreased from 11.9% without AR (7/59 passes) to 0% with AR (0/27 passes). First needle passes were closer to the ideal target trajectory with AR versus without AR (4.6° vs 8.0° offset, respectively, p = 0.018). AR reduced variability and elevated the performance of all operators to the same level irrespective of prior clinical experience. AR guidance can provide significant improvements in procedural efficiency and radiation dose savings for targeting out-of-plane lesions.

LEGO-compatible modular mapping phantom for magnetic resonance imaging

Physical phantoms have been widely used for performance evaluation of magnetic resonance imaging (MRI). Although there are many kinds of physical phantoms, most MRI phantoms use fixed configurations with specific sizes that may fit one or a few different types of radio frequency (RF) coils. Therefore, it has limitations for various image quality assessments of scanning areas. In this article, we report a novel design for a truly customizable MRI phantom called the LEGO-compatible Modular Mapping (MOMA) phantom, which not only serves as a general quality assurance phantom for a wide range of RF coils, but also a flexible calibration phantom for quantitative imaging. The MOMA phantom has a modular architecture which includes individual assessment functionality of the modules and LEGO-type assembly compatibility. We demonstrated the feasibility of the MOMA phantom for quantitative evaluation of image quality using customized module assembly compatible with head, breast, spine, knee, and body coil features. This unique approach allows comprehensive image quality evaluation with wide versatility. In addition, we provide detailed MOMA phantom development and imaging characteristics of the modules.

Augmented and Mixed Reality: Technologies for Enhancing the Future of IR

Augmented and mixed reality are emerging interactive and display technologies. These technologies are able to merge virtual objects, in either 2 or 3 dimensions, with the real world. Image guidance is the cornerstone of interventional radiology. With augmented or mixed reality, medical imaging can be more readily accessible or displayed in actual 3-dimensional space during procedures to enhance guidance, at times when this information is most needed. In this review, the current state of these technologies is addressed followed by a fundamental overview of their inner workings and challenges with 3-dimensional visualization. Finally, current and potential future applications in interventional radiology are highlighted.

HoloInjection: augmented reality support for CT-guided spinal needle injections

The correct placement of needles is decisive for the success of many minimally-invasive interventions and therapies. These needle insertions are usually only guided by radiological imaging and can benefit from additional navigation support. Augmented reality (AR) is a promising tool to conveniently provide needed information and may thus overcome the limitations of existing approaches. To this end, a prototypical AR application was developed to guide the insertion of needles to spinal targets using the mixed reality glasses Microsoft HoloLens. The system's registration accuracy was attempted to measure and three guidance visualisation concepts were evaluated concerning achievable in-plane and out-of-plane needle orientation errors in a comparison study. Results suggested high registration accuracy and showed that the AR prototype is suitable for reducing out-of-plane orientation errors. Limitations, like comparatively high in-plane orientation errors, effects of the viewing position and missing image slices indicate potential for improvement that needs to be addressed before transferring the application to clinical trials.

Comparison of Projective Augmented Reality Concepts to Support Medical Needle Insertion

Augmented reality (AR) is a promising tool to improve instrument navigation in needle-based interventions. Limited research has been conducted regarding suitable navigation visualizations. In this work, three navigation concepts based on existing approaches were compared in a user study using a projective AR setup. Each concept was implemented with three different scales for accuracy-to-color mapping and two methods of navigation indicator scaling. Participants were asked to perform simulated needle insertion tasks with each of the resulting 18 prototypes. Insertion angle and insertion depth accuracies were measured and analyzed, as well as task completion time and participants' subjectively perceived task difficulty. Results show a clear ranking of visualization concepts across variables. Less consistent results were obtained for the color and indicator scaling factors. Results suggest that logarithmic indicator scaling achieved better accuracy, but participants perceived it to be more difficult than linear scaling. With specific results for angle and depth accuracy, our study contributes to the future composition of improved navigation support and systems for precise needle insertion or similar applications.

Concepts for augmented reality visualisation to support needle guidance inside the MRI

During MRI-guided interventions, navigation support is often separated from the operating field on displays, which impedes the interpretation of positions and orientations of instruments inside the patient's body as well as hand–eye coordination. To overcome these issues projector-based augmented reality can be used to support needle guidance inside the MRI bore directly in the operating field. The authors present two visualisation concepts for needle navigation aids which were compared in an accuracy and usability study with eight participants, four of whom were experienced radiologists. The results show that both concepts are equally accurate (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$2.0 \pm 0.6$\end{document}2.0±0.6 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$1.7 \pm 0.5\, {\rm mm}$\end{document}1.7±0.5mm), useful and easy to use, with clear visual feedback about the state and success of the needle puncture. For easier clinical applicability, a dynamic projection on moving surfaces and organ movement tracking are needed. For now, tests with patients with respiratory arrest are feasible.

Augmented reality in neurosurgery

Neurosurgery is a medical specialty that relies heavily on imaging. The use of computed tomography and magnetic resonance images during preoperative planning and intraoperative surgical navigation is vital to the success of the surgery and positive patient outcome. Augmented reality application in neurosurgery has the potential to revolutionize and change the way neurosurgeons plan and perform surgical procedures in the future. Augmented reality technology is currently commercially available for neurosurgery for simulation and training. However, the use of augmented reality in the clinical setting is still in its infancy. Researchers are now testing augmented reality system prototypes to determine and address the barriers and limitations of the technology before it can be widely accepted and used in the clinical setting.

Percutaneous Computed Tomography-Guided Lung Biopsies: Preliminary Results using an Augmented Reality Navigation System

Aims and background

“Augmented reality” is a technique to create a composite view by augmenting the real intervention field, visualized by the doctor, with additional information coming from a virtual volume generated using computed tomography (CT), magnetic resonance or ultrasound images previously acquired from the same patient. In the present study we verified the accuracy and validated the clinical use of an augmented reality navigation system produced to perform percutaneous CT-guided lung biopsies.

Methods

One hundred and eighty consecutive patients with solitary parenchymal lung lesions, enrolled using a nonrandom enrollment system, underwent percutaneous CT-guided aspiration and core biopsy using a traditional technique (group C, 90 patients) and navigation system assistance (group S, 90 patients). For each patient we recorded the largest lesion diameter, procedure time, overall number of CT scans, radiation dose, and complications. The entire experimental project was evaluated and approved by the local institutional review board (ethics committee).

Results

Each procedure was concluded successfully and a pathological diagnosis was reached in 96% of cases in group S and 90% of cases in group C. Procedure time, overall number of CT scans and incident x-ray radiation dose (CTDIvol) were significantly reduced in navigation system-assisted procedures (P <0.001; z = 5.64) compared with traditional CT-guided procedures. The percentage of procedural complications was 14% in group S and 17% in group C.

Conclusion

The augmented reality navigation system used in this study was a highly safe, technically reliable and effective support tool in percutaneous CT-guided lung biopsy, allowing to shorten the procedure time and reduce the incident x-ray radiation dose to patients and the rate of insufficient specimens. Furthermore, it has the potential to increase the number of procedures executed in the allocated time without increasing the number of complications.

See It With Your Own Eyes: Markerless Mobile Augmented Reality for Radiation Awareness in the Hybrid Room

GOAL

We present an approach to provide awareness to the harmful ionizing radiation generated during X-ray-guided minimally invasive procedures.

METHODS

A hand-held screen is used to display directly in the user's view information related to radiation safety in a mobile augmented reality (AR) manner. Instead of using markers, we propose a method to track the observer's viewpoint, which relies on the use of multiple RGB-D sensors and combines equipment detection for tracking initialization with a KinectFusion-like approach for frame-to-frame tracking. Two of the sensors are ceiling-mounted and a third one is attached to the hand-held screen. The ceiling cameras keep an updated model of the room's layout, which is used to exploit context information and improve the relocalization procedure.

RESULTS

The system is evaluated on a multicamera dataset generated inside an operating room (OR) and containing ground-truth poses of the AR display. This dataset includes a wide variety of sequences with different scene configurations, occlusions, motion in the scene, and abrupt viewpoint changes. Qualitative results illustrating the different AR visualization modes for radiation awareness provided by the system are also presented.

CONCLUSION

Our approach allows the user to benefit from a large AR visualization area and permits to recover from tracking failure caused by vast motion or changes in the scene just by looking at a piece of equipment.

SIGNIFICANCE

The system enables the user to see the 3-D propagation of radiation, the medical staff's exposure, and/or the doses deposited on the patient's surface as seen through his own eyes.

Clinical application of navigation surgery using augmented reality in the abdominal field

This article presents general principles and recent advancements in the clinical application of augmented reality-based navigation surgery (AR based NS) for abdominal procedures and includes a description of our clinical trial and subsequent outcomes. Moreover, current problems and future aspects are discussed. The development of AR-based NS in the abdomen is delayed compared with another field because of the problem of intraoperative organ deformations or the existence of established modalities. Although there are a few reports on the clinical use of AR-based NS for digestive surgery, sophisticated technologies in urology have often been reported. However, the rapid widespread use of video- or robot assisted surgeries requires this technology. We have worked to develop a system of AR-based NS for hepatobiliary and pancreatic surgery. Then we developed a short rigid scope that enables surgeons to obtain 3D view. We recently focused on pancreatic surgery, because intraoperative organ shifting is minimal. The position of each organ in overlaid image almost corresponded with that of the actual organ with about 5 mm of mean registration errors. Intraoperative information generated from this system provided us with useful navigation. However, AR-based NS has several problems to overcome such as organ deformity, evaluation of utility, portability or cost.

Short rigid scope and stereo-scope designed specifically for open abdominal navigation surgery: clinical application for hepatobiliary and pancreatic surgery

BACKGROUND

We have reported the utility of an image display system using augmented reality (AR) technology in hepatobiliary surgery under laparotomy. Among several procedures, we herein report a system using a novel short rigid scope and stereo-scope, both designed specifically for open abdominal navigation surgery, and their clinical application for hepatobiliary and pancreatic surgery.

METHODS

The 3D reconstructed images were obtained from preoperative computed tomography data. In our specialized operating room, after paired-point matching registration, the reconstructed images are overlaid onto the operative field images captured by the short rigid scopes. The scopes, which are compact and sterilizable, can be used in the operative field. The stereo-scope provides depth information. Eight patients underwent operations using this system, including hepatectomy in two, distal pancreatectomy in three, and pancreaticoduodenectomy in three patients. The stereo-scope was used in five patients.

RESULTS

All eight operations were performed safely using the novel short rigid scopes, and stereo images were acquired in all five patients for whom the stereo-scope was used. The scopes were user friendly, and the intraoperative time requirement for our system was reduced compared with the conventional method.

CONCLUSIONS

The novel short rigid scope and stereo-scope seem to be suitable for clinical use in open abdominal navigation surgery. In hepatobiliary and pancreatic surgery, our novel system may improve the safety, accuracy and efficiency of operations.

On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic

Mixed reality environments for medical applications have been explored and developed over the past three decades in an effort to enhance the clinician's view of anatomy and facilitate the performance of minimally invasive procedures. These environments must faithfully represent the real surgical field and require seamless integration of pre- and intra-operative imaging, surgical instrument tracking, and display technology into a common framework centered around and registered to the patient. However, in spite of their reported benefits, few mixed reality environments have been successfully translated into clinical use. Several challenges that contribute to the difficulty in integrating such environments into clinical practice are presented here and discussed in terms of both technical and clinical limitations. This article should raise awareness among both developers and end-users toward facilitating a greater application of such environments in the surgical practice of the future.

DVV: a taxonomy for mixed reality visualization in image guided surgery

Mixed reality visualizations are increasingly studied for use in image guided surgery (IGS) systems, yet few mixed reality systems have been introduced for daily use into the operating room (OR). This may be the result of several factors: the systems are developed from a technical perspective, are rarely evaluated in the field, and/or lack consideration of the end user and the constraints of the OR. We introduce the Data, Visualization processing, View (DVV) taxonomy which defines each of the major components required to implement a mixed reality IGS system. We propose that these components be considered and used as validation criteria for introducing a mixed reality IGS system into the OR. A taxonomy of IGS visualization systems is a step toward developing a common language that will help developers and end users discuss and understand the constituents of a mixed reality visualization system, facilitating a greater presence of future systems in the OR. We evaluate the DVV taxonomy based on its goodness of fit and completeness. We demonstrate the utility of the DVV taxonomy by classifying 17 state-of-the-art research papers in the domain of mixed reality visualization IGS systems. Our classification shows that few IGS visualization systems' components have been validated and even fewer are evaluated.

Accuracy and diagnostic yield of CT-guided stereotactic liver biopsy of primary and secondary liver tumors

OBJECTIVE

CT-guided biopsy still plays a decisive role in the management of liver tumors, especially if the lesions are not visible or accessible by ultrasound. As CT-guided stereotaxy appears to be a very accurate targeting technique, the aim of this study was to evaluate the targeting accuracy, diagnostic yield, and complications of CT-guided stereotactic liver biopsy of primary and secondary liver tumors.

METHODS AND MATERIALS

Prior to stereotactic liver biopsy, patients under general anesthesia were immobilized using a vacuum cushion. Respiratory motion was controlled by temporary disconnections of the endotracheal tube. An optical-based navigation system was used for 3D trajectory planning and placement of a 15-G coaxial needle via a stereotactic aiming device. The histological samples were obtained using a 16-G Tru-Cut(™) biopsy needle system. For evaluation of targeting accuracy the control CT image with the needles in place was fused with the planning CT image. The lateral error at the tip and skin entry point and the angular error were calculated. In addition, the skin-to-liver-surface (SL) distance, the needle-to-liver-surface (NL) angle, and the presence of liver cirrhosis were evaluated. The diagnostic yield was evaluated by histological reports from the institutional pathologists.

RESULTS

The median lateral error was 2.5 mm (range: 0-6.5 mm) at the needle entry point and 3.2 mm (range: 0.01-9.4 mm) at the needle tip. The median angular error was 1.06° (range: 0-6.64°). Liver cirrhosis, NL angle and SL distance showed no significant impact on the targeting accuracy. Forty-five of the 46 liver biopsies (97.8%) were diagnostic according to the histological reports. No puncture-related complications such as bleeding or perforation of intestinal organs or lung tissue were recorded.

Morphological and functional MDCT: problem-solving tool and surrogate biomarker for hepatic disease clinical care and drug discovery in the era of personalized medicine

This article explains the significant role of morphological and functional multidetector computer tomography (MDCT) in combination with imaging postprocessing algorithms served as a problem-solving tool and noninvasive surrogate biomarker to effectively improve hepatic diseases characterization, detection, tumor staging and prognosis, therapy response assessment, and novel drug discovery programs, partial liver resection and transplantation, and MDCT-guided interventions in the era of personalized medicine. State-of-the-art MDCT depicts and quantifies hepatic disease over conventional CT for not only depicting lesion location, size, and extent but also detecting changes in tumor biologic behavior caused by therapy or tumor progression before morphologic changes. Color-encoded parameter display provides important functional information on blood flow, permeability, leakage space, and blood volume. Together with other relevant biomarkers and genomics, the imaging modality is being developed and validated as a biomarker to early response to novel, targeted anti-VEGF(R)/PDGFR or antivascular/angiogenesis agents as its parameters correlate with immunohistochemical surrogates of tumor angiogenesis and molecular features of malignancies. MDCT holds incremental value to World Health Organization response criteria and Response Evaluation Criteria in Solid Tumors in liver disease management. MDCT volumetric measurement of future remnant liver is the most important factor influencing the outcome of patients who underwent partial liver resection and transplantation. MDCT-guided interventional methods deliver personalized therapies locally in the human body. MDCT will hold more scientific impact when it is fused with other imaging probes to yield comprehensive information regarding changes in liver disease at different levels (anatomic, metabolic, molecular, histologic, and other levels).

Real-time tracking of liver motion and deformation using a flexible needle

PURPOSE

A real-time 3D image guidance system is needed to facilitate treatment of liver masses using radiofrequency ablation, for example. This study investigates the feasibility and accuracy of using an electromagnetically tracked flexible needle inserted into the liver to track liver motion and deformation.

METHODS

This proof-of-principle study was conducted both ex vivo and in vivo with a CT scanner taking the place of an electromagnetic tracking system as the spatial tracker. Deformations of excised livers were artificially created by altering the shape of the stage on which the excised livers rested. Free breathing or controlled ventilation created deformations of live swine livers. The positions of the needle and test targets were determined through CT scans. The shape of the needle was reconstructed using data simulating multiple embedded electromagnetic sensors. Displacement of liver tissues in the vicinity of the needle was derived from the change in the reconstructed shape of the needle.

RESULTS

The needle shape was successfully reconstructed with tracking information of two on-needle points. Within 30 mm of the needle, the registration error of implanted test targets was 2.4 ± 1.0 mm ex vivo and 2.8 ± 1.5 mm in vivo.

CONCLUSION

A practical approach was developed to measure the motion and deformation of the liver in real time within a region of interest. The approach relies on redesigning the often-used seeker needle to include embedded electromagnetic tracking sensors. With the nonrigid motion and deformation information of the tracked needle, a single- or multimodality 3D image of the intraprocedural liver, now clinically obtained with some delay, can be updated continuously to monitor intraprocedural changes in hepatic anatomy. This capability may be useful in radiofrequency ablation and other percutaneous ablative procedures.

Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery

BACKGROUND

We applied a new concept of "image overlay surgery" consisting of the integration of virtual reality (VR) and augmented reality (AR) technology, in which dynamic 3D images were superimposed on the patient's actual body surface and evaluated as a reference for surgical navigation in gastrointestinal, hepatobiliary and pancreatic surgery.

METHODS

We carried out seven surgeries, including three cholecystectomies, two gastrectomies and two colectomies. A Macintosh and a DICOM workstation OsiriX were used in the operating room for image analysis. Raw data of the preoperative patient information obtained via MDCT were reconstructed to volume rendering and projected onto the patient's body surface during the surgeries. For accurate registration, OsiriX was first set to reproduce the patient body surface, and the positional coordinates of the umbilicus, left and right nipples, and the inguinal region were fixed as physiological markers on the body surface to reduce the positional error.

RESULTS

The registration process was non-invasive and markerlesss, and was completed within 5 min. Image overlay navigation was helpful for 3D anatomical understanding of the surgical target in the gastrointestinal, hepatobiliary and pancreatic anatomies. The surgeon was able to minimize movement of the gaze and could utilize the image assistance without interfering with the forceps operation, reducing the gap from the VR. Unexpected organ injury could be avoided in all procedures. In biliary surgery, the projected virtual cholangiogram on the abdominal wall could advance safely with identification of the bile duct. For early gastric and colorectal cancer, the small tumors and blood vessels, which usually could not be found on the gastric serosa by laparoscopic view, were simultaneously detected on the body surface by carbon dioxide-enhanced MDCT. This provided accurate reconstructions of the tumor and involved lymph node, directly linked with optimization of the surgical procedures.

CONCLUSIONS

Our non-invasive markerless registration using physiological markers on the body surface reduced logistical efforts. The image overlay technique is a useful tool when highlighting hidden structures, giving more information.

FPGA-Accelerated Deformable Image Registration for Improved Target-Delineation During CT-Guided Interventions

Minimally invasive image-guided interventions (IGIs) are time and cost efficient, minimize unintended damage to healthy tissue, and lead to faster patient recovery. With the advent of multislice computed tomography (CT), many IGIs are now being performed under volumetric CT guidance. Registering pre-and intraprocedural images for improved intraprocedural target delineation is a fundamental need in the IGI workflow. Earlier approaches to meet this need primarily employed rigid body approximation, which may not be valid because of nonrigid tissue misalignment between these images. Intensity-based automatic deformable registration is a promising option to correct for this misalignment; however, the long execution times of these algorithms have prevented their use in clinical workflow. This article presents a field-programmable gate array-based architecture for accelerated implementation of mutual information (Ml)-based deformable registration. The reported implementation reduces the execution time of MI-based deformable registration from hours to a few minutes. This work also demonstrates successful registration of abdominal intraprocedural noncontrast CT (iCT) images with preprocedural contrast-enhanced CT (preCT) and positron emission tomography (PET) images using the reported solution. The registration accuracy for this application was evaluated using 5 iCT-preCT and 5 iCT-PET image pairs. The registration accuracy of the hardware implementation is comparable with that achieved using a software implementation and is on the order of a few millimeters. This registration accuracy, coupled with the execution speed and compact implementation of the reported solution, makes it suitable for integration in the IGI-workflow.

Renal Masses in the Adult Patient: The Role of Percutaneous Biopsy

Although percutaneous renal mass biopsy with cross-sectional imaging guidance has long been considered to be safe and accurate, there have been recent advances in imaging, interventional, and cytologic techniques that have increased the role of percutaneous biopsy in the diagnosis of renal masses. Today, biopsy plays a fundamental role in the care of patients with a renal mass. Biopsy results are used to confirm the diagnosis of renal cancers, metastases, and infections, and there is increasing evidence to suggest that biopsy can help subtype and grade many primary renal cancers. Because a considerable fraction of small solid renal masses are benign and do not need treatment, there is an increasing need to diagnose them. Biopsy after a full imaging work-up can help prevent unnecessary and potentially morbid surgical and ablation procedures in a substantial number of patients. Although more data are needed to understand the overall accuracy of biopsy for the diagnosis of benign lesions, many can be diagnosed with the aid of biopsy findings. This article reviews reported experience with percutaneous renal mass biopsy, discusses the technical factors that contribute to results, and details seven specific clinical settings that should prompt the clinician to consider percutaneous biopsy when encountering a renal mass.