MITK-US: real-time ultrasound support within MITK

The Essential Role of Open Data and Software for the Future of Ultrasound-Based Neuronavigation

With the recent developments in machine learning and modern graphics processing units (GPUs), there is a marked shift in the way intra-operative ultrasound (iUS) images can be processed and presented during surgery. Real-time processing of images to highlight important anatomical structures combined with in-situ display, has the potential to greatly facilitate the acquisition and interpretation of iUS images when guiding an operation. In order to take full advantage of the recent advances in machine learning, large amounts of high-quality annotated training data are necessary to develop and validate the algorithms. To ensure efficient collection of a sufficient number of patient images and external validity of the models, training data should be collected at several centers by different neurosurgeons, and stored in a standard format directly compatible with the most commonly used machine learning toolkits and libraries. In this paper, we argue that such effort to collect and organize large-scale multi-center datasets should be based on common open source software and databases. We first describe the development of existing open-source ultrasound based neuronavigation systems and how these systems have contributed to enhanced neurosurgical guidance over the last 15 years. We review the impact of the large number of projects worldwide that have benefited from the publicly available datasets “Brain Images of Tumors for Evaluation” (BITE) and “Retrospective evaluation of Cerebral Tumors” (RESECT) that include MR and US data from brain tumor cases. We also describe the need for continuous data collection and how this effort can be organized through the use of a well-adapted and user-friendly open-source software platform that integrates both continually improved guidance and automated data collection functionalities.

3D-XGuide: open-source X-ray navigation guidance system

PURPOSE

With the growing availability and variety of imaging modalities, new methods of intraoperative support have become available for all kinds of interventions. The basic principles of image fusion and image guidance have been widely adopted and are commercialized through a number of platforms. Although multimodal systems have been found to be useful for guiding interventional procedures, they all have their limitations. The integration of more advanced guidance techniques into the product functionality is, however, not easy due to the proprietary solutions of the vendors. Therefore, the purpose of this work is to introduce a software system for image fusion, real-time navigation, and working points documentation during transcatheter interventions performed under X-ray (XR) guidance.

METHODS

An interactive software system for cross-modal registration and image fusion of XR fluoroscopy with CT or MRI-derived anatomic 3D models is implemented using Qt application framework and VTK visualization pipeline. DICOM data can be imported in retrospective mode. Live XR data input is realized by a video capture card application interface.

RESULTS

The actual software release offers a graphical user interface with basic functionality including data import and handling, calculation of projection geometry and transformations between related coordinate systems, rigid 3D-3D registration, and template matching-based tracking and motion compensation algorithms in 2D and 3D. The link to the actual software release on GitHub including source code and executable is provided to support independent research and development in the field of intervention guidance.

CONCLUSION

The introduced system provides a common foundation for the rapid prototyping of new approaches in the field of XR fluoroscopic guidance. As a pure software solution, the developed system is potentially vendor-independent and can be easily extended to be used with the XR systems of different manufacturers.

iPad-assisted percutaneous nephrolithotomy (PCNL): a matched pair analysis compared to standard PCNL

PURPOSE

To compare iPad-assisted (Apple Inc., Cupertino, USA) percutaneous access to the kidney to the standard puncturing technique for percutaneous nephrolithotomy (PCNL).

METHODS

For the iPad-assisted PCNL, a computed tomography is performed prior to surgery, using fiducial radiopaque markers. The important anatomical structures (i.e. kidney, stones) are segmented using specific software enabling the superimposition of images semi-transparently on the iPad by marker-based navigation. Twenty-two patients underwent an iPad-assisted percutaneous puncture of the kidney for PCNL. Twenty-two patients of the clinical database from the Urological Department SLK Hospital Heilbronn, who underwent the standard puncturing technique, were matched to these patients. Matching criteria were age, gender, stone volume, body mass index, stone site and the absence of anatomical variation. Puncture time, radiation exposure and number of attempts for a successful puncture were evaluated. All procedures were performed by two experienced urologists. The standard puncturing method consisted of a combination of ultrasound and fluoroscopy guidance. Chi-square and t test were used to ensure that there was no difference in the matching criteria between the groups. To compare the two methods, U test, Kruskal-Wallis and Chi-square test were used.

RESULTS

Examination of radiation exposure showed a significant difference between the two groups in favour of the standard puncturing method (p < 0.01) and puncture time (p = 0.01). However, there was no significant difference in puncturing attempts (p = 0.45).

CONCLUSION

The iPad-assisted navigation, with the objective being to puncture the renal collecting system, represents a new technique (IDEAL criteria 2b), which proved to be applicable in clinical practice, but still has potential for technical improvement.

Versatile Low-Cost Volumetric 3-D Ultrasound Platform for Existing Clinical 2-D Systems

Ultrasound imaging has indications across many areas of medicine, but the need for training and the variability in skill and acquired image quality among 2-D ultrasound users have limited its wider adoption and utilization. Low-cost volumetric ultrasound with a known frame of reference has the potential to lower these operator-dependent barriers and enhance the clinical utility of ultrasound imaging. In this paper, we improve upon our previous research-scanner-based prototype to implement a versatile volumetric imaging platform for existing clinical 2-D ultrasound systems. We present improved data acquisition and image reconstruction schemes to increase quality, streamline workflow, and provide real-time visual feedback. We present initial results using the platform on a Vimedix simulator, as well as on phantom and in vivo targets using a variety of clinical ultrasound systems and probes.

Surgical data science: The new knowledge domain

Healthcare in general, and surgery/interventional care in particular, is evolving through rapid advances in technology and increasing complexity of care, with the goal of maximizing the quality and value of care. Whereas innovations in diagnostic and therapeutic technologies have driven past improvements in the quality of surgical care, future transformation in care will be enabled by data. Conventional methodologies, such as registry studies, are limited in their scope for discovery and research, extent and complexity of data, breadth of analytical techniques, and translation or integration of research findings into patient care. We foresee the emergence of surgical/interventional data science (SDS) as a key element to addressing these limitations and creating a sustainable path toward evidence-based improvement of interventional healthcare pathways. SDS will create tools to measure, model, and quantify the pathways or processes within the context of patient health states or outcomes and use information gained to inform healthcare decisions, guidelines, best practices, policy, and training, thereby improving the safety and quality of healthcare and its value. Data are pervasive throughout the surgical care pathway; thus, SDS can impact various aspects of care, including prevention, diagnosis, intervention, or postoperative recovery. The existing literature already provides preliminary results, suggesting how a data science approach to surgical decision-making could more accurately predict severe complications using complex data from preoperative, intraoperative, and postoperative contexts, how it could support intraoperative decision-making using both existing knowledge and continuous data streams throughout the surgical care pathway, and how it could enable effective collaboration between human care providers and intelligent technologies. In addition, SDS is poised to play a central role in surgical education, for example, through objective assessments, automated virtual coaching, and robot-assisted active learning of surgical skill. However, the potential for transforming surgical care and training through SDS may only be realized through a cultural shift that not only institutionalizes technology to seamlessly capture data but also assimilates individuals with expertise in data science into clinical research teams. Furthermore, collaboration with industry partners from the inception of the discovery process promotes optimal design of data products as well as their efficient translation and commercialization. As surgery continues to evolve through advances in technology that enhance delivery of care, SDS represents a new knowledge domain to engineer surgical care of the future.

Ultrasound-navigated radiofrequency ablation of thyroid nodules with integrated electromagnetic tracking: comparison with conventional ultrasound guidance in gelatin models

PURPOSE

A thyroid-like gelatin model was used to determine potential superiority of a new navigation system for ultrasound (US)-guided electrode insertion called EchoTrack, featuring a US probe with an integrated electromagnetic field generator, in comparison with conventional US when performing radiofrequency ablation.

METHODS

In order to compare 20 navigated ablations with 20 ablations under conventional US guidance, a thyroid-like gelatin model was used. In each group, 10 in-plane and 10 out-of-plane punctures were performed. Metal seeds measuring 8.5 [Formula: see text] 1.8 mm served as ablation targets. The number of redirections until final electrode placement, targeting accuracy and electrode placement time were measured.

RESULTS

The number of redirections could be significantly ([Formula: see text]) reduced from 2.7 ± 1.3 in the conventional group to 0.2 ± 0.5 in the EchoTrack group. Accuracy increased from 3.9 ± 4.7 to 2.0 ± 1.9 mm. The total placement time increased from 39 ± 20.5 to 79.2 ± 26 s.

CONCLUSIONS

EchoTrack is able to reduce the redirections needed to place the electrode in comparison with conventional US and provides high placement accuracy. Our new navigation system has high potential to reduce the risk of harming critical structures and to improve guidance during ablation of difficult nodules, as treatment planning as well as the safety of out-of-plane punctures are improved.

CustusX: an open-source research platform for image-guided therapy

Purpose

CustusX is an image-guided therapy (IGT) research platform dedicated to intraoperative navigation and ultrasound imaging. In this paper, we present CustusX as a robust, accurate, and extensible platform with full access to data and algorithms and show examples of application in technological and clinical IGT research.

Methods

CustusX has been developed continuously for more than 15 years based on requirements from clinical and technological researchers within the framework of a well-defined software quality process. The platform was designed as a layered architecture with plugins based on the CTK/OSGi framework, a superbuild that manages dependencies and features supporting the IGT workflow. We describe the use of the system in several different clinical settings and characterize major aspects of the system such as accuracy, frame rate, and latency.

Results

The validation experiments show a navigation system accuracy of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$<$$\end{document}<1.1 mm, a frame rate of 20 fps, and latency of 285 ms for a typical setup. The current platform is extensible, user-friendly and has a streamlined architecture and quality process. CustusX has successfully been used for IGT research in neurosurgery, laparoscopic surgery, vascular surgery, and bronchoscopy.

Conclusions

CustusX is now a mature research platform for intraoperative navigation and ultrasound imaging and is ready for use by the IGT research community. CustusX is open-source and freely available at http://www.custusx.org.

The NifTK software platform for image-guided interventions: platform overview and NiftyLink messaging

PURPOSE

To perform research in image-guided interventions, researchers need a wide variety of software components, and assembling these components into a flexible and reliable system can be a challenging task. In this paper, the NifTK software platform is presented. A key focus has been high-performance streaming of stereo laparoscopic video data, ultrasound data and tracking data simultaneously.

METHODS

A new messaging library called NiftyLink is introduced that uses the OpenIGTLink protocol and provides the user with easy-to-use asynchronous two-way messaging, high reliability and comprehensive error reporting. A small suite of applications called NiftyGuide has been developed, containing lightweight applications for grabbing data, currently from position trackers and ultrasound scanners. These applications use NiftyLink to stream data into NiftyIGI, which is a workstation-based application, built on top of MITK, for visualisation and user interaction. Design decisions, performance characteristics and initial applications are described in detail. NiftyLink was tested for latency when transmitting images, tracking data, and interleaved imaging and tracking data.

RESULTS

NiftyLink can transmit tracking data at 1,024 frames per second (fps) with latency of 0.31 milliseconds, and 512 KB images with latency of 6.06 milliseconds at 32 fps. NiftyIGI was tested, receiving stereo high-definition laparoscopic video at 30 fps, tracking data from 4 rigid bodies at 20-30 fps and ultrasound data at 20 fps with rendering refresh rates between 2 and 20 Hz with no loss of user interaction.

CONCLUSION

These packages form part of the NifTK platform and have proven to be successful in a variety of image-guided surgery projects. Code and documentation for the NifTK platform are available from http://www.niftk.org . NiftyLink is provided open-source under a BSD license and available from http://github.com/NifTK/NiftyLink . The code for this paper is tagged IJCARS-2014.

PLUS: open-source toolkit for ultrasound-guided intervention systems

A variety of advanced image analysis methods have been under the development for ultrasound-guided interventions. Unfortunately, the transition from an image analysis algorithm to clinical feasibility trials as part of an intervention system requires integration of many components, such as imaging and tracking devices, data processing algorithms, and visualization software. The objective of our paper is to provide a freely available open-source software platform-PLUS: Public software Library for Ultrasound-to facilitate rapid prototyping of ultrasound-guided intervention systems for translational clinical research. PLUS provides a variety of methods for interventional tool pose and ultrasound image acquisition from a wide range of tracking and imaging devices, spatial and temporal calibration, volume reconstruction, simulated image generation, and recording and live streaming of the acquired data. This paper introduces PLUS, explains its functionality and architecture, and presents typical uses and performance in ultrasound-guided intervention systems. PLUS fulfills the essential requirements for the development of ultrasound-guided intervention systems and it aspires to become a widely used translational research prototyping platform. PLUS is freely available as open source software under BSD license and can be downloaded from http://www.plustoolkit.org.