Efficient Photoacoustic Image Synthesis with Deep Learning

Photoacoustic imaging potentially allows for the real-time visualization of functional human tissue parameters such as oxygenation but is subject to a challenging underlying quantification problem. While in silico studies have revealed the great potential of deep learning (DL) methodology in solving this problem, the inherent lack of an efficient gold standard method for model training and validation remains a grand challenge. This work investigates whether DL can be leveraged to accurately and efficiently simulate photon propagation in biological tissue, enabling photoacoustic image synthesis. Our approach is based on estimating the initial pressure distribution of the photoacoustic waves from the underlying optical properties using a back-propagatable neural network trained on synthetic data. In proof-of-concept studies, we validated the performance of two complementary neural network architectures, namely a conventional U-Net-like model and a Fourier Neural Operator (FNO) network. Our in silico validation on multispectral human forearm images shows that DL methods can speed up image generation by a factor of 100 when compared to Monte Carlo simulations with 5×108 photons. While the FNO is slightly more accurate than the U-Net, when compared to Monte Carlo simulations performed with a reduced number of photons (5×106), both neural network architectures achieve equivalent accuracy. In contrast to Monte Carlo simulations, the proposed DL models can be used as inherently differentiable surrogate models in the photoacoustic image synthesis pipeline, allowing for back-propagation of the synthesis error and gradient-based optimization over the entire pipeline. Due to their efficiency, they have the potential to enable large-scale training data generation that can expedite the clinical application of photoacoustic imaging.

Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.

Standard error estimation in meta-analysis of studies reporting medians

We consider the setting of an aggregate data meta-analysis of a continuous outcome of interest. When the distribution of the outcome is skewed, it is often the case that some primary studies report the sample mean and standard deviation of the outcome and other studies report the sample median along with the first and third quartiles and/or minimum and maximum values. To perform meta-analysis in this context, a number of approaches have recently been developed to impute the sample mean and standard deviation from studies reporting medians. Then, standard meta-analytic approaches with inverse-variance weighting are applied based on the (imputed) study-specific sample means and standard deviations. In this article, we illustrate how this common practice can severely underestimate the within-study standard errors, which results in poor coverage for the pooled mean in common effect meta-analyses and overestimation of between-study heterogeneity in random effects meta-analyses. We propose a straightforward bootstrap approach to estimate the standard errors of the imputed sample means. Our simulation study illustrates how the proposed approach can improve the estimation of the within-study standard errors and consequently improve coverage for the pooled mean in common effect meta-analyses and estimation of between-study heterogeneity in random effects meta-analyses. Moreover, we apply the proposed approach in a meta-analysis to identify risk factors of a severe course of COVID-19.

Photoacoustic image synthesis with generative adversarial networks

Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties with high spatial resolution. However, previous attempts to solve the optical inverse problem with supervised machine learning were hampered by the absence of labeled reference data. While this bottleneck has been tackled by simulating training data, the domain gap between real and simulated images remains an unsolved challenge. We propose a novel approach to PAT image synthesis that involves subdividing the challenge of generating plausible simulations into two disjoint problems: (1) Probabilistic generation of realistic tissue morphology, and (2) pixel-wise assignment of corresponding optical and acoustic properties. The former is achieved with Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data. According to a validation study on a downstream task our approach yields more realistic synthetic images than the traditional model-based approach and could therefore become a fundamental step for deep learning-based quantitative PAT (qPAT).

Semantic segmentation of multispectral photoacoustic images using deep learning

Photoacoustic (PA) imaging has the potential to revolutionize functional medical imaging in healthcare due to the valuable information on tissue physiology contained in multispectral photoacoustic measurements. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. In this work, we present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images to facilitate image interpretability. Manually annotated photoacoustic and ultrasound imaging data are used as reference and enable the training of a deep learning-based segmentation algorithm in a supervised manner. Based on a validation study with experimentally acquired data from 16 healthy human volunteers, we show that automatic tissue segmentation can be used to create powerful analyses and visualizations of multispectral photoacoustic images. Due to the intuitive representation of high-dimensional information, such a preprocessing algorithm could be a valuable means to facilitate the clinical translation of photoacoustic imaging.

Robust hand tracking for surgical telestration

PURPOSE

As human failure has been shown to be one primary cause for post-operative death, surgical training is of the utmost socioeconomic importance. In this context, the concept of surgical telestration has been introduced to enable experienced surgeons to efficiently and effectively mentor trainees in an intuitive way. While previous approaches to telestration have concentrated on overlaying drawings on surgical videos, we explore the augmented reality (AR) visualization of surgical hands to imitate the direct interaction with the situs.

METHODS

We present a real-time hand tracking pipeline specifically designed for the application of surgical telestration. It comprises three modules, dedicated to (1) the coarse localization of the expert's hand and the subsequent (2) segmentation of the hand for AR visualization in the field of view of the trainee and (3) regression of keypoints making up the hand's skeleton. The semantic representation is obtained to offer the ability for structured reporting of the motions performed as part of the teaching.

RESULTS

According to a comprehensive validation based on a large data set comprising more than 14,000 annotated images with varying application-relevant conditions, our algorithm enables real-time hand tracking and is sufficiently accurate for the task of surgical telestration. In a retrospective validation study, a mean detection accuracy of 98%, a mean keypoint regression accuracy of 10.0 px and a mean Dice Similarity Coefficient of 0.95 were achieved. In a prospective validation study, it showed uncompromised performance when the sensor, operator or gesture varied.

CONCLUSION

Due to its high accuracy and fast inference time, our neural network-based approach to hand tracking is well suited for an AR approach to surgical telestration. Future work should be directed to evaluating the clinical value of the approach.

SIMPA: an open-source toolkit for simulation and image processing for photonics and acoustics

SIGNIFICANCE

Optical and acoustic imaging techniques enable noninvasive visualisation of structural and functional properties of tissue. The quantification of measurements, however, remains challenging due to the inverse problems that must be solved. Emerging data-driven approaches are promising, but they rely heavily on the presence of high-quality simulations across a range of wavelengths due to the lack of ground truth knowledge of tissue acoustical and optical properties in realistic settings.

AIM

To facilitate this process, we present the open-source simulation and image processing for photonics and acoustics (SIMPA) Python toolkit. SIMPA is being developed according to modern software design standards.

APPROACH

SIMPA enables the use of computational forward models, data processing algorithms, and digital device twins to simulate realistic images within a single pipeline. SIMPA's module implementations can be seamlessly exchanged as SIMPA abstracts from the concrete implementation of each forward model and builds the simulation pipeline in a modular fashion. Furthermore, SIMPA provides comprehensive libraries of biological structures, such as vessels, as well as optical and acoustic properties and other functionalities for the generation of realistic tissue models.

RESULTS

To showcase the capabilities of SIMPA, we show examples in the context of photoacoustic imaging: the diversity of creatable tissue models, the customisability of a simulation pipeline, and the degree of realism of the simulations.

CONCLUSIONS

SIMPA is an open-source toolkit that can be used to simulate optical and acoustic imaging modalities. The code is available at: https://github.com/IMSY-DKFZ/simpa, and all of the examples and experiments in this paper can be reproduced using the code available at: https://github.com/IMSY-DKFZ/simpa_paper_experiments.

Surgical data science - from concepts toward clinical translation

Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.

A Platform and Multisided Market for Translational, Software-Defined Medical Procedures in the Operating Room (OP 4.1): Proof-of-Concept Study

Background

Although digital and data-based technologies are widespread in various industries in the context of Industry 4.0, the use of smart connected devices in health care is still in its infancy. Innovative solutions for the medical environment are affected by difficult access to medical device data and high barriers to market entry because of proprietary systems.

Objective

In the proof-of-concept project OP 4.1, we show the business viability of connecting and augmenting medical devices and data through software add-ons by giving companies a technical and commercial platform for the development, implementation, distribution, and billing of innovative software solutions.

Methods

The creation of a central platform prototype requires the collaboration of several independent market contenders, including medical users, software developers, medical device manufacturers, and platform providers. A dedicated consortium of clinical and scientific partners as well as industry partners was set up.

Results

We demonstrate the successful development of the prototype of a user-centric, open, and extensible platform for the intelligent support of processes starting with the operating room. By connecting heterogeneous data sources and medical devices from different manufacturers and making them accessible for software developers and medical users, the cloud-based platform OP 4.1 enables the augmentation of medical devices and procedures through software-based solutions. The platform also allows for the demand-oriented billing of apps and medical devices, thus permitting software-based solutions to fast-track their economic development and become commercially successful.

Conclusions

The technology and business platform OP 4.1 creates a multisided market for the successful development, implementation, distribution, and billing of new software solutions in the operating room and in the health care sector in general. Consequently, software-based medical innovation can be translated into clinical routine quickly, efficiently, and cost-effectively, optimizing the treatment of patients through smartly assisted procedures.

Can we predict the severe course of COVID-19 - a systematic review and meta-analysis of indicators of clinical outcome?

BACKGROUND

COVID-19 has been reported in over 40million people globally with variable clinical outcomes. In this systematic review and meta-analysis, we assessed demographic, laboratory and clinical indicators as predictors for severe courses of COVID-19.

METHODS

This systematic review was registered at PROSPERO under CRD42020177154. We systematically searched multiple databases (PubMed, Web of Science Core Collection, MedRvix and bioRvix) for publications from December 2019 to May 31st 2020. Random-effects meta-analyses were used to calculate pooled odds ratios and differences of medians between (1) patients admitted to ICU versus non-ICU patients and (2) patients who died versus those who survived. We adapted an existing Cochrane risk-of-bias assessment tool for outcome studies.

RESULTS

Of 6,702 unique citations, we included 88 articles with 69,762 patients. There was concern for bias across all articles included. Age was strongly associated with mortality with a difference of medians (DoM) of 13.15 years (95% confidence interval (CI) 11.37 to 14.94) between those who died and those who survived. We found a clinically relevant difference between non-survivors and survivors for C-reactive protein (CRP; DoM 69.10 mg/L, CI 50.43 to 87.77), lactate dehydrogenase (LDH; DoM 189.49 U/L, CI 155.00 to 223.98), cardiac troponin I (cTnI; DoM 21.88 pg/mL, CI 9.78 to 33.99) and D-Dimer (DoM 1.29mg/L, CI 0.9 to 1.69). Furthermore, cerebrovascular disease was the co-morbidity most strongly associated with mortality (Odds Ratio 3.45, CI 2.42 to 4.91) and ICU admission (Odds Ratio 5.88, CI 2.35 to 14.73).

DISCUSSION

This comprehensive meta-analysis found age, cerebrovascular disease, CRP, LDH and cTnI to be the most important risk-factors that predict severe COVID-19 outcomes and will inform clinical scores to support early decision-making.

Can we predict the severe course of COVID-19 - a systematic review and meta-analysis of indicators of clinical outcome?

Background

COVID-19 has been reported in over 40million people globally with variable clinical outcomes. In this systematic review and meta-analysis, we assessed demographic, laboratory and clinical indicators as predictors for severe courses of COVID-19.

Methods

We systematically searched multiple databases (PubMed, Web of Science Core Collection, MedRvix and bioRvix) for publications from December 2019 to May 31st 2020. Random-effects meta-analyses were used to calculate pooled odds ratios and differences of medians between (1) patients admitted to ICU versus non-ICU patients and (2) patients who died versus those who survived. We adapted an existing Cochrane risk-of-bias assessment tool for outcome studies.

Results

Of 6,702 unique citations, we included 88 articles with 69,762 patients. There was concern for bias across all articles included. Age was strongly associated with mortality with a difference of medians (DoM) of 13.15 years (95% confidence interval (CI) 11.37 to 14.94) between those who died and those who survived. We found a clinically relevant difference between non-survivors and survivors for C-reactive protein (CRP; DoM 69.10, CI 50.43 to 87.77), lactate dehydrogenase (LDH; DoM 189.49, CI 155.00 to 223.98), cardiac troponin I (cTnI; DoM 21.88, CI 9.78 to 33.99) and D-Dimer (DoM 1.29mg/L, CI 0.9 - 1.69). Furthermore, cerebrovascular disease was the co-morbidity most strongly associated with mortality (Odds Ratio 3.45, CI 2.42 to 4.91) and ICU admission (Odds Ratio 5.88, CI 2.35 to 14.73).

Discussion

This comprehensive meta-analysis found age, cerebrovascular disease, CRP, LDH and cTnI to be the most important risk-factors in predicting severe COVID-19 outcomes and will inform decision analytical tools to support clinical decision-making.

[Surgery 4.0-are we ready?]

The increasing networking of data systems in medicine is not only leading to modern interdisciplinarity in the sense of cooperation between different medical departments, but also poses new challenges regarding the building and room infrastructure. The surgical operating room of the future expands or augments its reality, away from the pure building characteristics, towards an intelligent and communicative space platform. The building infrastructure (operating theatre) serves as sensor and actuator. Thus, it is possible to inform about missing diagnostics as well as to register them directly in the contextualization of the planned surgical intervention or to integrate them into the processes. Integrated operating theatres represent a comprehensive computer platform based on a corresponding system architecture with software-based protocols. An underlying modular system consisting of various modules for image acquisition and analysis, interaction and visualization supports the integration and merging of heterogeneous data that are generated in a hospital operation. Integral building data (e.g., air conditioning, lighting control, device registration) are merged with patient-related data (age, type of illness, concomitant diseases, existing diagnostic CT and MRI images). New systems coming onto the market, as well as already existing systems will have to be measured by the extent to which they will be able to guarantee this integration of information-similar to the development from mobile phone to smartphone. Cost reduction should not be the only legitimizing argument for the market launch, but the vision of a new quality of surgical perception and action.

Polhemus EM tracked Micro Sensor for CT-guided interventions

PURPOSE

Electromagnetic (EM) tracking is a key technology in image-guided therapy. A new EM Micro Sensor was presented by Polhemus Inc.; it is the first to enable localization of medical instruments through their trackers. Different field generators (FGs) are available by Polhemus, one being almost as small as a sugar cube. As accuracy and robustness of tracking are known challenges to using EM trackers in clinical environments, the goal of this study was a standardized assessment of the Micro Sensor in both a laboratory (lab) and a computed tomography (CT) environment.

METHODS

The Micro Sensor was assessed by means of Hummel et al.'s standardized protocol; it was assessed in conjunction with a Polhemus Liberty tracker and three FGs - with edge lengths of 1 (TX1), 2 (TX2), and 4 (TX4) inches. Precision as well as positional and rotational accuracy were determined in a lab and a CT suite. Distortions by four different metallic cylinders and tracking of two typical medical instruments - a hypodermic needle and a flexible endoscope - were also tested.

RESULTS

A jitter of 0.02 mm or less was found for all FGs in the different environments, except for the TX2 FG for which no valid data could be obtained in the CT. Errors of 5 cm distance measurements were 0.6 mm or less for all FGs in the lab. While the distance errors of the TX1 FG were only slightly increased up to 1.6 mm in the CT, those of the TX4 FG were found to be up to around 10% of the measured distance (5.4 mm on average). The mean orientation error was found to be 0.9^° /0.5^° /0.1^° for the TX4/TX2/TX1 FG in the lab. In the CT environment, rotation errors were in the same range: less than 1.2^° /0.1^° for the TX4/TX1 FG. Deviation under the presence of metallic cylinders stayed below 1 mm in most cases. Precision and orientational accuracy do not seem to be affected by instrument tracking and stayed in the same range as for the other measurements whereas distance errors were slightly increased up to 1.7 mm.

CONCLUSION

This study shows that accurate tracking of medical instruments is possible with the new Micro Sensor; it demonstrated a jitter of 0.01 mm or less, position errors below 2 mm, and rotation errors of less than 0.3^° . As with other EM trackers, errors increase when large tracking volumes with ranges of up to 50 cm are required in clinical environments. For smaller tracking volumes with ranges of up to 15 cm, a high accuracy and robustness was found. This is interesting especially for the TX1 FG which can easily be placed in close vicinity to the region of interest.

Clickstream Analysis for Crowd-Based Object Segmentation with Confidence

With the rapidly increasing interest in machine learning based solutions for automatic image annotation, the availability of reference annotations for algorithm training is one of the major bottlenecks in the field. Crowdsourcing has evolved as a valuable option for low-cost and large-scale data annotation; however, quality control remains a major issue which needs to be addressed. To our knowledge, we are the first to analyze the annotation process to improve crowd-sourced image segmentation. Our method involves training a regressor to estimate the quality of a segmentation from the annotator's clickstream data. The quality estimation can be used to identify spam and weight individual annotations by their (estimated) quality when merging multiple segmentations of one image. Using a total of 29,000 crowd annotations performed on publicly available data of different object classes, we show that (1) our method is highly accurate in estimating the segmentation quality based on clickstream data, (2) outperforms state-of-the-art methods for merging multiple annotations. As the regressor does not need to be trained on the object class that it is applied to it can be regarded as a low-cost option for quality control and confidence analysis in the context of crowd-based image annotation.

Large-scale medical image annotation with crowd-powered algorithms

Accurate segmentations in medical images are the foundations for various clinical applications. Advances in machine learning-based techniques show great potential for automatic image segmentation, but these techniques usually require a huge amount of accurately annotated reference segmentations for training. The guiding hypothesis of this paper was that crowd-algorithm collaboration could evolve as a key technique in large-scale medical data annotation. As an initial step toward this goal, we evaluated the performance of untrained individuals to detect and correct errors made by three-dimensional (3-D) medical segmentation algorithms. To this end, we developed a multistage segmentation pipeline incorporating a hybrid crowd-algorithm 3-D segmentation algorithm integrated into a medical imaging platform. In a pilot study of liver segmentation using a publicly available dataset of computed tomography scans, we show that the crowd is able to detect and refine inaccurate organ contours with a quality similar to that of experts (engineers with domain knowledge, medical students, and radiologists). Although the crowds need significantly more time for the annotation of a slice, the annotation rate is extremely high. This could render crowdsourcing a key tool for cost-effective large-scale medical image annotation.

Simultaneous Analysis of 2D Echo Views for Left Atrial Segmentation and Disease Detection

We propose a joint information approach for automatic analysis of 2D echocardiography (echo) data. The approach combines a priori images, their segmentations and patient diagnostic information within a unified framework to determine various clinical parameters, such as cardiac chamber volumes, and cardiac disease labels. The main idea behind the approach is to employ joint Independent Component Analysis of both echo image intensity information and corresponding segmentation labels to generate models that jointly describe the image and label space of echo patients on multiple apical views, instead of independently. These models are then both used for segmentation and volume estimation of cardiac chambers such as the left atrium and for detecting pathological abnormalities such as mitral regurgitation. We validate the approach on a large cohort of echoes obtained from 6,993 studies. We report performance of the proposed approach in estimation of the left-atrium volume and detection of mitral-regurgitation severity. A correlation coefficient of 0.87 was achieved for volume estimation of the left atrium when compared to the clinical report. Moreover, we classified patients that suffer from moderate or severe mitral regurgitation with an average accuracy of 82%.

Ultrasound-Guided Spine Anesthesia: Feasibility Study of a Guidance System

Spinal needle injections are guided by fluoroscopy or palpation, resulting in radiation exposure and/or multiple needle re-insertions. Consequently, guiding these procedures with live ultrasound has become more popular, but images are still challenging to interpret. We introduce a guidance system based on augmentation of ultrasound images with a patient-specific 3-D surface model of the lumbar spine. We assessed the feasibility of the system in a study on 12 patients. The system could accurately provide augmentations of the epidural space and the facet joint for all subjects. Following conventional, fluoroscopy-guided needle placement, augmentation accuracy was determined according to the electromagnetically tracked final position of the needle. In 9 of 12 cases, the accuracy was considered sufficient for successfully delivering anesthesia. The unsuccessful cases can be attributed to errors in the electromagnetic tracking reference, which can be avoided by a setup reducing the influence of the metal C-arm.

Automatic Segmentation of Wrist Bones in CT Using a Statistical Wrist Shape + Pose Model

Segmentation of the wrist bones in CT images has been frequently used in different clinical applications including arthritis evaluation, bone age assessment and image-guided interventions. The major challenges include non-uniformity and spongy textures of the bone tissue as well as narrow inter-bone spaces. In this work, we propose an automatic wrist bone segmentation technique for CT images based on a statistical model that captures the shape and pose variations of the wrist joint across 60 example wrists at nine different wrist positions. To establish the correspondences across the training shapes at neutral positions, the wrist bone surfaces are jointly aligned using a group-wise registration framework based on a Gaussian Mixture Model. Principal component analysis is then used to determine the major modes of shape variations. The variations in poses not only across the population but also across different wrist positions are incorporated in two pose models. An intra-subject pose model is developed by utilizing the similarity transforms at all wrist positions across the population. Further, an inter-subject pose model is used to model the pose variations across different wrist positions. For segmentation of the wrist bones in CT images, the developed model is registered to the edge point cloud extracted from the CT volume through an expectation maximization based probabilistic approach. Residual registration errors are corrected by application of a non-rigid registration technique. We validate the proposed segmentation method by registering the wrist model to a total of 66 unseen CT volumes of average voxel size of 0.38 mm. We report a mean surface distance error of 0.33 mm and a mean Jaccard index of 0.86.

Registration of a statistical model to intraoperative ultrasound for scaphoid screw fixation

PURPOSE

Volar percutaneous scaphoid fracture fixation is conventionally performed under fluoroscopy-based guidance, where surgeons need to mentally determine a trajectory for the insertion of the screw and its depth based on a series of 2D projection images. In addition to challenges associated with mapping 2D information to a 3D space, the process involves exposure to ionizing radiation. Three-dimensional ultrasound has been suggested as an alternative imaging tool for this procedure; however, it has not yet been integrated into clinical routine since ultrasound only provides a limited view of the scaphoid and its surrounding anatomy.

METHODS

We propose a registration of a statistical wrist shape + scale + pose model to a preoperative CT and intraoperative ultrasound to derive a patient-specific 3D model for guiding scaphoid fracture fixation. The registered model is then used to determine clinically important intervention parameters, including the screw length and the trajectory of screw insertion in the scaphoid bone.

RESULTS

Feasibility experiments are performed using 13 cadaver wrists. In 10 out of 13 cases, the trajectory of screw suggested by the registered model meets all clinically important intervention parameters. Overall, an average 94 % of maximum allowable screw length is obtained based on the measurements from gold standard CT. Also, we obtained an average 92 % successful volar accessibility, which indicates that the trajectory is not obstructed by the surrounding trapezium bone.

CONCLUSIONS

These promising results indicate that determining clinically important screw insertion parameters for scaphoid fracture fixation is feasible using 3D ultrasound imaging. This suggests the potential of this technology in replacing fluoroscopic guidance for this procedure in future applications.

A multi-center milestone study of clinical vertebral CT segmentation

A multiple center milestone study of clinical vertebra segmentation is presented in this paper. Vertebra segmentation is a fundamental step for spinal image analysis and intervention. The first half of the study was conducted in the spine segmentation challenge in 2014 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) Workshop on Computational Spine Imaging (CSI 2014). The objective was to evaluate the performance of several state-of-the-art vertebra segmentation algorithms on computed tomography (CT) scans using ten training and five testing dataset, all healthy cases; the second half of the study was conducted after the challenge, where additional 5 abnormal cases are used for testing to evaluate the performance under abnormal cases. Dice coefficients and absolute surface distances were used as evaluation metrics. Segmentation of each vertebra as a single geometric unit, as well as separate segmentation of vertebra substructures, was evaluated. Five teams participated in the comparative study. The top performers in the study achieved Dice coefficient of 0.93 in the upper thoracic, 0.95 in the lower thoracic and 0.96 in the lumbar spine for healthy cases, and 0.88 in the upper thoracic, 0.89 in the lower thoracic and 0.92 in the lumbar spine for osteoporotic and fractured cases. The strengths and weaknesses of each method as well as future suggestion for improvement are discussed. This is the first multi-center comparative study for vertebra segmentation methods, which will provide an up-to-date performance milestone for the fast growing spinal image analysis and intervention.

Lack of centrioles and primary cilia in STIL(-/-) mouse embryos

Although most animal cells contain centrosomes, consisting of a pair of centrioles, their precise contribution to cell division and embryonic development is unclear. Genetic ablation of STIL, an essential component of the centriole replication machinery in mammalian cells, causes embryonic lethality in mice around mid gestation associated with defective Hedgehog signaling. Here, we describe, by focused ion beam scanning electron microscopy, that STIL(-/-) mouse embryos do not contain centrioles or primary cilia, suggesting that these organelles are not essential for mammalian development until mid gestation. We further show that the lack of primary cilia explains the absence of Hedgehog signaling in STIL(-/-) cells. Exogenous re-expression of STIL or STIL microcephaly mutants compatible with human survival, induced non-templated, de novo generation of centrioles in STIL(-/-) cells. Thus, while the abscence of centrioles is compatible with mammalian gastrulation, lack of centrioles and primary cilia impairs Hedgehog signaling and further embryonic development.

Bone enhancement in ultrasound using local spectrum variations for guiding percutaneous scaphoid fracture fixation procedures

PURPOSE

The scaphoid bone is the most frequently fractured bone in the wrist. When fracture fixation is indicated, a screw is inserted into the bone either in an open surgical procedure or percutaneously under fluoroscopic guidance. Due to the complex geometry of the wrist, fracture fixation is a challenging task. Fluoroscopic guidance exposes both the patient and the physician to ionizing radiation. Ultrasound-based guidance has been suggested as a real-time, radiation-free alternative. The main challenge of using ultrasound is the difficulty in interpreting the images due to the low contrast and noisy nature of the data.

METHODS

We propose a bone enhancement method that exploits local spectrum features of the ultrasound image. These features are utilized to design a set of quadrature band-pass filters and subsequently estimate the local phase symmetry, where high symmetry is expected at the bone locations. We incorporate the shadow information below the bone surfaces to further enhance the bone responses. The extracted bone surfaces are then used to register a statistical wrist model to ultrasound volumes, allowing the localization and interpretation of the scaphoid bone in the volumes.

RESULTS

Feasibility experiments were performed using phantom and in vivo data. For phantoms, we obtain a surface distance error 1.08 mm and an angular deviation from the main axis of the scaphoid bone smaller than 5°, which are better compared to previously presented approaches.

CONCLUSION

The results are promising for further development of a surgical guidance system to enable accurate anatomy localization for guiding percutaneous scaphoid fracture fixations.

Standardized accuracy assessment of the calypso wireless transponder tracking system

Electromagnetic (EM) tracking allows localization of small EM sensors in a magnetic field of known geometry without line-of-sight. However, this technique requires a cable connection to the tracked object. A wireless alternative based on magnetic fields, referred to as transponder tracking, has been proposed by several authors. Although most of the transponder tracking systems are still in an early stage of development and not ready for clinical use yet, Varian Medical Systems Inc. (Palo Alto, California, USA) presented the Calypso system for tumor tracking in radiation therapy which includes transponder technology. But it has not been used for computer-assisted interventions (CAI) in general or been assessed for accuracy in a standardized manner, so far. In this study, we apply a standardized assessment protocol presented by Hummel et al (2005 Med. Phys. 32 2371-9) to the Calypso system for the first time. The results show that transponder tracking with the Calypso system provides a precision and accuracy below 1 mm in ideal clinical environments, which is comparable with other EM tracking systems. Similar to other systems the tracking accuracy was affected by metallic distortion, which led to errors of up to 3.2 mm. The potential of the wireless transponder tracking technology for use in many future CAI applications can be regarded as extremely high.

Comparative validation of single-shot optical techniques for laparoscopic 3-D surface reconstruction

Intra-operative imaging techniques for obtaining the shape and morphology of soft-tissue surfaces in vivo are a key enabling technology for advanced surgical systems. Different optical techniques for 3-D surface reconstruction in laparoscopy have been proposed, however, so far no quantitative and comparative validation has been performed. Furthermore, robustness of the methods to clinically important factors like smoke or bleeding has not yet been assessed. To address these issues, we have formed a joint international initiative with the aim of validating different state-of-the-art passive and active reconstruction methods in a comparative manner. In this comprehensive in vitro study, we investigated reconstruction accuracy using different organs with various shape and texture and also tested reconstruction robustness with respect to a number of factors like the pose of the endoscope as well as the amount of blood or smoke present in the scene. The study suggests complementary advantages of the different techniques with respect to accuracy, robustness, point density, hardware complexity and computation time. While reconstruction accuracy under ideal conditions was generally high, robustness is a remaining issue to be addressed. Future work should include sensor fusion and in vivo validation studies in a specific clinical context. To trigger further research in surface reconstruction, stereoscopic data of the study will be made publically available at www.open-CAS.com upon publication of the paper.

Navigated liver biopsy using a novel soft tissue navigation system versus CT-guided liver biopsy in a porcine model: a prospective randomized trial

RATIONALE AND OBJECTIVES

The aim of this prospective, randomized animal study was to compare a new computer guided needle-based navigation system for liver biopsy with conventional computed tomography (CT)-guided liver biopsy. Computer-navigated interventions provide continuous needle tracking during motion and deformation from patient respiration and movement.

MATERIALS AND METHODS

Twenty artificial tumors of about 5 mm in diameter were injected into the livers of five pigs, each at a different site. Each tumor was targeted by conventional CT-guided and computer navigated intervention. Intervention was considered complete after successful tumor biopsy. Data on procedure time, number of CT scans performed, accuracy, and success rate were recorded.

RESULTS

All tumors (100%) were biopsied successfully. Mean procedural time was comparable between the two techniques (20 ± 9 minutes conventional versus 20 ± 8 minutes navigation). Mean number of CT scans were 1.2 ± 0.4 with navigation and 6.1 ± 3.8 with the conventional technique (P < .01). The dose-length product in the conventional group was significantly higher (212 ± 116 mGy × cm) than in the navigated group (78 ± 22 mGy × cm; P < .001). Mean number of capsule penetrations was 4 ± 1 with navigation versus 2 ± 1 with the conventional technique (P < .001).

CONCLUSION

Computer-navigated liver biopsy may provide a promising and innovative device for easy, rapid, and successful liver biopsies with low morbidity. Further technical improvements and clinical studies in humans are required.

RepliExplore: coupling physical and virtual anatomy models

PURPOSE

We present a system which co-registers physical anatomy models with virtual three-dimensional (3D) representations. Interactions performed on the physical model by means of a 3D pointing device are directly reflected on its virtual counterpart. Complex anatomical information integrated into the virtual model thus becomes accessible through the physical interface in a simple and intuitive manner.

METHODS

Using an optical tracking system, we implemented and tested a reference application that includes several tools for the exploration and quantification of anatomical models. We theoretically evaluated the accuracy of the landmark-based registration for different landmark configurations.

RESULTS

Physicians and computer scientists found the system simple to learn and intuitive to use. By optimizing landmark configurations, the accuracy could be significantly increased, particularly for scenarios in which only selected regions required higher accuracy.

CONCLUSIONS

Physical anatomical models can benefit from the combination with a virtual counterpart in several ways. Applications include anatomical education and the study of patient-individual organ models. Optimizing the registration landmark configuration for specific applications can lower the accuracy requirements for the tracking system.

Towards a mixed reality environment for preoperative planning of cardiac surgery

We present a novel approach to studying physical heart models by coupling them with virtual 3D representations in a mixed reality environment. The limitations of standalone physical models (non-interactive, static) are overcome by the corresponding virtual models, which in turn become more natural to interact with. The potential of this approach is exemplified by a setup which enables cardiac surgeons to interactively trace the mitral annulus, a part of the cardiac skeleton playing a vital role in mitral valve surgery. We present results of a pilot study and discuss ways of improving and extending the system. The described mixed reality environment could easily be adapted to other fields and thus has the potential to become a new tool for investigating 3D medical data.

Precision targeting of liver lesions with a needle-based soft tissue navigation system

In this study, we assessed the targeting precision of a previously reported needle-based soft tissue navigation system. For this purpose, we implanted 10 2-ml agar nodules into three pig livers as tumor models, and two of the authors used the navigation system to target the center of gravity of each nodule. In order to obtain a realistic setting, we mounted the livers onto a respiratory liver motion simulator that models the human body. For each targeting procedure, we simulated the liver biopsy workflow, consisting of four steps: preparation, trajectory planning, registration, and navigation. The lesions were successfully hit in all 20 trials. The final distance between the applicator tip and the center of gravity of the lesion was determined from control computed tomography (CT) scans and was 3.5 +/- 1.1 mm on average. Robust targeting precision of this order of magnitude would significantly improve the clinical treatment standard for various CT-guided minimally invasive interventions in the liver.