Voice-Assisted Image Labeling for Endoscopic Ultrasound Classification Using Neural Networks

Ultrasound imaging is a commonly used technology for visualising patient anatomy in real-time during diagnostic and therapeutic procedures. High operator dependency and low reproducibility make ultrasound imaging and interpretation challenging with a steep learning curve. Automatic image classification using deep learning has the potential to overcome some of these challenges by supporting ultrasound training in novices, as well as aiding ultrasound image interpretation in patient with complex pathology for more experienced practitioners. However, the use of deep learning methods requires a large amount of data in order to provide accurate results. Labelling large ultrasound datasets is a challenging task because labels are retrospectively assigned to 2D images without the 3D spatial context available in vivo or that would be inferred while visually tracking structures between frames during the procedure. In this work, we propose a multi-modal convolutional neural network (CNN) architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. We use a CNN composed of two branches, one for voice data and another for image data, which are joined to predict image labels from the spoken names of anatomical landmarks. The network was trained using recorded verbal comments from expert operators. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels. We conclude that the addition of spoken commentaries can increase the performance of ultrasound image classification, and eliminate the burden of manually labelling large EUS datasets necessary for deep learning applications.

SciKit-SurgeryGlenoid, an Open Source Toolkit for Glenoid Version Measurement

Correct understanding of the geometry of the glenoid (the socket of the shoulder joint) is key to successful planning of shoulder replacement surgery. This surgery typically involves placing an implant in the shoulder joint to restore joint function. The most relevant geometry is the glenoid version, which is the angular orientation of the glenoid surface relative to the long axis of the scapula in the axial plane. However, measuring the glenoid version is not straightforward and there are multiple measurement methods in the literature and used in commercial planning software. In this paper we introduce SciKit-SurgeryGlenoid, an open source toolkit for the measurement of glenoid version. SciKit-SurgeryGlenoid contains implementations of the 4 most frequently used glenoid version measurement algorithms enabling easy and unbiased comparison of the different techniques. We present the results of using the software on 10 sets of pre-operative CT scans taken from patients who have subsequently undergone shoulder replacement surgery. We further compare these results with those obtained from a commercial implant planning software. SciKit-SurgeryGlenoid currently requires manual segmentation of the relevant anatomical features for each method. Future work will look at automating the segmentation process to build an automatic and repeatable pipeline from CT or radiograph to quantitative glenoid version measurement.

Deep hashing for global registration of untracked 2D laparoscopic ultrasound to CT

PURPOSE

The registration of Laparoscopic Ultrasound (LUS) to CT can enhance the safety of laparoscopic liver surgery by providing the surgeon with awareness on the relative positioning between critical vessels and a tumour. In an effort to provide a translatable solution for this poorly constrained problem, Content-based Image Retrieval (CBIR) based on vessel information has been suggested as a method for obtaining a global coarse registration without using tracking information. However, the performance of these frameworks is limited by the use of non-generalisable handcrafted vessel features.

METHODS

We propose the use of a Deep Hashing (DH) network to directly convert vessel images from both LUS and CT into fixed size hash codes. During training, these codes are learnt from a patient-specific CT scan by supplying the network with triplets of vessel images which include both a registered and a mis-registered pair. Once hash codes have been learnt, they can be used to perform registration with CBIR methods.

RESULTS

We test a CBIR pipeline on 11 sequences of untracked LUS distributed across 5 clinical cases. Compared to a handcrafted feature approach, our model improves the registration success rate significantly from 48% to 61%, considering a 20 mm error as the threshold for a successful coarse registration.

CONCLUSIONS

We present the first DH framework for interventional multi-modal registration tasks. The presented approach is easily generalisable to other registration problems, does not require annotated data for training, and may promote the translation of these techniques.

Large scale simulation of labeled intraoperative scenes in unity

PURPOSE

The use of synthetic or simulated data has the potential to greatly improve the availability and volume of training data for image guided surgery and other medical applications, where access to real-life training data is limited.

METHODS

By using the Unity game engine, complex intraoperative scenes can be simulated. The Unity Perception package allows for randomisation of paremeters within the scene, and automatic labelling, to make simulating large data sets a trivial operation. In this work, the approach has been prototyped for liver segmentation from laparoscopic video images. 50,000 simulated images were used to train a U-Net, without the need for any manual labelling. The use of simulated data was compared against a model trained with 950 manually labelled laparoscopic images.

RESULTS

When evaluated on data from 10 separate patients, synthetic data outperformed real data in 4 out of 10 cases. Average DICE scores across the 10 cases were 0.59 (synthetic data), 0.64 (real data) and 0.75 (both synthetic and real data).

CONCLUSION

Synthetic data generated using this method is able to make valid inferences on real data, with average performance slightly below models trained on real data. The use of the simulated data for pre-training boosts model performance, when compared with training on real data only.

Probabilistic landscape of seizure semiology localizing values

Semiology describes the evolution of symptoms and signs during epileptic seizures and contributes to the evaluation of individuals with focal drug-resistant epilepsy for curative resection. Semiology varies in complexity from elementary sensorimotor seizures arising from primary cortex to complex behaviours and automatisms emerging from distributed cerebral networks. Detailed semiology interpreted by expert epileptologists may point towards the likely site of seizure onset, but this process is subjective. No study has captured the variances in semiological localizing values in a data-driven manner to allow objective and probabilistic determinations of implicated networks and nodes. We curated an open data set from the epilepsy literature, in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, linking semiology to hierarchical brain localizations. A total of 11 230 data points were collected from 4643 patients across 309 articles, labelled using ground truths (postoperative seizure-freedom, concordance of imaging and neurophysiology, and/or invasive EEG) and a designation method that distinguished between semiologies arising from a predefined cortical region and descriptions of neuroanatomical localizations responsible for generating a particular semiology. This allowed us to mitigate temporal lobe publication bias by filtering studies that preselected patients based on prior knowledge of their seizure foci. Using this data set, we describe the probabilistic landscape of semiological localizing values as forest plots at the resolution of seven major brain regions: temporal, frontal, cingulate, parietal, occipital, insula, and hypothalamus, and five temporal subregions. We evaluated the intrinsic value of any one semiology over all other ictal manifestations. For example, epigastric auras implicated the temporal lobe with 83% probability when not accounting for the publication bias that favoured temporal lobe epilepsies. Unbiased results for a prior distribution of cortical localizations revised the prevalence of temporal lobe epilepsies from 66% to 44%. Therefore, knowledge about the presence of epigastric auras updates localization to the temporal lobe with an odds ratio (OR) of 2.4 [CI_95% (1.9, 2.9); and specifically, mesial temporal structures OR: 2.8 (2.3, 2.9)], attesting the value of epigastric auras. As a further example, although head version is thought to implicate the frontal lobes, it did not add localizing value compared with the prior distribution of cortical localizations [OR: 0.9 (0.7, 1.2)]. Objectification of the localizing values of the 12 most common semiologies provides a complementary view of brain dysfunction to that of lesion-deficit mappings, as instead of linking brain regions to phenotypic-deficits, semiological phenotypes are linked back to brain sources. This work enables coupling of seizure propagation with ictal manifestations, and clinical support algorithms for localizing seizure phenotypes.

Radiopaque drug-eluting embolisation beads as fiducial markers for stereotactic liver radiotherapy

Objective:

To determine the feasibility of using radiopaque (RO) beads as direct tumour surrogates for image-guided radiotherapy (IGRT) in patients with liver tumours after transarterial chemoembolisation (TACE).

Methods:

A novel vandetanib-eluting RO bead was delivered via TACE as part of a first-in-human clinical trial in patients with either hepatocellular carcinoma or liver metastases from colorectal cancer. Following TACE, patients underwent simulated radiotherapy imaging with four-dimensional computed tomography (4D-CT) and cone-beam CT (CBCT) imaging. RO beads were contoured using automated thresholding, and feasibility of matching between the simulated radiotherapy planning dataset (AVE-IP image from 4D data) and CBCT scans assessed. Additional kV, MV, helical CT and CBCT images of RO beads were obtained using an in-house phantom. Stability of RO bead position was assessed by comparing 4D-CT imaging to CT scans taken 6–20 days following TACE.

Results:

Eight patients were treated and 4D-CT and CBCT images acquired. RO beads were visible on 4D-CT and CBCT images in all cases and matching successfully performed. Differences in centre of mass of RO beads between CBCT and simulated radiotherapy planning scans (AVE-IP dataset) were 2.0 mm mediolaterally, 1.7 mm anteroposteriorally and 3.5 mm craniocaudally. RO beads in the phantom were visible on all imaging modalities assessed. RO bead position remained stable up to 29 days post TACE.

Conclusion:

RO beads are visible on IGRT imaging modalities, showing minimal artefact. They can be used for on-set matching with CBCT and remain stable over time.

Advances in knowledge:

The role of RO beads as fiducial markers for stereotactic liver radiotherapy is feasible and warrants further exploration as a combination therapy approach.

Zero-Shot Super-Resolution With a Physically-Motivated Downsampling Kernel for Endomicroscopy

Super-resolution (SR) methods have seen significant advances thanks to the development of convolutional neural networks (CNNs). CNNs have been successfully employed to improve the quality of endomicroscopy imaging. Yet, the inherent limitation of research on SR in endomicroscopy remains the lack of ground truth high-resolution (HR) images, commonly used for both supervised training and reference-based image quality assessment (IQA). Therefore, alternative methods, such as unsupervised SR are being explored. To address the need for non-reference image quality improvement, we designed a novel zero-shot super-resolution (ZSSR) approach that relies only on the endomicroscopy data to be processed in a self-supervised manner without the need for ground-truth HR images. We tailored the proposed pipeline to the idiosyncrasies of endomicroscopy by introducing both: a physically-motivated Voronoi downscaling kernel accounting for the endomicroscope's irregular fibre-based sampling pattern, and realistic noise patterns. We also took advantage of video sequences to exploit a sequence of images for self-supervised zero-shot image quality improvement. We run ablation studies to assess our contribution in regards to the downscaling kernel and noise simulation. We validate our methodology on both synthetic and original data. Synthetic experiments were assessed with reference-based IQA, while our results for original images were evaluated in a user study conducted with both expert and non-expert observers. The results demonstrated superior performance in image quality of ZSSR reconstructions in comparison to the baseline method. The ZSSR is also competitive when compared to supervised single-image SR, especially being the preferred reconstruction technique by experts.

Vessel segmentation for automatic registration of untracked laparoscopic ultrasound to CT of the liver

Purpose:

Registration of Laparoscopic Ultrasound (LUS) to a pre-operative scan such as Computed Tomography (CT) using blood vessel information has been proposed as a method to enable image-guidance for laparoscopic liver resection. Currently, there are solutions for this problem that can potentially enable clinical translation by bypassing the need for a manual initialisation and tracking information. However, no reliable framework for the segmentation of vessels in 2D untracked LUS images has been presented.

Methods:

We propose the use of 2D UNet for the segmentation of liver vessels in 2D LUS images. We integrate these results in a previously developed registration method, and show the feasibility of a fully automatic initialisation to the LUS to CT registration problem without a tracking device.

Results:

We validate our segmentation using LUS data from 6 patients. We test multiple models by placing patient datasets into different combinations of training, testing and hold-out, and obtain mean Dice scores ranging from 0.543 to 0.706. Using these segmentations, we obtain registration accuracies between 6.3 and 16.6 mm in 50% of cases.

Conclusions:

We demonstrate the first instance of deep learning (DL) for the segmentation of liver vessels in LUS. Our results show the feasibility of UNet in detecting multiple vessel instances in 2D LUS images, and potentially automating a LUS to CT registration pipeline.

Gesture Recognition in Robotic Surgery: A Review

OBJECTIVE

Surgical activity recognition is a fundamental step in computer-assisted interventions. This paper reviews the state-of-the-art in methods for automatic recognition of fine-grained gestures in robotic surgery focusing on recent data-driven approaches and outlines the open questions and future research directions.

METHODS

An article search was performed on 5 bibliographic databases with the following search terms: robotic, robot-assisted, JIGSAWS, surgery, surgical, gesture, fine-grained, surgeme, action, trajectory, segmentation, recognition, parsing. Selected articles were classified based on the level of supervision required for training and divided into different groups representing major frameworks for time series analysis and data modelling.

RESULTS

A total of 52 articles were reviewed. The research field is showing rapid expansion, with the majority of articles published in the last 4 years. Deep-learning-based temporal models with discriminative feature extraction and multi-modal data integration have demonstrated promising results on small surgical datasets. Currently, unsupervised methods perform significantly less well than the supervised approaches.

CONCLUSION

The development of large and diverse open-source datasets of annotated demonstrations is essential for development and validation of robust solutions for surgical gesture recognition. While new strategies for discriminative feature extraction and knowledge transfer, or unsupervised and semi-supervised approaches, can mitigate the need for data and labels, they have not yet been demonstrated to achieve comparable performance. Important future research directions include detection and forecast of gesture-specific errors and anomalies.

SIGNIFICANCE

This paper is a comprehensive and structured analysis of surgical gesture recognition methods aiming to summarize the status of this rapidly evolving field.

Integrated multi-modality image-guided navigation for neurosurgery: open-source software platform using state-of-the-art clinical hardware

PURPOSE

Image-guided surgery (IGS) is an integral part of modern neuro-oncology surgery. Navigated ultrasound provides the surgeon with reconstructed views of ultrasound data, but no commercial system presently permits its integration with other essential non-imaging-based intraoperative monitoring modalities such as intraoperative neuromonitoring. Such a system would be particularly useful in skull base neurosurgery.

METHODS

We established functional and technical requirements of an integrated multi-modality IGS system tailored for skull base surgery with the ability to incorporate: (1) preoperative MRI data and associated 3D volume reconstructions, (2) real-time intraoperative neurophysiological data and (3) live reconstructed 3D ultrasound. We created an open-source software platform to integrate with readily available commercial hardware. We tested the accuracy of the system's ultrasound navigation and reconstruction using a polyvinyl alcohol phantom model and simulated the use of the complete navigation system in a clinical operating room using a patient-specific phantom model.

RESULTS

Experimental validation of the system's navigated ultrasound component demonstrated accuracy of [Formula: see text] and a frame rate of 25 frames per second. Clinical simulation confirmed that system assembly was straightforward, could be achieved in a clinically acceptable time of [Formula: see text] and performed with a clinically acceptable level of accuracy.

CONCLUSION

We present an integrated open-source research platform for multi-modality IGS. The present prototype system was tailored for neurosurgery and met all minimum design requirements focused on skull base surgery. Future work aims to optimise the system further by addressing the remaining target requirements.

Registration of Untracked 2D Laparoscopic Ultrasound to CT Images of the Liver Using Multi-Labelled Content-Based Image Retrieval

Laparoscopic Ultrasound (LUS) is recommended as a standard-of-care when performing laparoscopic liver resections as it images sub-surface structures such as tumours and major vessels. Given that LUS probes are difficult to handle and some tumours are iso-echoic, registration of LUS images to a pre-operative CT has been proposed as an image-guidance method. This registration problem is particularly challenging due to the small field of view of LUS, and usually depends on both a manual initialisation and tracking to compose a volume, hindering clinical translation. In this paper, we extend a proposed registration approach using Content-Based Image Retrieval (CBIR), removing the requirement for tracking or manual initialisation. Pre-operatively, a set of possible LUS planes is simulated from CT and a descriptor generated for each image. Then, a Bayesian framework is employed to estimate the most likely sequence of CT simulations that matches a series of LUS images. We extend our CBIR formulation to use multiple labelled objects and constrain the registration by separating liver vessels into portal vein and hepatic vein branches. The value of this new labeled approach is demonstrated in retrospective data from 5 patients. Results show that, by including a series of 5 untracked images in time, a single LUS image can be registered with accuracies ranging from 5.7 to 16.4 mm with a success rate of 78%. Initialisation of the LUS to CT registration with the proposed framework could potentially enable the clinical translation of these image fusion techniques.

Are fiducial registration error and target registration error correlated? SciKit-SurgeryFRED for teaching and research

Understanding the relationship between fiducial registration error (FRE) and target registration error (TRE) is important for the correct use of interventional guidance systems. Whilst it is well established that TRE is statistically independent of FRE, system users still struggle against the intuitive assumption that a low FRE indicates a low TRE. We present the SciKit-Surgery Fiducial Registration Educational Demonstrator and describe its use. SciKit-SurgeryFRED was developed to enable remote teaching of key concepts in image registration. SciKit-SurgeryFRED also supports research into user interface design for image registration systems. SciKit-SurgeryFRED can be used to enable remote tutorials covering the statistics relevant to image guided interventions. Students are able to place fiducial markers on pre and intra-operative images and observe the effects of changes in marker geometry, marker count, and fiducial localisation error on TRE and FRE. SciKit-SurgeryFRED also calculates statistical measures for the expected values of TRE and FRE. Because many registrations can be performed quickly the students can then explore potential correlations between the different statistics. SciKit-SurgeryFRED also implements a registration based game, where participants are rewarded for complete treatment of a clinical target, whilst minimising the treatment margin. We used this game to perform a remote study on registration and simulated ablation, measuring how user performance changes depending on what error statistics are made available. The results support the assumption that knowing the exact value of target registration error leads to better treatment. Display of other statistics did not have a significant impact on the treatment performance.

Machine Learning for Localizing Epileptogenic-Zone in the Temporal Lobe: Quantifying the Value of Multimodal Clinical-Semiology and Imaging Concordance

Background: Epilepsy affects 50 million people worldwide and a third are refractory to medication. If a discrete cerebral focus or network can be identified, neurosurgical resection can be curative. Most excisions are in the temporal-lobe, and are more likely to result in seizure-freedom than extra-temporal resections. However, less than half of patients undergoing surgery become entirely seizure-free. Localizing the epileptogenic-zone and individualized outcome predictions are difficult, requiring detailed evaluations at specialist centers. Methods: We used bespoke natural language processing to text-mine 3,800 electronic health records, from 309 epilepsy surgery patients, evaluated over a decade, of whom 126 remained entirely seizure-free. We investigated the diagnostic performances of machine learning models using set-of-semiology (SoS) with and without hippocampal sclerosis (HS) on MRI as features, using STARD criteria. Findings: Support Vector Classifiers (SVC) and Gradient Boosted (GB) decision trees were the best performing algorithms for temporal-lobe epileptogenic zone localization (cross-validated Matthews correlation coefficient (MCC) SVC 0.73 ± 0.25, balanced accuracy 0.81 ± 0.14, AUC 0.95 ± 0.05). Models that only used seizure semiology were not always better than internal benchmarks. The combination of multimodal features, however, enhanced performance metrics including MCC and normalized mutual information (NMI) compared to either alone (p < 0.0001). This combination of semiology and HS on MRI increased both cross-validated MCC and NMI by over 25% (NMI, SVC SoS: 0.35 ± 0.28 vs. SVC SoS+HS: 0.61 ± 0.27). Interpretation: Machine learning models using only the set of seizure semiology (SoS) cannot unequivocally perform better than benchmarks in temporal epileptogenic-zone localization. However, the combination of SoS with an imaging feature (HS) enhance epileptogenic lobe localization. We quantified this added NMI value to be 25% in absolute terms. Despite good performance in localization, no model was able to predict seizure-freedom better than benchmarks. The methods used are widely applicable, and the performance enhancements by combining other clinical, imaging and neurophysiological features could be similarly quantified. Multicenter studies are required to confirm generalizability. Funding: Wellcome/EPSRC Center for Interventional and Surgical Sciences (WEISS) (203145Z/16/Z).

An Enhanced Visualization of DBT Imaging Using Blind Deconvolution and Total Variation Minimization Regularization

Digital Breast Tomosynthesis (DBT) presents out-of-plane artifacts caused by features of high intensity. Given observed data and knowledge about the point spread function (PSF), deconvolution techniques recover data from a blurred version. However, a correct PSF is difficult to achieve and these methods amplify noise. When no information is available about the PSF, blind deconvolution can be used. Additionally, Total Variation (TV) minimization algorithms have achieved great success due to its virtue of preserving edges while reducing image noise. This work presents a novel approach in DBT through the study of out-of-plane artifacts using blind deconvolution and noise regularization based on TV minimization. Gradient information was also included. The methodology was tested using real phantom data and one clinical data set. The results were investigated using conventional 2D slice-by-slice visualization and 3D volume rendering. For the 2D analysis, the artifact spread function (ASF) and Full Width at Half Maximum (FWHMM_ASF) of the ASF were considered. The 3D quantitative analysis was based on the FWHM of disks profiles at 90°, noise and signal to noise ratio (SNR) at 0° and 90°. A marked visual decrease of the artifact with reductions of FWHM_ASF (2D) and FWHM_90° (volume rendering) of 23.8% and 23.6%, respectively, was observed. Although there was an expected increase in noise level, SNR values were preserved after deconvolution. Regardless of the methodology and visualization approach, the objective of reducing the out-of-plane artifact was accomplished. Both for the phantom and clinical case, the artifact reduction in the z was markedly visible.

Impact of total variation minimization in volume rendering visualization of breast tomosynthesis data

BACKGROUND AND OBJECTIVE

Total Variation (TV) minimization algorithms have achieved great attention due to the virtue of decreasing noise while preserving edges. The purpose of this work is to implement and evaluate two TV minimization methods in 3D. Their performance is analyzed through 3D visualization of digital breast tomosynthesis (DBT) data with volume rendering.

METHODS

Both filters were studied with real phantom and one clinical DBT data. One algorithm was applied sequentially to all slices and the other was applied to the entire volume at once. The suitable Lagrange multiplier used in each filter equation was studied to reach the minimum 3D TV and the maximum contrast-to-noise ratio (CNR). Imaging blur was measured at 0° and 90° using two disks with different diameters (0.5 mm and 5.0 mm) and equal thickness. The quality of unfiltered and filtered data was analyzed with volume rendering at 0° and 90°.

RESULTS

For phantom data, with the sequential filter, a decrease of 25% in 3D TV value and an increase of 19% and 30% in CNR at 0° and 90°, respectively, were observed. When the filter is applied directly in 3D, TV value was reduced by 35% and an increase of 36% was achieved both for CNR at 0° and 90°. For the smaller disk, variations of 0% in width at half maximum (FWHM) at 0° and a decrease of about 2.5% for FWHM at 90° were observed for both filters. For the larger disk, there was a 2.5% increase in FWHM at 0° for both filters and a decrease of 6.28% and 1.69% in FWHM at 90° with the sequential filter and the 3D filter, respectively. When applied to clinical data, the performance of each filter was consistent with that obtained with the phantom.

CONCLUSIONS

Data analysis confirmed the relevance of these methods in improving quality of DBT images. Additionally, this type of 3D visualization showed that it may play an important complementary role in DBT imaging. It allows to visualize all DBT data at once and to analyze properly filters applied to all the three dimensions. Concise Abstract Total Variation (TV) minimization algorithms are one compressed sensing technique that has achieved great attention due to the virtue of decrease noise while preserve edges transitions. The purpose of this work is to solve the same TV minimization problem in DBT data, by studying two 3D filters. The obtained results were analyzed at 0° and 90° with a 3D visualization through volume rendering. The filters differ in their application. One considers a slice-by-slice optimization, sequentially traversing all slices of the data. The other considers the intensity values of adjacent slices to make this optimization on each voxel. The performance of each filter was also tested with a clinical case. The results obtained were very encouraging with a significantly increased contrast to noise ratio at 0° and 90° and a small reduction in blur at 90° (slight reduction of the out-of-plane artifact).

Optimization of Breast Tomosynthesis Visualization through 3D Volume Rendering

3D volume rendering may represent a complementary option in the visualization of Digital Breast Tomosynthesis (DBT) examinations by providing an understanding of the underlying data at once. Rendering parameters directly influence the quality of rendered images. The purpose of this work is to study the influence of two of these parameters (voxel dimension in z direction and sampling distance) on DBT rendered data. Both parameters were studied with a real phantom and one clinical DBT data set. The voxel size was changed from 0.085 × 0.085 × 1.0 mm³ to 0.085 × 0.085 × 0.085 mm³ using ten interpolation functions available in the Visualization Toolkit library (VTK) and several sampling distance values were evaluated. The results were investigated at 90º using volume rendering visualization with composite technique. For phantom quantitative analysis, degree of smoothness, contrast-to-noise ratio, and full width at half maximum of a Gaussian curve fitted to the profile of one disk were used. Additionally, the time required for each visualization was also recorded. Hamming interpolation function presented the best compromise in image quality. The sampling distance values that showed a better balance between time and image quality were 0.025 mm and 0.05 mm. With the appropriate rendering parameters, a significant improvement in rendered images was achieved.

SciKit-Surgery: compact libraries for surgical navigation

Purpose

This paper introduces the SciKit-Surgery libraries, designed to enable rapid development of clinical applications for image-guided interventions. SciKit-Surgery implements a family of compact, orthogonal, libraries accompanied by robust testing, documentation, and quality control. SciKit-Surgery libraries can be rapidly assembled into testable clinical applications and subsequently translated to production software without the need for software reimplementation. The aim is to support translation from single surgeon trials to multicentre trials in under 2 years.

Methods

At the time of publication, there were 13 SciKit-Surgery libraries provide functionality for visualisation and augmented reality in surgery, together with hardware interfaces for video, tracking, and ultrasound sources. The libraries are stand-alone, open source, and provide Python interfaces. This design approach enables fast development of robust applications and subsequent translation. The paper compares the libraries with existing platforms and uses two example applications to show how SciKit-Surgery libraries can be used in practice.

Results

Using the number of lines of code and the occurrence of cross-dependencies as proxy measurements of code complexity, two example applications using SciKit-Surgery libraries are analysed. The SciKit-Surgery libraries demonstrate ability to support rapid development of testable clinical applications. By maintaining stricter orthogonality between libraries, the number, and complexity of dependencies can be reduced. The SciKit-Surgery libraries also demonstrate the potential to support wider dissemination of novel research.

Conclusion

The SciKit-Surgery libraries utilise the modularity of the Python language and the standard data types of the NumPy package to provide an easy-to-use, well-tested, and extensible set of tools for the development of applications for image-guided interventions. The example application built on SciKit-Surgery has a simpler dependency structure than the same application built using a monolithic platform, making ongoing clinical translation more feasible.

Learning from irregularly sampled data for endomicroscopy super-resolution: a comparative study of sparse and dense approaches

Purpose

Probe-based confocal laser endomicroscopy (pCLE) enables performing an optical biopsy via a probe. pCLE probes consist of multiple optical fibres arranged in a bundle, which taken together generate signals in an irregularly sampled pattern. Current pCLE reconstruction is based on interpolating irregular signals onto an over-sampled Cartesian grid, using a naive linear interpolation. It was shown that convolutional neural networks (CNNs) could improve pCLE image quality. Yet classical CNNs may be suboptimal in regard to irregular data.

Methods

We compare pCLE reconstruction and super-resolution (SR) methods taking irregularly sampled or reconstructed pCLE images as input. We also propose to embed a Nadaraya–Watson (NW) kernel regression into the CNN framework as a novel trainable CNN layer. We design deep learning architectures allowing for reconstructing high-quality pCLE images directly from the irregularly sampled input data. We created synthetic sparse pCLE images to evaluate our methodology.

Results

The results were validated through an image quality assessment based on a combination of the following metrics: peak signal-to-noise ratio and the structural similarity index. Our analysis indicates that both dense and sparse CNNs outperform the reconstruction method currently used in the clinic.

Conclusion

The main contributions of our study are a comparison of sparse and dense approach in pCLE image reconstruction. We also implement trainable generalised NW kernel regression as a novel sparse approach. We also generated synthetic data for training pCLE SR.

Electronic supplementary material

The online version of this article (10.1007/s11548-020-02170-7) contains supplementary material, which is available to authorized users.

Is there rationale for the cuff pressures prescribed for blood flow restriction exercise? A systematic review

BACKGROUND

Blood flow restriction exercise has increasingly broad applications among healthy and clinical populations. Ensuring the technique is applied in a safe, controlled, and beneficial way for target populations is essential. Individualized cuff pressures are a favored method for achieving this. However, there remains marked inconsistency in how individualized cuff pressures are applied.

OBJECTIVES

To quantify the cuff pressures used in the broader blood flow restriction exercise literature, and determine whether there is clear justification for the choice of pressure prescribed.

METHODS

Studies were included in this review from database searches if they employed an experimental design using original data, involved either acute or chronic exercise using blood flow restriction, and they assessed limb or arterial occlusion pressure to determine an individualized cuff pressure. Methodologies of the studies were evaluated using a bespoke quality assessment tool.

RESULTS

Fifty-one studies met the inclusion criteria. Individualized cuff pressures ranged from 30% to 100% arterial occlusion pressure. Only 7 out of 52 studies attempted to justify the individualized cuff pressure applied during exercise. The mean quality rating for all studies was 11.1 ± 1.2 out of 13.

CONCLUSIONS

The broader blood flow restriction exercise literature uses markedly heterogeneous prescription variables despite using individualized cuff pressures. This is problematic in the absence of any clear justification for the individualized cuff pressures selected. Systematically measuring and reporting all relevant acute responses and training adaptations to the full spectrum of BFR pressures alongside increased clarity around the methodology used during blood flow restriction exercise is paramount.

SnappySonic: An Ultrasound Acquisition Replay Simulator

SnappySonic provides an ultrasound acquisition replay simulator designed for public engagement and training. It provides a simple interface to allow users to experience ultrasound acquisition without the need for specialist hardware or acoustically compatible phantoms. The software is implemented in Python, built on top of a set of open source Python modules targeted at surgical innovation. The library has high potential for reuse, most obviously for those who want to simulate ultrasound acquisition, but it could also be used as a user interface for displaying high dimensional images or video data.

Hemodynamic and perceptual responses to blood flow-restricted exercise among patients undergoing dialysis

End-stage kidney disease is associated with reduced exercise capacity, muscle atrophy, and impaired muscle function. While these may be improved with exercise, single modalities of exercise do not traditionally elicit improvements across all required physiological domains. Blood flow-restricted exercise may improve all of these physiological domains with low intensities traditionally considered insufficient for these adaptions. Investigation of this technique appeals, but is yet to be evaluated, in patients undergoing dialysis. With the use of a progressive crossover design, 10 satellite patients undergoing hemodialysis underwent three exercise conditions over 2 wk: two bouts (10 min) of unrestricted cycling during two consecutive hemodialysis sessions (condition 1), two bouts of cycling with blood flow restriction while off hemodialysis on 2 separate days (condition 2), and two bouts of cycling with blood flow restriction during two hemodialysis sessions (condition 3). Outcomes included hemodynamic responses (heart rate and blood pressure) throughout all sessions, participant-perceived exertion and discomfort on a Borg scale, and evaluation of ultrafiltration rates and dialysis adequacy (Kt/V) obtained post hoc. Hemodynamic responses were consistent regardless of condition. Significant increases in heart rate, systolic blood pressure, and mean arterial blood pressure (P < 0.05) were observed postexercise followed by a reduction in blood pressures during the 60-min recovery (12, 5, and 11 mmHg for systolic, diastolic, and mean arterial pressures, respectively). Blood pressures returned to predialysis ranges following the recovery period. Blood flow restriction did not affect ultrafiltration achieved or Kt/V. Hemodynamic safety and tolerability of blood flow restriction during aerobic exercise on hemodialysis is comparable to standard aerobic exercise.

Spectral Imaging Of Thermal Damage Induced During Microwave Ablation In The Liver

Induction of thermal damage to tissue through delivery of microwave energy is frequently applied in surgery to destroy diseased tissue such as cancer cells. Minimization of unwanted harm to healthy tissue is still achieved subjectively, and the surgeon has few tools at their disposal to monitor the spread of the induced damage. This work describes the use of optical methods to monitor the time course of changes to the tissue during delivery of microwave energy in the porcine liver. Multispectral imaging and diffuse reflectance spectroscopy are used to monitor temporal changes in optical properties in parallel with thermal imaging. The results demonstrate the ability to monitor the spatial extent of thermal damage on a whole organ, including possible secondary effects due to vascular damage. Future applications of this type of imaging may see the multispectral data used as a feedback mechanism to avoid collateral damage to critical healthy structures and to potentially verify sufficient application of energy to the diseased tissue.

VEROnA Protocol: A Pilot, Open-Label, Single-Arm, Phase 0, Window-of-Opportunity Study of Vandetanib-Eluting Radiopaque Embolic Beads (BTG-002814) in Patients With Resectable Liver Malignancies

BACKGROUND

Transarterial chemoembolization (TACE) is the current standard of care for patients with intermediate-stage hepatocellular carcinoma (HCC) and is also a treatment option for patients with liver metastases from colorectal cancer. However, TACE is not a curative treatment, and tumor progression occurs in more than half of the patients treated. Despite advances and technical refinements of TACE, including the introduction of drug-eluting beads-TACE, the clinical efficacy of TACE has not been optimized, and improved arterial therapies are required.

OBJECTIVE

The primary objectives of the VEROnA study are to evaluate the safety and tolerability of vandetanib-eluting radiopaque embolic beads (BTG-002814) in patients with resectable liver malignancies and to determine concentrations of vandetanib and the N-desmethyl metabolite in plasma and resected liver following treatment with BTG-002814.

METHODS

The VEROnA study is a first-in-human, open-label, single-arm, phase 0, window-of-opportunity study of BTG-002814 (containing 100 mg vandetanib) delivered transarterially, 7 to 21 days before surgery in patients with resectable liver malignancies. Eligible patients have a diagnosis of colorectal liver metastases, or HCC (Childs Pugh A), diagnosed histologically or radiologically, and are candidates for liver surgery. All patients are followed up for 28 days following surgery. Secondary objectives of this study are to evaluate the anatomical distribution of BTG-002814 on noncontrast-enhanced imaging, to evaluate histopathological features in the surgical specimen, and to assess changes in blood flow on dynamic contrast-enhanced magnetic resonance imaging following treatment with BTG-002814. Exploratory objectives of this study are to study blood biomarkers with the potential to identify patients likely to respond to treatment and to correlate the distribution of BTG-002814 on imaging with pathology by 3-dimensional modeling.

RESULTS

Enrollment for the study was completed in February 2019. Results of a planned interim analysis were reviewed by a safety committee after the first 3 patients completed follow-up. The recommendation of the committee was to continue the study without any changes to the dose or trial design, as there were no significant unexpected toxicities related to BTG-002814.

CONCLUSIONS

The VEROnA study is studying the feasibility of administering BTG-002814 to optimize the use of this novel technology as liver-directed therapy for patients with primary and secondary liver cancer.

TRIAL REGISTRATION

ClinicalTrial.gov NCT03291379; https://clinicaltrials.gov/ct2/show/NCT03291379.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/13696.