Catheter segmentation in X-ray fluoroscopy using synthetic data and transfer learning with light U-nets

Cardio-respiratory motion compensation for coronary roadmapping in fluoroscopic imaging

BACKGROUND

Inferring the shape and position of coronary artery poses challenges when using fluoroscopic image guidance during percutaneous coronary intervention (PCI) procedure. Although angiography enables coronary artery visualization, the use of injected contrast agent raises concerns about radiation exposure and the risk of contrast-induced nephropathy. To address these issues, dynamic coronary roadmapping overlaid on fluoroscopic images can provide coronary visual feedback without contrast injection.

PURPOSE

This paper proposes a novel cardio-respiratory motion compensation method that utilizes cardiac state synchronization and catheter motion estimation to achieve coronary roadmapping in fluoroscopic images.

METHODS

For more accurate cardiac state synchronization, video frame interpolation is applied to increase the frame rate of the original limited angiographic images, resulting in higher framerate and more adequate roadmaps. The proposed method also incorporates a multi-length cross-correlation based adaptive electrocardiogram (ECG) matching to address irregular cardiac motion situation. Furthermore, a shape-constrained path searching method is proposed to extract catheter structure from both fluoroscopic and angiographic image. Then catheter motion is estimated using a cascaded matching approach with an outlier removal strategy, leading to a final corrected roadmap.

RESULTS

Evaluation of the proposed method on clinical x-ray images demonstrates its effectiveness, achieving a 92.8% F1 score for catheter extraction on 589 fluoroscopic and angiographic images. Additionally, the method achieves a 5.6-pixel distance error of the coronary roadmap on 164 intraoperative fluoroscopic images.

CONCLUSIONS

Overall, the proposed method achieves accurate coronary roadmapping in fluoroscopic images and shows potential to overlay accurate coronary roadmap on fluoroscopic image in assisting PCI.

Real time artificial intelligence assisted carotid artery stenting: a preliminary experience

BACKGROUND

Neurointerventionalists must pay close attention to multiple devices on multiple screens simultaneously, which can lead to oversights and complications. Artificial intelligence (AI) has potential application in recognizing and monitoring these devices on fluoroscopic imaging.

METHODS

We report out preliminary experience with a real time AI assistance software, Neuro-Vascular Assist (iMed technologies, Tokyo, Japan), in six patients who underwent carotid artery stenting. This software provides real time assistance during endovascular procedures by tracking wires, guiding catheters, and embolic protection devices. The software provides notification when devices move out of a predefined region of interest or off the screen during the procedure. Efficacy, safety, and accuracy of the software were evaluated.

RESULTS

The software functioned well without problems and was easily used. Mean number of notifications per procedure was 21.0. The mean numbers of true positives, false positives, and false negatives per procedure were 17.2, 3.8, and 1.2, respectively. Precision and recall were 82% and 94%, respectively. Among the 103 true positive notifications, 24 caused the operator to adjust the inappropriate position of the device (23%), which is approximately four times per procedure. False notifications occurred because of false positive device detection. No adverse events related to the software occurred. No periprocedural complications occurred.

CONCLUSIONS

Neuro-Vascular Assist, a real time AI assistance software, worked appropriately and may be beneficial in carotid artery stenting procedures. Future large scale studies are warranted to confirm.

Catheter navigation support for mechanical thrombectomy guidance: 3D/2D multimodal catheter-based registration with no contrast dye fluoroscopy

PURPOSE

The fusion of pre-operative imaging and intra-operative fluoroscopy may support physicians during mechanical thrombectomy for catheter navigation from the aortic arch to carotids. Nevertheless, the aortic arch volume is too important for intra-operative contrast dye injection leading to a lack of common anatomical structure of interest that results in a challenging 3D/2D registration. The objective of this work is to propose a registration method between pre-operative 3D image and no contrast dye intra-operative fluoroscopy.

METHODS

The registration method exploits successive 2D fluoroscopic images of the catheter navigating in the aortic arch. The similarity measure is defined as the normalized cross-correlation between a binary combination of catheter images and a pseudo-DRR resulting from the 2D binary projection of the pre-operative 3D image (MRA or CTA). The 3D/2D transformation is decomposed in out-plane and in-plane transformations to reduce computational complexity. The 3D/2D transformation is then obtained by maximizing the similarity measure through multiresolution exhaustive search.

RESULTS

We evaluated the registration performance through dice score and mean landmark error. We evaluated the influence of parameters setting, aortic arch type and 2D navigation sequence duration. Results on a physical phantom and data from a patient who underwent a mechanical thrombectomy showed good registration accuracy with a dice score higher than 92% and a mean landmark error lower than the quarter of a carotid diameter (8-10 mm).

CONCLUSION

A new registration method compatible with no contrast dye fluoroscopy has been proposed to guide the crossing from aortic arch to a carotid in mechanical thrombectomy. First evaluation showed the feasibility and accuracy of the method as well as its compatibility with clinical routine practice.

Development of an Artificial Intelligence Tool for Intraoperative Guidance During Endovascular Abdominal Aortic Aneurysm Repair

BACKGROUND

Adverse events during surgery can occur in part due to errors in visual perception and judgment. Deep learning is a branch of artificial intelligence (AI) that has shown promise in providing real-time intraoperative guidance. This study aims to train and test the performance of a deep learning model that can identify inappropriate landing zones during endovascular aneurysm repair (EVAR).

METHODS

A deep learning model was trained to identify a "No-Go" landing zone during EVAR, defined by coverage of the lowest renal artery by the stent graft. Fluoroscopic images from elective EVAR procedures performed at a single institution and from open-access sources were selected. Annotations of the "No-Go" zone were performed by trained annotators. A 10-fold cross-validation technique was used to evaluate the performance of the model against human annotations. Primary outcomes were intersection-over-union (IoU) and F1 score and secondary outcomes were pixel-wise accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

RESULTS

The AI model was trained using 369 images procured from 110 different patients/videos, including 18 patients/videos (44 images) from open-access sources. For the primary outcomes, IoU and F1 were 0.43 (standard deviation ± 0.29) and 0.53 (±0.32), respectively. For the secondary outcomes, accuracy, sensitivity, specificity, NPV, and PPV were 0.97 (±0.002), 0.51 (±0.34), 0.99 (±0.001). 0.99 (±0.002), and 0.62 (±0.34), respectively.

CONCLUSIONS

AI can effectively identify suboptimal areas of stent deployment during EVAR. Further directions include validating the model on datasets from other institutions and assessing its ability to predict optimal stent graft placement and clinical outcomes.

Deep-learning based segmentation of ultrasound adipose image for liposuction

BACKGROUND

To develop an automatic and reliable ultrasonic visual system for robot- or computer-assisted liposuction, we examined the use of deep learning for the segmentation of adipose ultrasound images in clinical and educational settings.

METHODS

To segment adipose layers, it is proposed to use an Attention Skip-Convolutions ResU-Net (Attention SCResU-Net) consisting of SC residual blocks, attention gates and U-Net architecture. Transfer learning is utilised to compensate for the deficiency of clinical data. The Bama pig and clinical human adipose ultrasound image datasets are utilized, respectively.

RESULTS

The final model obtains a Dice of 99.06 ± 0.95% and an ASD of 0.19 ± 0.18 mm on clinical datasets, outperforming other methods. By fine-tuning the eight deepest layers, accurate and stable segmentation results are obtained.

CONCLUSIONS

The new deep-learning method achieves the accurate and automatic segmentation of adipose ultrasound images in real-time, thereby enhancing the safety of liposuction and enabling novice surgeons to better control the cannula.

Automatic image-based tracking of gadolinium-filled balloon wedge catheters for MRI-guided cardiac catheterization using deep learning

Introduction

Magnetic Resonance Imaging (MRI) is a promising alternative to standard x-ray fluoroscopy for the guidance of cardiac catheterization procedures as it enables soft tissue visualization, avoids ionizing radiation and provides improved hemodynamic data. MRI-guided cardiac catheterization procedures currently require frequent manual tracking of the imaging plane during navigation to follow the tip of a gadolinium-filled balloon wedge catheter, which unnecessarily prolongs and complicates the procedures. Therefore, real-time automatic image-based detection of the catheter balloon has the potential to improve catheter visualization and navigation through automatic slice tracking.

Methods

In this study, an automatic, parameter-free, deep-learning-based post-processing pipeline was developed for real-time detection of the catheter balloon. A U-Net architecture with a ResNet-34 encoder was trained on semi-artificial images for the segmentation of the catheter balloon. Post-processing steps were implemented to guarantee a unique estimate of the catheter tip coordinates. This approach was evaluated retrospectively in 7 patients (6M and 1F, age = 7 ± 5 year) who underwent an MRI-guided right heart catheterization procedure with all images acquired in an orientation unseen during training.

Results

The overall accuracy, specificity and sensitivity of the proposed catheter tracking strategy over all 7 patients were 98.4 ± 2.0%, 99.9 ± 0.2% and 95.4 ± 5.5%, respectively. The computation time of the deep-learning-based segmentation step was ∼10 ms/image, indicating its compatibility with real-time constraints.

Conclusion

Deep-learning-based catheter balloon tracking is feasible, accurate, parameter-free, and compatible with real-time conditions. Online integration of the technique and its evaluation in a larger patient cohort are now warranted to determine its benefit during MRI-guided cardiac catheterization.

Towards an Action Recognition Framework for Endovascular Surgery

Objective knowledge about instrument manoeuvres in endovascular surgery is essential for evaluating surgical skills and developing advanced technologies for cathlab routines. To the recent day, endovascular navigation has been exclusively assessed in laboratory scenarios. By contrast, information contained in available fluoroscopy data from clinical cases has been disregarded. In this work, we pioneer a learning-based framework for motion activity recognition in fluoroscopy sequences. The architecture is composed of two networks for instrument segmentation and action recognition. In this preliminary study, we demonstrate feasibility of recognising instrument manoeuvres automatically in our ex vivo datasets.Clinical relevance-The proposed framework contributes to image-based and automated assessment of endovascular tasks. This facilitates robotic control development, surgical education, and smart clinical documentation.

Real-time Pose Tracking for a Continuum Guidewire Robot under Fluoroscopic Imaging

Atherosclerosis is a medical condition that causes buildup of plaque in the blood vessels and narrowing of the arteries. Surgeons often treat this condition through angioplasty with catheter placements. Continuum guidewire robots offer significant advantages for catheter placements due to their dexterity. Tracking these guidewire robots and their surrounding workspace under fluoroscopy in real-time can be useful for visualization and accurate control. This paper discusses algorithms and methods to track the shape and orientation of the guidewire and the surrounding workspaces of phantom vasculatures in real-time under C-arm fluoroscopy. The shape of continuum guidewires is found through a semantic segmentation architecture based on MobileNetv2 with a Tversky loss function to deal with class imbalances. This shape is refined through medial axis filtering and parametric curve fitting to quantitatively describe the guidewire's pose. Using a constant curvature assumption for the guidewire's bending segments, the parameters that describe the joint variables are estimated in real-time for a tendon-actuated COaxially Aligned STeerable (COAST) guidewire robot and tracked through its traversal of an aortic bifurcation phantom. The accuracy of the tracking is ~90% and the execution times are within 100ms, and hence this method is deemed suitable for real-time tracking.

A Few-Shot Dental Object Detection Method Based on a Priori Knowledge Transfer

With the continuous improvement in oral health awareness, people’s demand for oral health diagnosis has also increased. Dental object detection is a key step in automated dental diagnosis; however, because of the particularity of medical data, researchers usually cannot obtain sufficient medical data. Therefore, this study proposes a dental object detection method for small-size datasets based on teeth semantics, structural information feature extraction, and an a priori knowledge migration, called a segmentation, points, segmentation, and classification network (SPSC-NET). In the region of interest area extraction method, the SPSC-NET method converts the teeth X-ray image into an a priori knowledge information image, composed of the edges of the teeth and the semantic segmentation image; the network structure used to extract the a priori knowledge information is a symmetric structure, which then generates the key points of the object instance. Next, it uses the key points of the object instance (i.e., the dental semantic segmentation image and the dental edge image) to obtain the object instance image (i.e., the positioning of the teeth). Using 10 training images, the test precision and recall rate of the tooth object center point of the SPSC-NET method were between 99–100%. In the classification method, the SPSC-NET identified the single instance segmentation image generated by migrating the dental object area, the edge image, and the semantic segmentation image as a priori knowledge. Under the premise of using the same deep neural network classification model, the model classification with a priori knowledge was 20% more accurate than the ordinary classification methods. For the overall object detection performance indicators, the SPSC-NET’s average precision (AP) value was more than 92%, which is better than that of the transfer-based faster region-based convolutional neural network (Faster-RCNN) object detection model; moreover, its AP and mean intersection-over-union (mIOU) were 14.72% and 19.68% better than the transfer-based Faster-CNN model, respectively.

Early Prediction of Sepsis Onset Using Neural Architecture Search Based on Genetic Algorithms

Sepsis is a life-threatening condition with a high mortality rate. Early prediction and treatment are the most effective strategies for increasing survival rates. This paper proposes a neural architecture search (NAS) model to predict the onset of sepsis with a low computational cost and high search performance by applying a genetic algorithm (GA). The proposed model shares the weights of all possible connection nodes internally within the neural network. Externally, the search cost is reduced through the weight-sharing effect between the genotypes of the GA. A predictive analysis was performed using the Medical Information Mart for Intensive Care III (MIMIC-III), a medical time-series dataset, with the primary objective of predicting sepsis onset 3 h before occurrence. In addition, experiments were conducted under various prediction times (0-12 h) for comparison. The proposed model exhibited an area under the receiver operating characteristic curve (AUROC) score of 0.94 (95% CI: 0.92-0.96) for 3 h, which is 0.31-0.26 higher than the scores obtained using the Sequential Organ Failure Assessment (SOFA), quick SOFA (qSOFA), and Simplified Acute Physiology Score (SAPS) II scoring systems. Furthermore, the proposed model exhibited a 12% improvement in the AUROC value over a simple model based on the long short-term memory neural network. Additionally, it is not only optimally searchable for sepsis onset prediction, but also outperforms conventional models that use similar predictive purposes and datasets. Notably, it is sufficiently robust to shape changes in the input data and has less structural dependence.

Implementing Machine Learning in Interventional Cardiology: The Benefits Are Worth the Trouble

Driven by recent innovations and technological progress, the increasing quality and amount of biomedical data coupled with the advances in computing power allowed for much progress in artificial intelligence (AI) approaches for health and biomedical research. In interventional cardiology, the hope is for AI to provide automated analysis and deeper interpretation of data from electrocardiography, computed tomography, magnetic resonance imaging, and electronic health records, among others. Furthermore, high-performance predictive models supporting decision-making hold the potential to improve safety, diagnostic and prognostic prediction in patients undergoing interventional cardiology procedures. These applications include robotic-assisted percutaneous coronary intervention procedures and automatic assessment of coronary stenosis during diagnostic coronary angiograms. Machine learning (ML) has been used in these innovations that have improved the field of interventional cardiology, and more recently, deep Learning (DL) has emerged as one of the most successful branches of ML in many applications. It remains to be seen if DL approaches will have a major impact on current and future practice. DL-based predictive systems also have several limitations, including lack of interpretability and lack of generalizability due to cohort heterogeneity and low sample sizes. There are also challenges for the clinical implementation of these systems, such as ethical limits and data privacy. This review is intended to bring the attention of health practitioners and interventional cardiologists to the broad and helpful applications of ML and DL algorithms to date in the field. Their implementation challenges in daily practice and future applications in the field of interventional cardiology are also discussed.

Towards real time guide wire shape extraction in fluoroscopic sequences: A two phase deep learning scheme to extract sparse curvilinear structures

BACKGROUND AND OBJECTIVE

Real time localization and shape extraction of guide wire in fluoroscopic images plays a significant role in the image guided navigation during cerebral and cardiovascular interventions. Given the complexity of the non-rigid and sparse characteristics of guide wire structures, and the low SNR(Signal Noise Ratio) of fluoroscopic images, traditional handcrafted guide wire tracking methods such as Frangi filter, Hessian Matrix, or open active contour usually produce insufficient accuracy with high computational cost, and may require extra human intervention for proper initialization or correction. The application of deep learning techniques to guide wire tracking is reported to produce significant improvement in guide wire localization accuracy, but the heavy calculation cost is still a concern.

METHOD

In this paper we propose a two phase deep learning scheme for accurate and real time guide wire shape extraction in fluoroscopic sequences. In the first phase we train a guide wire localization network to pick image regions containing guide wire structures. From the picked image regions, we train a guide wire shape extraction network in the second phase to mark the guide wire pixels.

RESULTS

We report that our proposed method can accurately distinguish about 99% of the guide wire structure pixels, and the falsely detected pixels in the background are close to 0, the average offset from the ground truth is less than 1 pixel. For extreme cases where traditional handcrafted method may fail, our proposed method can still extract guide wire completely and accurately. The processing time for a 512 × 512 pixels image is 78 ms.

CONCLUSION

Compared with the traditional filtering based method from our previous work, we show that our proposed method can achieve more accurate and stable performance. Compared with other deep learning methods, our proposed method significantly improve calculation efficiency to meet the real time requirement of clinical applications.

Deformable 3D-2D registration for high-precision guidance and verification of neuroelectrode placement

Purpose.Accurate neuroelectrode placement is essential to effective monitoring or stimulation of neurosurgery targets. This work presents and evaluates a method that combines deep learning and model-based deformable 3D-2D registration to guide and verify neuroelectrode placement using intraoperative imaging.Methods.The registration method consists of three stages: (1) detection of neuroelectrodes in a pair of fluoroscopy images using a deep learning approach; (2) determination of correspondence and initial 3D localization among neuroelectrode detections in the two projection images; and (3) deformable 3D-2D registration of neuroelectrodes according to a physical device model. The method was evaluated in phantom, cadaver, and clinical studies in terms of (a) the accuracy of neuroelectrode registration and (b) the quality of metal artifact reduction (MAR) in cone-beam CT (CBCT) in which the deformably registered neuroelectrode models are taken as input to the MAR.Results.The combined deep learning and model-based deformable 3D-2D registration approach achieved 0.2 ± 0.1 mm accuracy in cadaver studies and 0.6 ± 0.3 mm accuracy in clinical studies. The detection network and 3D correspondence provided initialization of 3D-2D registration within 2 mm, which facilitated end-to-end registration runtime within 10 s. Metal artifacts, quantified as the standard deviation in voxel values in tissue adjacent to neuroelectrodes, were reduced by 72% in phantom studies and by 60% in first clinical studies.Conclusions.The method combines the speed and generalizability of deep learning (for initialization) with the precision and reliability of physical model-based registration to achieve accurate deformable 3D-2D registration and MAR in functional neurosurgery. Accurate 3D-2D guidance from fluoroscopy could overcome limitations associated with deformation in conventional navigation, and improved MAR could improve CBCT verification of neuroelectrode placement.

Image-based shading correction for narrow-FOV truncated pelvic CBCT with deep convolutional neural networks and transfer learning

Purpose: Cone beam computed tomography (CBCT) is a standard solution for in‐room image guidance for radiation therapy. It is used to evaluate and compensate for anatomopathological changes between the dose delivery plan and the fraction delivery day. CBCT is a fast and versatile solution, but it suffers from drawbacks like low contrast and requires proper calibration to derive density values. Although these limitations are even more prominent with in‐room customized CBCT systems, strategies based on deep learning have shown potential in improving image quality. As such, this article presents a method based on a convolutional neural network and a novel two‐step supervised training based on the transfer learning paradigm for shading correction in CBCT volumes with narrow field of view (FOV) acquired with an ad hoc in‐room system.

Methods: We designed a U‐Net convolutional neural network, trained on axial slices of corresponding CT/CBCT couples. To improve the generalization capability of the network, we exploited two‐stage learning using two distinct data sets. At first, the network weights were trained using synthetic CBCT scans generated from a public data set, and then only the deepest layers of the network were trained again with real‐world clinical data to fine‐tune the weights. Synthetic data were generated according to real data acquisition parameters. The network takes a single grayscale volume as input and outputs the same volume with corrected shading and improved HU values.

Results: Evaluation was carried out with a leave‐one‐out cross‐validation, computed on 18 unique CT/CBCT pairs from six different patients from a real‐world dataset. Comparing original CBCT to CT and improved CBCT to CT, we obtained an average improvement of 6 dB on peak signal‐to‐noise ratio (PSNR), +2% on structural similarity index measure (SSIM). The median interquartile range (IQR) Hounsfield unit (HU) difference between CBCT and CT improved from 161.37 (162.54) HU to 49.41 (66.70) HU. Region of interest (ROI)‐based HU difference was narrowed by 75% in the spongy bone (femoral head), 89% in the bladder, 85% for fat, and 83% for muscle. The improvement in contrast‐to‐noise ratio for these ROIs was about 67%.

Conclusions: We demonstrated that shading correction obtaining CT‐compatible data from narrow‐FOV CBCTs acquired with a customized in‐room system is possible. Moreover, the transfer learning approach proved particularly beneficial for such a shading correction approach.

DeepWL: Robust EPID based Winston-Lutz analysis using deep learning, synthetic image generation and optical path-tracing

Radiation therapy requires clinical linear accelerators to be mechanically and dosimetrically calibrated to a high standard. One important quality assurance test is the Winston-Lutz test which localises the radiation isocentre of the linac. In the current work we demonstrate a novel method of analysing EPID based Winston-Lutz QA images using a deep learning model trained only on synthetic image data. In addition, we propose a novel method of generating the synthetic WL images and associated 'ground-truth' masks using an optical path-tracing engine to 'fake' mega-voltage EPID images. The model called DeepWL was trained on 1500 synthetic WL images using data augmentation techniques for 180 epochs. The model was built using Keras with a TensorFlow backend on an Intel Core i5-6500T CPU and trained in approximately 15 h. DeepWL was shown to produce ball bearing and multi-leaf collimator field segmentations with a mean dice coefficient of 0.964 and 0.994 respectively on previously unseen synthetic testing data. When DeepWL was applied to WL data measured on an EPID, the predicted mean displacements were shown to be statistically similar to the Canny Edge detection method. However, the DeepWL predictions for the ball bearing locations were shown to correlate better with manual annotations compared with the Canny edge detection algorithm. DeepWL was demonstrated to analyse Winston-Lutz images with an accuracy suitable for routine linac quality assurance with some statistical evidence that it may outperform Canny Edge detection methods in terms of segmentation robustness and the resultant displacement predictions.

Introducing frequency representation into convolution neural networks for medical image segmentation via twin-Kernel Fourier convolution

BACKGROUND AND OBJECTIVE

For medical image segmentation, deep learning-based methods have achieved state-of-the-art performance. However, the powerful spectral representation in the field of image processing is rarely considered in these models.

METHODS

In this work, we propose to introduce frequency representation into convolution neural networks (CNNs) and design a novel model, tKFC-Net, to combine powerful feature representation in both frequency and spatial domains. Through the Fast Fourier Transform (FFT) operation, frequency representation is employed on pooling, upsampling, and convolution without any adjustments to the network architecture. Furthermore, we replace original convolution with twin-Kernel Fourier Convolution (t-KFC), a new designed convolution layer, to specify the convolution kernels for particular functions and extract features from different frequency components.

RESULTS

We experimentally show that our method has an edge over other models in the task of medical image segmentation. Evaluated on four datasets-skin lesion segmentation (ISIC 2018), retinal blood vessel segmentation (DRIVE), lung segmentation (COVID-19-CT-Seg), and brain tumor segmentation (BraTS 2019), the proposed model achieves outstanding results: the metric F1-Score is 0.878 for ISIC 2018, 0.8185 for DRIVE, 0.9830 for COVID-19-CT-Seg, and 0.8457 for BraTS 2019.

CONCLUSION

The introduction of spectral representation retains spectral features which result in more accurate segmentation. The proposed method is orthogonal to other topology improvement methods and very convenient to be combined.

Deep neural network for automated simultaneous intervertebral disc (IVDs) identification and segmentation of multi-modal MR images

BACKGROUND AND OBJECTIVE

Lower back pain in humans has become a major risk. Classical approaches follow a non-invasive imaging technique for the assessment of spinal intervertebral disc (IVDs) abnormalities, where identification and segmentation of discs are done separately, making it a time-consuming phenomenon. This necessitates designing a robust automated and simultaneous IVDs identification and segmentation of multi-modality MRI images.

METHODS

We introduced a novel deep neural network architecture coined as 'RIMNet', a Region-to-Image Matching Network model, capable of performing an automated and simultaneous IVDs identification and segmentation of MRI images. The multi-modal input data is being fed to the network with a dropout strategy, by randomly disabling modalities in mini-batches. The performance accuracy as a function of the testing dataset was determined. The execution of the deep neural network model was evaluated by computing the IVDs Identification Accuracy, Dice coefficient, MDOC, Average Symmetric Surface Distance, Jaccard Coefficient, Hausdorff Distance and F1 Score.

RESULTS

Proposed model has attained 94% identification accuracy, dice coefficient value of 91.7±1% in segmentation and MDOC 90.2±1%. Our model also achieved 0.87±0.02 for Jaccard Coefficient, 0.54±0.04 for ASD and 0.62±0.02 mm Hausdorff Distance. The results have been validated and compared with other methodologies on dataset of MICCAI IVD 2018 challenge.

CONCLUSIONS

Our proposed deep-learning methodology is capable of performing simultaneous identification and segmentation on IVDs MRI images of the human spine with high accuracy.

Cryo-balloon catheter localization in X-Ray fluoroscopy using U-net

Purpose

Automatic identification of interventional devices in X-ray (XR) fluoroscopy offers the potential of improved navigation during transcatheter endovascular procedures. This paper presents a prototype implementation of fully automatic 3D reconstruction of a cryo-balloon catheter during pulmonary vein isolation (PVI) procedures by deep learning approaches.

Methods

We employ convolutional neural networks (CNN) to automatically identify the cryo-balloon XR marker and catheter shaft in 2D fluoroscopy during PVI. Training data are generated exploiting established semiautomatic techniques, including template-matching and analytical graph building. A first network of U-net architecture uses a single grayscale XR image as input and yields the mask of the XR marker. A second network of the similar architecture is trained using the mask of the XR marker as additional input to the grayscale XR image for the segmentation of the cryo-balloon catheter shaft mask. The structures automatically identified in two 2D images with different angulations are then used to reconstruct the cryo-balloon in 3D.

Results

Automatic identification of the XR marker was successful in 78% of test cases and in 100% for the catheter shaft. Training of the model for prediction of the XR marker mask was successful with 3426 training samples. Incorporation of the XR marker mask as additional input for the model predicting the catheter shaft allowed to achieve good training result with only 805 training samples. The average prediction time per frame was 14.47 ms for the XR marker and 78.22 ms for the catheter shaft. Localization accuracy for the XR marker yielded on average 1.52 pixels or 0.56 mm.

Conclusions

In this paper, we report a novel method for automatic detection and 3D reconstruction of the cryo-balloon catheter shaft and marker from 2D fluoroscopic images. Initial evaluation yields promising results thus indicating the high potential of CNNs as alternatives to the current state-of-the-art solutions.

Detection of peripherally inserted central catheter (PICC) in chest X-ray images: A multi-task deep learning model

BACKGROUND AND OBJECTIVE

Peripherally inserted central catheter (PICC) is a novel drug delivery mode which has been widely used in clinical practice. However, long-term retention and some improper actions of patients may cause some severe complications of PICC, such as the drift and prolapse of its catheter. Clinically, the postoperative care of PICC is mainly completed by nurses. However, they cannot recognize the correct position of PICC from X-ray chest images as soon as the complications happen, which may lead to improper treatment. Therefore, it is necessary to identify the position of the PICC catheter as soon as these complications occur. Here we proposed a novel multi-task deep learning framework to detect PICC automatically through X-ray images, which could help nurses to solve this problem.

METHODS

We collected 348 X-ray chest images from 326 patients with visible PICC. Then we proposed a multi-task deep learning framework for line segmentation and tip detection of PICC catheters simultaneously. The proposed deep learning model is composed of an extraction structure and three routes, an up-sampling route for segmentation, an RPNs route, and an RoI Pooling route for detection. We further compared the effectiveness of our model with the models previously proposed.

RESULTS

In the catheter segmentation task, 300 X-ray images were utilized for training the model, then 48 images were tested. In the tip detection task, 154 X-ray images were used for retraining and 20 images were used in the test. Our model achieved generally better results among several popular deep learning models previously proposed.

CONCLUSIONS

We proposed a multi-task deep learning model that could segment the catheter and detect the tip of PICC simultaneously from X-ray chest images. This model could help nurses to recognize the correct position of PICC, and therefore, to handle the potential complications properly.

Real-time segmentation and tracking of excised corneal contour by deep neural networks for DALK surgical navigation

OBJECTIVE

Corneal disease is one of the main causes of blindness for humans globally nowadays, and deep anterior lamellar keratoplasty (DALK) is a widely applied technique for corneal transplantation. However, the position of stitch points highly influences the success rate of such surgery, which would require accurate control and manipulation of surgical instruments.

METHODS

In this paper, we present a deep learning framework for augmented reality (AR) based surgery navigation to guide the suturing in DALK. It can robustly track the excised corneal contour by semantic segmentation and the reconstruction of occlusion. We propose a novel optical flow inpainting network to recover the missing motion caused by occlusion. The occluded regions are detected by weakly supervised segmentation of surgical instruments and reconstructed by key frame warping along the completed optical flow. Then we introduce two types of loss function to adapt the inpainting network in the optical flow space.

RESULTS

Our techniques are tested and evaluated by a number of real surgery videos from Shandong Eye Hospital in China. We compare our approaches with other typical methods in the corneal contour segmentation, optical flow inpainting and occlusion regions reconstruction. The tracking accuracy reachs 99.2% in average and PSNR reaches 25.52 for the reconstruction of the occluded frames.

CONCLUSION

From the experimental evaluations and user study, both the qualitative and quantitative results indicate that our techniques can achieve accurate detection and tracking of corneal contour under complex disturbance in real-time surgical scenes. Our prototype AR navigation system would be highly useful in clinical practice.