Automatic Catheter and Tube Detection in Pediatric X-ray Images Using a Scale-Recurrent Network and Synthetic Data

Supervised representation learning based on various levels of pediatric radiographic views for transfer learning

Transfer learning plays a pivotal role in addressing the paucity of data, expediting training processes, and enhancing model performance. Nonetheless, the prevailing practice of transfer learning predominantly relies on pre-trained models designed for the natural image domain, which may not be well-suited for the medical image domain in grayscale. Recognizing the significance of leveraging transfer learning in medical research, we undertook the construction of class-balanced pediatric radiograph datasets collectively referred to as PedXnets, grounded in radiographic views using the pediatric radiographs collected over 24 years at Asan Medical Center. For PedXnets pre-training, approximately 70,000 X-ray images were utilized. Three different pre-training weights of PedXnet were constructed using Inception V3 for various radiation perspective classifications: Model-PedXnet-7C, Model-PedXnet-30C, and Model-PedXnet-68C. We validated the transferability and positive effects of transfer learning of PedXnets through pediatric downstream tasks including fracture classification and bone age assessment (BAA). The evaluation of transfer learning effects through classification and regression metrics showed superior performance of Model-PedXnets in quantitative assessments. Additionally, visual analyses confirmed that the Model-PedXnets were more focused on meaningful regions of interest.

Automated precision localization of peripherally inserted central catheter tip through model-agnostic multi-stage networks

BACKGROUND

Peripherally inserted central catheters (PICCs) have been widely used as one of the representative central venous lines (CVCs) due to their long-term intravascular access with low infectivity. However, PICCs have a fatal drawback of a high frequency of tip mispositions, increasing the risk of puncture, embolism, and complications such as cardiac arrhythmias. To automatically and precisely detect it, various attempts have been made by using the latest deep learning (DL) technologies. However, even with these approaches, it is still practically difficult to determine the tip location because the multiple fragments phenomenon (MFP) occurs in the process of predicting and extracting the PICC line required before predicting the tip.

OBJECTIVE

This study aimed to develop a system generally applied to existing models and to restore the PICC line more exactly by removing the MFs of the model output, thereby precisely localizing the actual tip position for detecting its misposition.

METHODS

To achieve this, we proposed a multi-stage DL-based framework post-processing the PICC line extraction result of the existing technology. Our method consists of the following three stages: 1. Existing PICC line segmentation network for a baseline, 2. Patch-based PICC line refinement network, 3. PICC line reconnection network. The proposed second and third-stage models address MFs caused by the sparseness of the PICC line and the line disconnection due to confusion with anatomical structures respectively, thereby enhancing tip detection.

RESULTS

To verify the objective performance of the proposed MFCN, internal validation and external validation were conducted. For internal validation, learning (130 samples) and verification (150 samples) were performed with 280 data, including PICC among Chest X-ray (CXR) images taken at our institution. External validation was conducted using a public dataset called the Royal Australian and New Zealand College of Radiologists (RANZCR), and training (130 samples) and validation (150 samples) were performed with 280 data of CXR images, including PICC, which has the same number as that for internal validation. The performance was compared by root mean squared error (RMSE) and the ratio of single fragment images (RatioSFI) (i.e., the rate at which model predicts PICC as multiple sub-lines) according to whether or not MFCN is applied to seven conventional models (i.e., FCDN, UNET, AUNET, TUNET, FCDN-HT, UNET-ELL, and UNET-RPN). In internal validation, when MFCN was applied to the existing single model, MFP was improved by an average of 45 %. The RMSE improved over 63% from an average of 27.54 mm (17.16 to 35.80 mm) to 9.77 mm (9.11 to 10.98 mm). In external validation, when MFCN was applied, the MFP incidence rate decreased by an average of 32% and the RMSE decreased by an average of 65%. Therefore, by applying the proposed MFCN, we observed the consistent detection performance improvement of PICC tip location compared to the existing model.

CONCLUSION

In this study, we applied the proposed technique to the existing technique and demonstrated that it provides high tip detection performance, proving its high versatility and superiority. Therefore, we believe, in countries and regions where radiologists are scarce, that the proposed DL approach will be able to effectively detect PICC misposition on behalf of radiologists.

A principled representation of elongated structures using heatmaps

The detection of elongated structures like lines or edges is an essential component in semantic image analysis. Classical approaches that rely on significant image gradients quickly reach their limits when the structure is context-dependent, amorphous, or not directly visible. This study introduces a principled mathematical description of elongated structures with various origins and shapes. Among others, it serves as an expressive operational description of target functions that can be well approximated by Convolutional Neural Networks. The nominal position of a curve and its positional uncertainty are encoded as a heatmap by convolving the curve distribution with a filter function. We propose a low-error approximation to the expensive numerical integration by evaluating a distance-dependent function, enabling a lightweight implementation with linear time complexity. We analyze the method's numerical approximation error and behavior for different curve types and signal-to-noise levels. Application to surgical 2D and 3D data, semantic boundary detection, skeletonization, and other related tasks demonstrate the method's versatility at low errors.

Analysis of Line and Tube Detection Performance of a Chest X-ray Deep Learning Model to Evaluate Hidden Stratification

This retrospective case-control study evaluated the diagnostic performance of a commercially available chest radiography deep convolutional neural network (DCNN) in identifying the presence and position of central venous catheters, enteric tubes, and endotracheal tubes, in addition to a subgroup analysis of different types of lines/tubes. A held-out test dataset of 2568 studies was sourced from community radiology clinics and hospitals in Australia and the USA, and was then ground-truth labelled for the presence, position, and type of line or tube from the consensus of a thoracic specialist radiologist and an intensive care clinician. DCNN model performance for identifying and assessing the positioning of central venous catheters, enteric tubes, and endotracheal tubes over the entire dataset, as well as within each subgroup, was evaluated. The area under the receiver operating characteristic curve (AUC) was assessed. The DCNN algorithm displayed high performance in detecting the presence of lines and tubes in the test dataset with AUCs > 0.99, and good position classification performance over a subpopulation of ground truth positive cases with AUCs of 0.86-0.91. The subgroup analysis showed that model performance was robust across the various subtypes of lines or tubes, although position classification performance of peripherally inserted central catheters was relatively lower. Our findings indicated that the DCNN algorithm performed well in the detection and position classification of lines and tubes, supporting its use as an assistant for clinicians. Further work is required to evaluate performance in rarer scenarios, as well as in less common subgroups.

Automatic detection of punctate white matter lesions in infants using deep learning of composite images from two cases

Punctate white matter lesions (PWMLs) in infants may be related to neurodevelopmental outcomes based on the location or number of lesions. This study aimed to assess the automatic detectability of PWMLs in infants on deep learning using composite images created from several cases. To create the initial composite images, magnetic resonance (MR) images of two infants with the most PWMLs were used; their PWMLs were extracted and pasted onto MR images of infants without abnormality, creating many composite PWML images. Deep learning models based on a convolutional neural network, You Only Look Once v3 (YOLOv3), were constructed using the training set of 600, 1200, 2400, and 3600 composite images. As a result, a threshold of detection probability of 20% and 30% for all deep learning model sets yielded a relatively high sensitivity for automatic PWML detection (0.908-0.957). Although relatively high false-positive detections occurred with the lower threshold of detection probability, primarily, in the partial volume of the cerebral cortex (≥ 85.8%), those can be easily distinguished from the white matter lesions. Relatively highly sensitive automatic detection of PWMLs was achieved by creating composite images from two cases using deep learning.

Multi-mode information fusion navigation system for robot-assisted vascular interventional surgery

BACKGROUND

Minimally invasive vascular intervention (MIVI) is a powerful technique for the treatment of cardiovascular diseases, such as abdominal aortic aneurysm (AAA), thoracic aortic aneurysm (TAA) and aortic dissection (AD). Navigation of traditional MIVI surgery mainly relies only on 2D digital subtraction angiography (DSA) images, which is hard to observe the 3D morphology of blood vessels and position the interventional instruments. The multi-mode information fusion navigation system (MIFNS) proposed in this paper combines preoperative CT images and intraoperative DSA images together to increase the visualization information during operations.

RESULTS

The main functions of MIFNS were evaluated by real clinical data and a vascular model. The registration accuracy of preoperative CTA images and intraoperative DSA images were less than 1 mm. The positioning accuracy of surgical instruments was quantitatively assessed using a vascular model and was also less than 1 mm. Real clinical data used to assess the navigation results of MIFNS on AAA, TAA and AD.

CONCLUSIONS

A comprehensive and effective navigation system was developed to facilitate the operation of surgeon during MIVI. The registration accuracy and positioning accuracy of the proposed navigation system were both less than 1 mm, which met the accuracy requirements of robot assisted MIVI.

A Review of Synthetic Image Data and Its Use in Computer Vision

Development of computer vision algorithms using convolutional neural networks and deep learning has necessitated ever greater amounts of annotated and labelled data to produce high performance models. Large, public data sets have been instrumental in pushing forward computer vision by providing the data necessary for training. However, many computer vision applications cannot rely on general image data provided in the available public datasets to train models, instead requiring labelled image data that is not readily available in the public domain on a large scale. At the same time, acquiring such data from the real world can be difficult, costly to obtain, and manual labour intensive to label in large quantities. Because of this, synthetic image data has been pushed to the forefront as a potentially faster and cheaper alternative to collecting and annotating real data. This review provides general overview of types of synthetic image data, as categorised by synthesised output, common methods of synthesising different types of image data, existing applications and logical extensions, performance of synthetic image data in different applications and the associated difficulties in assessing data performance, and areas for further research.

Combination of UNet++ and ResNeSt to classify chronic inflammation of the choledochal cystic wall in patients with pancreaticobiliary maljunction

OBJECTIVES

The aim of this study was to establish an automatic classification model for chronic inflammation of the choledoch wall using deep learning with CT images in patients with pancreaticobiliary maljunction (PBM).

METHODS

CT images were obtained from 76 PBM patients, including 61 cases assigned to the training set and 15 cases assigned to the testing set. The region of interest (ROI) containing the choledochal lesion was extracted and segmented using the UNet++ network. The degree of severity of inflammation in the choledochal wall was initially classified using the ResNeSt network. The final classification result was determined per decision rules. Grad-CAM was used to explain the association between the classification basis of the network and clinical diagnosis.

RESULTS

Segmentation of the lesion on the common bile duct wall was roughly obtained with the UNet++ segmentation model and the average value of Dice coefficient of the segmentation model in the testing set was 0.839 ± 0.150, which was verified through fivefold cross-validation. Inflammation was initially classified with ResNeSt18, which resulted in accuracy = 0.756, sensitivity = 0.611, specificity = 0.852, precision = 0.733, and area under curve (AUC) = 0.711. The final classification sensitivity was 0.8. Grad-CAM revealed similar distribution of inflammation of the choledochal wall and verified the inflammation classification.

CONCLUSIONS

By combining the UNet++ network and the ResNeSt network, we achieved automatic classification of chronic inflammation of the choledoch in PBM patients and verified the robustness through cross-validation performed five times. This study provided an important basis for classification of inflammation severity of the choledoch in PBM patients.

ADVANCES IN KNOWLEDGE

We combined the UNet++ network and the ResNeSt network to achieve automatic classification of chronic inflammation of the choledoch in PBM. These results provided an important basis for classification of choledochal inflammation in PBM and for surgical therapy.

Assessment of germinal matrix hemorrhage on head ultrasound with deep learning algorithms

BACKGROUND

Germinal matrix hemorrhage-intraventricular hemorrhage is among the most common intracranial complications in premature infants. Early detection is important to guide clinical management for improved patient prognosis.

OBJECTIVE

The purpose of this study was to assess whether a convolutional neural network (CNN) can be trained via transfer learning to accurately diagnose germinal matrix hemorrhage on head ultrasound.

MATERIALS AND METHODS

Over a 10-year period, 400 head ultrasounds performed in patients ages 6 months or younger were reviewed. Key sagittal images at the level of the caudothalamic groove were obtained from 200 patients with germinal matrix hemorrhage and 200 patients without hemorrhage; all images were reviewed by a board-certified pediatric radiologist. One hundred cases were randomly allocated from the total for validation and an additional 100 for testing of a CNN binary classifier. Transfer learning and data augmentation were used to train the model.

RESULTS

The median age of patients was 0 weeks old with a median gestational age of 30 weeks. The final trained CNN model had a receiver operating characteristic area under the curve of 0.92 on the validation set and accuracy of 0.875 on the test set, with 95% confidence intervals of [0.86, 0.98] and [0.81, 0.94], respectively.

CONCLUSION

A CNN trained on a small set of images with data augmentation can detect germinal matrix hemorrhage on head ultrasounds with strong accuracy.

Pediatric chest radiograph interpretation: how far has artificial intelligence come? A systematic literature review

Most artificial intelligence (AI) studies have focused primarily on adult imaging, with less attention to the unique aspects of pediatric imaging. The objectives of this study were to (1) identify all publicly available pediatric datasets and determine their potential utility and limitations for pediatric AI studies and (2) systematically review the literature to assess the current state of AI in pediatric chest radiograph interpretation. We searched PubMed, Web of Science and Embase to retrieve all studies from 1990 to 2021 that assessed AI for pediatric chest radiograph interpretation and abstracted the datasets used to train and test AI algorithms, approaches and performance metrics. Of 29 publicly available chest radiograph datasets, 2 datasets included solely pediatric chest radiographs, and 7 datasets included pediatric and adult patients. We identified 55 articles that implemented an AI model to interpret pediatric chest radiographs or pediatric and adult chest radiographs. Classification of chest radiographs as pneumonia was the most common application of AI, evaluated in 65% of the studies. Although many studies report high diagnostic accuracy, most algorithms were not validated on external datasets. Most AI studies for pediatric chest radiograph interpretation have focused on a limited number of diseases, and progress is hindered by a lack of large-scale pediatric chest radiograph datasets.

Current and emerging artificial intelligence applications in chest imaging: a pediatric perspective

Artificial intelligence (AI) applications for chest radiography and chest CT are among the most developed applications in radiology. More than 40 certified AI products are available for chest radiography or chest CT. These AI products cover a wide range of abnormalities, including pneumonia, pneumothorax and lung cancer. Most applications are aimed at detecting disease, complemented by products that characterize or quantify tissue. At present, none of the thoracic AI products is specifically designed for the pediatric population. However, some products developed to detect tuberculosis in adults are also applicable to children. Software is under development to detect early changes of cystic fibrosis on chest CT, which could be an interesting application for pediatric radiology. In this review, we give an overview of current AI products in thoracic radiology and cover recent literature about AI in chest radiography, with a focus on pediatric radiology. We also discuss possible pediatric applications.

Automatic Detection and Classification of Multiple Catheters in Neonatal Radiographs with Deep Learning

We develop and evaluate a deep learning algorithm to classify multiple catheters on neonatal chest and abdominal radiographs. A convolutional neural network (CNN) was trained using a dataset of 777 neonatal chest and abdominal radiographs, with a split of 81%-9%-10% for training-validation-testing, respectively. We employed ResNet-50 (a CNN), pre-trained on ImageNet. Ground truth labelling was limited to tagging each image to indicate the presence or absence of endotracheal tubes (ETTs), nasogastric tubes (NGTs), and umbilical arterial and venous catheters (UACs, UVCs). The dataset included 561 images containing two or more catheters, 167 images with only one, and 49 with none. Performance was measured with average precision (AP), calculated from the area under the precision-recall curve. On our test data, the algorithm achieved an overall AP (95% confidence interval) of 0.977 (0.679-0.999) for NGTs, 0.989 (0.751-1.000) for ETTs, 0.979 (0.873-0.997) for UACs, and 0.937 (0.785-0.984) for UVCs. Performance was similar for the set of 58 test images consisting of two or more catheters, with an AP of 0.975 (0.255-1.000) for NGTs, 0.997 (0.009-1.000) for ETTs, 0.981 (0.797-0.998) for UACs, and 0.937 (0.689-0.990) for UVCs. Our network thus achieves strong performance in the simultaneous detection of these four catheter types. Radiologists may use such an algorithm as a time-saving mechanism to automate reporting of catheters on radiographs.

Deep learning for chest X-ray analysis: A survey

Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs published before March 2021, categorizing works by task: image-level prediction (classification and regression), segmentation, localization, image generation and domain adaptation. Detailed descriptions of all publicly available datasets are included and commercial systems in the field are described. A comprehensive discussion of the current state of the art is provided, including caveats on the use of public datasets, the requirements of clinically useful systems and gaps in the current literature.

An Intelligent Future for Medical Imaging: A Market Outlook on Artificial Intelligence for Medical Imaging

Radiologists today are under increasing work pressure. We surveyed radiologists in the United States across practice settings, and the overwhelming majority reported an increased workload. Artificial intelligence (AI), which includes machine learning, can help address these issues. It also has the potential to improve clinical outcomes and raise further the value of medical imaging in ways yet to be defined. In this article, we report on recent McKinsey & Company work to understand the growth of AI in medical imaging. We highlight progress in its clinical application, the investments that are backing it, and the barriers to broader adoption. We also offer a view on how the market will develop. AI is set to have a big impact on the medical imaging market and hence on how radiologists work, helping them to speed up scan time, make more accurate diagnoses, and ease their workload. As AI in medical imaging increasingly proves its worth, it is hard to imagine that AI will not ultimately transform radiology.

Synthesize and Segment: Towards Improved Catheter Segmentation via Adversarial Augmentation

Automatic catheter and guidewire segmentation plays an important role in robot-assisted interventions that are guided by fluoroscopy. Existing learning based methods addressing the task of segmentation or tracking are often limited by the scarcity of annotated samples and difficulty in data collection. In the case of deep learning based methods, the demand for large amounts of labeled data further impedes successful application. We propose a synthesize and segment approach with plug in possibilities for segmentation to address this. We show that an adversarially learned image-to-image translation network can synthesize catheters in X-ray fluoroscopy enabling data augmentation in order to alleviate a low data regime. To make realistic synthesized images, we train the translation network via a perceptual loss coupled with similarity constraints. Then existing segmentation networks are used to learn accurate localization of catheters in a semi-supervised setting with the generated images. The empirical results on collected medical datasets show the value of our approach with significant improvements over existing translation baseline methods.

Detection of peripherally inserted central catheter (PICC) in chest X-ray images: A multi-task deep learning model

BACKGROUND AND OBJECTIVE

Peripherally inserted central catheter (PICC) is a novel drug delivery mode which has been widely used in clinical practice. However, long-term retention and some improper actions of patients may cause some severe complications of PICC, such as the drift and prolapse of its catheter. Clinically, the postoperative care of PICC is mainly completed by nurses. However, they cannot recognize the correct position of PICC from X-ray chest images as soon as the complications happen, which may lead to improper treatment. Therefore, it is necessary to identify the position of the PICC catheter as soon as these complications occur. Here we proposed a novel multi-task deep learning framework to detect PICC automatically through X-ray images, which could help nurses to solve this problem.

METHODS

We collected 348 X-ray chest images from 326 patients with visible PICC. Then we proposed a multi-task deep learning framework for line segmentation and tip detection of PICC catheters simultaneously. The proposed deep learning model is composed of an extraction structure and three routes, an up-sampling route for segmentation, an RPNs route, and an RoI Pooling route for detection. We further compared the effectiveness of our model with the models previously proposed.

RESULTS

In the catheter segmentation task, 300 X-ray images were utilized for training the model, then 48 images were tested. In the tip detection task, 154 X-ray images were used for retraining and 20 images were used in the test. Our model achieved generally better results among several popular deep learning models previously proposed.

CONCLUSIONS

We proposed a multi-task deep learning model that could segment the catheter and detect the tip of PICC simultaneously from X-ray chest images. This model could help nurses to recognize the correct position of PICC, and therefore, to handle the potential complications properly.

Artificial Intelligence Solutions for Analysis of X-ray Images

Artificial intelligence (AI) presents a key opportunity for radiologists to improve quality of care and enhance the value of radiology in patient care and population health. The potential opportunity of AI to aid in triage and interpretation of conventional radiographs (X-ray images) is particularly significant, as radiographs are the most common imaging examinations performed in most radiology departments. Substantial progress has been made in the past few years in the development of AI algorithms for analysis of chest and musculoskeletal (MSK) radiographs, with deep learning now the dominant approach for image analysis. Large public and proprietary image data sets have been compiled and have aided the development of AI algorithms for analysis of radiographs, many of which demonstrate accuracy equivalent to radiologists for specific, focused tasks. This article describes (1) the basis for the development of AI solutions for radiograph analysis, (2) current AI solutions to aid in the triage and interpretation of chest radiographs and MSK radiographs, (3) opportunities for AI to aid in noninterpretive tasks related to radiographs, and (4) considerations for radiology practices selecting AI solutions for radiograph analysis and integrating them into existing IT systems. Although comprehensive AI solutions across modalities have yet to be developed, institutions can begin to select and integrate focused solutions which increase efficiency, increase quality and patient safety, and add value for their patients.

Computer-aided Assessment of Catheters and Tubes on Radiographs: How Good Is Artificial Intelligence for Assessment?

Catheters are the second most common abnormal finding on radiographs. The position of catheters must be assessed on all radiographs because serious complications can arise if catheters are malpositioned. However, due to the large number of radiographs obtained each day, there can be substantial delays between the time a radiograph is obtained and when it is interpreted by a radiologist. Computer-aided approaches hold the potential to assist in prioritizing radiographs with potentially malpositioned catheters for interpretation and automatically insert text indicating the placement of catheters in radiology reports, thereby improving radiologists' efficiency. After 50 years of research in computer-aided diagnosis, there is still a paucity of study in this area. With the development of deep learning approaches, the problem of catheter assessment is far more solvable. This review provides an overview of current algorithms and identifies key challenges in building a reliable computer-aided diagnosis system for assessment of catheters on radiographs. This review may serve to further the development of machine learning approaches for this important use case. Supplemental material is available for this article. © RSNA, 2020.