1
|
Kyung S, Jang M, Park S, Yoon HM, Hong GS, Kim N. Supervised representation learning based on various levels of pediatric radiographic views for transfer learning. Sci Rep 2024; 14:7551. [PMID: 38555414 PMCID: PMC10981659 DOI: 10.1038/s41598-024-58163-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2023] [Accepted: 03/26/2024] [Indexed: 04/02/2024] Open
Abstract
Transfer learning plays a pivotal role in addressing the paucity of data, expediting training processes, and enhancing model performance. Nonetheless, the prevailing practice of transfer learning predominantly relies on pre-trained models designed for the natural image domain, which may not be well-suited for the medical image domain in grayscale. Recognizing the significance of leveraging transfer learning in medical research, we undertook the construction of class-balanced pediatric radiograph datasets collectively referred to as PedXnets, grounded in radiographic views using the pediatric radiographs collected over 24 years at Asan Medical Center. For PedXnets pre-training, approximately 70,000 X-ray images were utilized. Three different pre-training weights of PedXnet were constructed using Inception V3 for various radiation perspective classifications: Model-PedXnet-7C, Model-PedXnet-30C, and Model-PedXnet-68C. We validated the transferability and positive effects of transfer learning of PedXnets through pediatric downstream tasks including fracture classification and bone age assessment (BAA). The evaluation of transfer learning effects through classification and regression metrics showed superior performance of Model-PedXnets in quantitative assessments. Additionally, visual analyses confirmed that the Model-PedXnets were more focused on meaningful regions of interest.
Collapse
Affiliation(s)
- Sunggu Kyung
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, College of Medicine, University of Ulsan, Seoul, Republic of Korea
| | - Miso Jang
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, 88 Olympic-Ro 43-Gil, Songpa-gu, Seoul, 05505, Republic of Korea
| | - Seungju Park
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, 88 Olympic-Ro 43-Gil, Songpa-gu, Seoul, 05505, Republic of Korea
| | - Hee Mang Yoon
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88 Olympic-Ro 43-Gil, Songpa-gu, Seoul, 05505, Republic of Korea
| | - Gil-Sun Hong
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88 Olympic-Ro 43-Gil, Songpa-gu, Seoul, 05505, Republic of Korea
| | - Namkug Kim
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, 88 Olympic-Ro 43-Gil, Songpa-gu, Seoul, 05505, Republic of Korea.
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88 Olympic-Ro 43-Gil, Songpa-gu, Seoul, 05505, Republic of Korea.
| |
Collapse
|
2
|
Park S, Cha YK, Park S, Chung MJ, Kim K. Automated precision localization of peripherally inserted central catheter tip through model-agnostic multi-stage networks. Artif Intell Med 2023; 144:102643. [PMID: 37783538 DOI: 10.1016/j.artmed.2023.102643] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2022] [Revised: 05/30/2023] [Accepted: 08/28/2023] [Indexed: 10/04/2023]
Abstract
BACKGROUND Peripherally inserted central catheters (PICCs) have been widely used as one of the representative central venous lines (CVCs) due to their long-term intravascular access with low infectivity. However, PICCs have a fatal drawback of a high frequency of tip mispositions, increasing the risk of puncture, embolism, and complications such as cardiac arrhythmias. To automatically and precisely detect it, various attempts have been made by using the latest deep learning (DL) technologies. However, even with these approaches, it is still practically difficult to determine the tip location because the multiple fragments phenomenon (MFP) occurs in the process of predicting and extracting the PICC line required before predicting the tip. OBJECTIVE This study aimed to develop a system generally applied to existing models and to restore the PICC line more exactly by removing the MFs of the model output, thereby precisely localizing the actual tip position for detecting its misposition. METHODS To achieve this, we proposed a multi-stage DL-based framework post-processing the PICC line extraction result of the existing technology. Our method consists of the following three stages: 1. Existing PICC line segmentation network for a baseline, 2. Patch-based PICC line refinement network, 3. PICC line reconnection network. The proposed second and third-stage models address MFs caused by the sparseness of the PICC line and the line disconnection due to confusion with anatomical structures respectively, thereby enhancing tip detection. RESULTS To verify the objective performance of the proposed MFCN, internal validation and external validation were conducted. For internal validation, learning (130 samples) and verification (150 samples) were performed with 280 data, including PICC among Chest X-ray (CXR) images taken at our institution. External validation was conducted using a public dataset called the Royal Australian and New Zealand College of Radiologists (RANZCR), and training (130 samples) and validation (150 samples) were performed with 280 data of CXR images, including PICC, which has the same number as that for internal validation. The performance was compared by root mean squared error (RMSE) and the ratio of single fragment images (RatioSFI) (i.e., the rate at which model predicts PICC as multiple sub-lines) according to whether or not MFCN is applied to seven conventional models (i.e., FCDN, UNET, AUNET, TUNET, FCDN-HT, UNET-ELL, and UNET-RPN). In internal validation, when MFCN was applied to the existing single model, MFP was improved by an average of 45 %. The RMSE improved over 63% from an average of 27.54 mm (17.16 to 35.80 mm) to 9.77 mm (9.11 to 10.98 mm). In external validation, when MFCN was applied, the MFP incidence rate decreased by an average of 32% and the RMSE decreased by an average of 65%. Therefore, by applying the proposed MFCN, we observed the consistent detection performance improvement of PICC tip location compared to the existing model. CONCLUSION In this study, we applied the proposed technique to the existing technique and demonstrated that it provides high tip detection performance, proving its high versatility and superiority. Therefore, we believe, in countries and regions where radiologists are scarce, that the proposed DL approach will be able to effectively detect PICC misposition on behalf of radiologists.
Collapse
Affiliation(s)
- Subin Park
- Department of Health Sciences and Technology, SAIHST, Sungkyunkwan University, Seoul 06351, Republic of Korea
| | - Yoon Ki Cha
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul 06351, Republic of Korea
| | - Soyoung Park
- Department of Health Sciences and Technology, SAIHST, Sungkyunkwan University, Seoul 06351, Republic of Korea
| | - Myung Jin Chung
- Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul 06351, Republic of Korea; Medical AI Research Center, Research Institute for Future Medicine, Samsung Medical Center, Seoul 06351, Republic of Korea.
| | - Kyungsu Kim
- Medical AI Research Center, Research Institute for Future Medicine, Samsung Medical Center, Seoul 06351, Republic of Korea; Department of Data Convergence and Future Medicine, Sungkyunkwan University School of Medicine, Seoul 06351, Republic of Korea.
| |
Collapse
|
3
|
Kordon F, Stiglmayr M, Maier A, Martín Vicario C, Pertlwieser T, Kunze H. A principled representation of elongated structures using heatmaps. Sci Rep 2023; 13:15253. [PMID: 37709790 PMCID: PMC10502041 DOI: 10.1038/s41598-023-41221-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Accepted: 08/23/2023] [Indexed: 09/16/2023] Open
Abstract
The detection of elongated structures like lines or edges is an essential component in semantic image analysis. Classical approaches that rely on significant image gradients quickly reach their limits when the structure is context-dependent, amorphous, or not directly visible. This study introduces a principled mathematical description of elongated structures with various origins and shapes. Among others, it serves as an expressive operational description of target functions that can be well approximated by Convolutional Neural Networks. The nominal position of a curve and its positional uncertainty are encoded as a heatmap by convolving the curve distribution with a filter function. We propose a low-error approximation to the expensive numerical integration by evaluating a distance-dependent function, enabling a lightweight implementation with linear time complexity. We analyze the method's numerical approximation error and behavior for different curve types and signal-to-noise levels. Application to surgical 2D and 3D data, semantic boundary detection, skeletonization, and other related tasks demonstrate the method's versatility at low errors.
Collapse
Affiliation(s)
- Florian Kordon
- Pattern Recognition Lab, Friedrich-Alexander Universität Erlangen-Nürnberg, 91058, Erlangen, Germany.
- Erlangen Graduate School in Advanced Optical Technologies (SAOT), Friedrich-Alexander Universität Erlangen-Nürnberg, 91052, Erlangen, Germany.
- Advanced Therapies, Siemens Healthcare GmbH, 91301, Forchheim, Germany.
| | - Michael Stiglmayr
- Optimization Group, Institute of Mathematical Modelling, Analysis and Computational Mathematics, University of Wuppertal, 42119, Wuppertal, Germany
| | - Andreas Maier
- Pattern Recognition Lab, Friedrich-Alexander Universität Erlangen-Nürnberg, 91058, Erlangen, Germany
- Erlangen Graduate School in Advanced Optical Technologies (SAOT), Friedrich-Alexander Universität Erlangen-Nürnberg, 91052, Erlangen, Germany
| | - Celia Martín Vicario
- Pattern Recognition Lab, Friedrich-Alexander Universität Erlangen-Nürnberg, 91058, Erlangen, Germany
| | - Tobias Pertlwieser
- Pattern Recognition Lab, Friedrich-Alexander Universität Erlangen-Nürnberg, 91058, Erlangen, Germany
| | - Holger Kunze
- Pattern Recognition Lab, Friedrich-Alexander Universität Erlangen-Nürnberg, 91058, Erlangen, Germany
- Advanced Therapies, Siemens Healthcare GmbH, 91301, Forchheim, Germany
| |
Collapse
|
4
|
Tang CHM, Seah JCY, Ahmad HK, Milne MR, Wardman JB, Buchlak QD, Esmaili N, Lambert JF, Jones CM. Analysis of Line and Tube Detection Performance of a Chest X-ray Deep Learning Model to Evaluate Hidden Stratification. Diagnostics (Basel) 2023; 13:2317. [PMID: 37510062 PMCID: PMC10378683 DOI: 10.3390/diagnostics13142317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Revised: 07/05/2023] [Accepted: 07/07/2023] [Indexed: 07/30/2023] Open
Abstract
This retrospective case-control study evaluated the diagnostic performance of a commercially available chest radiography deep convolutional neural network (DCNN) in identifying the presence and position of central venous catheters, enteric tubes, and endotracheal tubes, in addition to a subgroup analysis of different types of lines/tubes. A held-out test dataset of 2568 studies was sourced from community radiology clinics and hospitals in Australia and the USA, and was then ground-truth labelled for the presence, position, and type of line or tube from the consensus of a thoracic specialist radiologist and an intensive care clinician. DCNN model performance for identifying and assessing the positioning of central venous catheters, enteric tubes, and endotracheal tubes over the entire dataset, as well as within each subgroup, was evaluated. The area under the receiver operating characteristic curve (AUC) was assessed. The DCNN algorithm displayed high performance in detecting the presence of lines and tubes in the test dataset with AUCs > 0.99, and good position classification performance over a subpopulation of ground truth positive cases with AUCs of 0.86-0.91. The subgroup analysis showed that model performance was robust across the various subtypes of lines or tubes, although position classification performance of peripherally inserted central catheters was relatively lower. Our findings indicated that the DCNN algorithm performed well in the detection and position classification of lines and tubes, supporting its use as an assistant for clinicians. Further work is required to evaluate performance in rarer scenarios, as well as in less common subgroups.
Collapse
Affiliation(s)
- Cyril H M Tang
- Annalise.ai, Sydney, NSW 2000, Australia
- Intensive Care Unit, Gosford Hospital, Sydney, NSW 2250, Australia
| | - Jarrel C Y Seah
- Annalise.ai, Sydney, NSW 2000, Australia
- Department of Radiology, Alfred Health, Melbourne, VIC 3004, Australia
| | | | | | | | - Quinlan D Buchlak
- Annalise.ai, Sydney, NSW 2000, Australia
- School of Medicine, The University of Notre Dame Australia, Sydney, NSW 2007, Australia
- Department of Neurosurgery, Monash Health, Melbourne, VIC 3168, Australia
| | - Nazanin Esmaili
- School of Medicine, The University of Notre Dame Australia, Sydney, NSW 2007, Australia
- Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo, NSW 2007, Australia
| | | | - Catherine M Jones
- Annalise.ai, Sydney, NSW 2000, Australia
- I-MED Radiology Network, Brisbane, QLD 4006, Australia
- School of Public and Preventive Health, Monash University, Clayton, VIC 3800, Australia
- Department of Clinical Imaging Science, University of Sydney, Sydney, NSW 2006, Australia
| |
Collapse
|
5
|
Sun X, Niwa T, Okazaki T, Kameda S, Shibukawa S, Horie T, Kazama T, Uchiyama A, Hashimoto J. Automatic detection of punctate white matter lesions in infants using deep learning of composite images from two cases. Sci Rep 2023; 13:4426. [PMID: 36932141 PMCID: PMC10023755 DOI: 10.1038/s41598-023-31403-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2022] [Accepted: 03/11/2023] [Indexed: 03/19/2023] Open
Abstract
Punctate white matter lesions (PWMLs) in infants may be related to neurodevelopmental outcomes based on the location or number of lesions. This study aimed to assess the automatic detectability of PWMLs in infants on deep learning using composite images created from several cases. To create the initial composite images, magnetic resonance (MR) images of two infants with the most PWMLs were used; their PWMLs were extracted and pasted onto MR images of infants without abnormality, creating many composite PWML images. Deep learning models based on a convolutional neural network, You Only Look Once v3 (YOLOv3), were constructed using the training set of 600, 1200, 2400, and 3600 composite images. As a result, a threshold of detection probability of 20% and 30% for all deep learning model sets yielded a relatively high sensitivity for automatic PWML detection (0.908-0.957). Although relatively high false-positive detections occurred with the lower threshold of detection probability, primarily, in the partial volume of the cerebral cortex (≥ 85.8%), those can be easily distinguished from the white matter lesions. Relatively highly sensitive automatic detection of PWMLs was achieved by creating composite images from two cases using deep learning.
Collapse
Affiliation(s)
- Xuyang Sun
- Department of Radiology, Tokai University School of Medicine, 143 Shimokasuka, Isehara, 259-1193, Japan
| | - Tetsu Niwa
- Department of Radiology, Tokai University School of Medicine, 143 Shimokasuka, Isehara, 259-1193, Japan.
| | - Takashi Okazaki
- Department of Radiology, Tokai University School of Medicine, 143 Shimokasuka, Isehara, 259-1193, Japan
| | - Sadanori Kameda
- Department of Radiology, Tokai University School of Medicine, 143 Shimokasuka, Isehara, 259-1193, Japan
| | - Shuhei Shibukawa
- Department of Radiology, Tokai University School of Medicine, 143 Shimokasuka, Isehara, 259-1193, Japan
- Department of Radiological Technology, Faculty of Health Science, Juntendo University, Bunkyo-Ku, Tokyo, Japan
| | - Tomohiko Horie
- Department of Radiology, Tokai University Hospital, Isehara, Japan
| | - Toshiki Kazama
- Department of Radiology, Tokai University School of Medicine, 143 Shimokasuka, Isehara, 259-1193, Japan
| | - Atsushi Uchiyama
- Department of Pediatrics, Tokai University School of Medicine, Isehara, Japan
| | - Jun Hashimoto
- Department of Radiology, Tokai University School of Medicine, 143 Shimokasuka, Isehara, 259-1193, Japan
| |
Collapse
|
6
|
Guan S, Li T, Meng C, Ma L. Multi-mode information fusion navigation system for robot-assisted vascular interventional surgery. BMC Surg 2023; 23:51. [PMID: 36894932 PMCID: PMC9996930 DOI: 10.1186/s12893-023-01944-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2022] [Accepted: 02/23/2023] [Indexed: 03/11/2023] Open
Abstract
BACKGROUND Minimally invasive vascular intervention (MIVI) is a powerful technique for the treatment of cardiovascular diseases, such as abdominal aortic aneurysm (AAA), thoracic aortic aneurysm (TAA) and aortic dissection (AD). Navigation of traditional MIVI surgery mainly relies only on 2D digital subtraction angiography (DSA) images, which is hard to observe the 3D morphology of blood vessels and position the interventional instruments. The multi-mode information fusion navigation system (MIFNS) proposed in this paper combines preoperative CT images and intraoperative DSA images together to increase the visualization information during operations. RESULTS The main functions of MIFNS were evaluated by real clinical data and a vascular model. The registration accuracy of preoperative CTA images and intraoperative DSA images were less than 1 mm. The positioning accuracy of surgical instruments was quantitatively assessed using a vascular model and was also less than 1 mm. Real clinical data used to assess the navigation results of MIFNS on AAA, TAA and AD. CONCLUSIONS A comprehensive and effective navigation system was developed to facilitate the operation of surgeon during MIVI. The registration accuracy and positioning accuracy of the proposed navigation system were both less than 1 mm, which met the accuracy requirements of robot assisted MIVI.
Collapse
Affiliation(s)
- Shaoya Guan
- School of Engineers, Beijing Institute of Petrochemical Technology, Beijing, China
| | - Tianqi Li
- School of Information Engineering, Beijing Institute of Petrochemical Technology, Beijing, China
| | - Cai Meng
- School of Astronautics, Beihang University, Beijing, China
| | - Limei Ma
- School of Engineers, Beijing Institute of Petrochemical Technology, Beijing, China.
| |
Collapse
|
7
|
A Review of Synthetic Image Data and Its Use in Computer Vision. J Imaging 2022; 8:jimaging8110310. [PMID: 36422059 PMCID: PMC9698631 DOI: 10.3390/jimaging8110310] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Accepted: 11/15/2022] [Indexed: 11/23/2022] Open
Abstract
Development of computer vision algorithms using convolutional neural networks and deep learning has necessitated ever greater amounts of annotated and labelled data to produce high performance models. Large, public data sets have been instrumental in pushing forward computer vision by providing the data necessary for training. However, many computer vision applications cannot rely on general image data provided in the available public datasets to train models, instead requiring labelled image data that is not readily available in the public domain on a large scale. At the same time, acquiring such data from the real world can be difficult, costly to obtain, and manual labour intensive to label in large quantities. Because of this, synthetic image data has been pushed to the forefront as a potentially faster and cheaper alternative to collecting and annotating real data. This review provides general overview of types of synthetic image data, as categorised by synthesised output, common methods of synthesising different types of image data, existing applications and logical extensions, performance of synthetic image data in different applications and the associated difficulties in assessing data performance, and areas for further research.
Collapse
|
8
|
Guo WL, Geng AK, Geng C, Wang J, Dai YK. Combination of UNet++ and ResNeSt to classify chronic inflammation of the choledochal cystic wall in patients with pancreaticobiliary maljunction. Br J Radiol 2022; 95:20201189. [PMID: 35451311 PMCID: PMC10996311 DOI: 10.1259/bjr.20201189] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2020] [Revised: 03/10/2022] [Accepted: 04/01/2022] [Indexed: 11/05/2022] Open
Abstract
OBJECTIVES The aim of this study was to establish an automatic classification model for chronic inflammation of the choledoch wall using deep learning with CT images in patients with pancreaticobiliary maljunction (PBM). METHODS CT images were obtained from 76 PBM patients, including 61 cases assigned to the training set and 15 cases assigned to the testing set. The region of interest (ROI) containing the choledochal lesion was extracted and segmented using the UNet++ network. The degree of severity of inflammation in the choledochal wall was initially classified using the ResNeSt network. The final classification result was determined per decision rules. Grad-CAM was used to explain the association between the classification basis of the network and clinical diagnosis. RESULTS Segmentation of the lesion on the common bile duct wall was roughly obtained with the UNet++ segmentation model and the average value of Dice coefficient of the segmentation model in the testing set was 0.839 ± 0.150, which was verified through fivefold cross-validation. Inflammation was initially classified with ResNeSt18, which resulted in accuracy = 0.756, sensitivity = 0.611, specificity = 0.852, precision = 0.733, and area under curve (AUC) = 0.711. The final classification sensitivity was 0.8. Grad-CAM revealed similar distribution of inflammation of the choledochal wall and verified the inflammation classification. CONCLUSIONS By combining the UNet++ network and the ResNeSt network, we achieved automatic classification of chronic inflammation of the choledoch in PBM patients and verified the robustness through cross-validation performed five times. This study provided an important basis for classification of inflammation severity of the choledoch in PBM patients. ADVANCES IN KNOWLEDGE We combined the UNet++ network and the ResNeSt network to achieve automatic classification of chronic inflammation of the choledoch in PBM. These results provided an important basis for classification of choledochal inflammation in PBM and for surgical therapy.
Collapse
Affiliation(s)
- Wan-liang Guo
- Department of Radiology, Children’s Hospital of Soochow
University, Suzhou,
China
| | - An-kang Geng
- School of Biomedical Engineering (Suzhou), Division of Life
Sciences and Medicine, University of Science and Technology of China, 88
Keling Road, Suzhou,
China
- Suzhou Institute of Biomedical Engineering and Technology,
Chinese Academy of Sciences, 88 Keling Road,
Suzhou, China
| | - Chen Geng
- Suzhou Institute of Biomedical Engineering and Technology,
Chinese Academy of Sciences, 88 Keling Road,
Suzhou, China
| | - Jian Wang
- Pediatric Surgery, Children’s Hospital of Soochow
University, Suzhou,
China
| | - Ya-kang Dai
- Suzhou Institute of Biomedical Engineering and Technology,
Chinese Academy of Sciences, 88 Keling Road,
Suzhou, China
- Jinan Guoke Medical Engineering Technology Development Co.
LTD, Jinan,
China
| |
Collapse
|
9
|
Tubular shape aware data generation for segmentation in medical imaging. Int J Comput Assist Radiol Surg 2022; 17:1091-1099. [DOI: 10.1007/s11548-022-02621-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2021] [Accepted: 03/23/2022] [Indexed: 11/05/2022]
|
10
|
Kim KY, Nowrangi R, McGehee A, Joshi N, Acharya PT. Assessment of germinal matrix hemorrhage on head ultrasound with deep learning algorithms. Pediatr Radiol 2022; 52:533-538. [PMID: 35064324 DOI: 10.1007/s00247-021-05239-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/14/2021] [Revised: 09/15/2021] [Accepted: 10/31/2021] [Indexed: 11/30/2022]
Abstract
BACKGROUND Germinal matrix hemorrhage-intraventricular hemorrhage is among the most common intracranial complications in premature infants. Early detection is important to guide clinical management for improved patient prognosis. OBJECTIVE The purpose of this study was to assess whether a convolutional neural network (CNN) can be trained via transfer learning to accurately diagnose germinal matrix hemorrhage on head ultrasound. MATERIALS AND METHODS Over a 10-year period, 400 head ultrasounds performed in patients ages 6 months or younger were reviewed. Key sagittal images at the level of the caudothalamic groove were obtained from 200 patients with germinal matrix hemorrhage and 200 patients without hemorrhage; all images were reviewed by a board-certified pediatric radiologist. One hundred cases were randomly allocated from the total for validation and an additional 100 for testing of a CNN binary classifier. Transfer learning and data augmentation were used to train the model. RESULTS The median age of patients was 0 weeks old with a median gestational age of 30 weeks. The final trained CNN model had a receiver operating characteristic area under the curve of 0.92 on the validation set and accuracy of 0.875 on the test set, with 95% confidence intervals of [0.86, 0.98] and [0.81, 0.94], respectively. CONCLUSION A CNN trained on a small set of images with data augmentation can detect germinal matrix hemorrhage on head ultrasounds with strong accuracy.
Collapse
Affiliation(s)
- Kevin Y Kim
- Department of Radiology, Loma Linda University Medical Center, Loma Linda, CA, USA
| | - Rajeev Nowrangi
- Department of Radiology, Loma Linda University Medical Center, Loma Linda, CA, USA
| | - Arianna McGehee
- Loma Linda University School of Medicine, Loma Linda, CA, USA
| | - Neil Joshi
- Loma Linda University School of Medicine, Loma Linda, CA, USA
| | - Patricia T Acharya
- Loma Linda University School of Medicine, Loma Linda, CA, USA. .,Department of Radiology, Children's Hospital Los Angeles, 4650 Sunset Blvd., Mailstop #81, Los Angeles, CA, 90027, USA. .,Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
| |
Collapse
|
11
|
Padash S, Mohebbian MR, Adams SJ, Henderson RDE, Babyn P. Pediatric chest radiograph interpretation: how far has artificial intelligence come? A systematic literature review. Pediatr Radiol 2022; 52:1568-1580. [PMID: 35460035 PMCID: PMC9033522 DOI: 10.1007/s00247-022-05368-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Revised: 02/28/2022] [Accepted: 03/24/2022] [Indexed: 10/24/2022]
Abstract
Most artificial intelligence (AI) studies have focused primarily on adult imaging, with less attention to the unique aspects of pediatric imaging. The objectives of this study were to (1) identify all publicly available pediatric datasets and determine their potential utility and limitations for pediatric AI studies and (2) systematically review the literature to assess the current state of AI in pediatric chest radiograph interpretation. We searched PubMed, Web of Science and Embase to retrieve all studies from 1990 to 2021 that assessed AI for pediatric chest radiograph interpretation and abstracted the datasets used to train and test AI algorithms, approaches and performance metrics. Of 29 publicly available chest radiograph datasets, 2 datasets included solely pediatric chest radiographs, and 7 datasets included pediatric and adult patients. We identified 55 articles that implemented an AI model to interpret pediatric chest radiographs or pediatric and adult chest radiographs. Classification of chest radiographs as pneumonia was the most common application of AI, evaluated in 65% of the studies. Although many studies report high diagnostic accuracy, most algorithms were not validated on external datasets. Most AI studies for pediatric chest radiograph interpretation have focused on a limited number of diseases, and progress is hindered by a lack of large-scale pediatric chest radiograph datasets.
Collapse
Affiliation(s)
- Sirwa Padash
- Department of Medical Imaging, University of Saskatchewan, 103 Hospital Drive, Saskatoon, Saskatchewan, S7N 0W8, Canada. .,Department of Radiology, Mayo Clinic, Rochester, MN, USA.
| | - Mohammad Reza Mohebbian
- grid.25152.310000 0001 2154 235XDepartment of Electrical and Computer Engineering, University of Saskatchewan, Saskatoon, Saskatchewan Canada
| | - Scott J. Adams
- grid.25152.310000 0001 2154 235XDepartment of Medical Imaging, University of Saskatchewan, 103 Hospital Drive, Saskatoon, Saskatchewan S7N 0W8 Canada
| | - Robert D. E. Henderson
- grid.25152.310000 0001 2154 235XDepartment of Medical Imaging, University of Saskatchewan, 103 Hospital Drive, Saskatoon, Saskatchewan S7N 0W8 Canada
| | - Paul Babyn
- grid.25152.310000 0001 2154 235XDepartment of Medical Imaging, University of Saskatchewan, 103 Hospital Drive, Saskatoon, Saskatchewan S7N 0W8 Canada
| |
Collapse
|
12
|
Current and emerging artificial intelligence applications in chest imaging: a pediatric perspective. Pediatr Radiol 2022; 52:2120-2130. [PMID: 34471961 PMCID: PMC8409695 DOI: 10.1007/s00247-021-05146-0] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Revised: 05/22/2021] [Accepted: 06/28/2021] [Indexed: 12/19/2022]
Abstract
Artificial intelligence (AI) applications for chest radiography and chest CT are among the most developed applications in radiology. More than 40 certified AI products are available for chest radiography or chest CT. These AI products cover a wide range of abnormalities, including pneumonia, pneumothorax and lung cancer. Most applications are aimed at detecting disease, complemented by products that characterize or quantify tissue. At present, none of the thoracic AI products is specifically designed for the pediatric population. However, some products developed to detect tuberculosis in adults are also applicable to children. Software is under development to detect early changes of cystic fibrosis on chest CT, which could be an interesting application for pediatric radiology. In this review, we give an overview of current AI products in thoracic radiology and cover recent literature about AI in chest radiography, with a focus on pediatric radiology. We also discuss possible pediatric applications.
Collapse
|
13
|
Henderson RDE, Yi X, Adams SJ, Babyn P. Automatic Detection and Classification of Multiple Catheters in Neonatal Radiographs with Deep Learning. J Digit Imaging 2021; 34:888-897. [PMID: 34173089 DOI: 10.1007/s10278-021-00473-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2020] [Revised: 06/01/2021] [Accepted: 06/09/2021] [Indexed: 12/18/2022] Open
Abstract
We develop and evaluate a deep learning algorithm to classify multiple catheters on neonatal chest and abdominal radiographs. A convolutional neural network (CNN) was trained using a dataset of 777 neonatal chest and abdominal radiographs, with a split of 81%-9%-10% for training-validation-testing, respectively. We employed ResNet-50 (a CNN), pre-trained on ImageNet. Ground truth labelling was limited to tagging each image to indicate the presence or absence of endotracheal tubes (ETTs), nasogastric tubes (NGTs), and umbilical arterial and venous catheters (UACs, UVCs). The dataset included 561 images containing two or more catheters, 167 images with only one, and 49 with none. Performance was measured with average precision (AP), calculated from the area under the precision-recall curve. On our test data, the algorithm achieved an overall AP (95% confidence interval) of 0.977 (0.679-0.999) for NGTs, 0.989 (0.751-1.000) for ETTs, 0.979 (0.873-0.997) for UACs, and 0.937 (0.785-0.984) for UVCs. Performance was similar for the set of 58 test images consisting of two or more catheters, with an AP of 0.975 (0.255-1.000) for NGTs, 0.997 (0.009-1.000) for ETTs, 0.981 (0.797-0.998) for UACs, and 0.937 (0.689-0.990) for UVCs. Our network thus achieves strong performance in the simultaneous detection of these four catheter types. Radiologists may use such an algorithm as a time-saving mechanism to automate reporting of catheters on radiographs.
Collapse
Affiliation(s)
- Robert D E Henderson
- Department of Medical Imaging, University of Saskatchewan, 103 Hospital Drive, Room 1566, Saskatoon, SK, S7N 0W8, Canada.
| | - Xin Yi
- Department of Medical Imaging, University of Saskatchewan, 103 Hospital Drive, Room 1566, Saskatoon, SK, S7N 0W8, Canada
| | - Scott J Adams
- Department of Medical Imaging, University of Saskatchewan, 103 Hospital Drive, Room 1566, Saskatoon, SK, S7N 0W8, Canada
| | - Paul Babyn
- Department of Medical Imaging, University of Saskatchewan, 103 Hospital Drive, Room 1566, Saskatoon, SK, S7N 0W8, Canada
| |
Collapse
|
14
|
Çallı E, Sogancioglu E, van Ginneken B, van Leeuwen KG, Murphy K. Deep learning for chest X-ray analysis: A survey. Med Image Anal 2021; 72:102125. [PMID: 34171622 DOI: 10.1016/j.media.2021.102125] [Citation(s) in RCA: 103] [Impact Index Per Article: 34.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2021] [Revised: 05/17/2021] [Accepted: 05/27/2021] [Indexed: 12/14/2022]
Abstract
Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs published before March 2021, categorizing works by task: image-level prediction (classification and regression), segmentation, localization, image generation and domain adaptation. Detailed descriptions of all publicly available datasets are included and commercial systems in the field are described. A comprehensive discussion of the current state of the art is provided, including caveats on the use of public datasets, the requirements of clinically useful systems and gaps in the current literature.
Collapse
Affiliation(s)
- Erdi Çallı
- Radboud University Medical Center, Institute for Health Sciences, Department of Medical Imaging, Nijmegen, the Netherlands.
| | - Ecem Sogancioglu
- Radboud University Medical Center, Institute for Health Sciences, Department of Medical Imaging, Nijmegen, the Netherlands
| | - Bram van Ginneken
- Radboud University Medical Center, Institute for Health Sciences, Department of Medical Imaging, Nijmegen, the Netherlands
| | - Kicky G van Leeuwen
- Radboud University Medical Center, Institute for Health Sciences, Department of Medical Imaging, Nijmegen, the Netherlands
| | - Keelin Murphy
- Radboud University Medical Center, Institute for Health Sciences, Department of Medical Imaging, Nijmegen, the Netherlands
| |
Collapse
|
15
|
An Intelligent Future for Medical Imaging: A Market Outlook on Artificial Intelligence for Medical Imaging. J Am Coll Radiol 2021; 17:165-170. [PMID: 31918875 DOI: 10.1016/j.jacr.2019.07.019] [Citation(s) in RCA: 43] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2019] [Accepted: 07/23/2019] [Indexed: 01/03/2023]
Abstract
Radiologists today are under increasing work pressure. We surveyed radiologists in the United States across practice settings, and the overwhelming majority reported an increased workload. Artificial intelligence (AI), which includes machine learning, can help address these issues. It also has the potential to improve clinical outcomes and raise further the value of medical imaging in ways yet to be defined. In this article, we report on recent McKinsey & Company work to understand the growth of AI in medical imaging. We highlight progress in its clinical application, the investments that are backing it, and the barriers to broader adoption. We also offer a view on how the market will develop. AI is set to have a big impact on the medical imaging market and hence on how radiologists work, helping them to speed up scan time, make more accurate diagnoses, and ease their workload. As AI in medical imaging increasingly proves its worth, it is hard to imagine that AI will not ultimately transform radiology.
Collapse
|
16
|
Synthesize and Segment: Towards Improved Catheter Segmentation via Adversarial Augmentation. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11041638] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Automatic catheter and guidewire segmentation plays an important role in robot-assisted interventions that are guided by fluoroscopy. Existing learning based methods addressing the task of segmentation or tracking are often limited by the scarcity of annotated samples and difficulty in data collection. In the case of deep learning based methods, the demand for large amounts of labeled data further impedes successful application. We propose a synthesize and segment approach with plug in possibilities for segmentation to address this. We show that an adversarially learned image-to-image translation network can synthesize catheters in X-ray fluoroscopy enabling data augmentation in order to alleviate a low data regime. To make realistic synthesized images, we train the translation network via a perceptual loss coupled with similarity constraints. Then existing segmentation networks are used to learn accurate localization of catheters in a semi-supervised setting with the generated images. The empirical results on collected medical datasets show the value of our approach with significant improvements over existing translation baseline methods.
Collapse
|
17
|
Yu D, Zhang K, Huang L, Zhao B, Zhang X, Guo X, Li M, Gu Z, Fu G, Hu M, Ping Y, Sheng Y, Liu Z, Hu X, Zhao R. Detection of peripherally inserted central catheter (PICC) in chest X-ray images: A multi-task deep learning model. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 197:105674. [PMID: 32738678 DOI: 10.1016/j.cmpb.2020.105674] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/19/2020] [Accepted: 07/17/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND AND OBJECTIVE Peripherally inserted central catheter (PICC) is a novel drug delivery mode which has been widely used in clinical practice. However, long-term retention and some improper actions of patients may cause some severe complications of PICC, such as the drift and prolapse of its catheter. Clinically, the postoperative care of PICC is mainly completed by nurses. However, they cannot recognize the correct position of PICC from X-ray chest images as soon as the complications happen, which may lead to improper treatment. Therefore, it is necessary to identify the position of the PICC catheter as soon as these complications occur. Here we proposed a novel multi-task deep learning framework to detect PICC automatically through X-ray images, which could help nurses to solve this problem. METHODS We collected 348 X-ray chest images from 326 patients with visible PICC. Then we proposed a multi-task deep learning framework for line segmentation and tip detection of PICC catheters simultaneously. The proposed deep learning model is composed of an extraction structure and three routes, an up-sampling route for segmentation, an RPNs route, and an RoI Pooling route for detection. We further compared the effectiveness of our model with the models previously proposed. RESULTS In the catheter segmentation task, 300 X-ray images were utilized for training the model, then 48 images were tested. In the tip detection task, 154 X-ray images were used for retraining and 20 images were used in the test. Our model achieved generally better results among several popular deep learning models previously proposed. CONCLUSIONS We proposed a multi-task deep learning model that could segment the catheter and detect the tip of PICC simultaneously from X-ray chest images. This model could help nurses to recognize the correct position of PICC, and therefore, to handle the potential complications properly.
Collapse
Affiliation(s)
- Dingding Yu
- School of Mathematical Sciences, Zhejiang University. Hangzhou, Zhejiang Province, China, 310027
| | - Kaijie Zhang
- Department of Vascular Surgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang Province, China, 310009; Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province. Sir Run Shaw Hospital, School of Medicine, Zhejiang University. Hangzhou, Zhejiang Province, China, 310016
| | - Lingyan Huang
- Department of Radiation Oncology, Zhejiang Quhua Hospital, Quzhou, Zhejiang Province, China, 324000
| | - Bonan Zhao
- School of Mathematical Sciences, Zhejiang University. Hangzhou, Zhejiang Province, China, 310027
| | - Xiaoshan Zhang
- School of Mathematical Sciences, Zhejiang University. Hangzhou, Zhejiang Province, China, 310027
| | - Xin Guo
- Department of Vascular Surgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang Province, China, 310009; Bone Marrow Transplantation Center, The First Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang Province, China, 310000
| | - Miaomiao Li
- Department of Vascular Surgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang Province, China, 310009; Department of Reproductive Endocrinology, Women's Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang Province, China, 310019
| | - Zheng Gu
- Department of Vascular Surgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang Province, China, 310009
| | - Guosheng Fu
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province. Sir Run Shaw Hospital, School of Medicine, Zhejiang University. Hangzhou, Zhejiang Province, China, 310016
| | - Minchun Hu
- Department of Radiation Oncology, Zhejiang Quhua Hospital, Quzhou, Zhejiang Province, China, 324000
| | - Yan Ping
- Department of Radiation Oncology, Zhejiang Quhua Hospital, Quzhou, Zhejiang Province, China, 324000
| | - Ye Sheng
- Department of Nursing, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang Province, China, 310009
| | - Zhenjie Liu
- Department of Vascular Surgery, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang Province, China, 310009.
| | - Xianliang Hu
- School of Mathematical Sciences, Zhejiang University. Hangzhou, Zhejiang Province, China, 310027.
| | - Ruiyi Zhao
- Department of Nursing, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, Zhejiang Province, China, 310009.
| |
Collapse
|
18
|
Adams SJ, Henderson RDE, Yi X, Babyn P. Artificial Intelligence Solutions for Analysis of X-ray Images. Can Assoc Radiol J 2020; 72:60-72. [PMID: 32757950 DOI: 10.1177/0846537120941671] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Artificial intelligence (AI) presents a key opportunity for radiologists to improve quality of care and enhance the value of radiology in patient care and population health. The potential opportunity of AI to aid in triage and interpretation of conventional radiographs (X-ray images) is particularly significant, as radiographs are the most common imaging examinations performed in most radiology departments. Substantial progress has been made in the past few years in the development of AI algorithms for analysis of chest and musculoskeletal (MSK) radiographs, with deep learning now the dominant approach for image analysis. Large public and proprietary image data sets have been compiled and have aided the development of AI algorithms for analysis of radiographs, many of which demonstrate accuracy equivalent to radiologists for specific, focused tasks. This article describes (1) the basis for the development of AI solutions for radiograph analysis, (2) current AI solutions to aid in the triage and interpretation of chest radiographs and MSK radiographs, (3) opportunities for AI to aid in noninterpretive tasks related to radiographs, and (4) considerations for radiology practices selecting AI solutions for radiograph analysis and integrating them into existing IT systems. Although comprehensive AI solutions across modalities have yet to be developed, institutions can begin to select and integrate focused solutions which increase efficiency, increase quality and patient safety, and add value for their patients.
Collapse
Affiliation(s)
- Scott J Adams
- Department of Medical Imaging, Royal University Hospital, 7235University of Saskatchewan, Saskatoon, Canada
| | - Robert D E Henderson
- Department of Medical Imaging, Royal University Hospital, 7235University of Saskatchewan, Saskatoon, Canada
| | - Xin Yi
- Department of Medical Imaging, Royal University Hospital, 7235University of Saskatchewan, Saskatoon, Canada
| | - Paul Babyn
- Department of Medical Imaging, Royal University Hospital, 7235University of Saskatchewan, Saskatoon, Canada
| |
Collapse
|
19
|
Yi X, Adams SJ, Henderson RDE, Babyn P. Computer-aided Assessment of Catheters and Tubes on Radiographs: How Good Is Artificial Intelligence for Assessment? Radiol Artif Intell 2020; 2:e190082. [PMID: 33937813 DOI: 10.1148/ryai.2020190082] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2019] [Revised: 10/11/2019] [Accepted: 10/31/2019] [Indexed: 12/23/2022]
Abstract
Catheters are the second most common abnormal finding on radiographs. The position of catheters must be assessed on all radiographs because serious complications can arise if catheters are malpositioned. However, due to the large number of radiographs obtained each day, there can be substantial delays between the time a radiograph is obtained and when it is interpreted by a radiologist. Computer-aided approaches hold the potential to assist in prioritizing radiographs with potentially malpositioned catheters for interpretation and automatically insert text indicating the placement of catheters in radiology reports, thereby improving radiologists' efficiency. After 50 years of research in computer-aided diagnosis, there is still a paucity of study in this area. With the development of deep learning approaches, the problem of catheter assessment is far more solvable. This review provides an overview of current algorithms and identifies key challenges in building a reliable computer-aided diagnosis system for assessment of catheters on radiographs. This review may serve to further the development of machine learning approaches for this important use case. Supplemental material is available for this article. © RSNA, 2020.
Collapse
Affiliation(s)
- Xin Yi
- Department of Medical Imaging (X.Y., S.J.A., P.B.) and College of Medicine (R.D.E.H.), University of Saskatchewan, 103 Hospital Drive, Saskatoon, SK, Canada S7N 0W8
| | - Scott J Adams
- Department of Medical Imaging (X.Y., S.J.A., P.B.) and College of Medicine (R.D.E.H.), University of Saskatchewan, 103 Hospital Drive, Saskatoon, SK, Canada S7N 0W8
| | - Robert D E Henderson
- Department of Medical Imaging (X.Y., S.J.A., P.B.) and College of Medicine (R.D.E.H.), University of Saskatchewan, 103 Hospital Drive, Saskatoon, SK, Canada S7N 0W8
| | - Paul Babyn
- Department of Medical Imaging (X.Y., S.J.A., P.B.) and College of Medicine (R.D.E.H.), University of Saskatchewan, 103 Hospital Drive, Saskatoon, SK, Canada S7N 0W8
| |
Collapse
|