1
|
Chu M, De Maria GL, Dai R, Benenati S, Yu W, Zhong J, Kotronias R, Walsh J, Andreaggi S, Zuccarelli V, Chai J, Channon K, Banning A, Tu S. DCCAT: Dual-Coordinate Cross-Attention Transformer for thrombus segmentation on coronary OCT. Med Image Anal 2024; 97:103265. [PMID: 39029158 DOI: 10.1016/j.media.2024.103265] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 06/02/2024] [Accepted: 07/01/2024] [Indexed: 07/21/2024]
Abstract
Acute coronary syndromes (ACS) are one of the leading causes of mortality worldwide, with atherosclerotic plaque rupture and subsequent thrombus formation as the main underlying substrate. Thrombus burden evaluation is important for tailoring treatment therapy and predicting prognosis. Coronary optical coherence tomography (OCT) enables in-vivo visualization of thrombus that cannot otherwise be achieved by other image modalities. However, automatic quantification of thrombus on OCT has not been implemented. The main challenges are due to the variation in location, size and irregularities of thrombus in addition to the small data set. In this paper, we propose a novel dual-coordinate cross-attention transformer network, termed DCCAT, to overcome the above challenges and achieve the first automatic segmentation of thrombus on OCT. Imaging features from both Cartesian and polar coordinates are encoded and fused based on long-range correspondence via multi-head cross-attention mechanism. The dual-coordinate cross-attention block is hierarchically stacked amid convolutional layers at multiple levels, allowing comprehensive feature enhancement. The model was developed based on 5,649 OCT frames from 339 patients and tested using independent external OCT data from 548 frames of 52 patients. DCCAT achieved Dice similarity score (DSC) of 0.706 in segmenting thrombus, which is significantly higher than the CNN-based (0.656) and Transformer-based (0.584) models. We prove that the additional input of polar image not only leverages discriminative features from another coordinate but also improves model robustness for geometrical transformation.Experiment results show that DCCAT achieves competitive performance with only 10% of the total data, highlighting its data efficiency. The proposed dual-coordinate cross-attention design can be easily integrated into other developed Transformer models to boost performance.
Collapse
Affiliation(s)
- Miao Chu
- Biomedical Instrument Institute, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China; Oxford Heart Centre, Oxford University Hospitals NHS Trust, UK; Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK
| | - Giovanni Luigi De Maria
- Oxford Heart Centre, Oxford University Hospitals NHS Trust, UK; Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK; National Institute for Health Research, Oxford Biomedical Research Centre, UK.
| | - Ruobing Dai
- Biomedical Instrument Institute, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Stefano Benenati
- Oxford Heart Centre, Oxford University Hospitals NHS Trust, UK; Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK; University of Genoa, Genoa, Italy
| | - Wei Yu
- Biomedical Instrument Institute, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Jiaxin Zhong
- Biomedical Instrument Institute, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China; Department of Cardiology, Fujian Medical University Union Hospital, Fujian, China
| | - Rafail Kotronias
- Oxford Heart Centre, Oxford University Hospitals NHS Trust, UK; Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK; National Institute for Health Research, Oxford Biomedical Research Centre, UK
| | - Jason Walsh
- Oxford Heart Centre, Oxford University Hospitals NHS Trust, UK; Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK; National Institute for Health Research, Oxford Biomedical Research Centre, UK
| | - Stefano Andreaggi
- Oxford Heart Centre, Oxford University Hospitals NHS Trust, UK; Division of Cardiology, Department of Medicine, University of Verona, Italy
| | | | - Jason Chai
- Oxford Heart Centre, Oxford University Hospitals NHS Trust, UK; Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK
| | - Keith Channon
- Oxford Heart Centre, Oxford University Hospitals NHS Trust, UK; Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK; National Institute for Health Research, Oxford Biomedical Research Centre, UK
| | - Adrian Banning
- Oxford Heart Centre, Oxford University Hospitals NHS Trust, UK; Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK; National Institute for Health Research, Oxford Biomedical Research Centre, UK
| | - Shengxian Tu
- Biomedical Instrument Institute, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China; Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK.
| |
Collapse
|
2
|
Bayhaqi YA, Hamidi A, Navarini AA, Cattin PC, Canbaz F, Zam A. Real-time closed-loop tissue-specific laser osteotomy using deep-learning-assisted optical coherence tomography. BIOMEDICAL OPTICS EXPRESS 2023; 14:2986-3002. [PMID: 37342720 PMCID: PMC10278623 DOI: 10.1364/boe.486660] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 05/15/2023] [Accepted: 05/18/2023] [Indexed: 06/23/2023]
Abstract
This article presents a real-time noninvasive method for detecting bone and bone marrow in laser osteotomy. This is the first optical coherence tomography (OCT) implementation as an online feedback system for laser osteotomy. A deep-learning model has been trained to identify tissue types during laser ablation with a test accuracy of 96.28 %. For the hole ablation experiments, the average maximum depth of perforation and volume loss was 0.216 mm and 0.077 mm3, respectively. The contactless nature of OCT with the reported performance shows that it is becoming more feasible to utilize it as a real-time feedback system for laser osteotomy.
Collapse
Affiliation(s)
- Yakub. A. Bayhaqi
- Biomedical Laser and Optics Group (BLOG), Department of Biomedical Engineering, University of Basel, 4123 Allschwil, Switzerland
| | - Arsham Hamidi
- Biomedical Laser and Optics Group (BLOG), Department of Biomedical Engineering, University of Basel, 4123 Allschwil, Switzerland
| | - Alexander A. Navarini
- Digital Dermatology Group, Department of Biomedical Engineering, University of Basel, 4123 Allschwil, Switzerland
| | - Philippe C. Cattin
- Center for medical Image Analysis and Navigation (CIAN), Department of Biomedical Engineering, University of Basel, 4123 Allschwil, Switzerland
| | - Ferda Canbaz
- Biomedical Laser and Optics Group (BLOG), Department of Biomedical Engineering, University of Basel, 4123 Allschwil, Switzerland
| | - Azhar Zam
- Biomedical Laser and Optics Group (BLOG), Department of Biomedical Engineering, University of Basel, 4123 Allschwil, Switzerland
- Division of Engineering, New York University Abu Dhabi, Abu Dhabi, 129188, United Arab Emirates
- Tandon School of Engineering, New York University, Brooklyn, NY, 11201, USA
| |
Collapse
|
3
|
Wang Z, Zheng J, Jiang P, Gao D. Sk-Conv and SPP-based UNet for lesion segmentation of coronary optical coherence tomography. Technol Health Care 2023; 31:347-355. [PMID: 37066935 DOI: 10.3233/thc-236030] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/18/2023]
Abstract
BACKGROUND Coronary artery disease (CAD) manifests with a blockage the coronary arteries, usually due to plaque buildup, and has a serious impact on the human life. Atherosclerotic plaques, including fibrous plaques, lipid plaques, and calcified plaques can lead to occurrence of CAD. Optical coherence tomography (OCT) is employed in the clinical practice as it clearly provides a detailed display of the lesion plaques, thereby assessing the patient's condition. Analyzing the OCT images manually is a very tedious and time-consuming task for the clinicians. Therefore, automatic segmentation of the coronary OCT images is necessary. OBJECTIVE In view of the good utility of Unet network in the segmentation of medical images, the present study proposed the development of a Unet network based on Sk-Conv and spatial pyramid pooling modules to segment the coronary OCT images. METHODS In order to extract multi-scale features, these two modules were added at the bottom of UNet. Meanwhile, ablation experiments are designed to verify each module is effective. RESULTS After testing, our model achieves 0.8935 on f1 score and 0.7497 on mIOU. Compared to the current advanced models, our model performs better. CONCLUSION Our model achieves good results on OCT sequences.
Collapse
Affiliation(s)
- Zhan Wang
- School of Software, Xi'an Jiaotong University, Xi'an, Shaanxi, China
| | - Jiawei Zheng
- Department of Cardiovascular Medicine, Second Affiliated Hospital of Xi'an, Jiaotong University, Xi'an, Shaanxi, China
| | - Peilin Jiang
- School of Software, Xi'an Jiaotong University, Xi'an, Shaanxi, China
| | - Dengfeng Gao
- Department of Cardiovascular Medicine, Second Affiliated Hospital of Xi'an, Jiaotong University, Xi'an, Shaanxi, China
| |
Collapse
|
4
|
Shi P, Xin J, Zheng N. A-line-based thin-cap fibroatheroma detection with multi-view IVOCT images using multi-task learning and contrastive learning. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2022; 39:2298-2306. [PMID: 36520750 DOI: 10.1364/josaa.464303] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 10/21/2022] [Indexed: 06/17/2023]
Abstract
Automatic detection of thin-cap fibroatheroma (TCFA) is essential to prevent acute coronary syndrome. Hence, in this paper, a method is proposed to detect TCFAs by directly classifying each A-line using multi-view intravascular optical coherence tomography (IVOCT) images. To solve the problem of false positives, a multi-input-output network was developed to implement image-level classification and A-line-based classification at the same time, and a contrastive consistency term was designed to ensure consistency between two tasks. In addition, to learn spatial and global information and obtain the complete extent of TCFAs, an architecture and a regional connectivity constraint term are proposed to classify each A-line of IVOCT images. Experimental results obtained on the 2017 China Computer Vision Conference IVOCT dataset show that the proposed method achieved state-of-art performance with a total score of 88.7±0.88%, overlap rate of 88.64±0.26%, precision rate of 84.34±0.86%, and recall rate of 93.67±2.29%.
Collapse
|
5
|
Automatic assessment of calcified plaque and nodule by optical coherence tomography adopting deep learning model. Int J Cardiovasc Imaging 2022; 38:2501-2510. [DOI: 10.1007/s10554-022-02637-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Accepted: 04/27/2022] [Indexed: 11/05/2022]
|
6
|
Carpenter HJ, Ghayesh MH, Zander AC, Li J, Di Giovanni G, Psaltis PJ. Automated Coronary Optical Coherence Tomography Feature Extraction with Application to Three-Dimensional Reconstruction. Tomography 2022; 8:1307-1349. [PMID: 35645394 PMCID: PMC9149962 DOI: 10.3390/tomography8030108] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Revised: 05/03/2022] [Accepted: 05/10/2022] [Indexed: 11/16/2022] Open
Abstract
Coronary optical coherence tomography (OCT) is an intravascular, near-infrared light-based imaging modality capable of reaching axial resolutions of 10-20 µm. This resolution allows for accurate determination of high-risk plaque features, such as thin cap fibroatheroma; however, visualization of morphological features alone still provides unreliable positive predictive capability for plaque progression or future major adverse cardiovascular events (MACE). Biomechanical simulation could assist in this prediction, but this requires extracting morphological features from intravascular imaging to construct accurate three-dimensional (3D) simulations of patients' arteries. Extracting these features is a laborious process, often carried out manually by trained experts. To address this challenge, numerous techniques have emerged to automate these processes while simultaneously overcoming difficulties associated with OCT imaging, such as its limited penetration depth. This systematic review summarizes advances in automated segmentation techniques from the past five years (2016-2021) with a focus on their application to the 3D reconstruction of vessels and their subsequent simulation. We discuss four categories based on the feature being processed, namely: coronary lumen; artery layers; plaque characteristics and subtypes; and stents. Areas for future innovation are also discussed as well as their potential for future translation.
Collapse
Affiliation(s)
- Harry J. Carpenter
- School of Mechanical Engineering, University of Adelaide, Adelaide, SA 5005, Australia;
| | - Mergen H. Ghayesh
- School of Mechanical Engineering, University of Adelaide, Adelaide, SA 5005, Australia;
| | - Anthony C. Zander
- School of Mechanical Engineering, University of Adelaide, Adelaide, SA 5005, Australia;
| | - Jiawen Li
- School of Electrical Electronic Engineering, University of Adelaide, Adelaide, SA 5005, Australia;
- Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, SA 5005, Australia
- Institute for Photonics and Advanced Sensing, University of Adelaide, Adelaide, SA 5005, Australia
| | - Giuseppe Di Giovanni
- Vascular Research Centre, Lifelong Health Theme, South Australian Health and Medical Research Institute (SAHMRI), Adelaide, SA 5000, Australia; (G.D.G.); (P.J.P.)
| | - Peter J. Psaltis
- Vascular Research Centre, Lifelong Health Theme, South Australian Health and Medical Research Institute (SAHMRI), Adelaide, SA 5000, Australia; (G.D.G.); (P.J.P.)
- Adelaide Medical School, University of Adelaide, Adelaide, SA 5005, Australia
- Department of Cardiology, Central Adelaide Local Health Network, Adelaide, SA 5000, Australia
| |
Collapse
|
7
|
Zhang R, Fan Y, Qi W, Wang A, Tang X, Gao T. Current research and future prospects of IVOCT imaging-based detection of the vascular lumen and vulnerable plaque. JOURNAL OF BIOPHOTONICS 2022; 15:e202100376. [PMID: 35139263 DOI: 10.1002/jbio.202100376] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Revised: 01/17/2022] [Accepted: 02/07/2022] [Indexed: 06/14/2023]
Abstract
Intravascular optical coherence tomography (IVOCT) is an imaging method that has developed rapidly in recent years and is useful in coronary atherosclerosis diagnosis. It is widely used in the assessment of vulnerable plaque. This review summarizes the main research methods used in recent years for blood vessel lumen boundary detection and segmentation and vulnerable plaque segmentation and classification. This article aims to comprehensively and systematically introduce the research progress on internal tissues of blood vessels based on IVOCT images. The characteristics and advantages of various methods have been summarized to provide theoretical ideas and methods for the reference of relevant researchers and scholars.
Collapse
Affiliation(s)
- Ruolin Zhang
- School of Life Science, Beijing Institute of Technology, Beijing, China
| | - Yingwei Fan
- School of Medical Technology, Beijing Institute of Technology, Beijing, China
| | - Wenliu Qi
- School of Life Science, Beijing Institute of Technology, Beijing, China
| | - Ancong Wang
- School of Life Science, Beijing Institute of Technology, Beijing, China
| | - Xiaoying Tang
- School of Life Science, Beijing Institute of Technology, Beijing, China
- School of Medical Technology, Beijing Institute of Technology, Beijing, China
| | - Tianxin Gao
- School of Life Science, Beijing Institute of Technology, Beijing, China
| |
Collapse
|
8
|
Li C, Jia H, Tian J, He C, Lu F, Li K, Gong Y, Hu S, Yu B, Wang Z. Comprehensive Assessment of Coronary Calcification in Intravascular OCT Using a Spatial-Temporal Encoder-Decoder Network. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:857-868. [PMID: 34735339 DOI: 10.1109/tmi.2021.3125061] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
Coronary calcification is a strong indicator of coronary artery disease and a key determinant of the outcome of percutaneous coronary intervention. We propose a fully automated method to segment and quantify coronary calcification in intravascular OCT (IVOCT) images based on convolutional neural networks (CNN). All possible calcified plaques were segmented from IVOCT pullbacks using a spatial-temporal encoder-decoder network by exploiting the 3D continuity information of the plaques, which were then screened and classified by a DenseNet network to reduce false positives. A novel data augmentation method based on the IVOCT image acquisition pattern was also proposed to improve the performance and robustness of the segmentation. Clinically relevant metrics including calcification area, depth, angle, thickness, volume, and stent-deployment calcification score, were automatically computed. 13844 IVOCT images with 2627 calcification slices from 45 clinical OCT pullbacks were collected and used to train and test the model. The proposed method performed significantly better than existing state-of-the-art 2D and 3D CNN methods. The data augmentation method improved the Dice similarity coefficient for calcification segmentation from 0.615±0.332 to 0.756±0.222, reaching human-level inter-observer agreement. Our proposed region-based classifier improved image-level calcification classification precision and F1-score from 0.725±0.071 and 0.791±0.041 to 0.964±0.002 and 0.883±0.008, respectively. Bland-Altman analysis showed close agreement between manual and automatic calcification measurements. Our proposed method is valuable for automated assessment of coronary calcification lesions and in-procedure planning of stent deployment.
Collapse
|
9
|
Ding M, Pan SY, Huang J, Yuan C, Zhang Q, Zhu XL, Cai Y. Optical coherence tomography for identification of malignant pulmonary nodules based on random forest machine learning algorithm. PLoS One 2021; 16:e0260600. [PMID: 34971557 PMCID: PMC8719667 DOI: 10.1371/journal.pone.0260600] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Accepted: 11/14/2021] [Indexed: 11/24/2022] Open
Abstract
OBJECTIVE To explore the feasibility of using random forest (RF) machine learning algorithm in assessing normal and malignant peripheral pulmonary nodules based on in vivo endobronchial optical coherence tomography (EB-OCT). METHODS A total of 31 patients with pulmonary nodules were admitted to Department of Respiratory Medicine, Zhongda Hospital, Southeast University, and underwent chest CT, EB-OCT and biopsy. Attenuation coefficient and up to 56 different image features were extracted from A-line and B-scan of 1703 EB-OCT images. Attenuation coefficient and 29 image features with significant p-values were used to analyze the differences between normal and malignant samples. A RF classifier was trained using 70% images as training set, while 30% images were included in the testing set. The accuracy of the automated classification was validated by clinically proven pathological results. RESULTS Attenuation coefficient and 29 image features were found to present different properties with significant p-values between normal and malignant EB-OCT images. The RF algorithm successfully classified the malignant pulmonary nodules with sensitivity, specificity, and accuracy of 90.41%, 77.87% and 83.51% respectively. CONCLUSION It is clinically practical to distinguish the nature of pulmonary nodules by integrating EB-OCT imaging with automated machine learning algorithm. Diagnosis of malignant pulmonary nodules by analyzing quantitative features from EB-OCT images could be a potentially powerful way for early detection of lung cancer.
Collapse
Affiliation(s)
- Ming Ding
- Department of Respiratory Medicine, Southeast University Zhongda Hospital, Nanjing, Jiangsu, China
| | - Shi-yu Pan
- School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, Jiangsu, China
| | - Jing Huang
- Department of Respiratory Medicine, Southeast University Zhongda Hospital, Nanjing, Jiangsu, China
| | - Cheng Yuan
- Department of Respiratory Medicine, Southeast University Zhongda Hospital, Nanjing, Jiangsu, China
| | - Qiang Zhang
- Department of Respiratory Medicine, Southeast University Zhongda Hospital, Nanjing, Jiangsu, China
| | - Xiao-li Zhu
- Department of Respiratory Medicine, Southeast University Zhongda Hospital, Nanjing, Jiangsu, China
| | - Yan Cai
- School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, Jiangsu, China
| |
Collapse
|
10
|
Recent Trends in Artificial Intelligence-Assisted Coronary Atherosclerotic Plaque Characterization. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph181910003. [PMID: 34639303 PMCID: PMC8508413 DOI: 10.3390/ijerph181910003] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/09/2021] [Revised: 09/12/2021] [Accepted: 09/17/2021] [Indexed: 01/21/2023]
Abstract
Coronary artery disease is a major cause of morbidity and mortality worldwide. Its underlying histopathology is the atherosclerotic plaque, which comprises lipid, fibrous and—when chronic—calcium components. Intravascular ultrasound (IVUS) and intravascular optical coherence tomography (IVOCT) performed during invasive coronary angiography are reference standards for characterizing the atherosclerotic plaque. Fine image spatial resolution attainable with contemporary coronary computed tomographic angiography (CCTA) has enabled noninvasive plaque assessment, including identifying features associated with vulnerable plaques known to presage acute coronary events. Manual interpretation of IVUS, IVOCT and CCTA images demands scarce physician expertise and high time cost. This has motivated recent research into and development of artificial intelligence (AI)-assisted methods for image processing, feature extraction, plaque identification and characterization. We performed parallel searches of the medical and technical literature from 1995 to 2021 focusing respectively on human plaque characterization using various imaging modalities and the use of AI-assisted computer aided diagnosis (CAD) to detect and classify atherosclerotic plaques, including their composition and the presence of high-risk features denoting vulnerable plaques. A total of 122 publications were selected for evaluation and the analysis was summarized in terms of data sources, methods—machine versus deep learning—and performance metrics. Trends in AI-assisted plaque characterization are detailed and prospective research challenges discussed. Future directions for the development of accurate and efficient CAD systems to characterize plaque noninvasively using CCTA are proposed.
Collapse
|
11
|
Cannon TM, Bouma BE, Uribe-Patarroyo N. Layer-based, depth-resolved computation of attenuation coefficients and backscattering fractions in tissue using optical coherence tomography. BIOMEDICAL OPTICS EXPRESS 2021; 12:5037-5056. [PMID: 34513241 PMCID: PMC8407832 DOI: 10.1364/boe.427833] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/19/2021] [Revised: 07/06/2021] [Accepted: 07/11/2021] [Indexed: 05/18/2023]
Abstract
Structural optical coherence tomography (OCT) images of tissue stand to benefit from greater functionalization and quantitative interpretation. The OCT attenuation coefficient µ, an analogue of the imaged sample's scattering coefficient, offers potential functional contrast based on the relationship of µ to sub-resolution physical properties of the sample. Attenuation coefficients are computed either by fitting a representative µ over several depth-wise pixels of a sample's intensity decay, or by using previously-developed depth-resolved attenuation algorithms by Girard et al. [Invest. Ophthalmol. Vis. Sci.52, 7738 (2011). 10.1167/iovs.10-6925] and Vermeer et al. [Biomed. Opt. Express5, 322 (2014). 10.1364/BOE.5.000322]. However, the former method sacrifices axial information in the tomogram, while the latter relies on the stringent assumption that the sample's backscattering fraction, another optical property, does not vary along depth. This assumption may be violated by layered tissues commonly observed in clinical imaging applications. Our approach preserves the full depth resolution of the attenuation map but removes its dependence on backscattering fraction by performing signal analysis inside individual discrete layers over which the scattering properties (e.g., attenuation and backscattering fraction) vary minimally. Although this approach necessitates the detection of these layers, it removes the constant-backscattering-fraction assumption that has constrained quantitative attenuation coefficient analysis in the past, and additionally yields a layer-resolved backscattering fraction, providing complementary scattering information to the attenuation coefficient. We validate our approach using automated layer detection in layered phantoms, for which the measured optical properties were in good agreement with theoretical values calculated with Mie theory, and show preliminary results in tissue alongside corresponding histological analysis. Together, accurate backscattering fraction and attenuation coefficient measurements enable the estimation of both particle density and size, which is not possible from attenuation measurements alone. We hope that this improvement to depth-resolved attenuation coefficient measurement, augmented by a layer-resolved backscattering fraction, will increase the diagnostic power of quantitative OCT imaging.
Collapse
Affiliation(s)
- Taylor M. Cannon
- Massachusetts Institute of Technology, Institute of Medical Engineering and Science, 70 Massachusetts Avenue, Cambridge, MA 02141, USA
- Wellman Center for Photomedicine, Massachusetts General Hospital, 40 Blossom St, Boston, MA 02114, USA
| | - Brett E. Bouma
- Massachusetts Institute of Technology, Institute of Medical Engineering and Science, 70 Massachusetts Avenue, Cambridge, MA 02141, USA
- Wellman Center for Photomedicine, Massachusetts General Hospital, 40 Blossom St, Boston, MA 02114, USA
| | - Néstor Uribe-Patarroyo
- Wellman Center for Photomedicine, Massachusetts General Hospital, 40 Blossom St, Boston, MA 02114, USA
| |
Collapse
|
12
|
Yin Y, He C, Xu B, Li Z. Coronary Plaque Characterization From Optical Coherence Tomography Imaging With a Two-Pathway Cascade Convolutional Neural Network Architecture. Front Cardiovasc Med 2021; 8:670502. [PMID: 34222368 PMCID: PMC8241907 DOI: 10.3389/fcvm.2021.670502] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2021] [Accepted: 05/10/2021] [Indexed: 11/14/2022] Open
Abstract
Background: The morphological structure and tissue composition of a coronary atherosclerotic plaque determine its stability, which can be assessed by intravascular optical coherence tomography (OCT) imaging. However, plaque characterization relies on the interpretation of large datasets by well-trained observers. This study aims to develop a convolutional neural network (CNN) method to automatically extract tissue features from OCT images to characterize the main components of a coronary atherosclerotic plaque (fibrous, lipid, and calcification). The method is based on a novel CNN architecture called TwopathCNN, which is utilized in a cascaded structure. According to the evaluation, this proposed method is effective and robust in the characterization of coronary plaque composition from in vivo OCT imaging. On average, the method achieves 0.86 in F1-score and 0.88 in accuracy. The TwopathCNN architecture and cascaded structure show significant improvement in performance (p < 0.05). CNN with cascaded structure can greatly improve the performance of characterization compared to the conventional CNN methods and machine learning methods. This method has a higher efficiency, which may be proven to be a promising diagnostic tool in the detection of coronary plaques.
Collapse
Affiliation(s)
- Yifan Yin
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
| | - Chunliu He
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
| | - Biao Xu
- Department of Cardiology, Nanjing Drum Tower Hospital, Nanjing, China
| | - Zhiyong Li
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China.,School of Mechanical, Medical, and Process Engineering, Queensland University of Technology, Brisbane, QLD, Australia
| |
Collapse
|
13
|
Abdolmanafi A, Duong L, Ibrahim R, Dahdah N. A deep learning-based model for characterization of atherosclerotic plaque in coronary arteries using optical coherence tomography images. Med Phys 2021; 48:3511-3524. [PMID: 33914917 DOI: 10.1002/mp.14909] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Revised: 03/31/2021] [Accepted: 04/12/2021] [Indexed: 12/17/2022] Open
Abstract
PURPOSE Coronary artery events are mainly associated with atherosclerosis in adult population, which is recognized as accumulation of plaques in arterial wall tissues. Optical Coherence Tomography (OCT) is a light-based imaging system used in cardiology to analyze intracoronary tissue layers and pathological formations including plaque accumulation. This state-of-the-art catheter-based imaging system provides intracoronary cross-sectional images with high resolution of 10-15 µm. But interpretation of the acquired images is operator dependent, which is not only very time-consuming but also highly error prone from one observer to another. An automatic and accurate coronary plaque tagging using OCT image post-processing can contribute to wide adoption of the OCT system and reducing the diagnostic error rate. METHOD In this study, we propose a combination of spatial pyramid pooling module with dilated convolutions for semantic segmentation to extract atherosclerotic tissues regardless of their types and training a sparse auto-encoder to reconstruct the input features and enlarge the training data as well as plaque type characterization in OCT images. RESULTS The results demonstrate high precision of the proposed model with reduced computational complexity, which can be appropriate for real-time analysis of OCT images. At each step of the work, measured accuracy, sensitivity, specificity of more than 93% demonstrate high performance of the model. CONCLUSION The main focus of this study is atherosclerotic tissue characterization using OCT imaging. This contributes to wide adoption of the OCT imaging system by providing clinicians with a fully automatic interpretation of various atherosclerotic tissues. Future studies will be focused on analyzing atherosclerotic vulnerable plaques, those coronary plaques which are prone to rupture.
Collapse
Affiliation(s)
- Atefeh Abdolmanafi
- Department of Software and IT Engineering, École de technologie supérieure, Montreal, Canada
| | - Luc Duong
- Department of Software and IT Engineering, École de technologie supérieure, Montreal, Canada
| | - Ragui Ibrahim
- Division of Cardiology, Hôpital Pierre Boucher, Longueuil, Canada
| | - Nagib Dahdah
- Division of Pediatric Cardiology and Research Center, Centre Hospitalier Universitaire Sainte-Justine, Montreal, Canada
| |
Collapse
|
14
|
Zheng S, Fei Y, Jian S. Method for parametric imaging of attenuation by intravascular optical coherence tomography. BIOMEDICAL OPTICS EXPRESS 2021; 12:1882-1904. [PMID: 33996205 PMCID: PMC8086439 DOI: 10.1364/boe.420094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/19/2021] [Revised: 02/27/2021] [Accepted: 03/01/2021] [Indexed: 06/12/2023]
Abstract
Catheter-based intravascular optical coherence tomography (IVOCT) is a powerful imaging modality for visualization of atherosclerosis with high resolution. Quantitative characterization of various tissue types by attenuation coefficient (AC) extraction has been proven to be a potentially significant application of OCT attenuation imaging. However, existing methods for AC extraction from OCT suffer from the challenge of variability in complex tissue types in IVOCT pullback data such as healthy vessel wall, mixed atherosclerotic plaques, plaques with a single component and stent struts, etc. This challenge leads to the ineffectiveness in the tissue differentiation by AC representation based on single scattering model of OCT signal. In this paper, we propose a novel method based on multiple scattering model for parametric imaging of optical attenuation by AC retrieval from IVOCT images conventionally acquired during cardiac catheterization. The OCT signal characterized by the AC is physically modeled by Monte Carlo simulation. Then, the pixel-wise AC retrieval is achieved by iteratively minimizing an error function regarding the modeled and measured backscattered signal. This method provides a general scheme for AC extraction from IVOCT without the premise of complete attenuation of the incident light through the imaging depths. Results of computer-simulated and clinical images demonstrate that the method can avoid overestimation at the end of the depth profile in comparison with the approaches based on the depth-resolved (DR) model. The energy error depth and structural similarity are improved by about 30% and 10% respectively compared with DR. It provides a useful way to differentiate and characterize arterial tissue types in IVOCT images.
Collapse
Affiliation(s)
- Sun Zheng
- Department of Electronic and Communication Engineering, North China Electric Power University, Baoding 071003, Hebei, China
- Hebei Key Laboratory of Power Internet of Things Technology, North China Electric Power University, Baoding 071003, Hebei, China
| | - Yang Fei
- Department of Electronic and Communication Engineering, North China Electric Power University, Baoding 071003, Hebei, China
- Hebei Key Laboratory of Power Internet of Things Technology, North China Electric Power University, Baoding 071003, Hebei, China
| | - Sun Jian
- Department of Radiology, Hebei University Affiliated Hospital, Baoding 071003, Hebei, China
| |
Collapse
|
15
|
Munger E, Hickey JW, Dey AK, Jafri MS, Kinser JM, Mehta NN. Application of machine learning in understanding atherosclerosis: Emerging insights. APL Bioeng 2021; 5:011505. [PMID: 33644628 PMCID: PMC7889295 DOI: 10.1063/5.0028986] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Accepted: 01/21/2021] [Indexed: 01/18/2023] Open
Abstract
Biological processes are incredibly complex—integrating molecular signaling networks involved in multicellular communication and function, thus maintaining homeostasis. Dysfunction of these processes can result in the disruption of homeostasis, leading to the development of several disease processes including atherosclerosis. We have significantly advanced our understanding of bioprocesses in atherosclerosis, and in doing so, we are beginning to appreciate the complexities, intricacies, and heterogeneity atherosclerosi. We are also now better equipped to acquire, store, and process the vast amount of biological data needed to shed light on the biological circuitry involved. Such data can be analyzed within machine learning frameworks to better tease out such complex relationships. Indeed, there has been an increasing number of studies applying machine learning methods for patient risk stratification based on comorbidities, multi-modality image processing, and biomarker discovery pertaining to atherosclerotic plaque formation. Here, we focus on current applications of machine learning to provide insight into atherosclerotic plaque formation and better understand atherosclerotic plaque progression in patients with cardiovascular disease.
Collapse
Affiliation(s)
| | - John W Hickey
- Stanford University, Stanford, California 94306, USA
| | - Amit K Dey
- National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, Maryland 20892, USA
| | | | | | - Nehal N Mehta
- National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, Maryland 20892, USA
| |
Collapse
|
16
|
Kolluru C, Lee J, Gharaibeh Y, Bezerra HG, Wilson DL. Learning With Fewer Images via Image Clustering: Application to Intravascular OCT Image Segmentation. IEEE ACCESS : PRACTICAL INNOVATIONS, OPEN SOLUTIONS 2021; 9:37273-37280. [PMID: 33828934 PMCID: PMC8023588 DOI: 10.1109/access.2021.3058890] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Deep learning based methods are routinely used to segment various structures of interest in varied medical imaging modalities. Acquiring annotations for a large number of images requires a skilled analyst, and the process is both time consuming and challenging. Our approach to reduce effort is to reduce the number of images needing detailed annotation. For intravascular optical coherence tomography (IVOCT) image pullbacks, we tested 10% to 100% of training images derived from two schemes: equally-spaced image subsampling and deep-learning- based image clustering. The first strategy involves selecting images at equally spaced intervals from the volume, accounting for the high spatial correlation between neighboring images. In clustering, we used an autoencoder to create a deep feature space representation, performed k-medoids clustering, and then used the cluster medians for training. For coronary calcifications, a baseline U-net model was trained on all images from volumes of interest (VOIs) and compared with fewer images from the sub-sampling strategies. For a given sampling ratio, the clustering based method performed better or similar as compared to the equally spaced sampling approach (e.g., with 10% of images, mean F1 score for calcific class increased from 0.52 to 0.63, with equal spacing and with clustering, respectively). Additionally, for a fixed number of training images, sampling images from more VOIs performed better than otherwise. In conclusion, we recommend the clustering based approach to annotate a small fraction of images, creating a baseline model, which potentially can be improved further by annotating images selected using methods described in active learning research.
Collapse
Affiliation(s)
- Chaitanya Kolluru
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Juhwan Lee
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Yazan Gharaibeh
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Hiram G Bezerra
- Interventional Cardiology Center, Heart and Vascular Institute, The University of South Florida, Tampa, FL 33606, USA
| | - David L Wilson
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| |
Collapse
|
17
|
Lee J, Gharaibeh Y, Kolluru C, Zimin VN, Dallan LAP, Kim JN, Bezerra HG, Wilson DL. Segmentation of Coronary Calcified Plaque in Intravascular OCT Images Using a Two-Step Deep Learning Approach. IEEE ACCESS : PRACTICAL INNOVATIONS, OPEN SOLUTIONS 2020; 8:225581-225593. [PMID: 33598377 PMCID: PMC7885992 DOI: 10.1109/access.2020.3045285] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
We developed a fully automated, two-step deep learning approach for characterizing coronary calcified plaque in intravascular optical coherence tomography (IVOCT) images. First, major calcification lesions were detected from an entire pullback using a 3D convolutional neural network (CNN). Second, a SegNet deep learning model with the Tversky loss function was used to segment calcified plaques in the major calcification lesions. The fully connected conditional random field and the frame interpolation of the missing calcification frames were used to reduce classification errors. We trained/tested the networks on a large dataset comprising 8,231 clinical images from 68 patients with 68 vessels and 4,320 ex vivo cadaveric images from 4 hearts with 4 vessels. The 3D CNN model detected major calcifications with high sensitivity (97.7%), specificity (87.7%), and F1 score (0.922). Compared to the standard one-step approach, our two-step deep learning approach significantly improved sensitivity (from 77.5% to 86.2%), precision (from 73.5% to 75.8%), and F1 score (from 0.749 to 0.781). We investigated segmentation performance for varying numbers of training samples; at least 3,900 images were required to obtain stable segmentation results. We also found very small differences in calcification attributes (e.g., angle, thickness, and depth) and identical calcium scores on repetitive pullbacks, indicating excellent reproducibility. Applied to new clinical pullbacks, our method has implications for real-time treatment planning and imaging research.
Collapse
Affiliation(s)
- Juhwan Lee
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Yazan Gharaibeh
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Chaitanya Kolluru
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Vladislav N Zimin
- Cardiovascular Imaging Core Laboratory, Harrington Heart and Vascular Institute, University Hospitals Cleveland Medical Center, Cleveland, OH 44106, USA
| | - Luis Augusto Palma Dallan
- Cardiovascular Imaging Core Laboratory, Harrington Heart and Vascular Institute, University Hospitals Cleveland Medical Center, Cleveland, OH 44106, USA
| | - Justin Namuk Kim
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Hiram G Bezerra
- Interventional Cardiology Center, Heart and Vascular Institute, University of South Florida, Tampa, FL 33606, USA
| | - David L Wilson
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
- Department of Radiology, Case Western Reserve University, Cleveland, OH 44106, USA
| |
Collapse
|
18
|
Xie Z, Shu C, Yang D, Chen H, Chen C, Dai G, Lam KH, Zhang J, Wang X, Sheng Z, Gao D, Liu C, Song L, Gong X. In vivo intravascular photoacoustic imaging at a high speed of 100 frames per second. BIOMEDICAL OPTICS EXPRESS 2020; 11:6721-6731. [PMID: 33282520 PMCID: PMC7687943 DOI: 10.1364/boe.405792] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/20/2020] [Revised: 09/21/2020] [Accepted: 10/03/2020] [Indexed: 05/02/2023]
Abstract
Intravascular photoacoustic (IVPA) imaging technology enables the visualization of pathological characteristics (such as inflammation activities, lipid deposition) of the artery wall. Blood flushing is a necessary step in improving the imaging quality in in vivo IVPA imaging. But the limited imaging speed of the systems stretches their flushing time, which is an important obstacle of their clinical translations. In this paper, we report an improvement in IVPA/IVUS imaging speed to 100 frames per second. The high-speed imaging is demonstrated in rabbit in vivo, visualizing the nanoparticles accumulated on abdominal aorta wall at the wavelength of 1064 nm, in real time display. Blood flushing in vivo improves the IVPA signal-noise-ratio by around 3.5 dB. This study offers a stable, efficient and easy-to-use tool for instantaneous disease visualization and disease diagnosis in research and forwards IVPA/IVUS imaging technology towards clinical translations.
Collapse
Affiliation(s)
- Zhihua Xie
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Research Laboratory for Biomedical Optics and Molecular Imaging, Shenzhen Key Laboratory for Molecular Imaging, Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology, CAS Key Laboratory of Health Informatics, Shenzhen Institutes of Advanced Technology, Shenzhen 518055, China
- Equal contribution
| | - Chengyou Shu
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Research Laboratory for Biomedical Optics and Molecular Imaging, Shenzhen Key Laboratory for Molecular Imaging, Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology, CAS Key Laboratory of Health Informatics, Shenzhen Institutes of Advanced Technology, Shenzhen 518055, China
- Equal contribution
| | - Daya Yang
- Department of Cardiology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou 510080, China
- Key Laboratory of Assisted Circulation, National Health Commission, Sun Yat-sen University, Guangzhou 510080, China
- Equal contribution
| | - Hao Chen
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Chaobin Chen
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Gang Dai
- Department of Cardiology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou 510080, China
- Key Laboratory of Assisted Circulation, National Health Commission, Sun Yat-sen University, Guangzhou 510080, China
| | - Kwok Ho Lam
- Department of Electrical Engineering, The Hong Kong Polytechnic University, Hunghom, Hong Kong
| | - Jinke Zhang
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Research Laboratory for Biomedical Optics and Molecular Imaging, Shenzhen Key Laboratory for Molecular Imaging, Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology, CAS Key Laboratory of Health Informatics, Shenzhen Institutes of Advanced Technology, Shenzhen 518055, China
| | - Xiatian Wang
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Zonghai Sheng
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Research Laboratory for Biomedical Optics and Molecular Imaging, Shenzhen Key Laboratory for Molecular Imaging, Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology, CAS Key Laboratory of Health Informatics, Shenzhen Institutes of Advanced Technology, Shenzhen 518055, China
| | - Duyang Gao
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Research Laboratory for Biomedical Optics and Molecular Imaging, Shenzhen Key Laboratory for Molecular Imaging, Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology, CAS Key Laboratory of Health Informatics, Shenzhen Institutes of Advanced Technology, Shenzhen 518055, China
| | - Chengbo Liu
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Research Laboratory for Biomedical Optics and Molecular Imaging, Shenzhen Key Laboratory for Molecular Imaging, Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology, CAS Key Laboratory of Health Informatics, Shenzhen Institutes of Advanced Technology, Shenzhen 518055, China
| | - Liang Song
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Research Laboratory for Biomedical Optics and Molecular Imaging, Shenzhen Key Laboratory for Molecular Imaging, Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology, CAS Key Laboratory of Health Informatics, Shenzhen Institutes of Advanced Technology, Shenzhen 518055, China
| | - Xiaojing Gong
- Research Laboratory for Biomedical Optics and Molecular imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Research Laboratory for Biomedical Optics and Molecular Imaging, Shenzhen Key Laboratory for Molecular Imaging, Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology, CAS Key Laboratory of Health Informatics, Shenzhen Institutes of Advanced Technology, Shenzhen 518055, China
| |
Collapse
|
19
|
He C, Wang J, Yin Y, Li Z. Automated classification of coronary plaque calcification in OCT pullbacks with 3D deep neural networks. JOURNAL OF BIOMEDICAL OPTICS 2020; 25:JBO-200088R. [PMID: 32914606 PMCID: PMC7481437 DOI: 10.1117/1.jbo.25.9.095003] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/29/2020] [Accepted: 08/24/2020] [Indexed: 05/07/2023]
Abstract
SIGNIFICANCE Detection and characterization of coronary atherosclerotic plaques often need reviews of a large number of optical coherence tomography (OCT) imaging slices to make a clinical decision. However, it is a challenge to manually review all the slices and consider the interrelationship between adjacent slices. APPROACH Inspired by the recent success of deep convolutional network on the classification of medical images, we proposed a ResNet-3D network for classification of coronary plaque calcification in OCT pullbacks. The ResNet-3D network was initialized with a trained ResNet-50 network and a three-dimensional convolution filter filled with zeros padding and non-zeros padding with a convolutional filter. To retrain ResNet-50, we used a dataset of ∼4860 OCT images, derived by 18 entire pullbacks from different patients. In addition, we investigated a two-phase training method to address the data imbalance. For an improved performance, we evaluated different input sizes for the ResNet-3D network, such as 3, 5, and 7 OCT slices. Furthermore, we integrated all ResNet-3D results by majority voting. RESULTS A comparative analysis proved the effectiveness of the proposed ResNet-3D networks against ResNet-2D network in the OCT dataset. The classification performance (F1-scores = 94 % for non-zeros padding and F1-score = 96 % for zeros padding) demonstrated the potential of convolutional neural networks (CNNs) in classifying plaque calcification. CONCLUSIONS This work may provide a foundation for further work in extending the CNN to voxel segmentation, which may lead to a supportive diagnostic tool for assessment of coronary plaque vulnerability.
Collapse
Affiliation(s)
- Chunliu He
- Southeast University, School of Biological Science and Medical Engineering, Nanjing, China
| | - Jiaqiu Wang
- Queensland University of Technology, School of Mechanical, Medical and Process Engineering, Brisbane, Australia
| | - Yifan Yin
- Southeast University, School of Biological Science and Medical Engineering, Nanjing, China
| | - Zhiyong Li
- Southeast University, School of Biological Science and Medical Engineering, Nanjing, China
- Queensland University of Technology, School of Mechanical, Medical and Process Engineering, Brisbane, Australia
| |
Collapse
|
20
|
He C, Li Z, Wang J, Huang Y, Yin Y, Li Z. Atherosclerotic Plaque Tissue Characterization: An OCT-Based Machine Learning Algorithm With ex vivo Validation. Front Bioeng Biotechnol 2020; 8:749. [PMID: 32714918 PMCID: PMC7343706 DOI: 10.3389/fbioe.2020.00749] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2020] [Accepted: 06/11/2020] [Indexed: 11/13/2022] Open
Abstract
There is a need to develop a validated algorithm for plaque characterization which can help to facilitate the standardization of optical coherence tomography (OCT) image interpretation of plaque morphology, and improve the efficiency and accuracy in the application of OCT imaging for the quantitative assessment of plaque vulnerability. In this study, a machine learning algorithm was implemented for characterization of atherosclerotic plaque components by intravascular OCT using ex vivo carotid plaque tissue samples. A total of 31 patients underwent carotid endarterectomy and the ex vivo carotid plaques were imaged with OCT. Optical parameter, texture features and relative position of pixels were extracted within the region of interest and then used to quantify the tissue characterization of plaque components. The potential of individual and combined feature set to discriminate tissue components was quantified using sensitivity, specificity, accuracy. The results show there was a lower classification accuracy in the calcified tissue than the fibrous tissue and lipid tissue. The pixel-wise classification accuracy obtained by the developed method, to characterize the fibrous, calcified and lipid tissue by comparing with histology, were 80.0, 62.0, and 83.1, respectively. The developed algorithm was capable of characterizing plaque components with an excellent accuracy using the combined feature set.
Collapse
Affiliation(s)
- Chunliu He
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
| | - Zhonglin Li
- Department of Neurosurgery, The Affiliated Hospital of Xuzhou Medical College, Xuzhou, China
| | - Jiaqiu Wang
- School of Mechanical, Medical, Process Engineering, Queensland University of Technology, Brisbane, QLD, Australia
| | - Yuxiang Huang
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
| | - Yifan Yin
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
| | - Zhiyong Li
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
- School of Mechanical, Medical, Process Engineering, Queensland University of Technology, Brisbane, QLD, Australia
| |
Collapse
|
21
|
Foo KY, Chin L, Zilkens R, Lakhiani DD, Fang Q, Sanderson R, Dessauvagie BF, Latham B, McLaren S, Saunders CM, Kennedy BF. Three-dimensional mapping of the attenuation coefficient in optical coherence tomography to enhance breast tissue microarchitecture contrast. JOURNAL OF BIOPHOTONICS 2020; 13:e201960201. [PMID: 32141243 DOI: 10.1002/jbio.201960201] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/27/2019] [Revised: 02/16/2020] [Accepted: 03/04/2020] [Indexed: 06/10/2023]
Abstract
Effective intraoperative tumor margin assessment is needed to reduce re-excision rates in breast-conserving surgery (BCS). Mapping the attenuation coefficient in optical coherence tomography (OCT) throughout a sample to create an image (attenuation imaging) is one promising approach. For the first time, three-dimensional OCT attenuation imaging of human breast tissue microarchitecture using a wide-field (up to ~45 × 45 × 3.5 mm) imaging system is demonstrated. Representative results from three mastectomy and one BCS specimen (from 31 specimens) are presented with co-registered postoperative histology. Attenuation imaging is shown to provide substantially improved contrast over OCT, delineating nuanced features within tumors (including necrosis and variations in tumor cell density and growth patterns) and benign features (such as sclerosing adenosis). Additionally, quantitative micro-elastography (QME) images presented alongside OCT and attenuation images show that these techniques provide complementary contrast, suggesting that multimodal imaging could increase tissue identification accuracy and potentially improve tumor margin assessment.
Collapse
Affiliation(s)
- Ken Y Foo
- BRITElab, Harry Perkins Institute of Medical Research, QEII Medical Centre, Nedlands and Centre for Medical Research, The University of Western Australia, Crawley, Western Australia, Australia
- Department of Electrical, Electronic & Computer Engineering, School of Engineering, The University of Western Australia, Crawley, Western Australia, Australia
| | - Lixin Chin
- BRITElab, Harry Perkins Institute of Medical Research, QEII Medical Centre, Nedlands and Centre for Medical Research, The University of Western Australia, Crawley, Western Australia, Australia
- Department of Electrical, Electronic & Computer Engineering, School of Engineering, The University of Western Australia, Crawley, Western Australia, Australia
| | - Renate Zilkens
- BRITElab, Harry Perkins Institute of Medical Research, QEII Medical Centre, Nedlands and Centre for Medical Research, The University of Western Australia, Crawley, Western Australia, Australia
- Division of Surgery, Medical School, The University of Western Australia, Crawley, Western Australia, Australia
| | - Devina D Lakhiani
- BRITElab, Harry Perkins Institute of Medical Research, QEII Medical Centre, Nedlands and Centre for Medical Research, The University of Western Australia, Crawley, Western Australia, Australia
- Department of Electrical, Electronic & Computer Engineering, School of Engineering, The University of Western Australia, Crawley, Western Australia, Australia
| | - Qi Fang
- BRITElab, Harry Perkins Institute of Medical Research, QEII Medical Centre, Nedlands and Centre for Medical Research, The University of Western Australia, Crawley, Western Australia, Australia
- Department of Electrical, Electronic & Computer Engineering, School of Engineering, The University of Western Australia, Crawley, Western Australia, Australia
| | - Rowan Sanderson
- BRITElab, Harry Perkins Institute of Medical Research, QEII Medical Centre, Nedlands and Centre for Medical Research, The University of Western Australia, Crawley, Western Australia, Australia
- Department of Electrical, Electronic & Computer Engineering, School of Engineering, The University of Western Australia, Crawley, Western Australia, Australia
| | - Benjamin F Dessauvagie
- PathWest, Fiona Stanley Hospital, Murdoch, Western Australia, Australia
- Division of Pathology and Laboratory Medicine, The University of Western Australia, Crawley, Western Australia, Australia
| | - Bruce Latham
- PathWest, Fiona Stanley Hospital, Murdoch, Western Australia, Australia
- The University of Notre Dame, Fremantle, Western Australia, Australia
| | - Sally McLaren
- PathWest Laboratory Medicine WA, QEII Medical Centre, Nedlands, Western Australia, Australia
| | - Christobel M Saunders
- Division of Surgery, Medical School, The University of Western Australia, Crawley, Western Australia, Australia
- Breast Centre, Fiona Stanley Hospital, Murdoch, Western Australia, Australia
- Breast Clinic, Royal Perth Hospital, Perth, Western Australia, Australia
| | - Brendan F Kennedy
- BRITElab, Harry Perkins Institute of Medical Research, QEII Medical Centre, Nedlands and Centre for Medical Research, The University of Western Australia, Crawley, Western Australia, Australia
- Department of Electrical, Electronic & Computer Engineering, School of Engineering, The University of Western Australia, Crawley, Western Australia, Australia
- Australian Research Council Centre for Personalised Therapeutics Technologies, Perth, Western Australia, Australia
| |
Collapse
|
22
|
Fedewa R, Puri R, Fleischman E, Lee J, Prabhu D, Wilson DL, Vince DG, Fleischman A. Artificial Intelligence in Intracoronary Imaging. Curr Cardiol Rep 2020; 22:46. [DOI: 10.1007/s11886-020-01299-w] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
|
23
|
Fully automated plaque characterization in intravascular OCT images using hybrid convolutional and lumen morphology features. Sci Rep 2020; 10:2596. [PMID: 32054895 PMCID: PMC7018759 DOI: 10.1038/s41598-020-59315-6] [Citation(s) in RCA: 37] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2019] [Accepted: 01/17/2020] [Indexed: 11/28/2022] Open
Abstract
For intravascular OCT (IVOCT) images, we developed an automated atherosclerotic plaque characterization method that used a hybrid learning approach, which combined deep-learning convolutional and hand-crafted, lumen morphological features. Processing was done on innate A-line units with labels fibrolipidic (fibrous tissue followed by lipidous tissue), fibrocalcific (fibrous tissue followed by calcification), or other. We trained/tested on an expansive data set (6,556 images), and performed an active learning, relabeling step to improve noisy ground truth labels. Conditional random field was an important post-processing step to reduce classification errors. Sensitivities/specificities were 84.8%/97.8% and 91.4%/95.7% for fibrolipidic and fibrocalcific plaques, respectively. Over lesions, en face classification maps showed automated results that agreed favorably to manually labeled counterparts. Adding lumen morphological features gave statistically significant improvement (p < 0.05), as compared to classification with convolutional features alone. Automated assessments of clinically relevant plaque attributes (arc angle and length), compared favorably to those from manual labels. Our hybrid approach gave statistically improved results as compared to previous A-line classification methods using deep learning or hand-crafted features alone. This plaque characterization approach is fully automated, robust, and promising for live-time treatment planning and research applications.
Collapse
|
24
|
Lee J, Kolluru C, Gharaibeh Y, Prabhu D, Zimin VN, Bezerra H, Wilson D. Automatic A-line coronary plaque classification using combined deep learning and textural features in intravascular OCT images. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2020; 11315:1131513. [PMID: 35291576 PMCID: PMC8920332 DOI: 10.1117/12.2549066] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
We developed a fully automated method for classifying A-line coronary plaques in intravascular optical coherence tomography images using combined deep learning and textural features. The proposed method was trained on 4,292 images from 48 pullbacks giving 80 manually labeled, volumes of interest. Preprocessing steps including guidewire/shadow removal, lumen boundary detection, pixel shifting, and noise reduction were employed. We built a convolutional neural network to extract the deep learning features from the preprocessed image. Traditional textural features were also extracted and combined with deep learning features. Feature selection was performed using the minimum redundancy maximum relevance method. Combined features were utilized as inputs for a random forest classifier. After classification, conditional random field (CRF) method was used for classification noise cleaning. We determined a sub-feature set with the most predictive power. With CRF noise cleaning, sensitivities/specificities were 82.2%/90.8% and 82.4%/89.2% for fibrolipidic and fibrocalcific classes, respectively, with good Dice coefficients. The classification noise cleaning step improved performance metrics by nearly 10-15%. The predicted en face classification maps of entire pullbacks agreed favorably to the manually labeled counterparts. Both assessments suggested that our automated measurements gave clinically relevant results. The proposed method is very promising with regards to both clinical treatment planning and research applications.
Collapse
Affiliation(s)
- Juhwan Lee
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA 44106
| | - Chaitanya Kolluru
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA 44106
| | - Yazan Gharaibeh
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA 44106
| | - David Prabhu
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA 44106
| | - Vladislav N Zimin
- Cardiovascular Imaging Core Laboratory, Harrington Heart and Vascular Institute, University Hospitals Cleveland Medical Center, Cleveland, OH, USA 44106
| | - Hiram Bezerra
- Cardiovascular Imaging Core Laboratory, Harrington Heart and Vascular Institute, University Hospitals Cleveland Medical Center, Cleveland, OH, USA 44106
| | - David Wilson
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA 44106
- Department of Radiology, Case Western Reserve University, Cleveland, OH, USA 44106
| |
Collapse
|
25
|
Abdolmanafi A, Cheriet F, Duong L, Ibrahim R, Dahdah N. An automatic diagnostic system of coronary artery lesions in Kawasaki disease using intravascular optical coherence tomography imaging. JOURNAL OF BIOPHOTONICS 2020; 13:e201900112. [PMID: 31423740 DOI: 10.1002/jbio.201900112] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/25/2019] [Revised: 07/30/2019] [Accepted: 08/13/2019] [Indexed: 05/23/2023]
Abstract
Intravascular optical coherence tomography (IV-OCT) is a light-based imaging modality with high resolution, which employs near-infrared light to provide tomographic intracoronary images. Morbidity caused by coronary heart disease is a substantial cause of acute coronary syndrome and sudden cardiac death. The most common intracoronay complications caused by coronary artery disease are intimal hyperplasia, calcification, fibrosis, neovascularization and macrophage accumulation, which require efficient prevention strategies. OCT can provide discriminative information of the intracoronary tissues, which can be used to train a robust fully automatic tissue characterization model based on deep learning. In this study, we aimed to design a diagnostic model of coronary artery lesions. Particularly, we trained a random forest using convolutional neural network features to distinguish between normal and diseased arterial wall structure. Then, based on the arterial wall structure, fully convolutional network is designed to extract the tissue layers in normal cases, and pathological tissues regardless of lesion type in pathological cases. Then, the type of the lesions can be characterized with high precision using our previous model. The results demonstrate the robustness of the model with the approximate overall accuracy up to 90%.
Collapse
Affiliation(s)
- Atefeh Abdolmanafi
- Department of Computer Engineering, École Polytechnique de Montréal, Montréal, Canada
- Division of Pediatric Cardiology, Centre Hospitalier Universitaire Sainte-Justine, Montréal, Canada
| | - Farida Cheriet
- Department of Computer Engineering, École Polytechnique de Montréal, Montréal, Canada
- Division of Pediatric Cardiology, Centre Hospitalier Universitaire Sainte-Justine, Montréal, Canada
| | - Luc Duong
- Division of Pediatric Cardiology, Centre Hospitalier Universitaire Sainte-Justine, Montréal, Canada
- Department of Software and IT Engineering, École de technologie supérieure, Montréal, Canada
| | - Ragui Ibrahim
- Division of Cardiology, Hôpital Pierre Boucher, Longueuil, Canada
| | - Nagib Dahdah
- Division of Pediatric Cardiology, Centre Hospitalier Universitaire Sainte-Justine, Montréal, Canada
| |
Collapse
|
26
|
Lee J, Prabhu D, Kolluru C, Gharaibeh Y, Zimin VN, Bezerra HG, Wilson DL. Automated plaque characterization using deep learning on coronary intravascular optical coherence tomographic images. BIOMEDICAL OPTICS EXPRESS 2019; 10:6497-6515. [PMID: 31853413 PMCID: PMC6913416 DOI: 10.1364/boe.10.006497] [Citation(s) in RCA: 38] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/11/2019] [Revised: 11/09/2019] [Accepted: 11/10/2019] [Indexed: 05/28/2023]
Abstract
Accurate identification of coronary plaque is very important for cardiologists when treating patients with advanced atherosclerosis. We developed fully-automated semantic segmentation of plaque in intravascular OCT images. We trained/tested a deep learning model on a folded, large, manually annotated clinical dataset. The sensitivities/specificities were 87.4%/89.5% and 85.1%/94.2% for pixel-wise classification of lipidous and calcified plaque, respectively. Automated clinical lesion metrics, potentially useful for treatment planning and research, compared favorably (<4%) with those derived from ground-truth labels. When we converted the results to A-line classification, they were significantly better (p < 0.05) than those obtained previously by using deep learning classifications of A-lines.
Collapse
Affiliation(s)
- Juhwan Lee
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - David Prabhu
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Chaitanya Kolluru
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Yazan Gharaibeh
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
| | - Vladislav N. Zimin
- Cardiovascular Imaging Core Laboratory, Harrington Heart & Vascular Institute, University Hospitals Cleveland Medical Center, Cleveland, OH 44106, USA
| | - Hiram G. Bezerra
- Cardiovascular Imaging Core Laboratory, Harrington Heart & Vascular Institute, University Hospitals Cleveland Medical Center, Cleveland, OH 44106, USA
| | - David L. Wilson
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
- Department of Radiology, Case Western Reserve University, Cleveland, OH 44106, USA
| |
Collapse
|
27
|
Yan Q, Xu M, Wong DWK, Taruya A, Tanaka A, Liu J, Wong P, Cheng J. Automatic fibroatheroma identification in intravascular optical coherence tomography volumes. JOURNAL OF AMBIENT INTELLIGENCE AND HUMANIZED COMPUTING 2019. [DOI: 10.1007/s12652-019-01549-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/03/2018] [Accepted: 10/14/2019] [Indexed: 08/30/2023]
|
28
|
Gharaibeh Y, Prabhu D, Kolluru C, Lee J, Zimin V, Bezerra H, Wilson D. Coronary calcification segmentation in intravascular OCT images using deep learning: application to calcification scoring. J Med Imaging (Bellingham) 2019; 6:045002. [PMID: 31903407 PMCID: PMC6934132 DOI: 10.1117/1.jmi.6.4.045002] [Citation(s) in RCA: 34] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2019] [Accepted: 12/05/2019] [Indexed: 01/18/2023] Open
Abstract
Major calcifications are of great concern when performing percutaneous coronary interventions because they inhibit proper stent deployment. We created a comprehensive software to segment calcifications in intravascular optical coherence tomography (IVOCT) images and to calculate their impact using the stent-deployment calcification score, as reported by Fujino et al. We segmented the vascular lumen and calcifications using the pretrained SegNet, convolutional neural network, which was refined for our task. We cleaned segmentation results using conditional random field processing. We evaluated the method on manually annotated IVOCT volumes of interest (VOIs) without lesions and with calcifications, lipidous, or mixed lesions. The dataset included 48 VOIs taken from 34 clinical pullbacks, giving a total of 2640 in vivo images. Annotations were determined from consensus between two expert analysts. Keeping VOIs intact, we performed 10-fold cross-validation over all data. Following segmentation noise cleaning, we obtained sensitivities of 0.85 ± 0.04 , 0.99 ± 0.01 , and 0.97 ± 0.01 for calcified, lumen, and other tissue classes, respectively. From segmented regions, we automatically determined calcification depth, angle, and thickness attributes. Bland-Altman analysis suggested strong correlation between manually and automatically obtained lumen and calcification attributes. Agreement between manually and automatically obtained stent-deployment calcification scores was good (four of five lesions gave exact agreement). Results are encouraging and suggest our classification approach could be applied clinically for assessment and treatment planning of coronary calcification lesions.
Collapse
Affiliation(s)
- Yazan Gharaibeh
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - David Prabhu
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Chaitanya Kolluru
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Juhwan Lee
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Vladislav Zimin
- University Hospitals Cleveland Medical Center, Harrington Heart and Vascular Institute, Cardiovascular Imaging Core Laboratory, Cleveland, Ohio, United States
| | - Hiram Bezerra
- University Hospitals Cleveland Medical Center, Harrington Heart and Vascular Institute, Cardiovascular Imaging Core Laboratory, Cleveland, Ohio, United States
| | - David Wilson
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
- Case Western Reserve University, Department of Radiology, Cleveland, Ohio, United States
| |
Collapse
|
29
|
Prabhu D, Bezerra HG, Kolluru C, Gharaibeh Y, Mehanna E, Wu H, Wilson DL. Automated A-line coronary plaque classification of intravascular optical coherence tomography images using handcrafted features and large datasets. JOURNAL OF BIOMEDICAL OPTICS 2019; 24:1-15. [PMID: 31586357 PMCID: PMC6784787 DOI: 10.1117/1.jbo.24.10.106002] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/23/2019] [Accepted: 08/20/2019] [Indexed: 05/31/2023]
Abstract
We developed machine learning methods to identify fibrolipidic and fibrocalcific A-lines in intravascular optical coherence tomography (IVOCT) images using a comprehensive set of handcrafted features. We incorporated features developed in previous studies (e.g., optical attenuation and A-line peaks). In addition, we included vascular lumen morphology and three-dimensional (3-D) digital edge and texture features. Classification methods were developed using expansive datasets (∼7000 images), consisting of both clinical in-vivo images and an ex-vivo dataset, which was validated using 3-D cryo-imaging/histology. Conditional random field was used to perform 3-D classification noise cleaning of classification results. We tested various multiclass approaches, classifiers, and feature selection schemes and found that a three-class support vector machine with minimal-redundancy-maximal-relevance feature selection gave the best performance. We found that inclusion of our morphological and 3-D features improved overall classification accuracy. On a held-out test set consisting of >1700 images, we obtained an overall accuracy of 81.58%, with the following (sensitivity/specificity) for each class: other (81.43/89.59), fibrolipidic (94.48/87.32), and fibrocalcific (74.82/95.28). The en-face views of classification results showed that automated classification easily captured the preponderance of a disease segment (e.g., a calcified segment had large regions of fibrocalcific classifications). Finally, we demonstrated proof-of-concept for streamlining A-line classification output with existing fibrolipidic and fibrocalcific boundary segmentation methods, to enable fully automated plaque quantification. The results suggest that our classification approach is a viable step toward fully automated IVOCT plaque classification and segmentation for live-time treatment planning and for offline assessment of drug and biologic therapeutics.
Collapse
Affiliation(s)
- David Prabhu
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Hiram G. Bezerra
- University Hospitals Cleveland Medical Center, Harrington Heart and Vascular Institute, Cardiovascular Imaging Core Laboratory, Cleveland, Ohio, United States
| | - Chaitanya Kolluru
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Yazan Gharaibeh
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Emile Mehanna
- University Hospitals Cleveland Medical Center, Harrington Heart and Vascular Institute, Cardiovascular Imaging Core Laboratory, Cleveland, Ohio, United States
| | - Hao Wu
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - David L. Wilson
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
- Case Western Reserve University, Department of Radiology, Cleveland, Ohio, United States
| |
Collapse
|
30
|
Rico-Jimenez JJ, Campos-Delgado DU, Buja LM, Vela D, Jo JA. Intravascular optical coherence tomography method for automated detection of macrophage infiltration within atherosclerotic coronary plaques. Atherosclerosis 2019; 290:94-102. [PMID: 31604172 DOI: 10.1016/j.atherosclerosis.2019.09.023] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Revised: 08/22/2019] [Accepted: 09/27/2019] [Indexed: 02/05/2023]
Abstract
BACKGROUND AND AIMS Significant macrophages infiltration in advanced atherosclerotic plaques promotes acute coronary events. Hence, the clinical imaging of macrophage content in coronary atherosclerotic plaques could potentially aid in identifying patients most at risk of future acute coronary events. The aim of this study was to introduce and validate a simple intravascular optical coherence tomography (IV-OCT) image processing method for automated, accurate and fast detection of macrophage infiltration within coronary atherosclerotic plaques. METHODS This method calculates the ratio of the normalized-intensity standard deviation (NSD) values estimated over two axially-adjacent regions of interest in an IV-OCT cross-sectional image (B-scan). When applied to entire IV-OCT B-scans, this method highlights plaque areas with high NSD ratio values (NSDRatio), which was demonstrated to be correlated with the degree of coronary plaque macrophage infiltration. RESULTS Using an optimized NSDRatio threshold value, coronary plaque macrophage infiltration could be detected with ~88% sensitivity and specificity in a database of 28 IV-OCT scans from postmortem coronary segments. For comparison, using an optimized NSD threshold value, considered the standard IV-OCT signature for macrophages, coronary plaque macrophage infiltration could be detected with only ~55% sensitivity and specificity. CONCLUSIONS The proposed NSDRatio method significantly increases the sensitivity and specificity for the detection of coronary plaque macrophage infiltration compared to the standard NSD method. This computationally efficient method can be seamlessly implemented within standard IV-OCT imaging systems for in-vivo real-time imaging of macrophage content in coronary plaques, which could potentially aid in identifying patients most at risk of future acute coronary events.
Collapse
Affiliation(s)
- Jose J Rico-Jimenez
- Department of Biomedical Engineering, Texas A&M University, College Station, TX, USA
| | | | | | | | - Javier A Jo
- Department of Biomedical Engineering, Texas A&M University, College Station, TX, USA; School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK, USA.
| |
Collapse
|
31
|
Kozlowski KM, Sharma GK, Chen JJ, Qi L, Osann K, Jing JC, Ahuja GS, Heidari AE, Chung PS, Kim S, Chen Z, Wong BJF. Dynamic programming and automated segmentation of optical coherence tomography images of the neonatal subglottis: enabling efficient diagnostics to manage subglottic stenosis. JOURNAL OF BIOMEDICAL OPTICS 2019; 24:1-8. [PMID: 31493317 PMCID: PMC6732661 DOI: 10.1117/1.jbo.24.9.096001] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/30/2019] [Accepted: 08/20/2019] [Indexed: 05/25/2023]
Abstract
Subglottic stenosis (SGS) is a challenging disease to diagnose in neonates. Long-range optical coherence tomography (OCT) is an optical imaging modality that has been described to image the subglottis in intubated neonates. A major challenge associated with OCT imaging is the lack of an automated method for image analysis and micrometry of large volumes of data that are acquired with each airway scan (1 to 2 Gb). We developed a tissue segmentation algorithm that identifies, measures, and conducts image analysis on tissue layers within the mucosa and submucosa and compared these automated tissue measurements with manual tracings. We noted small but statistically significant differences in thickness measurements of the mucosa and submucosa layers in the larynx (p < 0.001), subglottis (p = 0.015), and trachea (p = 0.012). The automated algorithm was also shown to be over 8 times faster than the manual approach. Moderate Pearson correlations were found between different tissue texture parameters and the patient’s gestational age at birth, age in days, duration of intubation, and differences with age (mean age 17 days). Automated OCT data analysis is necessary in the diagnosis and monitoring of SGS, as it can provide vital information about the airway in real time and aid clinicians in making management decisions for intubated neonates.
Collapse
Affiliation(s)
- Konrad M. Kozlowski
- University of California Irvine, Beckman Laser Institute, Irvine, California, United States
| | - Giriraj K. Sharma
- University of California Irvine, Department of Otolaryngology-Head and Neck Surgery, Orange, California, United States
| | - Jason J. Chen
- University of California Irvine, Beckman Laser Institute, Irvine, California, United States
- University of California Irvine, Department of Biomedical Engineering, Irvine, California, United States
| | - Li Qi
- Southern Medical University, School of Biomedical Engineering, Guangzhou, China
| | - Kathryn Osann
- University of California Irvine, Department of Otolaryngology-Head and Neck Surgery, Orange, California, United States
- University of California Irvine, Department of Biomedical Engineering, Irvine, California, United States
| | - Joseph C. Jing
- University of California Irvine, Beckman Laser Institute, Irvine, California, United States
- University of California Irvine, Department of Biomedical Engineering, Irvine, California, United States
| | - Gurpreet S. Ahuja
- University of California Irvine, Department of Otolaryngology-Head and Neck Surgery, Orange, California, United States
- Children’s Hospital of Orange County, Orange, California, United States
| | - Andrew E. Heidari
- University of California Irvine, Beckman Laser Institute, Irvine, California, United States
| | - Phil-Sang Chung
- Dankook University, Beckman Laser Institute Korea, Cheoan, Republic of Korea
| | - Sehwan Kim
- Dankook University, Beckman Laser Institute Korea, Cheoan, Republic of Korea
- Dankook University, School of Medicine, Department of Biomedical Engineering, Cheoan, Republic of Korea
| | - Zhongping Chen
- University of California Irvine, Beckman Laser Institute, Irvine, California, United States
- University of California Irvine, Department of Biomedical Engineering, Irvine, California, United States
| | - Brian J.-F. Wong
- University of California Irvine, Beckman Laser Institute, Irvine, California, United States
- University of California Irvine, Department of Otolaryngology-Head and Neck Surgery, Orange, California, United States
- University of California Irvine, Department of Biomedical Engineering, Irvine, California, United States
| |
Collapse
|
32
|
Yang J, Zhang B, Wang H, Lin F, Han Y, Liu X. Automated characterization and classification of coronary atherosclerotic plaques for intravascular optical coherence tomography. Biocybern Biomed Eng 2019. [DOI: 10.1016/j.bbe.2019.06.006] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
33
|
Ali ZA, Karimi Galougahi K, Maehara A, Shlofmitz RA, Ben-Yehuda O, Mintz GS, Stone GW. Intracoronary Optical Coherence Tomography 2018: Current Status and Future Directions. JACC Cardiovasc Interv 2019; 10:2473-2487. [PMID: 29268880 DOI: 10.1016/j.jcin.2017.09.042] [Citation(s) in RCA: 73] [Impact Index Per Article: 14.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/07/2017] [Accepted: 09/13/2017] [Indexed: 01/14/2023]
Abstract
The advent of intravascular imaging has been a significant advancement in visualization of coronary arteries, particularly with optical coherence tomography (OCT) that allows for high-resolution imaging of intraluminal and transmural coronary structures. Accumulating data support a clinical role for OCT in a multitude of clinical scenarios, including assessing the natural history of atherosclerosis and modulating effects of therapies, mechanisms of acute coronary syndromes, mechanistic insights into the effects of novel interventional devices, and optimization of percutaneous coronary intervention. In this state-of-the-art review, we provide an overview of the published data on the clinical utility of OCT, highlighting the areas that need further investigation and the current barriers for further adoption of OCT in interventional cardiology practice.
Collapse
Affiliation(s)
- Ziad A Ali
- Center for Interventional Vascular Therapy, Division of Cardiology, Presbyterian Hospital and Columbia University, New York, New York; Cardiovascular Research Foundation, New York, New York.
| | - Keyvan Karimi Galougahi
- Center for Interventional Vascular Therapy, Division of Cardiology, Presbyterian Hospital and Columbia University, New York, New York
| | - Akiko Maehara
- Center for Interventional Vascular Therapy, Division of Cardiology, Presbyterian Hospital and Columbia University, New York, New York; Cardiovascular Research Foundation, New York, New York
| | | | - Ori Ben-Yehuda
- Center for Interventional Vascular Therapy, Division of Cardiology, Presbyterian Hospital and Columbia University, New York, New York; Cardiovascular Research Foundation, New York, New York
| | - Gary S Mintz
- Cardiovascular Research Foundation, New York, New York
| | - Gregg W Stone
- Center for Interventional Vascular Therapy, Division of Cardiology, Presbyterian Hospital and Columbia University, New York, New York; Cardiovascular Research Foundation, New York, New York
| |
Collapse
|
34
|
Kolluru C, Prabhu D, Gharaibeh Y, Bezerra H, Guagliumi G, Wilson D. Deep neural networks for A-line-based plaque classification in coronary intravascular optical coherence tomography images. J Med Imaging (Bellingham) 2018; 5:044504. [PMID: 30525060 PMCID: PMC6275844 DOI: 10.1117/1.jmi.5.4.044504] [Citation(s) in RCA: 41] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2018] [Accepted: 10/24/2018] [Indexed: 11/14/2022] Open
Abstract
We develop neural-network-based methods for classifying plaque types in clinical intravascular optical coherence tomography (IVOCT) images of coronary arteries. A single IVOCT pullback can consist of >500 microscopic-resolution images, creating both a challenge for physician interpretation during an interventional procedure and an opportunity for automated analysis. In the proposed method, we classify each A-line, a datum element that better captures physics and pathophysiology than a voxel, as a fibrous layer followed by calcification (fibrocalcific), a fibrous layer followed by a lipidous deposit (fibrolipidic), or other. For A-line classification, the usefulness of a convolutional neural network (CNN) is compared with that of a fully connected artificial neural network (ANN). A total of 4469 image frames across 48 pullbacks that are manually labeled using consensus labeling from two experts are used for training, evaluation, and testing. A 10-fold cross-validation using held-out pullbacks is applied to assess classifier performance. Noisy A-line classifications are cleaned by applying a conditional random field (CRF) and morphological processing to pullbacks in the en-face view. With CNN (ANN) approaches, we achieve an accuracy of 77.7%±4.1% (79.4%±2.9%) for fibrocalcific, 86.5%±2.3% (83.4%±2.6%) for fibrolipidic, and 85.3%±2.5% (82.4%±2.2%) for other, across all folds following CRF noise cleaning. The results without CRF cleaning are typically reduced by 10% to 15%. The enhanced performance of the CNN was likely due to spatial invariance of the convolution operation over the input A-line. The predicted en-face classification maps of entire pullbacks agree favorably to the annotated counterparts. In some instances, small error regions are actually hard to call when re-examined by human experts. Even in worst-case pullbacks, it can be argued that the results will not negatively impact usage by physicians, as there is a preponderance of correct calls.
Collapse
Affiliation(s)
- Chaitanya Kolluru
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - David Prabhu
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Yazan Gharaibeh
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Hiram Bezerra
- University Hospitals, Harrington Heart and Vascular Institute, Cardiovascular Imaging Core Laboratory, Cleveland, Ohio, United States
| | - Giulio Guagliumi
- Ospedali Riuniti di Bergamo, Cardiovascular Department, Bergamo, Italy
| | - David Wilson
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States.,Case Western Reserve University, Department of Radiology, Cleveland, Ohio, United States
| |
Collapse
|
35
|
Improving Plaque Classification With Optical Coherence Tomography. JACC Cardiovasc Imaging 2018; 11:1677-1678. [DOI: 10.1016/j.jcmg.2017.11.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/01/2017] [Revised: 11/13/2017] [Accepted: 11/14/2017] [Indexed: 11/21/2022]
|
36
|
Abdolmanafi A, Duong L, Dahdah N, Adib IR, Cheriet F. Characterization of coronary artery pathological formations from OCT imaging using deep learning. BIOMEDICAL OPTICS EXPRESS 2018; 9:4936-4960. [PMID: 30319913 PMCID: PMC6179392 DOI: 10.1364/boe.9.004936] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/24/2018] [Revised: 09/13/2018] [Accepted: 09/14/2018] [Indexed: 05/18/2023]
Abstract
Coronary artery disease is the number one health hazard leading to the pathological formations in coronary artery tissues. In severe cases, they can lead to myocardial infarction and sudden death. Optical Coherence Tomography (OCT) is an interferometric imaging modality, which has been recently used in cardiology to characterize coronary artery tissues providing high resolution ranging from 10 to 20 µm. In this study, we investigate different deep learning models for robust tissue characterization to learn the various intracoronary pathological formations caused by Kawasaki disease (KD) from OCT imaging. The experiments are performed on 33 retrospective cases comprising of pullbacks of intracoronary cross-sectional images obtained from different pediatric patients with KD. Our approach evaluates deep features computed from three different pre-trained convolutional networks. Then, a majority voting approach is applied to provide the final classification result. The results demonstrate high values of accuracy, sensitivity, and specificity for each tissue (up to 0.99 ± 0.01). Hence, deep learning models and especially, majority voting method are robust for automatic interpretation of the OCT images.
Collapse
Affiliation(s)
- Atefeh Abdolmanafi
- Dept. of Software and IT Engineering, École de technologie supérieure, Montréal,
Canada
| | - Luc Duong
- Dept. of Software and IT Engineering, École de technologie supérieure, Montréal,
Canada
| | - Nagib Dahdah
- Div. of Pediatric Cardiology and Research Center, Centre Hospitalier Universitaire Sainte-Justine, Montréal,
Canada
| | | | - Farida Cheriet
- Dept. of Computer Engineering, École Polytechnique de Montréal, Montréal,
Canada
| |
Collapse
|
37
|
Cao Y, Jin Q, Chen Y, Yin Q, Qin X, Li J, Zhu R, Zhao W. Automatic Side Branch Ostium Detection and Main Vascular Segmentation in Intravascular Optical Coherence Tomography Images. IEEE J Biomed Health Inform 2018; 22:1531-1539. [DOI: 10.1109/jbhi.2017.2771829] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
38
|
Liu S, Dzyubachyk O, Eggermont J, Nakatani S, Lelieveldt BPF, Dijkstra J. Histogram-based standardization of intravascular optical coherence tomography images acquired from different imaging systems. Med Phys 2018; 45:4158-4170. [PMID: 30039851 DOI: 10.1002/mp.13103] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2017] [Revised: 06/11/2018] [Accepted: 07/05/2018] [Indexed: 12/19/2022] Open
Abstract
PURPOSE Intravascular optical coherence tomography (OCT) is widely used for analysis of the coronary artery disease. Its high spatial resolution allows for visualization of arterial tissue components in detail. There are different OCT systems on the market, each of which produces data characterized by its own intensity range and distribution. These differences should be taken into account for the development of image processing algorithms. In order to overcome this difference in the intensity range and distribution, we developed a framework for matching intensities based on the exact histogram matching technique. METHODS In our method, the key step for using the exact histogram matching is to determine the target histogram. For this, we proposed two schemes: a global scheme that uses a single histogram as the target histogram for all the pullbacks, and a local scheme that selects for each single image a target histogram from a predefined database. These two schemes are compared on a unique dataset containing pairs of pullbacks that were acquired shortly after each other with systems from two vendors, St. Jude and Terumo. Pullbacks were aligned according to anatomical landmarks, and a database of matched histogram pairs was created. A leave-one-out cross validation was used to compare performance of the two schemes. The matching accuracy was evaluated by comparing: (a) histograms using Euclidean (dx2 ) and Kolmogorov-Smirnov (dKS ) distances, and (b) median intensity level within anatomical regions of interest. RESULTS Leave-one-out validation indicated that both matching schemes yield comparably high accuracies across the entire validation dataset. The local scheme outperforms the global scheme with marginally lower dissimilarities at both histogram level and intensity level. High visual similarity was observed when comparing the matched images to their aligned counterparts. CONCLUSION Both local and global schemes are robust and produce accurate intensity matching. While local scheme performs marginally better than the global scheme, it requires a predefined histogram dataset and is more time consuming. Thus, for offline standardization of the images, the local scheme should be preferred for being more accurate. For online standardization or when another system is involved, the global scheme can be used as a simple and nearly-as-accurate alternative.
Collapse
Affiliation(s)
- Shengnan Liu
- Division of Imaging Processing, Department of Radiology, Leiden University Medical Center, Leiden, 2300, RC, The Netherlands
| | - Oleh Dzyubachyk
- Division of Imaging Processing, Department of Radiology, Leiden University Medical Center, Leiden, 2300, RC, The Netherlands
| | - Jeroen Eggermont
- Division of Imaging Processing, Department of Radiology, Leiden University Medical Center, Leiden, 2300, RC, The Netherlands
| | - Shimpei Nakatani
- Division of Cardiology, Sakurabashi Watanabe Hospital, Osaka, 530-0001, Japan
| | - Boudewijn P F Lelieveldt
- Division of Imaging Processing, Department of Radiology, Leiden University Medical Center, Leiden, 2300, RC, The Netherlands
- Intelligent Systems Department, Delft University of Technology, Delft, 2628, CD, The Netherlands
| | - Jouke Dijkstra
- Division of Imaging Processing, Department of Radiology, Leiden University Medical Center, Leiden, 2300, RC, The Netherlands
| |
Collapse
|
39
|
Boi A, Jamthikar AD, Saba L, Gupta D, Sharma A, Loi B, Laird JR, Khanna NN, Suri JS. A Survey on Coronary Atherosclerotic Plaque Tissue Characterization in Intravascular Optical Coherence Tomography. Curr Atheroscler Rep 2018; 20:33. [PMID: 29781047 DOI: 10.1007/s11883-018-0736-8] [Citation(s) in RCA: 41] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
PURPOSE OF REVIEW Atherosclerotic plaque deposition within the coronary vessel wall leads to arterial stenosis and severe catastrophic events over time. Identification of these atherosclerotic plaque components is essential to pre-estimate the risk of cardiovascular disease (CVD) and stratify them as a high or low risk. The characterization and quantification of coronary plaque components are not only vital but also a challenging task which can be possible using high-resolution imaging techniques. RECENT FINDING Atherosclerotic plaque components such as thin cap fibroatheroma (TCFA), fibrous cap, macrophage infiltration, large necrotic core, and thrombus are the microstructural plaque components that can be detected with only high-resolution imaging modalities such as intravascular ultrasound (IVUS) and optical coherence tomography (OCT). Light-based OCT provides better visualization of plaque tissue layers of coronary vessel walls as compared to IVUS. Three dominant paradigms have been identified to characterize atherosclerotic plaque components based on optical attenuation coefficients, machine learning algorithms, and deep learning techniques. This review (condensation of 126 papers after downloading 150 articles) presents a detailed comparison among various methodologies utilized for plaque tissue characterization, classification, and arterial measurements in OCT. Furthermore, this review presents the different ways to predict and stratify the risk associated with the CVD based on plaque characterization and measurements in OCT. Moreover, this review discovers three different paradigms for plaque characterization and their pros and cons. Among all of the techniques, a combination of machine learning and deep learning techniques is a best possible solution that provides improved OCT-based risk stratification.
Collapse
Affiliation(s)
- Alberto Boi
- Department of Cardiology, University of Cagliari, Cagliari, Italy
| | - Ankush D Jamthikar
- Department of Electronics and Communication Engineering, Visvesvaraya National Institute of Technology Nagpur, Nagpur, Maharashtra, India
| | - Luca Saba
- Department of Radiology, University of Cagliari, Cagliari, Italy
| | - Deep Gupta
- Department of Electronics and Communication Engineering, Visvesvaraya National Institute of Technology Nagpur, Nagpur, Maharashtra, India
| | - Aditya Sharma
- Division of Cardiovascular Medicine, University of Virginia, Charlottesville, VA, USA
| | - Bruno Loi
- Department of Radiology, University of Cagliari, Cagliari, Italy
| | | | - Narendra N Khanna
- Department of Cardiology, Indraprastha Apollo Hospitals, New Delhi, India
| | - Jasjit S Suri
- Coronary Arterial Division, AtheroPoint™, Roseville, CA, USA.
| |
Collapse
|
40
|
Nam HS, Kang WJ, Lee MW, Song JW, Kim JW, Oh WY, Yoo H. Multispectral analog-mean-delay fluorescence lifetime imaging combined with optical coherence tomography. BIOMEDICAL OPTICS EXPRESS 2018; 9:1930-1947. [PMID: 29675330 PMCID: PMC5905935 DOI: 10.1364/boe.9.001930] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/14/2017] [Revised: 03/15/2018] [Accepted: 03/18/2018] [Indexed: 05/19/2023]
Abstract
The pathophysiological progression of chronic diseases, including atherosclerosis and cancer, is closely related to compositional changes in biological tissues containing endogenous fluorophores such as collagen, elastin, and NADH, which exhibit strong autofluorescence under ultraviolet excitation. Fluorescence lifetime imaging (FLIm) provides robust detection of the compositional changes by measuring fluorescence lifetime, which is an inherent property of a fluorophore. In this paper, we present a dual-modality system combining a multispectral analog-mean-delay (AMD) FLIm and a high-speed swept-source optical coherence tomography (OCT) to simultaneously visualize the cross-sectional morphology and biochemical compositional information of a biological tissue. Experiments using standard fluorescent solutions showed that the fluorescence lifetime could be measured with a precision of less than 40 psec using the multispectral AMD-FLIm without averaging. In addition, we performed ex vivo imaging on rabbit iliac normal-looking and atherosclerotic specimens to demonstrate the feasibility of the combined FLIm-OCT system for atherosclerosis imaging. We expect that the combined FLIm-OCT will be a promising next-generation imaging technique for diagnosing atherosclerosis and cancer due to the advantages of the proposed label-free high-precision multispectral lifetime measurement.
Collapse
Affiliation(s)
- Hyeong Soo Nam
- Department of Biomedical Engineering, Hanyang University, 222 Wangsimni-ro, Seongdong-gu, Seoul 04673, South Korea
- Equally contributed to this study
| | - Woo Jae Kang
- Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology, 291 Gwahang-no, Yuseong-gu, Daejeon 34141, South Korea
- Equally contributed to this study
| | - Min Woo Lee
- Department of Biomedical Engineering, Hanyang University, 222 Wangsimni-ro, Seongdong-gu, Seoul 04673, South Korea
| | - Joon Woo Song
- Cardiovascular Center, Korea University Guro Hospital, 148 Gurodong-ro, Guro-gu, Seoul 08308, South Korea
| | - Jin Won Kim
- Cardiovascular Center, Korea University Guro Hospital, 148 Gurodong-ro, Guro-gu, Seoul 08308, South Korea
| | - Wang-Yuhl Oh
- Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology, 291 Gwahang-no, Yuseong-gu, Daejeon 34141, South Korea
| | - Hongki Yoo
- Department of Biomedical Engineering, Hanyang University, 222 Wangsimni-ro, Seongdong-gu, Seoul 04673, South Korea
| |
Collapse
|
41
|
Kolluru C, Prabhu D, Gharaibeh Y, Wu H, Wilson DL. Voxel-based plaque classification in coronary intravascular optical coherence tomography images using decision trees. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2018; 10575. [PMID: 29568146 DOI: 10.1117/12.2293226] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Intravascular Optical Coherence Tomography (IVOCT) is a high contrast, 3D microscopic imaging technique that can be used to assess atherosclerosis and guide stent interventions. Despite its advantages, IVOCT image interpretation is challenging and time consuming with over 500 image frames generated in a single pullback volume. We have developed a method to classify voxel plaque types in IVOCT images using machine learning. To train and test the classifier, we have used our unique database of labeled cadaver vessel IVOCT images accurately registered to gold standard cryo-images. This database currently contains 300 images and is growing. Each voxel is labeled as fibrotic, lipid-rich, calcified or other. Optical attenuation, intensity and texture features were extracted for each voxel and were used to build a decision tree classifier for multi-class classification. Five-fold cross-validation across images gave accuracies of 96 % ± 0.01 %, 90 ± 0.02% and 90 % ± 0.01 % for fibrotic, lipid-rich and calcified classes respectively. To rectify performance degradation seen in left out vessel specimens as opposed to left out images, we are adding data and reducing features to limit overfitting. Following spatial noise cleaning, important vascular regions were unambiguous in display. We developed displays that enable physicians to make rapid determination of calcified and lipid regions. This will inform treatment decisions such as the need for devices (e.g., atherectomy or scoring balloon in the case of calcifications) or extended stent lengths to ensure coverage of lipid regions prone to injury at the edge of a stent.
Collapse
Affiliation(s)
- Chaitanya Kolluru
- Dept. of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA 44106
| | - David Prabhu
- Dept. of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA 44106
| | - Yazan Gharaibeh
- Dept. of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA 44106
| | - Hao Wu
- Dept. of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA 44106
| | - David L Wilson
- Dept. of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA 44106.,Dept. of Radiology, Case Western Reserve University, Cleveland, OH, USA 44106
| |
Collapse
|
42
|
Guha Mazumder A, Chatterjee S, Chatterjee S, Gonzalez JJ, Bag S, Ghosh S, Mukherjee A, Chatterjee J. Spectropathology-corroborated multimodal quantitative imaging biomarkers for neuroretinal degeneration in diabetic retinopathy. Clin Ophthalmol 2017; 11:2073-2089. [PMID: 29200821 PMCID: PMC5703157 DOI: 10.2147/opth.s140110] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022] Open
Abstract
Introduction Image-based early detection for diabetic retinopathy (DR) needs value addition due to lack of well-defined disease-specific quantitative imaging biomarkers (QIBs) for neuroretinal degeneration and spectropathological information at the systemic level. Retinal neurodegeneration is an early event in the pathogenesis of DR. Therefore, development of an integrated assessment method for detecting neuroretinal degeneration using spectropathology and QIBs is necessary for the early diagnosis of DR. Methods The present work explored the efficacy of intensity and textural features extracted from optical coherence tomography (OCT) images after selecting a specific subset of features for the precise classification of retinal layers using variants of support vector machine (SVM). Fourier transform infrared (FTIR) spectroscopy and nuclear magnetic resonance (NMR) spectroscopy were also performed to confirm the spectropathological attributes of serum for further value addition to the OCT, fundoscopy, and fluorescein angiography (FA) findings. The serum metabolomic findings were also incorporated for characterizing retinal layer thickness alterations and vascular asymmetries. Results Results suggested that OCT features could differentiate the retinal lesions indicating retinal neurodegeneration with high sensitivity and specificity. OCT, fundoscopy, and FA provided geometrical as well as optical features. NMR revealed elevated levels of ribitol, glycerophosphocholine, and uridine diphosphate N-acetyl glucosamine, while the FTIR of serum samples confirmed the higher expressions of lipids and β-sheet-containing proteins responsible for neoangiogenesis, vascular fragility, vascular asymmetry, and subsequent neuroretinal degeneration in DR. Conclusion Our data indicated that disease-specific spectropathological alterations could be the major phenomena behind the vascular attenuations observed through fundoscopy and FA, as well as the variations in the intensity and textural features observed in OCT images. Finally, we propose a model that uses spectropathology corroborated with specific QIBs for detecting neuroretinal degeneration in early diagnosis of DR.
Collapse
Affiliation(s)
- Arpan Guha Mazumder
- Multimodal Imaging and Computing for Theranostics Laboratory, School of Medical Science and Technology, Indian Institute of Technology-Kharagpur, Kharagpur, West Bengal, India.,Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Swarnadip Chatterjee
- Advanced Technology Development Centre, Indian Institute of Technology-Kharagpur, Kharagpur, West Bengal, India
| | - Saunak Chatterjee
- Multimodal Imaging and Computing for Theranostics Laboratory, School of Medical Science and Technology, Indian Institute of Technology-Kharagpur, Kharagpur, West Bengal, India
| | - Juan Jose Gonzalez
- Department of Computer and Electrical Engineering, Rice University, Houston, TX, USA
| | - Swarnendu Bag
- Department of Biotechnology, National Institute of Technology Sikkim, Ravangla Sub-Division, South Sikkim
| | - Sambuddha Ghosh
- Department of Ophthalmology, Calcutta National Medical College and Hospital, Kolkata, West Bengal
| | - Anirban Mukherjee
- Department of Electrical Engineering, Indian Institute of Technology-Kharagpur, Kharagpur, West Bengal, India
| | - Jyotirmoy Chatterjee
- Multimodal Imaging and Computing for Theranostics Laboratory, School of Medical Science and Technology, Indian Institute of Technology-Kharagpur, Kharagpur, West Bengal, India
| |
Collapse
|
43
|
Lee JA, Wong DWK, Taruya A, Tanaka A, Foin N, Wong P. Fibroatheroma identification in Intravascular Optical Coherence Tomography images using deep features. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2017:1501-1504. [PMID: 29060164 DOI: 10.1109/embc.2017.8037120] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Identifying vulnerable plaque is important in coronary heart disease diagnosis. Recent emerged imaging modality, Intravascular Optical Coherence Tomography (IVOCT), has been proved to be able to characterize the appearance of vulnerable plaques. Comparing with the manual method, automated fibroatheroma identification would be more efficient and objective. Deep convolutional neural networks have been adopted in many medical image analysis tasks. In this paper, we introduce deep features to resolve fibroatheroma identification problem. Deep features which extracted using four deep convolutional neural networks, AlexNet, GoogLeNet, VGG-16 and VGG-19, are studied. And a dataset of 360 IVOCT images from 18 pullbacks are constructed to evaluate these features. Within these 360 images, 180 images are normal IVOCT images and the rest 180 images are IVOCT images with fibroatheroma. Here, one pullback belongs to one patient; leave-one-patient-out cross-validation is employed for evaluation. Data augmentation is applied on training set for each classification scheme. Linear support vector machine is conducted to classify the normal IVOCT image and IVOCT image with fibroatheroma. The experimental results show that deep features could achieve relatively high accuracy in fibroatheroma identification.
Collapse
|
44
|
Liu S, Sotomi Y, Eggermont J, Nakazawa G, Torii S, Ijichi T, Onuma Y, Serruys PW, Lelieveldt BPF, Dijkstra J. Tissue characterization with depth-resolved attenuation coefficient and backscatter term in intravascular optical coherence tomography images. JOURNAL OF BIOMEDICAL OPTICS 2017; 22:1-16. [PMID: 28901053 DOI: 10.1117/1.jbo.22.9.096004] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/06/2017] [Accepted: 08/21/2017] [Indexed: 05/08/2023]
Abstract
An important application of intravascular optical coherence tomography (IVOCT) for atherosclerotic tissue analysis is using it to estimate attenuation and backscatter coefficients. This work aims at exploring the potential of the attenuation coefficient, a proposed backscatter term, and image intensities in distinguishing different atherosclerotic tissue types with a robust implementation of depth-resolved (DR) approach. Therefore, the DR model is introduced to estimate the attenuation coefficient and further extended to estimate the backscatter-related term in IVOCT images, such that values can be estimated per pixel without predefining any delineation for the estimation. In order to exclude noisy regions with a weak signal, an automated algorithm is implemented to determine the cut-off border in IVOCT images. The attenuation coefficient, backscatter term, and the image intensity are further analyzed in regions of interest, which have been delineated referring to their pathology counterparts. Local statistical values were reported and their distributions were further compared with a two-sample t-test to evaluate the potential for distinguishing six types of tissues. Results show that the IVOCT intensity, DR attenuation coefficient, and backscatter term extracted with the reported implementation are complementary to each other on characterizing six tissue types: mixed, calcification, fibrous, lipid-rich, macrophages, and necrotic core.
Collapse
Affiliation(s)
- Shengnan Liu
- Leiden University Medical Center, Division of Imaging Processing, Department of Radiology, Leiden, The Netherlands
| | - Yohei Sotomi
- University of Amsterdam, Academic Medical Center, Amsterdam, The Netherlands
| | - Jeroen Eggermont
- Leiden University Medical Center, Division of Imaging Processing, Department of Radiology, Leiden, The Netherlands
| | - Gaku Nakazawa
- Tokai University School of Medicine, Department of Cardiology, Kanaagawa, Japan
| | - Sho Torii
- Tokai University School of Medicine, Department of Cardiology, Kanaagawa, Japan
| | - Takeshi Ijichi
- Tokai University School of Medicine, Department of Cardiology, Kanaagawa, Japan
| | - Yoshinobu Onuma
- Thoraxcenter, Erasmus Medical Center, Rotterdam, The Netherlands
- Cardialysis, Rotterdam, The Netherlands
| | - Patrick W Serruys
- International Centre for Circulatory Health, the National Heart and Lung Institute, Imperial College, United Kingdom
| | - Boudewijn P F Lelieveldt
- Leiden University Medical Center, Division of Imaging Processing, Department of Radiology, Leiden, The Netherlands
- Delft University of Technology, Department of Intelligent Systems, Delft, The Netherlands
| | - Jouke Dijkstra
- Leiden University Medical Center, Division of Imaging Processing, Department of Radiology, Leiden, The Netherlands
| |
Collapse
|
45
|
Sotomi Y, Suwannasom P, Tenekecioglu E, Collet C, Nakatani S, Okamura T, Muramatsu T, Ishibashi Y, Tateishi H, Miyazaki Y, Asano T, Katagiri Y, von zur Muehlen C, Tanabe K, Kozuma K, Ozaki Y, Serruys PW, Onuma Y. Imaging assessment of bioresorbable vascular scaffolds. Cardiovasc Interv Ther 2017; 33:11-22. [DOI: 10.1007/s12928-017-0486-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2017] [Accepted: 07/26/2017] [Indexed: 10/19/2022]
|
46
|
Zhou P, Zhu T, He C, Li Z. Automatic classification of atherosclerotic tissue in intravascular optical coherence tomography images. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2017; 34:1152-1159. [PMID: 29036125 DOI: 10.1364/josaa.34.001152] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/13/2017] [Accepted: 05/26/2017] [Indexed: 06/07/2023]
Abstract
Intravascular optical coherence tomography (IVOCT) has been successfully utilized for in vivo diagnostics of coronary plaques. However, classification of atherosclerotic tissues is mainly performed manually by experienced experts, which is time-consuming and subjective. To overcome these limitations, an automatic method of segmentation and classification of IVOCT images is developed in this paper. The method is capable of detecting the plaque contour between the fibrous tissues and other components. Subsequently, the method classifies the tissues based on their texture features described by Fourier transform and discrete wavelet transform. The experimental results of 103 images show that an overall classification accuracy of over 80% in the indicator of depth and span angle is achieved in comparison to manual results. The validation suggests that this method is objective, accurate, and automatic without any manual intervention. The proposed method is able to demonstrate the artery wall morphology successfully, which is valuable for the research of atherosclerotic disease.
Collapse
|
47
|
|
48
|
Reconstruction of stented coronary arteries from optical coherence tomography images: Feasibility, validation, and repeatability of a segmentation method. PLoS One 2017; 12:e0177495. [PMID: 28574987 PMCID: PMC5456060 DOI: 10.1371/journal.pone.0177495] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2017] [Accepted: 04/27/2017] [Indexed: 11/19/2022] Open
Abstract
Optical coherence tomography (OCT) is an established catheter-based imaging modality for the assessment of coronary artery disease and the guidance of stent placement during percutaneous coronary intervention. Manual analysis of large OCT datasets for vessel contours or stent struts detection is time-consuming and unsuitable for real-time applications. In this study, a fully automatic method was developed for detection of both vessel contours and stent struts. The method was applied to in vitro OCT scans of eight stented silicone bifurcation phantoms for validation purposes. The proposed algorithm comprised four main steps, namely pre-processing, lumen border detection, stent strut detection, and three-dimensional point cloud creation. The algorithm was validated against manual segmentation performed by two independent image readers. Linear regression showed good agreement between automatic and manual segmentations in terms of lumen area (r>0.99). No statistically significant differences in the number of detected struts were found between the segmentations. Mean values of similarity indexes were >95% and >85% for the lumen and stent detection, respectively. Stent point clouds of two selected cases, obtained after OCT image processing, were compared to the centerline points of the corresponding stent reconstructions from micro computed tomography, used as ground-truth. Quantitative comparison between the corresponding stent points resulted in median values of ~150 μm and ~40 μm for the total and radial distances of both cases, respectively. The repeatability of the detection method was investigated by calculating the lumen volume and the mean number of detected struts per frame for seven repeated OCT scans of one selected case. Results showed low deviation of values from the median for both analyzed quantities. In conclusion, this study presents a robust automatic method for detection of lumen contours and stent struts from OCT as supported by focused validation against both manual segmentation and micro computed tomography and by good repeatability.
Collapse
|
49
|
Li Z, Tang Q, Jin L, Andrews PM, Chen Y. Monitoring kidney microanatomy changes during ischemia-reperfusion process using texture analysis of OCT images. IEEE PHOTONICS JOURNAL 2017; 9:4000110. [PMID: 29104734 PMCID: PMC5665408 DOI: 10.1109/jphot.2017.2669482] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/17/2023]
Abstract
kidney ischemia-reperfusion (I/R) accounts for the majority of acute kidney injury cases, whose consequences are commonly encountered after kidney transplantation. Optical coherence tomography (OCT) has been applied to image changes in kidney microanatomy and microcirculation. In this paper, we demonstrate a quantitative method for monitoring kidney status during ischemia-reperfusion process using texture properties of OCT images. This approach employs skewness to measure the distribution of en face OCT image intensities at different depths, thus allowing differentiating ischemia-reperfusion status of kidney. The skewness analysis based on quantitative intensity shows promise for monitoring kidney status during ischemia-reperfusion, and the potential for evaluating the viability of transplant kidney.
Collapse
Affiliation(s)
- Zhifang Li
- College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou, 350007, China
- Fischell Department of Bioengineering, University of Maryland, College Park, MD, 20742
| | - Qinggong Tang
- Fischell Department of Bioengineering, University of Maryland, College Park, MD, 20742
| | - Lili Jin
- Fischell Department of Bioengineering, University of Maryland, College Park, MD, 20742
| | | | - Yu Chen
- College of Photonic and Electronic Engineering, Fujian Normal University, Fuzhou, 350007, China
- Fischell Department of Bioengineering, University of Maryland, College Park, MD, 20742
| |
Collapse
|
50
|
Abdolmanafi A, Duong L, Dahdah N, Cheriet F. Deep feature learning for automatic tissue classification of coronary artery using optical coherence tomography. BIOMEDICAL OPTICS EXPRESS 2017; 8:1203-1220. [PMID: 28271012 PMCID: PMC5330543 DOI: 10.1364/boe.8.001203] [Citation(s) in RCA: 79] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/27/2016] [Revised: 01/20/2017] [Accepted: 01/20/2017] [Indexed: 05/03/2023]
Abstract
Kawasaki disease (KD) is an acute childhood disease complicated by coronary artery aneurysms, intima thickening, thrombi, stenosis, lamellar calcifications, and disappearance of the media border. Automatic classification of the coronary artery layers (intima, media, and scar features) is important for analyzing optical coherence tomography (OCT) images recorded in pediatric patients. OCT has been known as an intracoronary imaging modality using near-infrared light which has recently been used to image the inner coronary artery tissues of pediatric patients, providing high spatial resolution (ranging from 10 to 20 μm). This study aims to develop a robust and fully automated tissue classification method by using the convolutional neural networks (CNNs) as feature extractor and comparing the predictions of three state-of-the-art classifiers, CNN, random forest (RF), and support vector machine (SVM). The results show the robustness of CNN as the feature extractor and random forest as the classifier with classification rate up to 96%, especially to characterize the second layer of coronary arteries (media), which is a very thin layer and it is challenging to be recognized and specified from other tissues.
Collapse
Affiliation(s)
- Atefeh Abdolmanafi
- Dept. of Software and IT Engineering, École de Technologie Supérieure, Montréal,
Canada
| | - Luc Duong
- Dept. of Software and IT Engineering, École de Technologie Supérieure, Montréal,
Canada
| | - Nagib Dahdah
- Div. of Pediatric Cardiology and Research Center, Centre Hospitalier Universitaire Sainte-Justine, Montréal,
Canada
| | - Farida Cheriet
- Dept. of Computer Engineering, École Polytechnique de Montréal, Montréal,
Canada
| |
Collapse
|