Deep feature learning for automatic tissue classification of coronary artery using optical coherence tomography

Artificial Intelligence for the Interventional Cardiologist: Powering and Enabling OCT Image Interpretation

Intravascular optical coherence tomography (IVOCT) is a form of intra-coronary imaging that uses near-infrared light to generate high-resolution, cross-sectional, and 3D volumetric images of the vessel. Given its high spatial resolution, IVOCT is well-placed to characterise coronary plaques and aid with decision-making during percutaneous coronary intervention. IVOCT requires significant interpretation skills, which themselves require extensive education and training for effective utilisation, and this would appear to be the biggest barrier to its widespread adoption. Various artificial intelligence-based tools have been utilised in the most contemporary clinical IVOCT systems to facilitate better human interaction, interpretation and decision-making. The purpose of this article is to review the existing and future technological developments in IVOCT and demonstrate how they could aid the operator.

Rapid automated lumen segmentation of coronary optical coherence tomography images followed by 3D reconstruction of coronary arteries

Purpose

Optical coherence tomography has emerged as an important intracoronary imaging technique for coronary artery disease diagnosis as it produces high-resolution cross-sectional images of luminal and plaque morphology. Precise and fast lumen segmentation is essential for efficient OCT morphometric analysis. However, due to the presence of various image artifacts, including side branches, luminal blood artifacts, and complicated lesions, this remains a challenging task.

Approach

Our research study proposes a rapid automatic segmentation method that utilizes nonuniform rational B-spline to connect limited pixel points and identify the edges of the OCT lumen. The proposed method suppresses image noise and accurately extracts the lumen border with a high correlation to ground truth images based on the area, minimal diameter, and maximal diameter.

Results

We evaluated the method using 3300 OCT frames from 10 patients and found that it achieved favorable results. The average time taken for automatic segmentation by the proposed method is 0.17 s per frame. Additionally, the proposed method includes seamless vessel reconstruction following the lumen segmentation.

Conclusions

The developed automated system provides an accurate, efficient, robust, and user-friendly platform for coronary lumen segmentation and reconstruction, which can pave the way for improved assessment of the coronary artery lumen morphology.

Machine Learning-Assisted Short-Wave InfraRed (SWIR) Techniques for Biomedical Applications: Towards Personalized Medicine

Personalized medicine transforms healthcare by adapting interventions to individuals' unique genetic, molecular, and clinical profiles. To maximize diagnostic and/or therapeutic efficacy, personalized medicine requires advanced imaging devices and sensors for accurate assessment and monitoring of individual patient conditions or responses to therapeutics. In the field of biomedical optics, short-wave infrared (SWIR) techniques offer an array of capabilities that hold promise to significantly enhance diagnostics, imaging, and therapeutic interventions. SWIR techniques provide in vivo information, which was previously inaccessible, by making use of its capacity to penetrate biological tissues with reduced attenuation and enable researchers and clinicians to delve deeper into anatomical structures, physiological processes, and molecular interactions. Combining SWIR techniques with machine learning (ML), which is a powerful tool for analyzing information, holds the potential to provide unprecedented accuracy for disease detection, precision in treatment guidance, and correlations of complex biological features, opening the way for the data-driven personalized medicine field. Despite numerous biomedical demonstrations that utilize cutting-edge SWIR techniques, the clinical potential of this approach has remained significantly underexplored. This paper demonstrates how the synergy between SWIR imaging and ML is reshaping biomedical research and clinical applications. As the paper showcases the growing significance of SWIR imaging techniques that are empowered by ML, it calls for continued collaboration between researchers, engineers, and clinicians to boost the translation of this technology into clinics, ultimately bridging the gap between cutting-edge technology and its potential for personalized medicine.

Symmetry and asymmetry in atherosclerosis

Atherosclerosis remains the main cause of death worldwide. Most important issues concerning atherosclerosis are hemodynamics and how it affects plaque prevalence and distribution, as well as the symmetry and asymmetry of vasculature and plaques. To present the symmetry in the vascular system an analysis of PubMed and MEDLINE databases was performed. As of February 21, 2023, the results were as follows: for "symmetry" AND "atherosclerosis" there were 47 results; for "symmetry" AND "atherosclerotic lesions" - 20 results; for "symmetry" AND "artery stenosis" - 82 results; for "asymmetry" AND "atherosclerosis" - 87 results. Not without meaning are preventive measures. In the light of the Fourth Industrial Revolution artificial intelligence (AI) solutions help to develop new tools outperforming already existing cardiovascular risk scales. The aim of this paper is to present a current view on symmetry within vasculature and atherosclerosis as well as present a new approach to assess individuals' cardiovascular risk in accordance with precision medicine assumptions. Symmetry and asymmetry within the human vascular system play a crucial role in understanding of arterial diseases, including atherosclerosis. Moreover, it is unavoidable to use AI in cardiovascular risk stratification. Int J Occup Med Environ Health. 2023;36(6):693-703.

Automatic lumen and anatomical layers segmentation in IVOCT images using meta learning

Automated analysis of the vessel structure in intravascular optical coherence tomography (IVOCT) images is critical to assess the health status of vessels and monitor coronary artery disease progression. However, deep learning-based methods usually require well-annotated large datasets, which are difficult to obtain in the field of medical image analysis. Hence, an automatic layers segmentation method based on meta-learning was proposed, which can simultaneously extract the surfaces of the lumen, intima, media, and adventitia using a handful of annotated samples. Specifically, we leverage a bi-level gradient strategy to train a meta-learner for capturing the shared meta-knowledge among different anatomical layers and quickly adapting to unknown anatomical layers. Then, a Claw-type network and a contrast consistency loss were designed to better learn the meta-knowledge according to the characteristic of annotation of the lumen and anatomical layers. Experimental results on the two cardiovascular IVOCT datasets show that the proposed method achieved state-of-art performance.

Optical Coherence Tomography in Cerebrovascular Disease: Open up New Horizons

Optical coherence tomography (OCT), based on the backscattering or reflection of near-infrared light, enables an ultra-high resolution of up to 10 μm. The successful application of OCT in coronary artery diseases has sparked increasing interest in its implementation in cerebrovascular diseases. OCT has shown promising potential in the atherosclerotic plaque structure characterization, plaque rupture risk stratification, pre-stenting and post-stenting evaluation, and long-term follow-up in extracranial and intracranial atherosclerotic stenosis (ICAS). In hemorrhagic cerebrovascular diseases, OCT plays an important role in the structure evaluation, rupture risk stratification, and healing and occlusion evaluation following initial treatment in intracranial aneurysms (IAs). In this study, we summarized the applications of OCT in the diagnosis, treatment, and follow-up of cerebrovascular diseases, especially in ICAS and IAs. The current limitations and future directions of OCT in the endovascular treatment of cerebrovascular diseases were also discussed.

Diagnosis of coronary layered plaque by deep learning

Healed coronary plaques, morphologically characterized by a layered phenotype, are signs of previous plaque destabilization and healing. Recent optical coherence tomography (OCT) studies demonstrated that layered plaque is associated with higher levels of local and systemic inflammation and rapid plaque progression. However, the diagnosis of layered plaque needs expertise in OCT image analysis and is susceptible to inter-observer variability. We developed a deep learning (DL) model for an accurate diagnosis of layered plaque. A Visual Transformer (ViT)-based DL model that integrates information from adjacent frames emulating the cardiologists who review consecutive OCT frames to make a diagnosis was developed and compared with the standard convolutional neural network (CNN) model. A total of 237,021 cross-sectional OCT images from 581 patients collected from 8 sites were used for training and internal validation, and 65,394 images from 292 patients collected from another site were used for external validation. In the five-fold cross-validation, the ViT-based model provided better performance (area under the curve [AUC]: 0.860; 95% confidence interval [CI]: 0.855-0.866) than the standard CNN-based model (AUC: 0.799; 95% CI: 0.792-0.805). The ViT-based model (AUC: 0.845; 95% CI: 0.837-0.853) also surpassed the standard CNN-based model (AUC: 0.791; 95% CI: 0.782-0.800) in the external validation. The ViT-based DL model can accurately diagnose a layered plaque, which could help risk stratification for cardiac events.

One-class machine learning classification of skin tissue based on manually scanned optical coherence tomography imaging

We investigated a method for automatic skin tissue characterization based on optical coherence tomography (OCT) imaging. We developed a manually scanned single fiber OCT instrument to perform in vivo skin imaging and tumor boundary assessment. The goal is to achieve more accurate tissue excision in Mohs micrographic surgery (MMS) and reduce the time required for MMS. The focus of this study was to develop a novel machine learning classification method to automatically identify abnormal skin tissues through one-class classification. We trained a deep convolutional neural network (CNN) with a U-Net architecture for automatic skin segmentation, used the pre-trained U-Net as a feature extractor, and trained one-class support vector machine (SVM) classifiers to detect abnormal tissues. The novelty of this study is the use of a neural network as a feature extractor and the use of a one-class SVM for abnormal tissue detection. Our approach eliminated the need to engineer the features for classification and eliminated the need to train the classifier with data obtained from abnormal tissues. To validate the effectiveness of the one-class classification method, we assessed the performance of our algorithm using computer synthesized data, and experimental data. We also performed a pilot study on a patient with skin cancer.

Classification of neck tissues in OCT images by using convolutional neural network

Identification and classification of surrounding neck tissues are very important in thyroid surgery. The advantages of optical coherence tomography (OCT), high resolution, non-invasion, and non-destruction make it have great potential in identifying different neck tissues during thyroidectomy. We studied the automatic classification for neck tissues in OCT images based on convolutional neural network in this paper. OCT images of five kinds of neck tissues were collected firstly by our home-made swept source (SS-OCT) system, and a dataset was built for neural network training. Three image classification neural networks: LeNet, VGGNet, and ResNet, were used to train and test the dataset. The impact of transfer learning on the classification of neck tissue OCT images was also studied. Through the comparison of accuracy, it was found that ResNet has the best classification accuracy among the three networks. In addition, transfer learning did not significantly improve the accuracy, but it can somewhat accelerate the convergence of the network and shorten the network training time.

Rapid lipid-laden plaque identification in intravascular optical coherence tomography imaging based on time-series deep learning

SIGNIFICANCE

Coronary heart disease has the highest rate of death and morbidity in the Western world. Atherosclerosis is an asymptomatic condition that is considered the primary cause of cardiovascular diseases. The accumulation of low-density lipoprotein triggers an inflammatory process in focal areas of arteries, which leads to the formation of plaques. Lipid-laden plaques containing a necrotic core may eventually rupture, causing heart attack and stroke. Lately, intravascular optical coherence tomography (IV-OCT) imaging has been used for plaque assessment. The interpretation of the IV-OCT images is performed visually, which is burdensome and requires highly trained physicians for accurate plaque identification.

AIM

Our study aims to provide high throughput lipid-laden plaque identification that can assist in vivo imaging by offering faster screening and guided decision making during percutaneous coronary interventions.

APPROACH

An A-line-wise classification methodology based on time-series deep learning is presented to fulfill this aim. The classifier was trained and validated with a database consisting of IV-OCT images of 98 artery sections. A trained physician with expertise in the analysis of IV-OCT imaging provided the visual evaluation of the database that was used as ground truth for training and validation.

RESULTS

This method showed an accuracy, sensitivity, and specificity of 89.6%, 83.6%, and 91.1%, respectively. This deep learning methodology has the potential to increase the speed of lipid-laden plaques identification to provide a high throughput of more than 100 B-scans/s.

CONCLUSIONS

These encouraging results suggest that this method will allow for high throughput video-rate atherosclerotic plaque assessment through automated tissue characterization for in vivo imaging by providing faster screening to assist in guided decision making during percutaneous coronary interventions.

Interplay between Artificial Intelligence and Biomechanics Modeling in the Cardiovascular Disease Prediction

Cardiovascular disease (CVD) is the most common cause of morbidity and mortality worldwide, and early accurate diagnosis is the key point for improving and optimizing the prognosis of CVD. Recent progress in artificial intelligence (AI), especially machine learning (ML) technology, makes it possible to predict CVD. In this review, we first briefly introduced the overview development of artificial intelligence. Then we summarized some ML applications in cardiovascular diseases, including ML-based models to directly predict CVD based on risk factors or medical imaging findings and the ML-based hemodynamics with vascular geometries, equations, and methods for indirect assessment of CVD. We also discussed case studies where ML could be used as the surrogate for computational fluid dynamics in data-driven models and physics-driven models. ML models could be a surrogate for computational fluid dynamics, accelerate the process of disease prediction, and reduce manual intervention. Lastly, we briefly summarized the research difficulties and prospected the future development of AI technology in cardiovascular diseases.

Classifying breast cancer in ultrahigh-resolution optical coherence tomography images using convolutional neural networks

Optical coherence tomography (OCT) is being investigated in breast cancer diagnostics as a real-time histology evaluation tool. We present a customized deep convolutional neural network (CNN) for classification of breast tissues in OCT B-scans. Images of human breast samples from mastectomies and breast reductions were acquired using a custom ultrahigh-resolution OCT system with 2.72 µm axial resolution and 5.52 µm lateral resolution. The network achieved 96.7% accuracy, 92% sensitivity, and 99.7% specificity on a dataset of 23 patients. The usage of deep learning will be important for the practical integration of OCT into clinical practice.

Automated Coronary Optical Coherence Tomography Feature Extraction with Application to Three-Dimensional Reconstruction

Coronary optical coherence tomography (OCT) is an intravascular, near-infrared light-based imaging modality capable of reaching axial resolutions of 10–20 µm. This resolution allows for accurate determination of high-risk plaque features, such as thin cap fibroatheroma; however, visualization of morphological features alone still provides unreliable positive predictive capability for plaque progression or future major adverse cardiovascular events (MACE). Biomechanical simulation could assist in this prediction, but this requires extracting morphological features from intravascular imaging to construct accurate three-dimensional (3D) simulations of patients’ arteries. Extracting these features is a laborious process, often carried out manually by trained experts. To address this challenge, numerous techniques have emerged to automate these processes while simultaneously overcoming difficulties associated with OCT imaging, such as its limited penetration depth. This systematic review summarizes advances in automated segmentation techniques from the past five years (2016–2021) with a focus on their application to the 3D reconstruction of vessels and their subsequent simulation. We discuss four categories based on the feature being processed, namely: coronary lumen; artery layers; plaque characteristics and subtypes; and stents. Areas for future innovation are also discussed as well as their potential for future translation.

Current research and future prospects of IVOCT imaging-based detection of the vascular lumen and vulnerable plaque

Intravascular optical coherence tomography (IVOCT) is an imaging method that has developed rapidly in recent years and is useful in coronary atherosclerosis diagnosis. It is widely used in the assessment of vulnerable plaque. This review summarizes the main research methods used in recent years for blood vessel lumen boundary detection and segmentation and vulnerable plaque segmentation and classification. This article aims to comprehensively and systematically introduce the research progress on internal tissues of blood vessels based on IVOCT images. The characteristics and advantages of various methods have been summarized to provide theoretical ideas and methods for the reference of relevant researchers and scholars.

Classification of gastric cancerous tissues by a residual network based on optical coherence tomography images

Optical coherence tomography (OCT) is a noninvasive, radiation-free, and high-resolution imaging technology. The intraoperative classification of normal and cancerous tissue is critical for surgeons to guide surgical operations. Accurate classification of gastric cancerous OCT images is beneficial to improve the effect of surgical treatment based on the deep learning method. The OCT system was used to collect images of cancerous tissues removed from patients. An intelligent classification method of gastric cancerous tissues based on the residual network is proposed in this study and optimized with the ResNet18 model. Four residual blocks are used to reset the model structure of ResNet18 and reduce the number of network layers to identify cancerous tissues. The model performance of different residual networks is evaluated by accuracy, precision, recall, specificity, F1 value, ROC curve, and model parameters. The classification accuracies of the proposed method and ResNet18 both reach 99.90%. Also, the model parameters of the proposed method are 44% of ResNet18, which occupies fewer system resources and is more efficient. In this study, the proposed deep learning method was used to automatically recognize OCT images of gastric cancerous tissue. This artificial intelligence method could help promote the clinical application of gastric cancerous tissue classification in the future.

Artificial intelligence opportunities in cardio-oncology: Overview with spotlight on electrocardiography

Cardiovascular disease is a leading cause of death among cancer survivors, second only to cancer recurrence or development of new tumors. Cardio-oncology has therefore emerged as a relatively new specialty focused on prevention and management of cardiovascular consequences of cancer therapies. Yet challenges remain regarding precision and accuracy with predicting individuals at highest risk for cardiotoxicity. Barriers such as access to care also limit screening and early diagnosis to improve prognosis. Thus, developing innovative approaches for prediction and early detection of cardiovascular illness in this population is critical. In this review, we provide an overview of the present state of machine learning applications in cardio-oncology. We begin by outlining some factors that should be considered while utilizing machine learning algorithms. We then examine research in which machine learning has been applied to improve prediction of cardiac dysfunction in cancer survivors. We also highlight the use of artificial intelligence (AI) in conjunction with electrocardiogram (ECG) to predict cardiac malfunction and also atrial fibrillation (AF), and we discuss the potential role of wearables. Additionally, the article summarizes future prospects and critical takeaways for the application of machine learning in cardio-oncology. This study is the first in a series on artificial intelligence in cardio-oncology, and complements our manuscript on echocardiography and other forms of imaging relevant to cancer survivors cared for in cardiology clinical practice.

Ovarian cancer detection using optical coherence tomography and convolutional neural networks

Ovarian cancer has the sixth-largest fatality rate in the United States among all cancers. A non-surgical assay capable of detecting ovarian cancer with acceptable sensitivity and specificity has yet to be developed. However, such a discovery would profoundly impact the pace of the treatment and improvement to patients’ quality of life. Achieving such a solution requires high-quality imaging, image processing, and machine learning to support an acceptably robust automated diagnosis. In this work, we propose an automated framework that learns to identify ovarian cancer in transgenic mice from optical coherence tomography (OCT) recordings. Classification is accomplished using a neural network that perceives spatially ordered sequences of tomograms. We present three neural network-based approaches, namely a VGG-supported feed-forward network, a 3D convolutional neural network, and a convolutional LSTM (Long Short-Term Memory) network. Our experimental results show that our models achieve a favorable performance with no manual tuning or feature crafting, despite the challenging noise inherent in OCT images. Specifically, our best performing model, the convolutional LSTM-based neural network, achieves a mean AUC (± standard error) of 0.81 ± 0.037. To the best of the authors’ knowledge, no application of machine learning to analyze depth-resolved OCT images of whole ovaries has been documented in the literature. A significant broader impact of this research is the potential transferability of the proposed diagnostic system from transgenic mice to human organs, which would enable medical intervention from early detection of an extremely deadly affliction.

Gastrointestinal Disease Classification in Endoscopic Images Using Attention-Guided Convolutional Neural Networks

Gastrointestinal (GI) diseases constitute a leading problem in the human digestive system. Consequently, several studies have explored automatic classification of GI diseases as a means of minimizing the burden on clinicians and improving patient outcomes, for both diagnostic and treatment purposes. The challenge in using deep learning-based (DL) approaches, specifically a convolutional neural network (CNN), is that spatial information is not fully utilized due to the inherent mechanism of CNNs. This paper proposes the application of spatial factors in improving classification performance. Specifically, we propose a deep CNN-based spatial attention mechanism for the classification of GI diseases, implemented with encoder–decoder layers. To overcome the data imbalance problem, we adapt data-augmentation techniques. A total of 12,147 multi-sited, multi-diseased GI images, drawn from publicly available and private sources, were used to validate the proposed approach. Furthermore, a five-fold cross-validation approach was adopted to minimize inconsistencies in intra- and inter-class variability and to ensure that results were robustly assessed. Our results, compared with other state-of-the-art models in terms of mean accuracy (ResNet50 = 90.28, GoogLeNet = 91.38, DenseNets = 91.60, and baseline = 92.84), demonstrated better outcomes (Precision = 92.8, Recall = 92.7, F1-score = 92.8, and Accuracy = 93.19). We also implemented t-distributed stochastic neighbor embedding (t–SNE) and confusion matrix analysis techniques for better visualization and performance validation. Overall, the results showed that the attention mechanism improved the automatic classification of multi-sited GI disease images. We validated clinical tests based on the proposed method by overcoming previous limitations, with the goal of improving automatic classification accuracy in future work.

Synthetic OCT data in challenging conditions: three-dimensional OCT and presence of abnormalities

Nowadays, retinal optical coherence tomography (OCT) plays an important role in ophthalmology and automatic analysis of the OCT is of real importance: image denoising facilitates a better diagnosis and image segmentation and classification are undeniably critical in treatment evaluation. Synthetic OCT was recently considered to provide a benchmark for quantitative comparison of automatic algorithms and to be utilized in the training stage of novel solutions based on deep learning. Due to complicated data structure in retinal OCTs, a limited number of delineated OCT datasets are already available in presence of abnormalities; furthermore, the intrinsic three-dimensional (3D) structure of OCT is ignored in many public 2D datasets. We propose a new synthetic method, applicable to 3D data and feasible in presence of abnormalities like diabetic macular edema (DME). In this method, a limited number of OCT data is used during the training step and the Active Shape Model is used to produce synthetic OCTs plus delineation of retinal boundaries and location of abnormalities. Statistical comparison of thickness maps showed that synthetic dataset can be used as a statistically acceptable representative of the original dataset (p > 0.05). Visual inspection of the synthesized vessels was also promising. Regarding the texture features of the synthesized datasets, Q-Q plots were used, and even in cases that the points have slightly digressed from the straight line, the p-values of the Kolmogorov–Smirnov test rejected the null hypothesis and showed the same distribution in texture features of the real and the synthetic data. The proposed algorithm provides a unique benchmark for comparison of OCT enhancement methods and a tailored augmentation method to overcome the limited number of OCTs in deep learning algorithms.

Hybrid Transfer Learning for Classification of Uterine Cervix Images for Cervical Cancer Screening

Transfer learning using deep pre-trained convolutional neural networks is increasingly used to solve a large number of problems in the medical field. In spite of being trained using images with entirely different domain, these networks are flexible to adapt to solve a problem in a different domain too. Transfer learning involves fine-tuning a pre-trained network with optimal values of hyperparameters such as learning rate, batch size, and number of training epochs. The process of training the network identifies the relevant features for solving a specific problem. Adapting the pre-trained network to solve a different problem requires fine-tuning until relevant features are obtained. This is facilitated through the use of large number of filters present in the convolutional layers of pre-trained network. A very few features out of these features are useful for solving the problem in a different domain, while others are irrelevant, use of which may only reduce the efficacy of the network. However, by minimizing the number of filters required to solve the problem, the efficiency of the training the network can be improved. In this study, we consider identification of relevant filters using the pre-trained networks namely AlexNet and VGG-16 net to detect cervical cancer from cervix images. This paper presents a novel hybrid transfer learning technique, in which a CNN is built and trained from scratch, with initial weights of only those filters which were identified as relevant using AlexNet and VGG-16 net. This study used 2198 cervix images with 1090 belonging to negative class and 1108 to positive class. Our experiment using hybrid transfer learning achieved an accuracy of 91.46%.

Automatic Classification of A-Lines in Intravascular OCT Images Using Deep Learning and Estimation of Attenuation Coefficients

Intravascular Optical Coherence Tomography (IVOCT) images provide important insight into every aspect of atherosclerosis. Specifically, the extent of plaque and its type, which are indicative of the patient’s condition, are better assessed by OCT images in comparison to other in vivo modalities. A large amount of imaging data per patient require automatic methods for rapid results. An effective step towards automatic plaque detection and plaque characterization is axial lines (A-lines) based classification into normal and various plaque types. In this work, a novel automatic method for A-line classification is proposed. The method employed convolutional neural networks (CNNs) for classification in its core and comprised the following pre-processing steps: arterial wall segmentation and an OCT-specific (depth-resolved) transformation and a post-processing step based on the majority of classifications. The important step was the OCT-specific transformation, which was based on the estimation of the attenuation coefficient in every pixel of the OCT image. The dataset used for training and testing consisted of 183 images from 33 patients. In these images, four different plaque types were delineated. The method was evaluated by cross-validation. The mean values of accuracy, sensitivity and specificity were 74.73%, 87.78%, and 61.45%, respectively, when classifying into plaque and normal A-lines. When plaque A-lines were classified into fibrolipidic and fibrocalcific, the overall accuracy was 83.47% for A-lines of OCT-specific transformed images and 74.94% for A-lines of original images. This large improvement in accuracy indicates the advantage of using attenuation coefficients when characterizing plaque types. The proposed automatic deep-learning pipeline constitutes a positive contribution to the accurate classification of A-lines in intravascular OCT images.

Role of Artificial Intelligence in Video Capsule Endoscopy

Capsule endoscopy (CE) has been increasingly utilised in recent years as a minimally invasive tool to investigate the whole gastrointestinal (GI) tract and a range of capsules are currently available for evaluation of upper GI, small bowel, and lower GI pathology. Although CE is undoubtedly an invaluable test for the investigation of small bowel pathology, it presents considerable challenges and limitations, such as long and laborious reading times, risk of missing lesions, lack of bowel cleansing score and lack of locomotion. Artificial intelligence (AI) seems to be a promising tool that may help improve the performance metrics of CE, and consequently translate to better patient care. In the last decade, significant progress has been made to apply AI in the field of endoscopy, including CE. Although it is certain that AI will find soon its place in day-to-day endoscopy clinical practice, there are still some open questions and barriers limiting its widespread application. In this review, we provide some general information about AI, and outline recent advances in AI and CE, issues around implementation of AI in medical practice and potential future applications of AI-aided CE.

Automated classification of coronary atherosclerotic plaque in optical frequency domain imaging based on deep learning

BACKGROUND AND AIMS

We developed a deep learning (DL) model for automated atherosclerotic plaque categorization using optical frequency domain imaging (OFDI) and performed quantitative and visual evaluations.

METHODS

A total of 1103 histological cross-sections from 45 autopsy hearts were examined to compare the ex vivo OFDI scans. The images were segmented and annotated considering four histological categories: pathological intimal thickening (PIT), fibrous cap atheroma (FA), fibrocalcific plaque (FC), and healed erosion/rupture (HER). The DL model was developed based on pyramid scene parsing network (PSPNet). Given an input image, a convolutional neural network (ResNet50) was used as an encoder to generate feature maps of the last convolutional layer.

RESULTS

For the quantitative evaluation, the mean F-score and IoU values, which are used to evaluate how close the predicted results are to the ground truth, were used. The validation and test dataset had F-score and IoU values of 0.63, 0.49, and 0.66, 0.52, respectively. For the section-level diagnostic accuracy, the areas under the receiver-operating characteristic curve produced by the DL model for FC, PIT, FA, and HER were 0.91, 0.85, 0.86, and 0.86, respectively, and were comparable to those of an expert observer.

CONCLUSIONS

DL semantic segmentation of coronary plaques in OFDI images was used as a tool to automatically categorize atherosclerotic plaques using histological findings as the gold standard. The proposed method can support interventional cardiologists in understanding histological properties of plaques.

A deep learning-based model for characterization of atherosclerotic plaque in coronary arteries using optical coherence tomography images

PURPOSE

Coronary artery events are mainly associated with atherosclerosis in adult population, which is recognized as accumulation of plaques in arterial wall tissues. Optical Coherence Tomography (OCT) is a light-based imaging system used in cardiology to analyze intracoronary tissue layers and pathological formations including plaque accumulation. This state-of-the-art catheter-based imaging system provides intracoronary cross-sectional images with high resolution of 10-15 µm. But interpretation of the acquired images is operator dependent, which is not only very time-consuming but also highly error prone from one observer to another. An automatic and accurate coronary plaque tagging using OCT image post-processing can contribute to wide adoption of the OCT system and reducing the diagnostic error rate.

METHOD

In this study, we propose a combination of spatial pyramid pooling module with dilated convolutions for semantic segmentation to extract atherosclerotic tissues regardless of their types and training a sparse auto-encoder to reconstruct the input features and enlarge the training data as well as plaque type characterization in OCT images.

RESULTS

The results demonstrate high precision of the proposed model with reduced computational complexity, which can be appropriate for real-time analysis of OCT images. At each step of the work, measured accuracy, sensitivity, specificity of more than 93% demonstrate high performance of the model.

CONCLUSION

The main focus of this study is atherosclerotic tissue characterization using OCT imaging. This contributes to wide adoption of the OCT imaging system by providing clinicians with a fully automatic interpretation of various atherosclerotic tissues. Future studies will be focused on analyzing atherosclerotic vulnerable plaques, those coronary plaques which are prone to rupture.

Artificial intelligence and optical coherence tomography for the automatic characterisation of human atherosclerotic plaques

BACKGROUND

Intravascular optical coherence tomography (IVOCT) enables detailed plaque characterisation in vivo, but visual assessment is time-consuming and subjective.

AIMS

This study aimed to develop and validate an automatic framework for IVOCT plaque characterisation using artificial intelligence (AI).

METHODS

IVOCT pullbacks from five international centres were analysed in a core lab, annotating basic plaque components, inflammatory markers and other structures. A deep convolutional network with encoding-decoding architecture and pseudo-3D input was developed and trained using hybrid loss. The proposed network was integrated into commercial software to be externally validated on additional IVOCT pullbacks from three international core labs, taking the consensus among core labs as reference.

RESULTS

Annotated images from 509 pullbacks (391 patients) were divided into 10,517 and 1,156 cross-sections for the training and testing data sets, respectively. The Dice coefficient of the model was 0.906 for fibrous plaque, 0.848 for calcium and 0.772 for lipid in the testing data set. Excellent agreement in plaque burden quantification was observed between the model and manual measurements (R2=0.98). In the external validation, the software correctly identified 518 out of 598 plaque regions from 300 IVOCT cross-sections, with a diagnostic accuracy of 97.6% (95% CI: 93.4-99.3%) in fibrous plaque, 90.5% (95% CI: 85.2-94.1%) in lipid and 88.5% (95% CI: 82.4-92.7%) in calcium. The median time required for analysis was 21.4 (18.6-25.0) seconds per pullback.

CONCLUSIONS

A novel AI framework for automatic plaque characterisation in IVOCT was developed, providing excellent diagnostic accuracy in both internal and external validation. This model might reduce subjectivity in image interpretation and facilitate IVOCT quantification of plaque composition, with potential applications in research and IVOCT-guided PCI.

Automatic classification of esophageal disease in gastroscopic images using an efficient channel attention deep dense convolutional neural network

The accurate diagnosis of various esophageal diseases at different stages is crucial for providing precision therapy planning and improving 5-year survival rate of esophageal cancer patients. Automatic classification of various esophageal diseases in gastroscopic images can assist doctors to improve the diagnosis efficiency and accuracy. The existing deep learning-based classification method can only classify very few categories of esophageal diseases at the same time. Hence, we proposed a novel efficient channel attention deep dense convolutional neural network (ECA-DDCNN), which can classify the esophageal gastroscopic images into four main categories including normal esophagus (NE), precancerous esophageal diseases (PEDs), early esophageal cancer (EEC) and advanced esophageal cancer (AEC), covering six common sub-categories of esophageal diseases and one normal esophagus (seven sub-categories). In total, 20,965 gastroscopic images were collected from 4,077 patients and used to train and test our proposed method. Extensive experiments results have demonstrated convincingly that our proposed ECA-DDCNN outperforms the other state-of-art methods. The classification accuracy (Acc) of our method is 90.63% and the averaged area under curve (AUC) is 0.9877. Compared with other state-of-art methods, our method shows better performance in the classification of various esophageal disease. Particularly for these esophageal diseases with similar mucosal features, our method also achieves higher true positive (TP) rates. In conclusion, our proposed classification method has confirmed its potential ability in a wide variety of esophageal disease diagnosis.

Coronary artery segmentation from intravascular optical coherence tomography using deep capsules

The segmentation and analysis of coronary arteries from intravascular optical coherence tomography (IVOCT) is an important aspect of diagnosing and managing coronary artery disease. Current image processing methods are hindered by the time needed to generate expert-labelled datasets and the potential for bias during the analysis. Therefore, automated, robust, unbiased and timely geometry extraction from IVOCT, using image processing, would be beneficial to clinicians. With clinical application in mind, we aim to develop a model with a small memory footprint that is fast at inference time without sacrificing segmentation quality. Using a large IVOCT dataset of 12,011 expert-labelled images from 22 patients, we construct a new deep learning method based on capsules which automatically produces lumen segmentations. Our dataset contains images with both blood and light artefacts (22.8 %), as well as metallic (23.1 %) and bioresorbable stents (2.5 %). We split the dataset into a training (70 %), validation (20 %) and test (10 %) set and rigorously investigate design variations with respect to upsampling regimes and input selection. We show that our developments lead to a model, DeepCap, that is on par with state-of-the-art machine learning methods in terms of segmentation quality and robustness, while using as little as 12 % of the parameters. This enables DeepCap to have per image inference times up to 70 % faster on GPU and up to 95 % faster on CPU compared to other state-of-the-art models. DeepCap is a robust automated segmentation tool that can aid clinicians to extract unbiased geometrical data from IVOCT.

Application of machine learning in understanding atherosclerosis: Emerging insights

Biological processes are incredibly complex—integrating molecular signaling networks involved in multicellular communication and function, thus maintaining homeostasis. Dysfunction of these processes can result in the disruption of homeostasis, leading to the development of several disease processes including atherosclerosis. We have significantly advanced our understanding of bioprocesses in atherosclerosis, and in doing so, we are beginning to appreciate the complexities, intricacies, and heterogeneity atherosclerosi. We are also now better equipped to acquire, store, and process the vast amount of biological data needed to shed light on the biological circuitry involved. Such data can be analyzed within machine learning frameworks to better tease out such complex relationships. Indeed, there has been an increasing number of studies applying machine learning methods for patient risk stratification based on comorbidities, multi-modality image processing, and biomarker discovery pertaining to atherosclerotic plaque formation. Here, we focus on current applications of machine learning to provide insight into atherosclerotic plaque formation and better understand atherosclerotic plaque progression in patients with cardiovascular disease.

Learning With Fewer Images via Image Clustering: Application to Intravascular OCT Image Segmentation

Deep learning based methods are routinely used to segment various structures of interest in varied medical imaging modalities. Acquiring annotations for a large number of images requires a skilled analyst, and the process is both time consuming and challenging. Our approach to reduce effort is to reduce the number of images needing detailed annotation. For intravascular optical coherence tomography (IVOCT) image pullbacks, we tested 10% to 100% of training images derived from two schemes: equally-spaced image subsampling and deep-learning- based image clustering. The first strategy involves selecting images at equally spaced intervals from the volume, accounting for the high spatial correlation between neighboring images. In clustering, we used an autoencoder to create a deep feature space representation, performed k-medoids clustering, and then used the cluster medians for training. For coronary calcifications, a baseline U-net model was trained on all images from volumes of interest (VOIs) and compared with fewer images from the sub-sampling strategies. For a given sampling ratio, the clustering based method performed better or similar as compared to the equally spaced sampling approach (e.g., with 10% of images, mean F1 score for calcific class increased from 0.52 to 0.63, with equal spacing and with clustering, respectively). Additionally, for a fixed number of training images, sampling images from more VOIs performed better than otherwise. In conclusion, we recommend the clustering based approach to annotate a small fraction of images, creating a baseline model, which potentially can be improved further by annotating images selected using methods described in active learning research.

Artificial intelligence in cardiovascular medicine

PURPOSE OF REVIEW

Artificial intelligence is a broad set of sophisticated computer-based statistical tools that have become widely available. Cardiovascular medicine with its large data repositories, need for operational efficiency and growing focus on precision care is set to be transformed by artificial intelligence. Applications range from new pathophysiologic discoveries to decision support for individual patient care to optimization of system-wide logistical processes.

RECENT FINDINGS

Machine learning is the dominant form of artificial intelligence wherein complex statistical algorithms 'learn' by deducing patterns in datasets. Supervised machine learning uses classified large data to train an algorithm to accurately predict the outcome, whereas in unsupervised machine learning, the algorithm uncovers mathematical relationships within unclassified data. Artificial multilayered neural networks or deep learning is one of the most successful tools. Artificial intelligence has demonstrated superior efficacy in disease phenomapping, early warning systems, risk prediction, automated processing and interpretation of imaging, and increasing operational efficiency.

SUMMARY

Artificial intelligence demonstrates the ability to learn through assimilation of large datasets to unravel complex relationships, discover prior unfound pathophysiological states and develop predictive models. Artificial intelligence needs widespread exploration and adoption for large-scale implementation in cardiovascular practice.

Segmentation of Coronary Calcified Plaque in Intravascular OCT Images Using a Two-Step Deep Learning Approach

We developed a fully automated, two-step deep learning approach for characterizing coronary calcified plaque in intravascular optical coherence tomography (IVOCT) images. First, major calcification lesions were detected from an entire pullback using a 3D convolutional neural network (CNN). Second, a SegNet deep learning model with the Tversky loss function was used to segment calcified plaques in the major calcification lesions. The fully connected conditional random field and the frame interpolation of the missing calcification frames were used to reduce classification errors. We trained/tested the networks on a large dataset comprising 8,231 clinical images from 68 patients with 68 vessels and 4,320 ex vivo cadaveric images from 4 hearts with 4 vessels. The 3D CNN model detected major calcifications with high sensitivity (97.7%), specificity (87.7%), and F1 score (0.922). Compared to the standard one-step approach, our two-step deep learning approach significantly improved sensitivity (from 77.5% to 86.2%), precision (from 73.5% to 75.8%), and F1 score (from 0.749 to 0.781). We investigated segmentation performance for varying numbers of training samples; at least 3,900 images were required to obtain stable segmentation results. We also found very small differences in calcification attributes (e.g., angle, thickness, and depth) and identical calcium scores on repetitive pullbacks, indicating excellent reproducibility. Applied to new clinical pullbacks, our method has implications for real-time treatment planning and imaging research.

Artificial intelligence in cardiology

This review examines the current state and application of artificial intelligence (AI) and machine learning (ML) in cardiovascular medicine. AI is changing the clinical practice of medicine in other specialties. With progress continuing in this emerging technology, the impact for cardiovascular medicine is highlighted to provide insight for the practicing clinician and to identify potential patient benefits.

Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging

Artificial intelligence (AI) has transformed key aspects of human life. Machine learning (ML), which is a subset of AI wherein machines autonomously acquire information by extracting patterns from large databases, has been increasingly used within the medical community, and specifically within the domain of cardiovascular diseases. In this review, we present a brief overview of ML methodologies that are used for the construction of inferential and predictive data-driven models. We highlight several domains of ML application such as echocardiography, electrocardiography, and recently developed non-invasive imaging modalities such as coronary artery calcium scoring and coronary computed tomography angiography. We conclude by reviewing the limitations associated with contemporary application of ML algorithms within the cardiovascular disease field.

Automated classification of coronary plaque calcification in OCT pullbacks with 3D deep neural networks

SIGNIFICANCE

Detection and characterization of coronary atherosclerotic plaques often need reviews of a large number of optical coherence tomography (OCT) imaging slices to make a clinical decision. However, it is a challenge to manually review all the slices and consider the interrelationship between adjacent slices.

APPROACH

Inspired by the recent success of deep convolutional network on the classification of medical images, we proposed a ResNet-3D network for classification of coronary plaque calcification in OCT pullbacks. The ResNet-3D network was initialized with a trained ResNet-50 network and a three-dimensional convolution filter filled with zeros padding and non-zeros padding with a convolutional filter. To retrain ResNet-50, we used a dataset of ∼4860 OCT images, derived by 18 entire pullbacks from different patients. In addition, we investigated a two-phase training method to address the data imbalance. For an improved performance, we evaluated different input sizes for the ResNet-3D network, such as 3, 5, and 7 OCT slices. Furthermore, we integrated all ResNet-3D results by majority voting.

RESULTS

A comparative analysis proved the effectiveness of the proposed ResNet-3D networks against ResNet-2D network in the OCT dataset. The classification performance (F1-scores  =  94  %   for non-zeros padding and F1-score  =  96  %   for zeros padding) demonstrated the potential of convolutional neural networks (CNNs) in classifying plaque calcification.

CONCLUSIONS

This work may provide a foundation for further work in extending the CNN to voxel segmentation, which may lead to a supportive diagnostic tool for assessment of coronary plaque vulnerability.

Optical coherence tomography for thyroid pathology: 3D analysis of tissue microstructure

To investigate the potential of optical coherence tomography (OCT) to distinguish between normal and pathologic thyroid tissue, 3D OCT images were acquired on ex vivo thyroid samples from adult subjects (n=22) diagnosed with a variety of pathologies. The follicular structure was analyzed in terms of count, size, density and sphericity. Results showed that OCT images highly agreed with the corresponding histopatology and the calculated parameters were representative of the follicular structure variation. The analysis of OCT volumes provides quantitative information that could make automatic classification possible. Thus, OCT can be beneficial for intraoperative surgical guidance or in the pathology assessment routine.

Atherosclerotic Plaque Tissue Characterization: An OCT-Based Machine Learning Algorithm With ex vivo Validation

There is a need to develop a validated algorithm for plaque characterization which can help to facilitate the standardization of optical coherence tomography (OCT) image interpretation of plaque morphology, and improve the efficiency and accuracy in the application of OCT imaging for the quantitative assessment of plaque vulnerability. In this study, a machine learning algorithm was implemented for characterization of atherosclerotic plaque components by intravascular OCT using ex vivo carotid plaque tissue samples. A total of 31 patients underwent carotid endarterectomy and the ex vivo carotid plaques were imaged with OCT. Optical parameter, texture features and relative position of pixels were extracted within the region of interest and then used to quantify the tissue characterization of plaque components. The potential of individual and combined feature set to discriminate tissue components was quantified using sensitivity, specificity, accuracy. The results show there was a lower classification accuracy in the calcified tissue than the fibrous tissue and lipid tissue. The pixel-wise classification accuracy obtained by the developed method, to characterize the fibrous, calcified and lipid tissue by comparing with histology, were 80.0, 62.0, and 83.1, respectively. The developed algorithm was capable of characterizing plaque components with an excellent accuracy using the combined feature set.

A primer on artificial intelligence for the paediatric cardiologist

The combination of pediatric cardiology being both a perceptual and a cognitive subspecialty demands a complex decision-making model which makes artificial intelligence a particularly attractive technology with great potential. The prototypical artificial intelligence system would autonomously impute patient data into a collaborative database that stores, syncs, interprets and ultimately classifies the patient's profile to specific disease phenotypes to compare against a large aggregate of shared peer health data and outcomes, the current medical body of literature and ongoing trials to offer morbidity and mortality prediction, drug therapy options targeted to each patient's genetic profile, tailored surgical plans and recommendations for timing of sequential imaging. The focus of this review paper is to offer a primer on artificial intelligence and paediatric cardiology by briefly discussing the history of artificial intelligence in medicine, modern and future applications in adult and paediatric cardiology across selected concentrations, and current barriers to implementation of these technologies.

A digital health industry cohort across the health continuum

The digital health industry has grown rapidly in the past decade. There will be few future aspects of healthcare untouched by digital health. Thus, the current status of the industry, the implications of companies' directions and clinical focus, and their external funding are increasingly relevant to healthcare policy, regulation, research, and all healthcare stakeholders. Yet, little is known about the degree to which the digital health industry has focused on the key domains in the health continuum, including prevention, detection, and management. We performed a cross-sectional study of a US digital health industry cohort that received publicly disclosed funding from 2011-2018. We assessed the number of companies; respective funding within each part of the health continuum; and products and services by technology type, clinical indication, purchasers, and end users. In this emerging industry, most companies focused on management of disease and the minority on prevention or detection. This asymmetry, which is similar to the traditional healthcare system, represents an opportunity to focus on earlier parts of the health continuum. Patients were a common purchaser of all products, but especially prevention-focused digital health products, implying a large unmet need not yet served by the traditional healthcare system.

Convolutional Neural Networks for Automatic Risser Stage Assessment

PURPOSE

To develop an automatic method for the assessment of the Risser stage using deep learning that could be used in the management panel of adolescent idiopathic scoliosis (AIS).

MATERIALS AND METHODS

In this institutional review board approved-study, a total of 1830 posteroanterior radiographs of patients with AIS (age range, 10-18 years, 70% female) were collected retrospectively and graded manually by six trained readers using the United States Risser staging system. Each radiograph was preprocessed and cropped to include the entire pelvic region. A convolutional neural network was trained to automatically grade conventional radiographs according to the Risser classification. The network was then validated by comparing its accuracy against the interobserver variability of six trained graders from the authors' institution using the Fleiss κ statistical measure.

RESULTS

Overall agreement between the six observers was fair, with a κ coefficient of 0.65 for the experienced graders and agreement of 74.5%. The automatic grading method obtained a κ coefficient of 0.72, which is a substantial agreement with the ground truth, and an overall accuracy of 78.0%.

CONCLUSION

The high accuracy of the model presented here compared with human readers suggests that this work may provide a new method for standardization of Risser grading. The model could assist physicians with the task, as well as provide additional insights in the assessment of bone maturity based on radiographs.© RSNA, 2020.

Fully Automated Postlumpectomy Breast Margin Assessment Utilizing Convolutional Neural Network Based Optical Coherence Tomography Image Classification Method

BACKGROUND

The purpose of this study was to develop a deep learning classification approach to distinguish cancerous from noncancerous regions within optical coherence tomography (OCT) images of breast tissue for potential use in an intraoperative setting for margin assessment.

METHODS

A custom ultrahigh-resolution OCT (UHR-OCT) system with an axial resolution of 2.7 μm and a lateral resolution of 5.5 μm was used in this study. The algorithm used an A-scan-based classification scheme and the convolutional neural network (CNN) was implemented using an 11-layer architecture consisting of serial 3 × 3 convolution kernels. Four tissue types were classified, including adipose, stroma, ductal carcinoma in situ, and invasive ductal carcinoma.

RESULTS

The binary classification of cancer versus noncancer with the proposed CNN achieved 94% accuracy, 96% sensitivity, and 92% specificity. The mean five-fold validation F1 score was highest for invasive ductal carcinoma (mean standard deviation, 0.89 ± 0.09) and adipose (0.79 ± 0.17), followed by stroma (0.74 ± 0.18), and ductal carcinoma in situ (0.65 ± 0.15).

CONCLUSION

It is feasible to use CNN based algorithm to accurately distinguish cancerous regions in OCT images. This fully automated method can overcome limitations of manual interpretation including interobserver variability and speed of interpretation and may enable real-time intraoperative margin assessment.

Fully automated plaque characterization in intravascular OCT images using hybrid convolutional and lumen morphology features

For intravascular OCT (IVOCT) images, we developed an automated atherosclerotic plaque characterization method that used a hybrid learning approach, which combined deep-learning convolutional and hand-crafted, lumen morphological features. Processing was done on innate A-line units with labels fibrolipidic (fibrous tissue followed by lipidous tissue), fibrocalcific (fibrous tissue followed by calcification), or other. We trained/tested on an expansive data set (6,556 images), and performed an active learning, relabeling step to improve noisy ground truth labels. Conditional random field was an important post-processing step to reduce classification errors. Sensitivities/specificities were 84.8%/97.8% and 91.4%/95.7% for fibrolipidic and fibrocalcific plaques, respectively. Over lesions, en face classification maps showed automated results that agreed favorably to manually labeled counterparts. Adding lumen morphological features gave statistically significant improvement (p < 0.05), as compared to classification with convolutional features alone. Automated assessments of clinically relevant plaque attributes (arc angle and length), compared favorably to those from manual labels. Our hybrid approach gave statistically improved results as compared to previous A-line classification methods using deep learning or hand-crafted features alone. This plaque characterization approach is fully automated, robust, and promising for live-time treatment planning and research applications.

Real-time colorectal cancer diagnosis using PR-OCT with deep learning

Prior reports have shown optical coherence tomography (OCT) can differentiate normal colonic mucosa from neoplasia, potentially offering an alternative technique to endoscopic biopsy - the current gold-standard colorectal cancer screening and surveillance modality. To help clinical translation limited by processing the large volume of generated data, we designed a deep learning-based pattern recognition (PR) OCT system that automates image processing and provides accurate diagnosis potentially in real-time. Method: OCT is an emerging imaging technique to obtain 3-dimensional (3D) "optical biopsies" of biological samples with high resolution. We designed a convolutional neural network to capture the structure patterns in human colon OCT images. The network is trained and tested using around 26,000 OCT images acquired from 20 tumor areas, 16 benign areas, and 6 other abnormal areas. Results: The trained network successfully detected patterns that identify normal and neoplastic colorectal tissue. Experimental diagnoses predicted by the PR-OCT system were compared to the known histologic findings and quantitatively evaluated. A sensitivity of 100% and specificity of 99.7% can be reached. Further, the area under the receiver operating characteristic (ROC) curves (AUC) of 0.998 is achieved. Conclusions: Our results demonstrate that PR-OCT can be used to give an accurate real-time computer-aided diagnosis of colonic neoplastic mucosa. Future development of this system as an "optical biopsy" tool to assist doctors in real-time for early mucosal neoplasms screening and treatment evaluation following initial oncologic therapy is planned.

Automatic A-line coronary plaque classification using combined deep learning and textural features in intravascular OCT images

We developed a fully automated method for classifying A-line coronary plaques in intravascular optical coherence tomography images using combined deep learning and textural features. The proposed method was trained on 4,292 images from 48 pullbacks giving 80 manually labeled, volumes of interest. Preprocessing steps including guidewire/shadow removal, lumen boundary detection, pixel shifting, and noise reduction were employed. We built a convolutional neural network to extract the deep learning features from the preprocessed image. Traditional textural features were also extracted and combined with deep learning features. Feature selection was performed using the minimum redundancy maximum relevance method. Combined features were utilized as inputs for a random forest classifier. After classification, conditional random field (CRF) method was used for classification noise cleaning. We determined a sub-feature set with the most predictive power. With CRF noise cleaning, sensitivities/specificities were 82.2%/90.8% and 82.4%/89.2% for fibrolipidic and fibrocalcific classes, respectively, with good Dice coefficients. The classification noise cleaning step improved performance metrics by nearly 10-15%. The predicted en face classification maps of entire pullbacks agreed favorably to the manually labeled counterparts. Both assessments suggested that our automated measurements gave clinically relevant results. The proposed method is very promising with regards to both clinical treatment planning and research applications.

An automatic diagnostic system of coronary artery lesions in Kawasaki disease using intravascular optical coherence tomography imaging

Intravascular optical coherence tomography (IV-OCT) is a light-based imaging modality with high resolution, which employs near-infrared light to provide tomographic intracoronary images. Morbidity caused by coronary heart disease is a substantial cause of acute coronary syndrome and sudden cardiac death. The most common intracoronay complications caused by coronary artery disease are intimal hyperplasia, calcification, fibrosis, neovascularization and macrophage accumulation, which require efficient prevention strategies. OCT can provide discriminative information of the intracoronary tissues, which can be used to train a robust fully automatic tissue characterization model based on deep learning. In this study, we aimed to design a diagnostic model of coronary artery lesions. Particularly, we trained a random forest using convolutional neural network features to distinguish between normal and diseased arterial wall structure. Then, based on the arterial wall structure, fully convolutional network is designed to extract the tissue layers in normal cases, and pathological tissues regardless of lesion type in pathological cases. Then, the type of the lesions can be characterized with high precision using our previous model. The results demonstrate the robustness of the model with the approximate overall accuracy up to 90%.

Automated plaque characterization using deep learning on coronary intravascular optical coherence tomographic images

Accurate identification of coronary plaque is very important for cardiologists when treating patients with advanced atherosclerosis. We developed fully-automated semantic segmentation of plaque in intravascular OCT images. We trained/tested a deep learning model on a folded, large, manually annotated clinical dataset. The sensitivities/specificities were 87.4%/89.5% and 85.1%/94.2% for pixel-wise classification of lipidous and calcified plaque, respectively. Automated clinical lesion metrics, potentially useful for treatment planning and research, compared favorably (<4%) with those derived from ground-truth labels. When we converted the results to A-line classification, they were significantly better (p < 0.05) than those obtained previously by using deep learning classifications of A-lines.