High signal-to-noise ratio reconstruction of low bit-depth optical coherence tomography using deep learning

Artificial neural network for enhancing signal-to-noise ratio and contrast in photothermal optical coherence tomography

Optical coherence tomography (OCT) is a medical imaging method that generates micron-resolution 3D volumetric images of tissues in-vivo. Photothermal (PT)-OCT is a functional extension of OCT with the potential to provide depth-resolved molecular information complementary to the OCT structural images. PT-OCT typically requires long acquisition times to measure small fluctuations in the OCT phase signal. Here, we use machine learning with a neural network to infer the amplitude of the photothermal phase modulation from a short signal trace, trained in a supervised fashion with the ground truth signal obtained by conventional reconstruction of the PT-OCT signal from a longer acquisition trace. Results from phantom and tissue studies show that the developed network improves signal to noise ratio (SNR) and contrast, enabling PT-OCT imaging with short acquisition times and without any hardware modification to the PT-OCT system. The developed network removes one of the key barriers in translation of PT-OCT (i.e., long acquisition time) to the clinic.

Adaptive convolutional dictionary learning for denoising seismocardiogram to enhance the classification performance of aortic stenosis

BACKGROUND

Aortic stenosis (AS) is the most prevalent type of valvular heart disease (VHD), traditionally diagnosed using echocardiogram or phonocardiogram. Seismocardiogram (SCG), an emerging wearable cardiac monitoring modality, is proved to be feasible in non-invasive and cost-effective AS diagnosis. However, SCG waveforms acquired from patients with heart diseases are typically weak, making them more susceptible to noise contamination. While most related researches focus on motion artifacts, sensor noise and quantization noise have been mostly overlooked. These noises pose additional challenges for extracting features from the SCG, especially impeding accurate AS classification.

METHOD

To address this challenge, we present a convolutional dictionary learning-based method. Based on sparse modeling of SCG, the proposed method generates a personalized adaptive-size dictionary from noisy measurements. The dictionary is used for sparse coding of the noisy SCG into a transform domain. Reconstruction from the domain removes the noise while preserving the individual waveform pattern of SCG.

RESULTS

Using two self-collected SCG datasets, we established optimal dictionary learning parameters and validated the denoising performance. Subsequently, the proposed method denoised SCG from 50 subjects (25 AS and 25 non-AS). Leave-one-subject-out cross-validation (LOOCV) was applied to 5 machine learning classifiers. Among the classifiers, a bi-layer neural network achieved a moderate accuracy of 90.2%, with an improvement of 13.8% from the denoising.

CONCLUSIONS

The proposed sparsity-based denoising technique effectively removes stochastic sensor noise and quantization noise from SCG, consequently improving AS classification performance. This approach shows promise for overcoming instrumentation constraints of SCG-based diagnosis.

StruNet: Perceptual and low-rank regularized transformer for medical image denoising

BACKGROUND

Various types of noise artifacts inevitably exist in some medical imaging modalities due to limitations of imaging techniques, which impair either clinical diagnosis or subsequent analysis. Recently, deep learning approaches have been rapidly developed and applied on medical images for noise removal or image quality enhancement. Nevertheless, due to complexity and diversity of noise distribution representations in different medical imaging modalities, most of the existing deep learning frameworks are incapable to flexibly remove noise artifacts while retaining detailed information. As a result, it remains challenging to design an effective and unified medical image denoising method that will work across a variety of noise artifacts for different imaging modalities without requiring specialized knowledge in performing the task.

PURPOSE

In this paper, we propose a novel encoder-decoder architecture called Swin transformer-based residual u-shape Network (StruNet), for medical image denoising.

METHODS

Our StruNet adopts a well-designed block as the backbone of the encoder-decoder architecture, which integrates Swin Transformer modules with residual block in parallel connection. Swin Transformer modules could effectively learn hierarchical representations of noise artifacts via self-attention mechanism in non-overlapping shifted windows and cross-window connection, while residual block is advantageous to compensate loss of detailed information via shortcut connection. Furthermore, perceptual loss and low-rank regularization are incorporated into loss function respectively in order to constrain the denoising results on feature-level consistency and low-rank characteristics.

RESULTS

To evaluate the performance of the proposed method, we have conducted experiments on three medical imaging modalities including computed tomography (CT), optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA).

CONCLUSIONS

The results demonstrate that the proposed architecture yields a promising performance of suppressing multiform noise artifacts existing in different imaging modalities.

Frequency-aware optical coherence tomography image super-resolution via conditional generative adversarial neural network

Optical coherence tomography (OCT) has stimulated a wide range of medical image-based diagnosis and treatment in fields such as cardiology and ophthalmology. Such applications can be further facilitated by deep learning-based super-resolution technology, which improves the capability of resolving morphological structures. However, existing deep learning-based method only focuses on spatial distribution and disregards frequency fidelity in image reconstruction, leading to a frequency bias. To overcome this limitation, we propose a frequency-aware super-resolution framework that integrates three critical frequency-based modules (i.e., frequency transformation, frequency skip connection, and frequency alignment) and frequency-based loss function into a conditional generative adversarial network (cGAN). We conducted a large-scale quantitative study from an existing coronary OCT dataset to demonstrate the superiority of our proposed framework over existing deep learning frameworks. In addition, we confirmed the generalizability of our framework by applying it to fish corneal images and rat retinal images, demonstrating its capability to super-resolve morphological details in eye imaging.

Fast Component Tree Computation for Images of Limited Levels

Component trees have many applications. We introduce a new component tree computation algorithm, applicable to 4-/8-connectivity and 6-connectivity. The algorithm consists of two steps: building level line trees using an optimized top-down algorithm, and computing components from level lines by a novel line-by-line approach. As compared with traditional component computation algorithms, the new algorithm is fast for images of limited levels. It represents components by level lines, offering boundary information which traditional algorithms do not provide.

Multi-Scale Reconstruction of Undersampled Spectral-Spatial OCT Data for Coronary Imaging Using Deep Learning

Coronary artery disease (CAD) is a cardiovascular condition with high morbidity and mortality. Intravascular optical coherence tomography (IVOCT) has been considered as an optimal imagining system for the diagnosis and treatment of CAD. Constrained by Nyquist theorem, dense sampling in IVOCT attains high resolving power to delineate cellular structures/features. There is a trade-off between high spatial resolution and fast scanning rate for coronary imaging. In this paper, we propose a viable spectral-spatial acquisition method that down-scales the sampling process in both spectral and spatial domain while maintaining high quality in image reconstruction. The down-scaling schedule boosts data acquisition speed without any hardware modifications. Additionally, we propose a unified multi-scale reconstruction framework, namely Multiscale-Spectral-Spatial-Magnification Network (MSSMN), to resolve highly down-scaled (compressed) OCT images with flexible magnification factors. We incorporate the proposed methods into Spectral Domain OCT (SD-OCT) imaging of human coronary samples with clinical features such as stent and calcified lesions. Our experimental results demonstrate that spectral-spatial down-scaled data can be better reconstructed than data that are down-scaled solely in either spectral or spatial domain. Moreover, we observe better reconstruction performance using MSSMN than using existing reconstruction methods. Our acquisition method and multi-scale reconstruction framework, in combination, may allow faster SD-OCT inspection with high resolution during coronary intervention.

Choroidal layer segmentation in OCT images by a boundary enhancement network

Morphological changes of the choroid have been proved to be associated with the occurrence and pathological mechanism of many ophthalmic diseases. Optical Coherence Tomography (OCT) is a non-invasive technique for imaging of ocular biological tissues, that can reveal the structure of the retinal and choroidal layers in micron-scale resolution. However, unlike the retinal layer, the interface between the choroidal layer and the sclera is ambiguous in OCT, which makes it difficult for ophthalmologists to identify with certainty. In this paper, we propose a novel boundary-enhanced encoder-decoder architecture for choroid segmentation in retinal OCT images, in which a Boundary Enhancement Module (BEM) forms the backbone of each encoder-decoder layer. The BEM consists of three parallel branches: 1) a Feature Extraction Branch (FEB) to obtain feature maps with different receptive fields; 2) a Channel Enhancement Branch (CEB) to extract the boundary information of different channels; and 3) a Boundary Activation Branch (BAB) to enhance the boundary information via a novel activation function. In addition, in order to incorporate expert knowledge into the segmentation network, soft key point maps are generated on the choroidal boundary, and are combined with the predicted images to facilitate precise choroidal boundary segmentation. In order to validate the effectiveness and superiority of the proposed method, both qualitative and quantitative evaluations are employed on three retinal OCT datasets for choroid segmentation. The experimental results demonstrate that the proposed method yields better choroid segmentation performance than other deep learning approaches. Moreover, both 2D and 3D features are extracted for statistical analysis from normal and highly myopic subjects based on the choroid segmentation results, which is helpful in revealing the pathology of high myopia. Code is available at https://github.com/iMED-Lab/Choroid-segmentation.

A review of generative adversarial network applications in optical coherence tomography image analysis

Optical coherence tomography (OCT) has revolutionized ophthalmic clinical practice and research, as a result of the high-resolution images that the method is able to capture in a fast, non-invasive manner. Although clinicians can interpret OCT images qualitatively, the ability to quantitatively and automatically analyse these images represents a key goal for eye care by providing clinicians with immediate and relevant metrics to inform best clinical practice. The range of applications and methods to analyse OCT images is rich and rapidly expanding. With the advent of deep learning methods, the field has experienced significant progress with state-of-the-art-performance for several OCT image analysis tasks. Generative adversarial networks (GANs) represent a subfield of deep learning that allows for a range of novel applications not possible in most other deep learning methods, with the potential to provide more accurate and robust analyses. In this review, the progress in this field and clinical impact are reviewed and the potential future development of applications of GANs to OCT image processing are discussed.

Novel technical and privacy-preserving technology for artificial intelligence in ophthalmology

PURPOSE OF REVIEW

The application of artificial intelligence (AI) in medicine and ophthalmology has experienced exponential breakthroughs in recent years in diagnosis, prognosis, and aiding clinical decision-making. The use of digital data has also heralded the need for privacy-preserving technology to protect patient confidentiality and to guard against threats such as adversarial attacks. Hence, this review aims to outline novel AI-based systems for ophthalmology use, privacy-preserving measures, potential challenges, and future directions of each.

RECENT FINDINGS

Several key AI algorithms used to improve disease detection and outcomes include: Data-driven, imagedriven, natural language processing (NLP)-driven, genomics-driven, and multimodality algorithms. However, deep learning systems are susceptible to adversarial attacks, and use of data for training models is associated with privacy concerns. Several data protection methods address these concerns in the form of blockchain technology, federated learning, and generative adversarial networks.

SUMMARY

AI-applications have vast potential to meet many eyecare needs, consequently reducing burden on scarce healthcare resources. A pertinent challenge would be to maintain data privacy and confidentiality while supporting AI endeavors, where data protection methods would need to rapidly evolve with AI technology needs. Ultimately, for AI to succeed in medicine and ophthalmology, a balance would need to be found between innovation and privacy.

The study on the inverse problem of applied current thermoacoustic imaging based on generative adversarial network

Applied Current Thermoacoustic Imaging (ACTAI) is a new imaging method which combines electromagnetic excitation with ultrasound imaging, and takes ultrasonic signal as medium and biological tissue conductivity as detection target. Taking the high contrast advantage of Electrical Impedance Tomography (EIT) and high resolution advantage of ultrasound imaging, ACTAI has broad application prospects in the field of biomedical imaging. Although ACTAI has high excitation efficiency and strong detectable Signal-to-Noise Ratio, yet while under low frequency electromagnetic excitation, it is still a big challenge to reconstruct a high-resolution image of target conductivity. This paper proposes a new method for reconstructing conductivity based on Generative Adversarial Network, and it consists of three main steps: firstly, use Wiener filtering deconvolution to restore the electrical signal output by the ultrasonic probe to a real acoustic signal. Then obtain the initial acoustic source image with filtered backprojection technology. Finally, match the conductivity image with the initial sound source image, which are used as training samples for generating the adversarial network to establish a deep learning model for conductivity reconstruction. After theoretical analysis and simulation research, it is found that by introducing machine learning, the new method can dig out the inverse problem solving model contained in the data, which further reconstruct a high-resolution conductivity image and has strong anti-interference characteristics. The new method provides a new way to solve the problem of conductivity reconstruction in Applied Current Thermoacoustic Imaging.

Effect of optical coherence tomography and angiography sampling rate towards diabetic retinopathy severity classification

Optical coherence tomography (OCT) and OCT angiography (OCT-A) may benefit the screening of diabetic retinopathy (DR). This study investigated the effect of laterally subsampling OCT/OCT-A en face scans by up to a factor of 8 when using deep neural networks for automated referable DR classification. There was no significant difference in the classification performance across all evaluation metrics when subsampling up to a factor of 3, and only minimal differences up to a factor of 8. Our findings suggest that OCT/OCT-A can reduce the number of samples (and hence the acquisition time) for a volume for a given field of view on the retina that is acquired for rDR classification.

Synthetic polarization-sensitive optical coherence tomography by deep learning

Polarization-sensitive optical coherence tomography (PS-OCT) is a high-resolution label-free optical biomedical imaging modality that is sensitive to the microstructural architecture in tissue that gives rise to form birefringence, such as collagen or muscle fibers. To enable polarization sensitivity in an OCT system, however, requires additional hardware and complexity. We developed a deep-learning method to synthesize PS-OCT images by training a generative adversarial network (GAN) on OCT intensity and PS-OCT images. The synthesis accuracy was first evaluated by the structural similarity index (SSIM) between the synthetic and real PS-OCT images. Furthermore, the effectiveness of the computational PS-OCT images was validated by separately training two image classifiers using the real and synthetic PS-OCT images for cancer/normal classification. The similar classification results of the two trained classifiers demonstrate that the predicted PS-OCT images can be potentially used interchangeably in cancer diagnosis applications. In addition, we applied the trained GAN models on OCT images collected from a separate OCT imaging system, and the synthetic PS-OCT images correlate well with the real PS-OCT image collected from the same sample sites using the PS-OCT imaging system. This computational PS-OCT imaging method has the potential to reduce the cost, complexity, and need for hardware-based PS-OCT imaging systems.

Super-resolution technology to simultaneously improve optical & digital resolution of optical coherence tomography via deep learning

Optical coherence tomography (OCT) has stimulated a wide range of medical image-based diagnosis and treatment. In cardiac imaging, OCT has been used in assessing plaques before and after stenting. While needed in many scenarios, high resolution comes at the costs of demanding optical design and data storage/transmission. In OCT, there are two types of resolutions to characterize image quality: optical and digital resolutions. Although multiple existing works have heavily emphasized on improving the digital resolution, the studies on improving optical resolution or both resolutions remain scarce. In this paper, we focus on improving both resolutions. In particular, we investigate a deep learning method to address the problem of generating a high-resolution (HR) OCT image from a low optical and low digital resolution (L2R) image. To this end, we have modified the existing super-resolution generative adversarial network (SR-GAN) for OCT image reconstruction. Experimental results from the human coronary OCT images have demonstrated that the reconstructed images from highly compressed data could achieve high structural similarity and accuracy in comparison with the HR images. Besides, our method has obtained better denoising performance than the block-matching and 3D filtering (BM3D) and Denoising Convolutional Neural Networks (DnCNN) denoising method.