Robust anisotropic diffusion

A 3D dual-echo spiral sequence for simultaneous dynamic susceptibility contrast and dynamic contrast-enhanced MRI with single bolus injection

PURPOSE

Perfusion MRI reveals important tumor physiological and pathophysiologic information, making it a critical component in managing brain tumor patients. This study aimed to develop a dual-echo 3D spiral technique with a single-bolus scheme to simultaneously acquire both dynamic susceptibility contrast (DSC) and dynamic contrast-enhanced (DCE) data and overcome the limitations of current EPI-based techniques.

METHODS

A 3D spiral-based technique with dual-echo acquisition was implemented and optimized on a 3T MRI scanner with a spiral staircase trajectory and through-plane SENSE acceleration for improved speed and image quality, in-plane variable-density undersampling combined with a sliding-window acquisition and reconstruction approach for increased speed, and an advanced iterative deblurring algorithm. Four volunteers were scanned and compared with the standard of care (SOC) single-echo EPI and a dual-echo EPI technique. Two patients were scanned with the spiral technique during a preload bolus and compared with the SOC single-echo EPI collected during the second bolus injection.

RESULTS

Volunteer data demonstrated that the spiral technique achieved high image quality, reduced geometric artifacts, and high temporal SNR compared with both single-echo and dual-echo EPI. Patient perfusion data showed that the spiral acquisition achieved accurate DSC quantification comparable to SOC single-echo dual-dose EPI, with the additional DCE information.

CONCLUSION

A 3D dual-echo spiral technique was developed to simultaneously acquire both DSC and DCE data in a single-bolus injection with reduced contrast use. Preliminary volunteer and patient data demonstrated increased temporal SNR, reduced geometric artifacts, and accurate perfusion quantification, suggesting a competitive alternative to SOC-EPI techniques for brain perfusion MRI.

Effective image fusion strategies in scientific signal processing disciplines: Application to cancer and carcinoma treatment planning

Multimodal medical image fusion is a perennially prominent research topic that can obtain informative medical images and aid radiologists in diagnosing and treating disease more effectively. However, the recent state-of-the-art methods extract and fuse features by subjectively defining constraints, which easily distort the exclusive information of source images. To overcome these problems and get a better fusion method, this study proposes a 2D data fusion method that uses salient structure extraction (SSE) and a swift algorithm via normalized convolution to fuse different types of medical images. First, salient structure extraction (SSE) is used to attenuate the effect of noise and irrelevant data in the source images by preserving the significant structures. The salient structure extraction is performed to ensure that the pixels with a higher gradient magnitude impact the choices of their neighbors and further provide a way to restore the sharply altered pixels to their neighbors. In addition, a Swift algorithm is used to overcome the excessive pixel values and modify the contrast of the source images. Furthermore, the method proposes an efficient method for performing edge-preserving filtering using normalized convolution. In the end,the fused image are obtained through linear combination of the processed image and the input images based on the properties of the filters. A quantitative function composed of structural loss and region mutual data loss is designed to produce restrictions for preserving data at feature level and the structural level. Extensive experiments on CT-MRI images demonstrate that the proposed algorithm exhibits superior performance when compared to some of the state-of-the-art methods in terms of providing detailed information, edge contour, and overall contrasts.

Incomplete Gamma Kernels: Generalizing Locally Optimal Projection Operators

We present incomplete gamma kernels, a generalization of Locally Optimal Projection (LOP) operators. In particular, we reveal the relation of the classical localized L1 estimator, used in the LOP operator for point cloud denoising, to the common Mean Shift framework via a novel kernel. Furthermore, we generalize this result to a whole family of kernels that are built upon the incomplete gamma function and each represents a localized Lp estimator. By deriving various properties of the kernel family concerning distributional, Mean Shift induced, and other aspects such as strict positive definiteness, we obtain a deeper understanding of the operator's projection behavior. From these theoretical insights, we illustrate several applications ranging from an improved Weighted LOP (WLOP) density weighting scheme and a more accurate Continuous LOP (CLOP) kernel approximation to the definition of a novel set of robust loss functions. These incomplete gamma losses include the Gaussian and LOP loss as special cases and can be applied to various tasks including normal filtering. Furthermore, we show that the novel kernels can be included as priors into neural networks. We demonstrate the effects of each application in a range of quantitative and qualitative experiments that highlight the benefits induced by our modifications.

Improving Structural MRI Preprocessing with Hybrid Transformer GANs

Magnetic resonance imaging (MRI) is a technique that is widely used in practice to evaluate any pathologies in the human body. One of the areas of interest is the human brain. Naturally, MR images are low-resolution and contain noise due to signal interference, the patient's body's radio-frequency emissions and smaller Tesla coil counts in the machinery. There is a need to solve this problem, as MR tomographs that have the capability of capturing high-resolution images are extremely expensive and the length of the procedure to capture such images increases by the order of magnitude. Vision transformers have lately shown state-of-the-art results in super-resolution tasks; therefore, we decided to evaluate whether we can employ them for structural MRI super-resolution tasks. A literature review showed that similar methods do not focus on perceptual image quality because upscaled images are often blurry and are subjectively of poor quality. Knowing this, we propose a methodology called HR-MRI-GAN, which is a hybrid transformer generative adversarial network capable of increasing resolution and removing noise from 2D T1w MRI slice images. Experiments show that our method quantitatively outperforms other SOTA methods in terms of perceptual image quality and is capable of subjectively generalizing to unseen data. During the experiments, we additionally identified that the visual saliency-induced index metric is not applicable to MRI perceptual quality assessment and that general-purpose denoising networks are effective when removing noise from MR images.

Improving segmentation and detection of lesions in CT scans using intensity distribution supervision

We propose a method to incorporate the intensity information of a target lesion on CT scans in training segmentation and detection networks. We first build an intensity-based lesion probability (ILP) function from an intensity histogram of the target lesion. It is used to compute the probability of being the lesion for each voxel based on its intensity. Finally, the computed ILP map of each input CT scan is provided as additional supervision for network training, which aims to inform the network about possible lesion locations in terms of intensity values at no additional labeling cost. The method was applied to improve the segmentation of three different lesion types, namely, small bowel carcinoid tumor, kidney tumor, and lung nodule. The effectiveness of the proposed method on a detection task was also investigated. We observed improvements of 41.3% → 47.8%, 74.2% → 76.0%, and 26.4% → 32.7% in segmenting small bowel carcinoid tumor, kidney tumor, and lung nodule, respectively, in terms of per case Dice scores. An improvement of 64.6% → 75.5% was achieved in detecting kidney tumors in terms of average precision. The results of different usages of the ILP map and the effect of varied amount of training data are also presented.

Improving Small Lesion Segmentation in CT Scans using Intensity Distribution Supervision: Application to Small Bowel Carcinoid Tumor

Finding small lesions is very challenging due to lack of noticeable features, severe class imbalance, as well as the size itself. One approach to improve small lesion segmentation is to reduce the region of interest and inspect it at a higher sensitivity rather than performing it for the entire region. It is usually implemented as sequential or joint segmentation of organ and lesion, which requires additional supervision on organ segmentation. Instead, we propose to utilize an intensity distribution of a target lesion at no additional labeling cost to effectively separate regions where the lesions are possibly located from the background. It is incorporated into network training as an auxiliary task. We applied the proposed method to segmentation of small bowel carcinoid tumors in CT scans. We observed improvements for all metrics (33.5% → 38.2%, 41.3% → 47.8%, 30.0% → 35.9% for the global, per case, and per tumor Dice scores, respectively.) compared to the baseline method, which proves the validity of our idea. Our method can be one option for explicitly incorporating intensity distribution information of a target in network training.

Dictionary learning compressed sensing reconstruction: pilot validation of accelerated echo planar J-resolved spectroscopic imaging in prostate cancer

Objectives

This study aimed at developing dictionary learning (DL) based compressed sensing (CS) reconstruction for randomly undersampled five-dimensional (5D) MR Spectroscopic Imaging (3D spatial + 2D spectral) data acquired in prostate cancer patients and healthy controls, and test its feasibility at 8x and 12x undersampling factors.

Materials and methods

Prospectively undersampled 5D echo-planar J-resolved spectroscopic imaging (EP-JRESI) data were acquired in nine prostate cancer (PCa) patients and three healthy males. The 5D EP-JRESI data were reconstructed using DL and compared with gradient sparsity-based Total Variation (TV) and Perona-Malik (PM) methods. A hybrid reconstruction technique, Dictionary Learning-Total Variation (DLTV), was also designed to further improve the quality of reconstructed spectra.

Results

The CS reconstruction of prospectively undersampled (8x and 12x) 5D EP-JRESI data acquired in prostate cancer and healthy subjects were performed using DL, DLTV, TV and PM. It is evident that the hybrid DLTV method can unambiguously resolve 2D J-resolved peaks including myo-inositol, citrate, creatine, spermine and choline.

Conclusion

Improved reconstruction of the accelerated 5D EP-JRESI data was observed using the hybrid DLTV. Accelerated acquisition of in vivo 5D data with as low as 8.33% samples (12x) corresponds to a total scan time of 14 min as opposed to a fully sampled scan that needs a total duration of 2.4 h (TR = 1.2 s, 32 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${k}_{x}$$\end{document}kx×16 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${k}_{y}$$\end{document}ky×8 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${k}_{z}$$\end{document}kz, 512 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${t}_{2}$$\end{document}t2 and 64 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${t}_{1}$$\end{document}t1).

Supplementary Information

The online version contains supplementary material available at 10.1007/s10334-022-01029-z.

PDE Learning of Filtering and Propagation for Task-Aware Facial Intrinsic Image Analysis

Filtering and propagation are two basic operations in image analysis and rendering, and they are also widely used in computer graphics and machine learning. However, the models of filtering and propagation were based on diverse mathematical formulations, which have not been fully understood. This article aims to explore the properties of both filtering and propagation models from a partial differential equation (PDE) learning perspective. We propose a unified PDE learning framework based on nonlinear reaction-diffusion with a guided map, graph Laplacian, and reaction weight. It reveals that: 1) the guided map and reaction weight determines whether the PDE produces filtering or propagation diffusion and 2) the kernel of graph Laplacian controls the diffusion pattern. Based on the proposed PDE framework, we derive the mathematical relations between different models, including learning to diffusion (LTD) model, label propagation, edit propagation, and edge-aware filter. In practical verification, we apply the PDE framework to design diffusion operations with the adaptive kernel to tackle the ill-posed problem of facial intrinsic image analysis (FIIA). A flexible task-aware FIIA system is built to achieve various facial rendering effects, such as face image relighting and delighting, artistic illumination transfer, illumination-aware face swapping, or transfiguring. Qualitative and quantitative experiments show the effectiveness and flexibility of task-aware FIIA and provide new insights on PDE learning for visual analysis and rendering.

SNR enhancement for Brillouin distributed optical fiber sensors based on asynchronous control

We propose the asynchronous control of anisotropic diffusion (AD) algorithm, and such asynchronous anisotropic diffusion (AAD) algorithm is demonstrated experimentally to reduce noise from the sensing signals obtained from Brillouin distributed optical fiber sensors. The performance of the proposed AAD algorithm is analyzed in detail for different experimental conditions and compared with that of block-matching and 3D filtering, two-dimensional wavelet denoising, AD, and non-local means algorithms. Some key factors of the proposed algorithm, such as the impact of convolution kernel size on the performance of AD algorithms, the influence of low sampling point number (SPN) on the quality of Brillouin frequency shift and the selection of diffusion thresholds are analyzed and discussed with experimental results. The experimental results validate that the AAD algorithm can provide better root-mean-square error (RMSE) and spatial resolution (SR) than the other four algorithms, especially for higher signal-to-noise ratio (SNR) improvement and higher SPNs. For lower SPNs, the performance of AAD is also not inferior to the RMSE performance of NLM and AD. The runtime of the AAD algorithm is also quite low. Moreover, the proposed algorithm offers the best SR performance as compared to other noise reduction algorithms investigated in this study. Thus, the proposed AAD algorithm can be an effective candidate to improve the measurement accuracy of Brillouin distributed optical fiber sensors.

3D Interactive Segmentation With Semi-Implicit Representation and Active Learning

Segmenting complex 3D geometry is a challenging task due to rich structural details and complex appearance variations of target object. Shape representation and foreground-background delineation are two of the core components of segmentation. Explicit shape models, such as mesh based representations, suffer from poor handling of topological changes. On the other hand, implicit shape models, such as level-set based representations, have limited capacity for interactive manipulation. Fully automatic segmentation for separating foreground objects from background generally utilizes non-interoperable machine learning methods, which heavily rely on the off-line training dataset and are limited to the discrimination power of the chosen model. To address these issues, we propose a novel semi-implicit representation method, namely Non-Uniform Implicit B-spline Surface (NU-IBS), which adaptively distributes parametrically blended patches according to geometrical complexity. Then, a two-stage cascade classifier is introduced to carry out efficient foreground and background delineation, where a simplistic Naïve-Bayesian model is trained for fast background elimination, followed by a stronger pseudo-3D Convolutional Neural Network (CNN) multi-scale classifier to precisely identify the foreground objects. A localized interactive and adaptive segmentation scheme is incorporated to boost the delineation accuracy by utilizing the information iteratively gained from user intervention. The segmentation result is obtained via deforming an NU-IBS according to the probabilistic interpretation of delineated regions, which also imposes a homogeneity constrain for individual segments. The proposed method is evaluated on a 3D cardiovascular Computed Tomography Angiography (CTA) image dataset and Brain Tumor Image Segmentation Benchmark 2015 (BraTS2015) 3D Magnetic Resonance Imaging (MRI) dataset.

Image Denoising Based on Fractional Gradient Vector Flow and Overlapping Group Sparsity as Priors

In this paper, a new regularization term in the form of L1-norm based fractional gradient vector flow (LF-GGVF) is presented for the task of image denoising. A fractional order variational method is formulated, which is then utilized for estimating the proposed LF-GGVF. Overlapping group sparsity along with LF-GGVF is used as priors in image denoising optimization framework. The Riemann-Liouville derivative is used for approximating the fractional order derivatives present in the optimization framework. Its role in the framework helps in boosting the denoising performance. The numerical optimization is performed in an alternating manner using the well-known alternating direction method of multipliers (ADMM) and split Bregman techniques. The resulting system of linear equations is then solved using an efficient numerical scheme. A variety of simulated data that includes test images contaminated by additive white Gaussian noise are used for experimental validation. The results of numerical solutions obtained from experimental work demonstrate that the performance of the proposed approach in terms of noise suppression and edge preservation is better when compared with that of several other methods.

12-year prediction of mild cognitive impairment aided by Alzheimer's brain signatures at mean age 56

Neuroimaging signatures based on composite scores of cortical thickness and hippocampal volume predict progression from mild cognitive impairment to Alzheimer's disease. However, little is known about the ability of these signatures among cognitively normal adults to predict progression to mild cognitive impairment. Towards that end, a signature sensitive to microstructural changes that may predate macrostructural atrophy should be useful. We hypothesized that: (i) a validated MRI-derived Alzheimer's disease signature based on cortical thickness and hippocampal volume in cognitively normal middle-aged adults would predict progression to mild cognitive impairment; and (ii) a novel grey matter mean diffusivity signature would be a better predictor than the thickness/volume signature. This cohort study was part of the Vietnam Era Twin Study of Aging. Concurrent analyses compared cognitively normal and mild cognitive impairment groups at each of three study waves (ns = 246-367). Predictive analyses included 169 cognitively normal men at baseline (age = 56.1, range = 51-60). Our previously published thickness/volume signature derived from independent data, a novel mean diffusivity signature using the same regions and weights as the thickness/volume signature, age, and an Alzheimer's disease polygenic risk score were used to predict incident mild cognitive impairment an average of 12 years after baseline (follow-up age = 67.2, range = 61-71). Additional analyses adjusted for predicted brain age difference scores (chronological age minus predicted brain age) to determine if signatures were Alzheimer-related and not simply ageing-related. In concurrent analyses, individuals with mild cognitive impairment had higher (worse) mean diffusivity signature scores than cognitively normal participants, but thickness/volume signature scores did not differ between groups. In predictive analyses, age and polygenic risk score yielded an area under the curve of 0.74 (sensitivity = 80.00%; specificity = 65.10%). Prediction was significantly improved with addition of the mean diffusivity signature (area under the curve = 0.83; sensitivity = 85.00%; specificity = 77.85%; P = 0.007), but not with addition of the thickness/volume signature. A model including both signatures did not improve prediction over a model with only the mean diffusivity signature. Results held up after adjusting for predicted brain age difference scores. The novel mean diffusivity signature was limited by being yoked to the thickness/volume signature weightings. An independently derived mean diffusivity signature may thus provide even stronger prediction. The young age of the sample at baseline is particularly notable. Given that the brain signatures were examined when participants were only in their 50 s, our results suggest a promising step towards improving very early identification of Alzheimer's disease risk and the potential value of mean diffusivity and/or multimodal brain signatures.

A Generalized Framework for Edge-preserving and Structure-preserving Image Smoothing

Image smoothing is a fundamental procedure in applications of both computer vision and graphics. The required smoothing properties can be different or even contradictive among different tasks. Nevertheless, the inherent smoothing nature of one smoothing operator is usually fixed and thus cannot meet the various requirements of different applications. In this paper, we first introduce the truncated Huber penalty function which shows strong flexibility under different parameter settings. A generalized framework is then proposed with the introduced truncated Huber penalty function. When combined with its strong flexibility, our framework is able to achieve diverse smoothing natures where contradictive smoothing behaviors can even be achieved. It can also yield the smoothing behavior that can seldom be achieved by previous methods, and superior performance is thus achieved in challenging cases. These together enable our framework capable of a range of applications and able to outperform the state-of-the-art approaches in several tasks. In addition, an efficient numerical solution is provided and its convergence is theoretically guaranteed even the optimization framework is non-convex and non-smooth. A simple yet effective approach is further proposed to reduce the computational cost of our method while maintaining its performance. The effectiveness and superior performance of our approach are validated through comprehensive experiments in a range of applications.

Hyperspectral Anomaly Detection via Graph Dictionary-Based Low Rank Decomposition with Texture Feature Extraction

The accuracy of anomaly detection in hyperspectral images (HSIs) faces great challenges due to the high dimensionality, redundancy of data, and correlation of spectral bands. In this paper, to further improve the detection accuracy, we propose a novel anomaly detection method based on texture feature extraction and a graph dictionary-based low rank decomposition (LRD). First, instead of using traditional clustering methods for the dictionary, the proposed method employs the graph theory and designs a graph Laplacian matrix-based dictionary for LRD. The robust information of the background matrix in the LRD model is retained, and both the low rank matrix and the sparse matrix are well separated while preserving the correlation of background pixels. To further improve the detection performance, we explore and extract texture features from HSIs and integrate with the low-rank model to obtain the sparse components by decomposition. The detection results from feature maps are generated in order to suppress background components similar to anomalies in the sparse matrix and increase the strength of real anomalies. Experiments were run on one synthetic dataset and three real datasets to evaluate the performance. The results show that the performance of the proposed method yields competitive results in terms of average area under the curve (AUC) for receiver operating characteristic (ROC), i.e., 0.9845, 0.9962, 0.9699, and 0.9900 for different datasets, respectively. Compared with seven other state-of-the-art algorithms, our method yielded the highest average AUC for ROC in all datasets.

Anisotropic diffusion technique to eliminate speckle noise in continuous-wave Doppler ultrasound spectrogram

Paper described a simplified speckled elimination algorithm for the reconstruction of continuous-wave (CW) Doppler ultrasound spectrogram. Standard Nonlinear Anisotropic Diffusion (SNAD) technique fails to eliminate the speckle noise, blurs the interregional area of neighbouring speckle clusters and regulates the heat flow to preserve the systolic and diastolic peaks in the spectrogram. To resolve this problem, author modelled a kernel, conductance and diffusion function to regulate the blurring of speckle clusters and sharpen the high-contrast edges over low-contrast ones. Paper analysed the performance based on the variation of conductance (C), signal-to-noise ratio (SNR), peak signal-to-noise ratio (PSNR), mean square error (MSE), root mean square error (RMSE) and structural similarity index (SSIM) of each pairs of speckled and despeckled spectrograms. Quantitative analysis reveals that the minimised version of the kernel effectively regulates the flow of heat at edges with a high SNR (8 dB) and PSNR (>20 dB) for a low-frequency range of 20-25 Hz. SSIM (>0.9) shows an efficient structural reconstruction that identifies the lower and higher local SSIM indices as dark and bright pixels on SSIM maps. Hence, the algorithm performs its best at lower peak frequencies in the Doppler spectrograms.

Compressed sensing MRI using an interpolation-free nonlinear diffusion model

PURPOSE

Constraints in extended neighborhood system demand the use of a large number of interpolations in directionality-guided compressed-sensing nonlinear diffusion MR image reconstruction technique. This limits its practical application in terms of computational complexity. The proposed method aims at multifold improvement in its runtime without compromising the image quality.

THEORY AND METHODS

Conventional approach to extended neighborhood computation requires 108 linear interpolations per pixel for 10 sets of neighborhoods. We propose a neighborhood stretching technique that systematically extends the location of neighboring pixels such that 66% to 100% fewer interpolations are required to compute the gradients along multiple directions. A spatial frequency-based deviation measure is then used to choose the most reliable edges from the set of images generated by diffusion along different directions.

RESULTS

The semi-interpolated and interpolation-free diffusion techniques proposed in this paper are compared with the fully interpolated diffusion-based reconstruction by reconstruing multiple multichannel in vivo datasets, undersampled using different sampling patterns at various sampling rates. Results indicate a two- to fivefold increase in reconstruction speed with a potential to generate 1 to 2 dB improvement in peak SNR measure.

CONCLUSION

The proposed method outperforms the state-of-the-art fully interpolated diffusion model and generates high-quality reconstructions for different sampling patterns and acceleration factors with a two- to fivefold increment in reconstruction speed. This makes it the most suitable candidate for edge-preserving penalties used in the compressed sensing MRI reconstruction methods.

Speckle Noise Diffusion in Knee Articular Cartilage Ultrasound Images

BACKGROUND

Ultrasound (US) imaging can be a convenient and reliable substitute for magnetic resonance imaging in the investigation or screening of articular cartilage injury. However, US images suffer from two main impediments, i.e., low contrast ratio and presence of speckle noise.

AIMS

A variation of anisotropic diffusion is proposed that can reduce speckle noise without compromising the image quality of the edges and other important details.

METHODS

For this technique, four gradient thresholds were adopted instead of one. A new diffusivity function that preserves the edge of the resultant image is also proposed. To automatically terminate the iterative procedures, the Mean Absolute Error as its stopping criterion was implemented.

RESULTS

Numerical results obtained by simulations unanimously indicate that the proposed method outperforms conventional speckle reduction techniques. Nevertheless, this preliminary study has been conducted based on a small number of asymptomatic subjects.

CONCLUSION

Future work must investigate the feasibility of this method in a large cohort and its clinical validity through testing subjects with a symptomatic cartilage injury.

Automated labeling of the airway tree in terms of lobes based on deep learning of bifurcation point detection

This paper presents an automatic lobe-based labeling of airway tree method, which can detect the bifurcation points for reconstructing and labeling the airway tree from a computed tomography image. A deep learning-based network structure is designed to identify the four key bifurcation points. Then, based on the detected bifurcation points, the entire airway tree is reconstructed by a new region-growing method. Finally, with the basic airway tree anatomy and topology knowledge, individual branches of the airway tree are classified into different categories in terms of pulmonary lobes. There are several advantages in our method such as the detection of the bifurcation points does not depend on the segmentation of airway tree and only four bifurcation points need to be manually labeled for each sample to prepare the training dataset. The segmentation of airway tree is guided by the detected points, which overcomes the difficulty of manual seed selection of conventional region-growing algorithm. In addition, the bifurcation points can help analyze the tree structure, which provides a basis for effective airway tree labeling. Experimental results show that our method is fast, stable, and the accuracy of our method is 97.85%, which is higher than that of the traditional skeleton-based method. Graphical Abstract The pipeline of our proposed lobe-based airway tree labeling method. Given a raw CT volume, a neural network structure is designed to predict major bifurcation points of airway tree. Based on the detected points, airway tree is reconstructed and labeled in terms of lobes.

Feature Keypoint-Based Image Compression Technique Using a Well-Posed Nonlinear Fourth-Order PDE-Based Model

A digital image compression framework based on nonlinear partial differential equations (PDEs) is proposed in this research article. First, a feature keypoint-based sparsification algorithm is proposed for the image coding stage. The interest keypoints corresponding to various scale-invariant image feature descriptors, such as SIFT, SURF, MSER, ORB, and BRIEF, are extracted, and the points from their neighborhoods are then used as sparse pixels and coded using a lossless encoding scheme. An effective nonlinear fourth-order PDE-based scattered data interpolation is proposed for solving the decompression task. A rigorous mathematical investigation of the considered PDE model is also performed, with the well-posedness of this model being demonstrated. It is then solved numerically by applying a consistent finite difference method-based numerical approximation algorithm that is next successfully applied in the image compression and decompression experiments, which are also discussed in this work.

High-ISO Long-Exposure Image Denoising Based on Quantitative Blob Characterization

Blob detection and image denoising are fundamental, sometimes related tasks in computer vision. In this paper, we present a computational method to quantitatively measure blob characteristics using normalized unilateral second-order Gaussian kernels. This method suppresses non-blob structures while yielding a quantitative measurement of the position, prominence and scale of blobs, which can facilitate the tasks of blob reconstruction and blob reduction. Subsequently, we propose a denoising scheme to address high-ISO long-exposure noise, which sometimes spatially shows a blob appearance, employing a blob reduction procedure as a cheap preprocessing for conventional denoising methods. We apply the proposed denoising methods to real-world noisy images as well as standard images that are corrupted by real noise. The experimental results demonstrate the superiority of the proposed methods over state-of-the-art denoising methods.

Fully automatic 3D segmentation of the thoracolumbar spinal cord and the vertebral canal from T2-weighted MRI using K-means clustering algorithm

STUDY DESIGN

Method development.

OBJECTIVES

To develop a reliable protocol for automatic segmentation of Thoracolumbar spinal cord using MRI based on K-means clustering algorithm in 3D images.

SETTING

University-based laboratory, Tehran, Iran.

METHODS

T2 structural volumes acquired from the spinal cord of 20 uninjured volunteers on a 3T MR scanner. We proposed an automatic method for spinal cord segmentation based on the K-means clustering algorithm in 3D images and compare our results with two available segmentation methods (PropSeg, DeepSeg) implemented in the Spinal Cord Toolbox. Dice and Hausdorff were used to compare the results of our method (K-Seg) with the manual segmentation, PropSeg, and DeepSeg.

RESULTS

The accuracy of our automatic segmentation method for T2-weighted images was significantly better or similar to the SCT methods, in terms of 3D DC (p < 0.001). The 3D DCs were respectively (0.81 ± 0.04) and Hausdorff Distance (12.3 ± 2.48) by the K-Seg method in contrary to other SCT methods for T2-weighted images.

CONCLUSIONS

The output with similar protocols showed that K-Seg results match the manual segmentation better than the other methods especially on the thoracolumbar levels in the spinal cord due to the low image contrast as a result of poor SNR in these areas.

Image denoising via a non-local patch graph total variation

Total variation (TV) based models are very popular in image denoising but suffer from some drawbacks. For example, local TV methods often cannot preserve edges and textures well when they face excessive smoothing. Non-local TV methods constitute an alternative, but their computational cost is huge. To overcome these issues, we propose an image denoising method named non-local patch graph total variation (NPGTV). Its main originality stands for the graph total variation method, which combines the total variation with graph signal processing. Schematically, we first construct a K-nearest graph from the original image using a non-local patch-based method. Then the model is solved with the Douglas-Rachford Splitting algorithm. By doing so, the image details can be well preserved while being denoised. Experiments conducted on several standard natural images illustrate the effectiveness of our method when compared to some other state-of-the-art denoising methods like classical total variation, non-local means filter (NLM), non-local graph based transform (NLGBT), adaptive graph-based total variation (AGTV).

A new approach to automatic fetal brain extraction from MRI using a variational level set method

BACKGROUND AND PURPOSE

Appropriate images extracted from the MRI of mothers' wombs can be of great help in the medical diagnosis of fetal abnormalities. As maternal tissue may appear in such images, affecting visualization of myelination of the fetal brain, it is not possible to use methods routinely used for extraction of adult brains for fetal brains. The aim of the present study was to use a variational level set approach to extract fetal brain from T2-weighted MR images of the womb.

METHODS

Coronal T2-weighted images were acquired using fast MRI protocols (to avoid artifacts). The database includes 105 MR images from eight subjects. After correcting the inhomogeneity of the images, the fetal eyes were located, and from that information, the location of the fetus brain was automatically determined. Then, the variational level set was used for fetus brain extraction. The results were analyzed by a clinical specialist (radiologist) and the similarity (Dice and Jaccard coefficients), sensitivity and specificity were calculated.

RESULTS AND CONCLUSIONS

The means of the statistical analysis for the Dice and Jaccard coefficients, sensitivity and specificity, were 99.56%, 96.89%, 95.71%, and 97.96%, respectively. Thus, extraction of fetal brain from MR images was confirmed, both statistically and visually through cross-validation.

Spatio-Temporal Feature Extraction/Recognition in Videos Based on Energy Optimization

Videos are spatio-temporally rich in static to dynamic objects/scenes, sparse to dense, and periodic to non-periodic motions. Particularly, the dynamic texture (DT) exhibits complex appearance and motion changes that remain a challenge to deal with. This paper presents an energy optimization method for feature extraction and recognition in videos. For noise and background jitter, the Tikhonov regularization with eigen-vector and Frenet-Serret formula-based energy constraints is also proposed. The different periodicity of DT can be adapted by the time-varying number of learning temporal frames. The optimal duration of an image sequence is determined from the temporal property of its eigen-values. Unlike the state-of-the-art recognition methods, i.e., sparse coding and slow feature analysis, the proposed method can capture the physical property of objects and scenes: velocity, acceleration, and orientation. Also, the static and dynamic image regions can be locally classified. Owing to these spatio-temporal features, stability, robustness, and accuracy of feature extraction and recognition are enhanced. Using DT videos, the superiority of the proposed method compared to the state-of-the-art recognition methods is experimentally shown.

Robust and High Fidelity Mesh Denoising

This paper presents a simple and effective two-stage mesh denoising algorithm, where in the first stage, face normal filtering is done by using bilateral normal filtering in a robust statistics framework. Tukey's bi-weight function is used as similarity function in the bilateral weighting, which is a robust estimator and stops the diffusion at sharp edges to retain features and removes noise from flat regions effectively. In the second stage, an edge-weighted Laplace operator is introduced to compute a differential coordinate. This differential coordinate helps the algorithm to produce a high-quality mesh without any face normal flips and makes the method robust against high-intensity noise.

Motion parallax for 360° RGBD video

We present a method for adding parallax and real-time playback of 360° videos in Virtual Reality headsets. In current video players, the playback does not respond to translational head movement, which reduces the feeling of immersion, and causes motion sickness for some viewers. Given a 360° video and its corresponding depth (provided by current stereo 360° stitching algorithms), a naive image-based rendering approach would use the depth to generate a 3D mesh around the viewer, then translate it appropriately as the viewer moves their head. However, this approach breaks at depth discontinuities, showing visible distortions, whereas cutting the mesh at such discontinuities leads to ragged silhouettes and holes at disocclusions. We address these issues by improving the given initial depth map to yield cleaner, more natural silhouettes. We rely on a three-layer scene representation, made up of a foreground layer and two static background layers, to handle disocclusions by propagating information from multiple frames for the first background layer, and then inpainting for the second one. Our system works with input from many of today's most popular 360° stereo capture devices (e.g., Yi Halo or GoPro Odyssey), and works well even if the original video does not provide depth information. Our user studies confirm that our method provides a more compelling viewing experience than without parallax, increasing immersion while reducing discomfort and nausea.

FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence

We present a descriptor, called fully convolutional self-similarity (FCSS), for dense semantic correspondence. Unlike traditional dense correspondence approaches for estimating depth or optical flow, semantic correspondence estimation poses additional challenges due to intra-class appearance and shape variations among different instances within the same object or scene category. To robustly match points across semantically similar images, we formulate FCSS using local self-similarity (LSS), which is inherently insensitive to intra-class appearance variations. LSS is incorporated through a proposed convolutional self-similarity (CSS) layer, where the sampling patterns and the self-similarity measure are jointly learned in an end-to-end and multi-scale manner. Furthermore, to address shape variations among different object instances, we propose a convolutional affine transformer (CAT) layer that estimates explicit affine transformation fields at each pixel to transform the sampling patterns and corresponding receptive fields. As training data for semantic correspondence is rather limited, we propose to leverage object candidate priors provided in most existing datasets and also correspondence consistency between object pairs to enable weakly-supervised learning. Experiments demonstrate that FCSS significantly outperforms conventional handcrafted descriptors and CNN-based descriptors on various benchmarks.

Local Kernels that Approximate Bayesian Regularization and Proximal Operators

In this work, we broadly connect kernel-based filtering (e.g. approaches such as the bilateral filter and nonlocal means, but also many more) with general variational formulations of Bayesian regularized least squares, and the related concept of proximal operators. Variational/Bayesian/proximal formulations often result in optimization problems that do not have closed-form solutions, and therefore typically require global iterative solutions. Our main contribution here is to establish how one can approximate the solution of the resulting global optimization problems using locally adaptive filters with specific kernels. Our results are valid for small regularization strength (i.e. weak noise) but the approach is powerful enough to be useful for a wide range of applications because we expose how to derive a "kernelized" solution to these problems that approximates the global solution in one shot, using only local operations. As another side benefit in the reverse direction, given a local data-adaptive filter constructed with a particular choice of kernel, we enable the interpretation of such filters in the variational/Bayesian/proximal framework.

Robust Learning With Kernel Mean -Power Error Loss

Correntropy is a second order statistical measure in kernel space, which has been successfully applied in robust learning and signal processing. In this paper, we define a nonsecond order statistical measure in kernel space, called the kernel mean- power error (KMPE), including the correntropic loss (C-Loss) as a special case. Some basic properties of KMPE are presented. In particular, we apply the KMPE to extreme learning machine (ELM) and principal component analysis (PCA), and develop two robust learning algorithms, namely ELM-KMPE and PCA-KMPE. Experimental results on synthetic and benchmark data show that the developed algorithms can achieve better performance when compared with some existing methods.

Comparison Study of Regularizations in Spectral Computed Tomography Reconstruction

The energy-resolving photon-counting detectors in spectral computed tomography (CT) can acquire projections of an object in different energy channels. In other words, they are able to reliably distinguish the received photon energies. These detectors lead to the emerging spectral CT, which is also called multi-energy CT, energy-selective CT, color CT, etc. Spectral CT can provide additional information in comparison with the conventional CT in which energy integrating detectors are used to acquire polychromatic projections of an object being investigated. The measurements obtained by X-ray CT detectors are noisy in reality, especially in spectral CT where the photon number is low in each energy channel. Therefore, some regularization should be applied to obtain a better image quality for this ill-posed problem in spectral CT image reconstruction. Quadratic-based regularizations are not often satisfactory as they blur the edges in the reconstructed images. As a result, different edge-preserving regularization methods have been adopted for reconstructing high quality images in the last decade. In this work, we numerically evaluate the performance of different regularizers in spectral CT, including total variation, non-local means and anisotropic diffusion. The goal is to provide some practical guidance to accurately reconstruct the attenuation distribution in each energy channel of the spectral CT data.

Edge Probability and Pixel Relativity-Based Speckle Reducing Anisotropic Diffusion

Anisotropic diffusion filters are one of the best choices for speckle reduction in the ultrasound images. These filters control the diffusion flux flow using local image statistics and provide the desired speckle suppression. However, inefficient use of edge characteristics results in either oversmooth image or an image containing misinterpreted spurious edges. As a result, the diagnostic quality of the images becomes a concern. To alleviate such problems, a novel anisotropic diffusion-based speckle reducing filter is proposed in this paper. A probability density function of the edges along with pixel relativity information is used to control the diffusion flux flow. The probability density function helps in removing the spurious edges and the pixel relativity reduces the oversmoothing effects. Furthermore, the filtering is performed in superpixel domain to reduce the execution time, wherein a minimum of 15% of the total number of image pixels can be used. For performance evaluation, 31 frames of three synthetic images and 40 real ultrasound images are used. In most of the experiments, the proposed filter shows a better performance as compared to the state-of-the-art filters in terms of the speckle region's signal-to-noise ratio and mean square error. It also shows a comparative performance for figure of merit and structural similarity measure index. Furthermore, in the subjective evaluation, performed by the expert radiologists, the proposed filter's outputs are preferred for the improved contrast and sharpness of the object boundaries. Hence, the proposed filtering framework is suitable to reduce the unwanted speckle and improve the quality of the ultrasound images.

Robust Guided Image Filtering Using Nonconvex Potentials

Filtering images using a guidance signal, a process called guided or joint image filtering, has been used in various tasks in computer vision and computational photography, particularly for noise reduction and joint upsampling. This uses an additional guidance signal as a structure prior, and transfers the structure of the guidance signal to an input image, restoring noisy or altered image structure. The main drawbacks of such a data-dependent framework are that it does not consider structural differences between guidance and input images, and that it is not robust to outliers. We propose a novel SD (for static/dynamic) filter to address these problems in a unified framework, and jointly leverage structural information from guidance and input images. Guided image filtering is formulated as a nonconvex optimization problem, which is solved by the majorize-minimization algorithm. The proposed algorithm converges quickly while guaranteeing a local minimum. The SD filter effectively controls the underlying image structure at different scales, and can handle a variety of types of data from different sensors. It is robust to outliers and other artifacts such as gradient reversal and global intensity shift, and has good edge-preserving smoothing properties. We demonstrate the flexibility and effectiveness of the proposed SD filter in a variety of applications, including depth upsampling, scale-space filtering, texture removal, flash/non-flash denoising, and RGB/NIR denoising.

DASC: Robust Dense Descriptor for Multi-Modal and Multi-Spectral Correspondence Estimation

Establishing dense correspondences between multiple images is a fundamental task in many applications. However, finding a reliable correspondence between multi-modal or multi-spectral images still remains unsolved due to their challenging photometric and geometric variations. In this paper, we propose a novel dense descriptor, called dense adaptive self-correlation (DASC), to estimate dense multi-modal and multi-spectral correspondences. Based on an observation that self-similarity existing within images is robust to imaging modality variations, we define the descriptor with a series of an adaptive self-correlation similarity measure between patches sampled by a randomized receptive field pooling, in which a sampling pattern is obtained using a discriminative learning. The computational redundancy of dense descriptors is dramatically reduced by applying fast edge-aware filtering. Furthermore, in order to address geometric variations including scale and rotation, we propose a geometry-invariant DASC (GI-DASC) descriptor that effectively leverages the DASC through a superpixel-based representation. For a quantitative evaluation of the GI-DASC, we build a novel multi-modal benchmark as varying photometric and geometric conditions. Experimental results demonstrate the outstanding performance of the DASC and GI-DASC in many cases of dense multi-modal and multi-spectral correspondences.

Fully automated breast boundary and pectoral muscle segmentation in mammograms

Breast and pectoral muscle segmentation is an essential pre-processing step for the subsequent processes in computer aided diagnosis (CAD) systems. Estimating the breast and pectoral boundaries is a difficult task especially in mammograms due to artifacts, homogeneity between the pectoral and breast regions, and low contrast along the skin-air boundary. In this paper, a breast boundary and pectoral muscle segmentation method in mammograms is proposed. For breast boundary estimation, we determine the initial breast boundary via thresholding and employ Active Contour Models without edges to search for the actual boundary. A post-processing technique is proposed to correct the overestimated boundary caused by artifacts. The pectoral muscle boundary is estimated using Canny edge detection and a pre-processing technique is proposed to remove noisy edges. Subsequently, we identify five edge features to find the edge that has the highest probability of being the initial pectoral contour and search for the actual boundary via contour growing. The segmentation results for the proposed method are compared with manual segmentations using 322, 208 and 100mammograms from the Mammographic Image Analysis Society (MIAS), INBreast and Breast Cancer Digital Repository (BCDR) databases, respectively. Experimental results show that the breast boundary and pectoral muscle estimation methods achieved dice similarity coefficients of 98.8% and 97.8% (MIAS), 98.9% and 89.6% (INBreast) and 99.2% and 91.9% (BCDR), respectively.

Real-Time Nonlocal Means-Based Despeckling

In this paper, we propose a multiscale nonlocal means-based despeckling method for medical ultrasound. The multiscale approach leads to large computational savings and improves despeckling results over single-scale iterative approaches. We present two variants of the method. The first, denoted multiscale nonlocal means (MNLM), yields uniform robust filtering of speckle both in structured and homogeneous regions. The second, denoted unnormalized MNLM (UMNLM), is more conservative in regions of structure assuring minimal disruption of salient image details. Due to the popularity of anisotropic diffusion-based methods in the despeckling literature, we review the connection between anisotropic diffusion and iterative variants of NLM. These iterative variants in turn relate to our multiscale variant. As part of our evaluation, we conduct a simulation study making use of ground truth phantoms generated from clinical B-mode ultrasound images. We evaluate our method against a set of popular methods from the despeckling literature on both fine and coarse speckle noise. In terms of computational efficiency, our method outperforms the other considered methods. Quantitatively on simulations and on a tissue-mimicking phantom, our method is found to be competitive with the state-of-the-art. On clinical B-mode images, our method is found to effectively smooth speckle while preserving low-contrast and highly localized salient image detail.

$L_{0}$ Gradient Projection

Minimizing L₀ gradient, the number of the non-zero gradients of an image, together with a quadratic data-fidelity to an input image has been recognized as a powerful edge-preserving filtering method. However, the L₀ gradient minimization has an inherent difficulty: a user-given parameter controlling the degree of flatness does not have a physical meaning since the parameter just balances the relative importance of the L₀ gradient term to the quadratic data-fidelity term. As a result, the setting of the parameter is a troublesome work in the L₀ gradient minimization. To circumvent the difficulty, we propose a new edge-preserving filtering method with a novel use of the L₀ gradient. Our method is formulated as the minimization of the quadratic data-fidelity subject to the hard constraint that the L₀ gradient is less than a user-given parameter α . This strategy is much more intuitive than the L₀ gradient minimization because the parameter α has a clear meaning: the L₀ gradient value of the output image itself, so that one can directly impose a desired degree of flatness by α . We also provide an efficient algorithm based on the so-called alternating direction method of multipliers for computing an approximate solution of the nonconvex problem, where we decompose it into two subproblems and derive closed-form solutions to them. The advantages of our method are demonstrated through extensive experiments.

Theory of cortical function

Most models of sensory processing in the brain have a feedforward architecture in which each stage comprises simple linear filtering operations and nonlinearities. Models of this form have been used to explain a wide range of neurophysiological and psychophysical data, and many recent successes in artificial intelligence (with deep convolutional neural nets) are based on this architecture. However, neocortex is not a feedforward architecture. This paper proposes a first step toward an alternative computational framework in which neural activity in each brain area depends on a combination of feedforward drive (bottom-up from the previous processing stage), feedback drive (top-down context from the next stage), and prior drive (expectation). The relative contributions of feedforward drive, feedback drive, and prior drive are controlled by a handful of state parameters, which I hypothesize correspond to neuromodulators and oscillatory activity. In some states, neural responses are dominated by the feedforward drive and the theory is identical to a conventional feedforward model, thereby preserving all of the desirable features of those models. In other states, the theory is a generative model that constructs a sensory representation from an abstract representation, like memory recall. In still other states, the theory combines prior expectation with sensory input, explores different possible perceptual interpretations of ambiguous sensory inputs, and predicts forward in time. The theory, therefore, offers an empirically testable framework for understanding how the cortex accomplishes inference, exploration, and prediction.

Genetic and environmental influences on cortical mean diffusivity

Magnetic resonance imaging (MRI) has become an important tool in the early detection of age-related and neuropathological brain changes. Recent studies suggest that changes in mean diffusivity (MD) of cortical gray matter derived from diffusion MRI scans may be useful in detecting early effects of Alzheimer's disease (AD), and that these changes may be detected earlier than alterations associated with standard structural MRI measures such as cortical thickness. Thus, due to its potential clinical relevance, we examined the genetic and environmental influences on cortical MD in middle-aged men to provide support for the biological relevance of this measure and to guide future gene association studies. It is not clear whether individual differences in cortical MD reflect neuroanatomical variability similarly detected by other MRI measures, or whether unique features are captured. For instance, variability in cortical MD may reflect morphological variability more commonly measured by cortical thickness. Differences among individuals in cortical MD may also arise from breakdowns in myelinated fibers running through the cortical mantle. Thus, we investigated whether genetic influences on variation in cortical MD are the same or different from those influencing cortical thickness and MD of white matter (WM) subjacent to the cortical ribbon. Univariate twin analyses indicated that cortical MD is heritable in the majority of brain regions; the average of regional heritability estimates ranged from 0.38 in the cingulate cortex to 0.66 in the occipital cortex, consistent with the heritability of other MRI measures of the brain. Trivariate analyses found that, while there was some shared genetic variance between cortical MD and each of the other two measures, this overlap was not complete (i.e., the correlation was statistically different from 1). A significant amount of distinct genetic variance influences inter-individual variability in cortical MD; therefore, this measure could be useful for further investigation in studies of neurodegenerative diseases and gene association studies.

Structure Selective Depth Superresolution for RGB-D Cameras

This paper describes a method for high-quality depth superresolution. The standard formulations of image-guided depth upsampling, using simple joint filtering or quadratic optimization, lead to texture copying and depth bleeding artifacts. These artifacts are caused by inherent discrepancy of structures in data from different sensors. Although there exists some correlation between depth and intensity discontinuities, they are different in distribution and formation. To tackle this problem, we formulate an optimization model using a nonconvex regularizer. A nonlocal affinity established in a high-dimensional feature space is used to offer precisely localized depth boundaries. We show that the proposed method iteratively handles differences in structure between depth and intensity images. This property enables reducing texture copying and depth bleeding artifacts significantly on a variety of range data sets. We also propose a fast alternating direction method of multipliers algorithm to solve our optimization problem. Our solver shows a noticeable speed up compared with the conventional majorize-minimize algorithm. Extensive experiments with synthetic and real-world data sets demonstrate that the proposed method is superior to the existing methods.