Dynamic programming and graph algorithms in computer vision

A dictionary-based graph-cut algorithm for MRI reconstruction

PURPOSE

Compressive sensing (CS)-based image reconstruction methods have proposed random undersampling schemes that produce incoherent, noise-like aliasing artifacts, which are easier to remove. The denoising process is critically assisted by imposing sparsity-enforcing priors. Sparsity is known to be induced if the prior is in the form of the L_p (0 ≤ p ≤ 1) norm. CS methods generally use a convex relaxation of these priors such as the L₁ norm, which may not exploit the full power of CS. An efficient, discrete optimization formulation is proposed, which works not only on arbitrary L_p -norm priors as some non-convex CS methods do, but also on highly non-convex truncated penalty functions, resulting in a specific type of edge-preserving prior. These advanced features make the minimization problem highly non-convex, and thus call for more sophisticated minimization routines.

THEORY AND METHODS

The work combines edge-preserving priors with random undersampling, and solves the resulting optimization using a set of discrete optimization methods called graph cuts. The resulting optimization problem is solved by applying graph cuts iteratively within a dictionary, defined here as an appropriately constructed set of vectors relevant to brain MRI data used here.

RESULTS

Experimental results with in vivo data are presented.

CONCLUSION

The proposed algorithm produces better results than regularized SENSE or standard CS for reconstruction of in vivo data.

Membrane System-Based Improved Neural Networks for Time-Series Anomaly Detection

Anomaly detection in time series has attracted much attention recently and is quite a challenging task. In this paper, a novel deep-learning approach (AL-CNN) that classifies the time series as normal or abnormal with less domain knowledge is proposed. The proposed algorithm combines Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) to effectively model the spatial and temporal information contained in time-series data, the techniques of Squeeze-and-Excitation are applied to implement the feature recalibration. However, the difficulty of selecting multiple parameters and the long training time of a single model make AL-CNN less effective. To alleviate these challenges, a hybrid dynamic membrane system (HM-AL-CNN) is designed which is a new distributed and parallel computing model. We have performed a detailed evaluation of this proposed approach on three well-known benchmarks including the Yahoo S5 datasets. Experiments show that the proposed method possessed a robust and superior performance than the state-of-the-art methods and improved the average on three used indicators significantly.

A Scan-Line Forest Growing-Based Hand Segmentation Framework With Multipriority Vertex Stereo Matching for Wearable Devices

A hand segmentation framework is proposed for 3-D hand gesture interaction for wearable devices. In this framework, all the objects in a scene are regarded as directed trees in a forest, and the problem of the hand segmentation can be converted into finding the target tree (called hand tree) in the forest with proper hand properties including color consistency, space consistency, disparity, and hand shape constraints. The forest grows scan-line by scan-line from high reliable regions to low reliable regions. First, the vertices in each tree are generated through boundary detection. To get rid of the interference of skin-colored objects, an under-segment vertex splitting step is added in. Second, a scan-line-based forest growing method is proposed by constructing the maximum spanning forest (MSF) with the edges weighted by the consistencies of color and space. Meanwhile, the high-order history information of the tree construction propagates along each growing tree to the current scan-line to predict the 3-D location of the growing hand tree, so as to achieve semiglobal optimization. At last, the growing hand tree is trimmed and labeled with the disparity and hand shape constraints to correct the misconnection in the MSF. The vertex's disparity is obtained by our multipriority vertex stereo matching algorithm using the 3-D location prediction to overcome the problem of occlusion and high similarity between fingers. Experimental results demonstrate that by using our method, the hand can be well segmented. The score and accuracy of the hand segmentation result is about 96.3% and 92.9%, respectively.

Feature Augmentation for Learning Confidence Measure in Stereo Matching

Confidence estimation is essential for refining stereo matching results through a post-processing step. This problem has recently been studied using a learning-based approach, which demonstrates a substantial improvement on conventional simple non-learning based methods. However, the formulation of learning-based methods that individually estimates the confidence of each pixel disregards spatial coherency that might exist in the confidence map, thus providing a limited performance under challenging conditions. Our key observation is that the confidence features and resulting confidence maps are smoothly varying in the spatial domain, and highly correlated within the local regions of an image. We present a new approach that imposes spatial consistency on the confidence estimation. Specifically, a set of robust confidence features is extracted from each superpixel decomposed using the Gaussian mixture model, and then these features are concatenated with pixel-level confidence features. The features are then enhanced through adaptive filtering in the feature domain. In addition, the resulting confidence map, estimated using the confidence features with a random regression forest, is further improved through K-nearest neighbor based aggregation scheme on both pixel- and superpixel-level. To validate the proposed confidence estimation scheme, we employ cost modulation or ground control points based optimization in stereo matching. Experimental results demonstrate that the proposed method outperforms state-of-the-art approaches on various benchmarks including challenging outdoor scenes.

Robust Object Tracking With Discrete Graph-Based Multiple Experts

Variations of target appearances due to illumination changes, heavy occlusions, and target deformations are the major factors for tracking drift. In this paper, we show that the tracking drift can be effectively corrected by exploiting the relationship between the current tracker and its historical tracker snapshots. Here, a multi-expert framework is established by the current tracker and its historical trained tracker snapshots. The proposed scheme is formulated into a unified discrete graph optimization framework, whose nodes are modeled by the hypotheses of the multiple experts. Furthermore, an exact solution of the discrete graph exists giving the object state estimation at each time step. With the unary and binary compatibility graph scores defined properly, the proposed framework corrects the tracker drift via selecting the best expert hypothesis, which implicitly analyzes the recent performance of the multi-expert by only evaluating graph scores at the current frame. Three base trackers are integrated into the proposed framework to validate its effectiveness. We first integrate the online SVM on a budget algorithm into the framework with significant improvement. Then, the regression correlation filters with hand-crafted features and deep convolutional neural network features are introduced, respectively, to further boost the tracking performance. The proposed three trackers are extensively evaluated on three data sets: TB-50, TB-100, and VOT2015. The experimental results demonstrate the excellent performance of the proposed approaches against the state-of-the-art methods.

Parameter identifiability-based optimal observation remedy for biological networks

Background

To systematically understand the interactions between numerous biological components, a variety of biological networks on different levels and scales have been constructed and made available in public databases or knowledge repositories. Graphical models such as structural equation models have long been used to describe biological networks for various quantitative analysis tasks, especially key biological parameter estimation. However, limited by resources or technical capacities, partial observation is a common problem in experimental observations of biological networks, and it thus becomes an important problem how to select unobserved nodes for additional measurements such that all unknown model parameters become identifiable. To the best knowledge of our authors, a solution to this problem does not exist until this study.

Results

The identifiability-based observation problem for biological networks is mathematically formulated for the first time based on linear recursive structural equation models, and then a dynamic programming strategy is developed to obtain the optimal observation strategies. The efficiency of the dynamic programming algorithm is achieved by avoiding both symbolic computation and matrix operations as used in other studies. We also provided necessary theoretical justifications to the proposed method. Finally, we verified the algorithm using synthetic network structures and illustrated the application of the proposed method in practice using a real biological network related to influenza A virus infection.

Conclusions

The proposed approach is the first solution to the structural identifiability-based optimal observation remedy problem. It is applicable to an arbitrary directed acyclic biological network (recursive SEMs) without bidirectional edges, and it is a computerizable method. Observation remedy is an important issue in experiment design for biological networks, and we believe that this study provides a solid basis for dealing with more challenging design issues (e.g., feedback loops, dynamic or nonlinear networks) in the future. We implemented our method in R, which is freely accessible at https://github.com/Hongyu-Miao/SIOOR.

Electronic supplementary material

The online version of this article (doi:10.1186/s12918-017-0432-2) contains supplementary material, which is available to authorized users.

Dynamic Programming Based Segmentation in Biomedical Imaging

Many applications in biomedical imaging have a demand on automatic detection of lines, contours, or boundaries of bones, organs, vessels, and cells. Aim is to support expert decisions in interactive applications or to include it as part of a processing pipeline for automatic image analysis. Biomedical images often suffer from noisy data and fuzzy edges. Therefore, there is a need for robust methods for contour and line detection. Dynamic programming is a popular technique that satisfies these requirements in many ways. This work gives a brief overview over approaches and applications that utilize dynamic programming to solve problems in the challenging field of biomedical imaging.

Graph-based segmentation of abnormal nuclei in cervical cytology

A general method is reported for improving the segmentation of abnormal cell nuclei in cervical cytology images. In automation-assisted reading of cervical cytology, one of the essential steps is the segmentation of nuclei. Despite some progress, there is a need to improve the sensitivity, particularly the segmentation of abnormal nuclei. Our method starts with pre-segmenting the nucleus to define the coarse center and size of nucleus, which is used to construct a graph by image unfolding that maps ellipse-like border in the Cartesian coordinate system to lines in the polar coordinate system. The cost function jointly reflects properties of nucleus border and nucleus region. The prior constraints regarding the context of nucleus-cytoplasm position are utilized to modify the local cost functions. The globally optimal path in the constructed graph is then identified by dynamic programming with an iterative approach ensuring an optimal closed contour. Validation of our method was performed on abnormal nuclei from two cervical cell image datasets, Herlev and H&E stained manual liquid-based cytology (HEMLBC). Compared with five state-of-the-art approaches, our graph-search based method shows superior performance.

Detection of Multiple Innervation Zones from Multi-Channel Surface EMG Recordings with Low Signal-to-Noise Ratio Using Graph-Cut Segmentation

Knowledge of the location of muscle Innervation Zones (IZs) is important in many applications, e.g. for minimizing the quantity of injected botulinum toxin for the treatment of spasticity or for deciding on the type of episiotomy during child delivery. Surface EMG (sEMG) can be noninvasively recorded to assess physiological and morphological characteristics of contracting muscles. However, it is not often possible to record signals of high quality. Moreover, muscles could have multiple IZs, which should all be identified. We designed a fully-automatic algorithm based on the enhanced image Graph-Cut segmentation and morphological image processing methods to identify up to five IZs in 60-ms intervals of very-low to moderate quality sEMG signal detected with multi-channel electrodes (20 bipolar channels with Inter Electrode Distance (IED) of 5 mm). An anisotropic multilayered cylinder model was used to simulate 750 sEMG signals with signal-to-noise ratio ranging from -5 to 15 dB (using Gaussian noise) and in each 60-ms signal frame, 1 to 5 IZs were included. The micro- and macro- averaged performance indices were then reported for the proposed IZ detection algorithm. In the micro-averaging procedure, the number of True Positives, False Positives and False Negatives in each frame were summed up to generate cumulative measures. In the macro-averaging, on the other hand, precision and recall were calculated for each frame and their averages are used to determine F1-score. Overall, the micro (macro)-averaged sensitivity, precision and F1-score of the algorithm for IZ channel identification were 82.7% (87.5%), 92.9% (94.0%) and 87.5% (90.6%), respectively. For the correctly identified IZ locations, the average bias error was of 0.02±0.10 IED ratio. Also, the average absolute conduction velocity estimation error was 0.41±0.40 m/s for such frames. The sensitivity analysis including increasing IED and reducing interpolation coefficient for time samples was performed. Meanwhile, the effect of adding power-line interference and using other image interpolation methods on the deterioration of the performance of the proposed algorithm was investigated. The average running time of the proposed algorithm on each 60-ms sEMG frame was 25.5±8.9 (s) on an Intel dual-core 1.83 GHz CPU with 2 GB of RAM. The proposed algorithm correctly and precisely identified multiple IZs in each signal epoch in a wide range of signal quality and is thus a promising new offline tool for electrophysiological studies.

Lung diaphragm tracking in CBCT images using spatio-temporal MRF

In EBRT in order to monitor the intra fraction motion of thoracic and abdominal tumors, one of the standard approaches is to use the lung diaphragm apex as an internal marker. However, tracking the position of the apex from image based observations is a challenging problem, as it undergoes both position and shape variation. The purpose of this paper is to propose an alternative method for tracking the ipsi-lateral hemidiaphragm apex (IHDA) position on Cone Beam Computed Tomography (CBCT) projection images. A hierarchical method is proposed to track the IHDA position across the frames. The diaphragm state is modeled as a spatio-temporal Markov Random Field (MRF). The likelihood function is derived from the votes based on 4D-Hough space. The optimal state of the diaphragm is obtained by solving the associated energy minimization problem using graph-cuts. A heterogeneous GPU implementation is provided for the method using CUDA framework and the performance is compared with that of CPU implementation. The method was tested using 15 clinical CBCT images. The results demonstrate that the MRF formulation outperforms the full search method in terms of accuracy. The GPU based heterogeneous implementation of the proposed algorithm takes about 25s, which is 16% improvement over the existing benchmark. The proposed MRF formulation considers all the possible combinations from the 4D-Hough space and therefore results in better tracking accuracy. The GPU based implementation exploits the inherent parallelism in our algorithm to accelerate the performance thereby increasing the viability of the approach for clinical use.

Simultaneous Registration of Location and Orientation in Intravascular Ultrasound Pullbacks Pairs Via 3D Graph-Based Optimization

A novel method is reported for simultaneous registration of location (axial direction) and orientation (circumferential direction) of two intravascular ultrasound (IVUS) pullbacks of the same vessel taken at different times. Monitoring plaque progression or regression (e.g., during lipid treatment) is of high clinical relevance. Our method uses a 3D graph optimization approach, in which the cost function jointly reflects similarity of plaque morphology and plaque/perivascular image appearance. Graph arcs incorporate prior information about temporal correspondence of the two IVUS sequences and limited angular twisting between consecutive IVUS images. Additionally, our approach automatically identifies starting and ending frame pairs in the two IVUS pullbacks. Validation of our method was performed in 29 pairs of IVUS baseline/follow-up pullback sequences consisting of 8 622 IVUS image frames in total. In comparison to manual registration by three experts, the average location and orientation registration errors ranged from 0.72 mm to 0.79 mm and from 7.3(°) to 9.3(°), respectively, all close to the inter-observer variability with no difference being statistically significant (p = NS). Rotation angles determined by our automated approach and expert observers showed high correlation (r(2) of 0.97 to 0.98) and agreed closely (mutual bias between the automated method and expert observers ranged from -1.57(°) to 0.15(°)). Compared with state-of-the-art approaches, the new method offers lower errors in both location and orientation registration. Our method offers highly automated and accurate IVUS pullback registration and can be employed in IVUS-based studies of coronary disease progression, enabling more focal studies of coronary plaque development and transition of vulnerability.

A VOTING SCHEME FOR PARTIAL OBJECT EXTRACTION UNDER CLUTTERED ENVIRONMENT

Shape extraction aims to detect and localize objects via the shape information. The paper presents a novel voting scheme that can extract partially occluded objects under cluttered environments using a single shape. It works by jointly figuring out the boundaries and resolving the geometric configurations. To model the missing part lead by occlusion, we discretize the shape template into a set of its subpart, named portions. Our representation of shape template is through a set of portion together with their interconnections. Instead of forming a fully connected network, our interconnections make the portions consistent with the chain along the boundary of shape template. Based on the representation, we formulate an auto-locked objective function that contains both the unary and pairwise terms and balances the effects of missing parts. Min-sum voting scheme with strategy driven by bottom–up information is then proposed to minimize the objective function. Conducted experiments show that proposed algorithm is promising for shape extraction with occlusion and noisy backgrounds and allows the non-rigid deformations.

Adaptable image cuts for motility inspection using WCE

The Wireless Capsule Endoscopy (WCE) technology allows the visualization of the whole small intestine tract. Since the capsule is freely moving, mainly by the means of peristalsis, the data acquired during the study gives a lot of information about the intestinal motility. However, due to: (1) huge amount of frames, (2) complex intestinal scene appearance and (3) intestinal dynamics that make difficult the visualization of the small intestine physiological phenomena, the analysis of the WCE data requires computer-aided systems to speed up the analysis. In this paper, we propose an efficient algorithm for building a novel representation of the WCE video data, optimal for motility analysis and inspection. The algorithm transforms the 3D video data into 2D longitudinal view by choosing the most informative, from the intestinal motility point of view, part of each frame. This step maximizes the lumen visibility in its longitudinal extension. The task of finding "the best longitudinal view" has been defined as a cost function optimization problem which global minimum is obtained by using Dynamic Programming. Validation on both synthetic data and WCE data shows that the adaptive longitudinal view is a good alternative to the traditional motility analysis done by video analysis. The proposed novel data representation a new, holistic insight into the small intestine motility, allowing to easily define and analyze motility events that are difficult to spot by analyzing WCE video. Moreover, the visual inspection of small intestine motility is 4 times faster then by means of video skimming of the WCE.

Rotation-invariant nonrigid point set matching in cluttered scenes

This paper addresses the problem of rotation-invariant nonrigid point set matching. The shape context (SC) feature descriptor is used because of its strong discriminative nature, whereas edges in the graphs constructed by point sets are used to determine the orientations of SCs. Similar to lengths or directions, oriented SCs constructed this way can be regarded as attributes of edges. By matching edges between two point sets, rotation invariance is achieved. Two novel ways of constructing graphs on a model point set are proposed, aiming at making the orientations of SCs as robust to disturbances as possible. The structures of these graphs facilitate the use of dynamic programming (DP) for optimization. The strong discriminative nature of SC, the special structure of the model graphs, and the global optimality of DP make our methods robust to various types of disturbances, particularly clutters. The extensive experiments on both synthetic and real data validated the robustness of the proposed methods to various types of disturbances. They can robustly detect the desired shapes in complex and highly cluttered scenes.