Democratic integration: self-organized integration of adaptive cues

Online Object Tracking via Novel Adaptive Multicue Based Particle Filter Framework for Video Surveillance

Multicue based object tracking frameworks have been extensively explored due to their numerous applications in the field of computer vision. However, the online adaptive fusion of multicue under scale and illumination variations, partial or full occlusion, background clutters and object deformation remains an open challenge problem. In order to address this, we propose an online visual tracking algorithm using adaptive integration of multicue in a particle filter framework. The particle level fusion process is modelled as Shafer’s model with a power set defined over two focal elements. Partial conflicting masses and conjunctive consensus among three cues are estimated for each evaluated particle. Partial conflicts among cues are redistributed using Dezert-Smarandache Theory (DSmT) based proportional conflict redistribution rules (PCR-6). Additionally, context sensitive transductive cues reliabilities are used for discounting the particle likelihoods for quick adaptation of tracker. In the proposed model, automatic boosting of good particles and suppression of low performing particles not only improves resampling process but also enhances tracker accuracy. Experimental validation over benchmarked video sequences reveals that the proposed multicue tracking framework outperforms state-of-the-art trackers under various dynamic environmental challenges.

Electroencephalographic Motor Imagery Brain Connectivity Analysis for BCI: A Review

Recent research has reached a consensus on the feasibility of motor imagery brain-computer interface (MI-BCI) for different applications, especially in stroke rehabilitation. Most MI-BCI systems rely on temporal, spectral, and spatial features of single channels to distinguish different MI patterns. However, no successful communication has been established for a completely locked-in subject. To provide more useful and informative features, it has been recommended to take into account the relationships among electroencephalographic (EEG) sensor/source signals in the form of brain connectivity as an efficient tool of neuroscience. In this review, we briefly report the challenges and limitations of conventional MI-BCIs. Brain connectivity analysis, particularly functional and effective, has been described as one of the most promising approaches for improving MI-BCI performance. An extensive literature on EEG-based MI brain connectivity analysis of healthy subjects is reviewed. We subsequently discuss the brain connectomes during left and right hand, feet, and tongue MI movements. Moreover, key components involved in brain connectivity analysis that considerably affect the results are explained. Finally, possible technical shortcomings that may have influenced the results in previous research are addressed and suggestions are provided.

Optimal Appearance Model for Visual Tracking

Many studies argue that integrating multiple cues in an adaptive way increases tracking performance. However, what is the definition of adaptiveness and how to realize it remains an open issue. On the premise that the model with optimal discriminative ability is also optimal for tracking the target, this work realizes adaptiveness and robustness through the optimization of multi-cue integration models. Specifically, based on prior knowledge and current observation, a set of discrete samples are generated to approximate the foreground and background distribution. With the goal of optimizing the classification margin, an objective function is defined, and the appearance model is optimized by introducing optimization algorithms. The proposed optimized appearance model framework is embedded into a particle filter for a field test, and it is demonstrated to be robust against various kinds of complex tracking conditions. This model is general and can be easily extended to other parameterized multi-cue models.

EEG oscillatory states: universality, uniqueness and specificity across healthy-normal, altered and pathological brain conditions

For the first time the dynamic repertoires and oscillatory types of local EEG states in 13 diverse conditions (examined over 9 studies) that covered healthy-normal, altered and pathological brain states were quantified within the same methodological and conceptual framework. EEG oscillatory states were assessed by the probability-classification analysis of short-term EEG spectral patterns. The results demonstrated that brain activity consists of a limited repertoire of local EEG states in any of the examined conditions. The size of the state repertoires was associated with changes in cognition and vigilance or neuropsychopathologic conditions. Additionally universal, optional and unique EEG states across 13 diverse conditions were observed. It was demonstrated also that EEG oscillations which constituted EEG states were characteristic for different groups of conditions in accordance to oscillations' functional significance. The results suggested that (a) there is a limit in the number of local states available to the cortex and many ways in which these local states can rearrange themselves and still produce the same global state and (b) EEG individuality is determined by varying proportions of universal, optional and unique oscillatory states. The results enriched our understanding about dynamic microstructure of EEG-signal.

EEG oscillatory states: universality, uniqueness and specificity across healthy-normal, altered and pathological brain conditions

For the first time the dynamic repertoires and oscillatory types of local EEG states in 13 diverse conditions (examined over 9 studies) that covered healthy-normal, altered and pathological brain states were quantified within the same methodological and conceptual framework. EEG oscillatory states were assessed by the probability-classification analysis of short-term EEG spectral patterns. The results demonstrated that brain activity consists of a limited repertoire of local EEG states in any of the examined conditions. The size of the state repertoires was associated with changes in cognition and vigilance or neuropsychopathologic conditions. Additionally universal, optional and unique EEG states across 13 diverse conditions were observed. It was demonstrated also that EEG oscillations which constituted EEG states were characteristic for different groups of conditions in accordance to oscillations' functional significance. The results suggested that (a) there is a limit in the number of local states available to the cortex and many ways in which these local states can rearrange themselves and still produce the same global state and (b) EEG individuality is determined by varying proportions of universal, optional and unique oscillatory states. The results enriched our understanding about dynamic microstructure of EEG-signal.

PHENOMENOLOGICAL ARCHITECTURE OF A MIND AND OPERATIONAL ARCHITECTONICS OF THE BRAIN: THE UNIFIED METASTABLE CONTINUUM

In our contribution we will observe phenomenal architecture of a mind and operational architectonics of the brain and will show their intimate connectedness within a single integrated metastable continuum. The notion of operation of different complexity is the fundamental and central one in bridging the gap between brain and mind: it is precisely by means of this notion that it is possible to identify what at the same time belongs to the phenomenal conscious level and to the neurophysiological level of brain activity organization, and what mediates between them. Implications for linguistic semantics, self-organized distributed computing algorithms, artificial machine consciousness, and diagnosis of dynamic brain diseases will be discussed briefly.

Tracking by third-order tensor representation

This paper proposes a robust tracking algorithm by third-order tensor representation and adaptive appearance modeling. In this method, the target in each video frame is represented by a third-order tensor. This representation preserves the spatial correlation inside the target region and can integrate multiple appearance cues for target description. Based on this representation, a multilinear subspace is learned online to model the target appearance variations during tracking. Compared to other methods, our approach can detect local spatial structure in the target tensor space and fuse information from different feature spaces. Therefore, the learned appearance model is more discriminative when there are significant appearance variations of the target or when the background gets cluttered. Applying the multilinear algebra, our appearance model can efficiently be learned and updated online, without causing high-dimensional data-learning problems. Then, tracking is implemented in the Bayesian inference framework, where a likelihood model is defined to measure the similarity between a test sample and the learned appearance model, and a particle filter is used to recursively estimate the target state over time. Theoretic analysis and experiments compared with other state-of-the-art methods demonstrate the effectiveness of the proposed approach.

Automatic robust medical image registration using a new democratic vector optimization approach with multiple measures

The registration of various data is a challenging task in medical image processing and a highly frequented area of research. Most of the published approaches tend to fail sporadically on different data sets. This happens due to two major problems. First, local optimization strategies induce a high risk when optimizing nonconvex functions. Second, similarity measures might fail if they are not suitable for the data. Thus, researchers began to combine multiple measures by weighted sums. In this paper, we show severe limitations of such summation approaches. We address both issues by a gradient-based vector optimization algorithm that uses multiple similarity measures. It gathers context information from the iteration process to detect and suppress failing measures. The new approach is evaluated by experiments from the field of 2D-3D registration. Besides its generic character with respect to arbitrary data, the main benefit is a highly robust iteration behavior, where even very poor initial guesses of the transform result in good solutions.

Learning an intrinsic-variable preserving manifold for dynamic visual tracking

Manifold learning is a hot topic in the field of computer science, particularly since nonlinear dimensionality reduction based on manifold learning was proposed in Science in 2000. The work has achieved great success. The main purpose of current manifold-learning approaches is to search for independent intrinsic variables underlying high dimensional inputs which lie on a low dimensional manifold. In this paper, a new manifold is built up in the training step of the process, on which the input training samples are set to be close to each other if the values of their intrinsic variables are close to each other. Then, the process of dimensionality reduction is transformed into a procedure of preserving the continuity of the intrinsic variables. By utilizing the new manifold, the dynamic tracking of a human who can move and rotate freely is achieved. From the theoretical point of view, it is the first approach to transfer the manifold-learning framework to dynamic tracking. From the application point of view, a new and low dimensional feature for visual tracking is obtained and successfully applied to the real-time tracking of a free-moving object from a dynamic vision system. Experimental results from a dynamic tracking system which is mounted on a dynamic robot validate the effectiveness of the new algorithm.

The coordination dynamics of economic decision making: a multilevel approach to social neuroeconomics

The basic reciprocity between individual parts and collective organization constitutes a key scientific question spanning the biological and social sciences. Such reciprocity is accompanied by the absence of direct linkages between levels of description giving rise to what is often referred to as the aggregation or nonequivalence problem between levels of analysis. This issue is encountered both in neuroscience and economics. So far, in spite of being identified and extensively discussed in various (other) scientific fields, the problem of understanding the nature of the interactions and coordination dynamics between individual (neuron approximately agent) and collective (neural networks approximately population of humans) behaviors has received little, if any attention in the growing field of neuroeconomics. The present contribution focuses on bringing a theoretical perspective to the interpretation of experiments recently published in this field and addressing how the concepts and methods of coordination dynamics may impact future research. First, we very briefly discuss the links between biology and economics. Second, we address the nonequivalence problem between different levels of analysis and the concept of reciprocal causality. Third, neuroeconomics studies that investigate the neural underpinnings of social decision making in the context of two economic games (trust and ultimatum) are reviewed to highlight issues that arise when experimental results exist at multiple scales of observation and description. Finally, in the last two sections, we discuss how coordination dynamics might provide novel routes to studying and modelling the relation between brain activity and decision making.

Figure-Ground Separation by Cue Integration

This letter presents an improved cue integration approach to reliably separate coherent moving objects from their background scene in video sequences. The proposed method uses a probabilistic framework to unify bottom-up and top-down cues in a parallel, “democratic” fashion. The algorithm makes use of a modified Bayes rule where each pixel's posterior probabilities of figure or ground layer assignment are derived from likelihood models of three bottom-up cues and a prior model provided by a top-down cue. Each cue is treated as independent evidence for figure-ground separation. They compete with and complement each other dynamically by adjusting relative weights from frame to frame according to cue quality measured against the overall integration. At the same time, the likelihood or prior models of individual cues adapt toward the integrated result. These mechanisms enable the system to organize under the influence of visual scene structure without manual intervention. A novel contribution here is the incorporation of a top-down cue. It improves the system's robustness and accuracy and helps handle difficult and ambiguous situations, such as abrupt lighting changes or occlusion among multiple objects. Results on various video sequences are demonstrated and discussed. (Video demos are available at http://organic.usc.edu:8376/∼tangx/neco/index.html .)

Dependent multiple cue integration for robust tracking

We propose a new technique for fusing multiple cues to robustly segment an object from its background in video sequences that suffer from abrupt changes of both illumination and position of the target. Robustness is achieved by the integration of appearance and geometric object features and by their estimation using Bayesian filters, such as Kalman or particle filters. In particular, each filter estimates the state of a specific object feature, conditionally dependent on another feature estimated by a distinct filter. This dependence provides improved target representations, permitting to segment it out from the background even in non-stationary sequences. Considering that the procedure of the Bayesian filters may be described by a "hypotheses generation--hypotheses correction" strategy, the major novelty of our methodology compared to previous approaches is that the mutual dependence between filters is considered during the feature observation, i.e, into the "hypotheses correction" stage,instead of considering it when generating the hypotheses. This proves to be much more effective in terms of accuracy and reliability. The proposed method is analytically justified and applied to develop a robust tracking system that adapts online and simultaneously the color space where the image points are represented, the color distributions, the contour of the object and its bounding box. Results with synthetic data and real video sequences demonstrate the robustness and versatility of our method.

Driven from distraction: How infants respond to parents’ attempts to elicit and re-direct their attention

This experiment examined how parents' verbal and non-verbal behavioral cues cause infants to shift and share attention within environments where many objects compete for infants' attention. Fifteen- and 21-month-old infants played with toys while their parent periodically shifted attention to a distal object within a larger array. Parents' attention-shifts were indicated by a change in direction of gaze, a pointing gesture, and/or verbalizations. Verbalizations were either attention-eliciting or attention-directing. In some trials parents covered their eyes to occlude line-of-gaze. Both ages seldom followed simple gaze shifts, but frequently followed gaze with-points or gaze-with-directing verbalizations. Attention-eliciting verbalizations increased infants' looks to the parent. Gaze occlusion reduced infants' responses to directing verbalizations. Responses to eliciting verbalizations increased with age. Infant receptive vocabulary did not predict attention-sharing, even when parents named objects (i.e., directing verbalizations). Implications for development of attention-sharing, language and understanding of visual attention are discussed.

Estimation of contour motion and deformation for nonrigid object tracking

We present an algorithm for nonrigid contour tracking in heavily cluttered background scenes. Based on the properties of nonrigid contour movements, a sequential framework for estimating contour motion and deformation is proposed. We solve the nonrigid contour tracking problem by decomposing it into three subproblems: motion estimation, deformation estimation, and shape regulation. First, we employ a particle filter to estimate the global motion parameters of the affine transform between successive frames. Then we generate a probabilistic deformation map to deform the contour. To improve robustness, multiple cues are used for deformation probability estimation. Finally, we use a shape prior model to constrain the deformed contour. This enables us to retrieve the occluded parts of the contours and accurately track them while allowing shape changes specific to the given object types. Our experiments show that the proposed algorithm significantly improves the tracker performance.

Quality-Based Fusion of Multiple Video Sensors for Video Surveillance

In this correspondence, we address the problem of fusing data for object tracking for video surveillance. The fusion process is dynamically regulated to take into account the performance of the sensors in detecting and tracking the targets. This is performed through a function that adjusts the measurement error covariance associated with the position information of each target according to the quality of its segmentation. In this manner, localization errors due to incorrect segmentation of the blobs are reduced thus improving tracking accuracy. Experimental results on video sequences of outdoor environments show the effectiveness of the proposed approach.

Timing in cognition and EEG brain dynamics: discreteness versus continuity

This article provides an overview of recent developments in solving the timing problem (discreteness vs. continuity) in cognitive neuroscience. Both theoretical and empirical studies have been considered, with an emphasis on the framework of operational architectonics (OA) of brain functioning (Fingelkurts and Fingelkurts in Brain Mind 2:291-29, 2001; Neurosci Biobehav Rev 28:827-836, 2005). This framework explores the temporal structure of information flow and interarea interactions within the network of functional neuronal populations by examining topographic sharp transition processes in the scalp EEG, on the millisecond scale. We conclude, based on the OA framework, that brain functioning is best conceptualized in terms of continuity-discreteness unity which is also the characteristic property of cognition. At the end we emphasize where one might productively proceed for the future research.

Functional connectivity in the brain--is it an elusive concept?

Even though functional brain connectivity is an influential concept in modern cognitive neuroscience, it is a very controversial notion. This is why further theoretical and methodological clarification are needed to help define precisely what is meant by functional connectivity and to help frame-associated issues. In this review we present the neurophysiological concept of functional connectivity, which utilizes in a plausible manner the notion of neural assemblies, as well as local and large-scale levels of description. Here functional connectivity is the mechanism for the coordination of activity between different neural assemblies in order to achieve a complex cognitive task or perceptual process. Our theoretical and empirical findings offer new insights into possible implications of the concept of functional connectivity for cognitive neuroscience.

Making complexity simpler: multivariability and metastability in the brain

This article provides a retrospective, current, and prospective overview on developments in brain research and neuroscience. Both theoretical and empirical studies are considered, with emphasis in the concept of multivariability and metastability in the brain. In this new view on the human brain, the potential multivariability of the neuronal networks appears to be far from continuous in time, but confined by the dynamics of short-term local and global metastable brain states. The article closes by suggesting some of the implications of this view in future multidisciplinary brain research.

Local and remote functional connectivity of neocortex under the inhibition influence

The current paper focuses on a relatively new and promising area of the study of EEG transformations during brain information processing based on the reduction of the signal to the discrete quasi-stationary segment sequences which may reflect individual brain microstates or discrete operations. In this framework, the complex brain functions require integration of several operations throughout the whole neocortex. However, the role of inhibitory brain systems in such processes is still unsettled. The effects of a single dose (30 microg/kg) of lorazepam on the operational activity of neuronal populations and on the temporal binding between them were examined in a double-blind randomized crossover placebo-controlled study with eight healthy volunteers. EEG measures at 20 channels were evaluated on two occasions: (1) eyes closed, (2) eyes open. In short, we conducted a two-by-two factorial study where one factor manipulated GABAergic neurotransmission (lorazepam vs. placebo), and the other factor was simply brain state (eyes closed vs. eyes opened). We were primarily interested in the main effect of lorazepam. In the present study, a connection between the mesoscopic level, described by the local functional processes (neuronal assemblies or populations) and the macroscopic level, described as a sequence of metastable brain states (remote functionally synchronized neuronal populations) was established. The role of inhibitory brain systems facilitated by lorazepam in the operational dynamics of neuronal populations and in the process of EEG structural synchrony (SS) (topological peculiarities) was addressed for the first time. It was shown that GABA signaling reorganized the dynamics of local neuronal populations and the remote functional connectivity between them.

Three-dimensional tracking of multiple skin-colored regions by a moving stereoscopic system

A system that performs three-dimensional (3D) tracking of multiple skin-colored regions (SCRs) in images acquired by a calibrated, possibly moving stereoscopic rig is described. The system consists of a collection of techniques that permit the modeling and detection of SCRs, the determination of their temporal association in monocular image sequences, the establishment of their correspondence between stereo images, and the extraction of their 3D positions in a world-centered coordinate system. The development of these techniques has been motivated by the need for robust, near-real-time tracking performance. SCRs are detected by use of a Bayesian classifier that is trained with the aid of a novel technique. More specifically, the classifier is bootstrapped with a small set of training data. Then, as new images are being processed, an iterative training procedure is employed to refine the classifier. Furthermore, a technique is proposed to enable the classifier to cope with changes in illumination. Tracking of SCRs in time as well as matching of SCRs in the images of the employed stereo rig is performed through computationally inexpensive and robust techniques. One of the main characteristics of the skin-colored region tracker (SCRT) instrument is its ability to report the 3D positions of SCRs in a world-centered coordinate system by employing a possibly moving stereo rig with independently verging CCD cameras. The system operates on images of dimensions 640 x 480 pixels at a rate of 13 Hz on a conventional Pentium 4 processor at 1.8 GHz. Representative experimental results from the application of the SCRT to image sequences are also provided.

Fast temporal dynamics of visual cue integration

The integration of information from different sensors, cues, or modalities lies at the very heart of perception. We are studying adaptive phenomena in visual cue integration. To this end, we have designed a visual tracking task, where subjects track a target object among distractors and try to identify the target after an occlusion. Objects are defined by three different attributes (color, shape, size) which change randomly within a single trial. When the attributes differ in their reliability (two change frequently, one is stable), our results show that subjects dynamically adapt their processing. The results are consistent with the hypothesis that subjects rapidly re-weight the information provided by the different cues by emphasizing the information from the stable cue. This effect seems to be automatic, ie not requiring subjects' awareness of the differential reliabilities of the cues. The hypothesized re-weighting seems to take place in about 1 s. Our results suggest that cue integration can exhibit adaptive phenomena on a very fast time scale. We propose a probabilistic model with temporal dynamics that accounts for the observed effect.