Real-time object tracking based on scale-invariant features employing bio-inspired hardware

High-throughput automated methods for classical and operant conditioning of Drosophila larvae

Learning which stimuli (classical conditioning) or which actions (operant conditioning) predict rewards or punishments can improve chances of survival. However, the circuit mechanisms that underlie distinct types of associative learning are still not fully understood. Automated, high-throughput paradigms for studying different types of associative learning, combined with manipulation of specific neurons in freely behaving animals, can help advance this field. The Drosophila melanogaster larva is a tractable model system for studying the circuit basis of behaviour, but many forms of associative learning have not yet been demonstrated in this animal. Here, we developed a high-throughput (i.e. multi-larva) training system that combines real-time behaviour detection of freely moving larvae with targeted opto- and thermogenetic stimulation of tracked animals. Both stimuli are controlled in either open- or closed-loop, and delivered with high temporal and spatial precision. Using this tracker, we show for the first time that Drosophila larvae can perform classical conditioning with no overlap between sensory stimuli (i.e. trace conditioning). We also demonstrate that larvae are capable of operant conditioning by inducing a bend direction preference through optogenetic activation of reward-encoding serotonergic neurons. Our results extend the known associative learning capacities of Drosophila larvae. Our automated training rig will facilitate the study of many different forms of associative learning and the identification of the neural circuits that underpin them.

Role of Noise in Spontaneous Activity of Networks of Neurons on Patterned Silicon Emulated by Noise–activated CMOS Neural Nanoelectronic Circuits

Background noise in biological cortical microcircuits constitutes a powerful resource to assess their computational tasks, including, for instance, the synchronization of spiking activity, the enhancement of the speed of information transmission, and the minimization of the corruption of signals. We explore the correlation of spontaneous firing activity of ≈ 100 biological neurons adhering to engineered scaffolds by governing the number of functionalized patterned connection pathways among groups of neurons. We then emulate the biological system by a series of noise-activated silicon neural network simulations. We show that by suitably tuning both the amplitude of noise and the number of synapses between the silicon neurons, the same controlled correlation of the biological population is achieved. Our results extend to a realistic silicon nanoelectronics neuron design using noise injection to be exploited in artificial spiking neural networks such as liquid state machines and recurrent neural networks for stochastic computation.

Adaptive ensemble perception tracking

Recently, tracking models based on bounding box regression (such as region proposal networks), built on the Siamese network, have attracted much attention. Despite their promising performance, these trackers are less effective in perceiving the target information in the following two aspects. First, existing regression models cannot take a global view of a large-scale target since the effective receptive field of a neuron is too small to cover the target with a large scale. Second, the neurons with a fixed receptive field (RF) size in these models cannot adapt to the scale and aspect ratio changes of the target. In this paper, we propose an adaptive ensemble perception tracking framework to address these issues. Specifically, we first construct a per-pixel prediction model, which predicts the target state at each pixel of the correlated feature. On top of the per-pixel prediction model, we then develop a confidence-guided ensemble prediction mechanism. The ensemble mechanism adaptively fuses the predictions of multiple pixels with the guidance of confidence maps, which enlarges the perception range and enhances the adaptive perception ability at the object-level. In addition, we introduce a receptive field adaption model to enhance the adaptive perception ability at the neuron-level, which adjusts the RF by adaptively integrating the features with different RFs. Extensive experimental results on the VOT2018, VOT2016, UAV123, LaSOT, and TC128 datasets demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods in terms of accuracy and speed.

Learning dual-margin model for visual tracking

Existing trackers usually exploit robust features or online updating mechanisms to deal with target variations which is a key challenge in visual tracking. However, the features being robust to variations remain little spatial information, and existing online updating methods are prone to overfitting. In this paper, we propose a dual-margin model for robust and accurate visual tracking. The dual-margin model comprises an intra-object margin between different target appearances and an inter-object margin between the target and the background. The proposed method is able to not only distinguish the target from the background but also perceive the target changes, which tracks target appearance changing and facilitates accurate target state estimation. In addition, to exploit rich off-line video data and learn general rules of target appearance variations, we train the dual-margin model on a large off-line video dataset. We perform tracking under a Siamese framework using the constructed appearance set as templates. The proposed method achieves accurate and robust tracking performance on five public datasets while running in real-time. The favorable performance against the state-of-the-art methods demonstrates the effectiveness of the proposed algorithm.