Modelling Task-Dependent Eye Guidance to Objects in Pictures

Modeling eye movement in dynamic interactive tasks for maximizing situation awareness based on Markov decision process

For complex dynamic interactive tasks (such as aviating), operators need to continuously extract information from areas of interest (AOIs) through eye movement to maintain high level of situation awareness (SA), as failures of SA may cause task performance degradation, even system accident. Most of the current eye movement models focus on either static tasks (such as image viewing) or simple dynamic tasks (such as video watching), without considering SA. In this study, an eye movement model with the goal of maximizing SA is proposed based on Markov decision process (MDP), which is designed to describe the dynamic eye movement of experienced operators in dynamic interactive tasks. Two top-down factors, expectancy and value, are introduced into this model to represent the update probability and the importance of information in AOIs, respectively. In particular, the model regards sequence of eye fixations to different AOIs as sequential decisions to maximize the SA-related reward (value) in the context of uncertain information update (expectancy). Further, this model was validated with a flight simulation experiment. Results show that the predicted probabilities of fixation on and shift between AOIs are highly correlated (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R = 0.928$$\end{document}R=0.928 and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R = 0.951$$\end{document}R=0.951, respectively) with those of the experiment data.

U-Net Modelling-Based Imaging MAP Score for Tl Stage Nephrectomy: An Exploratory Study

We evaluate the stability of the clinical application of the MAP scoring system based on anatomical features of renal tumour images, explore the relevance of this scoring system to the choice of surgical procedure for patients with limited renal tumours, and investigate the effectiveness of automated segmentation and reconstruction 3D models of renal tumour images based on U-net for interpretative cognitive navigation during laparoscopy Tl stage radical renal tumour cancer surgery. A total of 5 000 kidney tumour images containing manual annotations were applied to the training set, and a stable and efficient full CNN algorithm model oriented to clinical needs was constructed to regionalism and multistructure and to finely automate segmentation of kidney tumour images, output modelling information in STL format, and apply a tablet computer to intraoperatively display the Tl stage kidney tumour model for cognitive navigation. Based on a training sample of MR images from 201 patients with stage Tl renal tumour cancer, an adaptation of the classical U-net allows individual segmentation of important structures such as renal tumours and 3D visualisation to visualise the structural relationships and the extent of tumour invasion at key surgical sites. The preoperative CT and clinical data of 225 patients with limited renal tumours treated surgically at our hospital from August 2011 to August 2012 were retrospectively analysed by three imaging physicians using the MAP scoring system for the total score and the variables R (maximum diameter), E (exogenous/endogenous), N (distance from the renal sinus), A (ventral/dorsal), L (relationship along the longitudinal axis of the kidney), and h (whether in contact with the renal hilum). The score for each variable (contact with the renal hilum) was statistically compared with each other for the three observers. Patients were divided into three groups according to the total score-low, medium, and high-and according to the surgical procedure-radical and partial resection. The correlation between the total score and the score of each variable and the choice of surgical procedure was analysed. The agreement rate of the total score and the score of each variable for all three observers was over 90% (P ≤ 0.001). The map scoring system based on the anatomical features of renal tumour imaging was well stabilized, and the scores were significantly correlated with the surgical approach.

Theoretical perspectives on active sensing

A key component of interacting with the world is how to direct ones' sensors so as to extract task-relevant information - a process referred to as active sensing. In this review, we present a framework for active sensing that forms a closed loop between an ideal observer, that extracts task-relevant information from a sequence of observations, and an ideal planner which specifies the actions that lead to the most informative observations. We discuss active sensing as an approximation to exploration in the wider framework of reinforcement learning, and conversely, discuss several sensory, perceptual, and motor processes as approximations to active sensing. Based on this framework, we introduce a taxonomy of sensing strategies, identify hallmarks of active sensing, and discuss recent advances in formalizing and quantifying active sensing.

Attentive Monitoring of Multiple Video Streams Driven by a Bayesian Foraging Strategy

In this paper, we shall consider the problem of deploying attention to the subsets of the video streams for collating the most relevant data and information of interest related to a given task. We formalize this monitoring problem as a foraging problem. We propose a probabilistic framework to model observer's attentive behavior as the behavior of a forager. The forager, moment to moment, focuses its attention on the most informative stream/camera, detects interesting objects or activities, or switches to a more profitable stream. The approach proposed here is suitable to be exploited for multistream video summarization. Meanwhile, it can serve as a preliminary step for more sophisticated video surveillance, e.g., activity and behavior analysis. Experimental results achieved on the UCR Videoweb Activities Data Set, a publicly available data set, are presented to illustrate the utility of the proposed technique.