The schema system

Towards a Computational Comparative Neuroprimatology: Framing the language-ready brain

We make the case for developing a Computational Comparative Neuroprimatology to inform the analysis of the function and evolution of the human brain. First, we update the mirror system hypothesis on the evolution of the language-ready brain by (i) modeling action and action recognition and opportunistic scheduling of macaque brains to hypothesize the nature of the last common ancestor of macaque and human (LCA-m); and then we (ii) introduce dynamic brain modeling to show how apes could acquire gesture through ontogenetic ritualization, hypothesizing the nature of evolution from LCA-m to the last common ancestor of chimpanzee and human (LCA-c). We then (iii) hypothesize the role of imitation, pantomime, protosign and protospeech in biological and cultural evolution from LCA-c to Homo sapiens with a language-ready brain. Second, we suggest how cultural evolution in Homo sapiens led from protolanguages to full languages with grammar and compositional semantics. Third, we assess the similarities and differences between the dorsal and ventral streams in audition and vision as the basis for presenting and comparing two models of language processing in the human brain: A model of (i) the auditory dorsal and ventral streams in sentence comprehension; and (ii) the visual dorsal and ventral streams in defining "what language is about" in both production and perception of utterances related to visual scenes provide the basis for (iii) a first step towards a synthesis and a look at challenges for further research.

KNOWLEDGE-BASED SHAPE-FROM-SHADING

In this paper, we study the problem of recovering approximate shape from the shading of a three-dimensional object in a single image when knowledge about the object is available. The application of knowledge-based methods to low-level image processing tasks will help overcome problems that arise from processing images using a pixel-based approach. Shape-from-shading has generally been approached by precognitive vision methods where a standard operator is applied to the image based on assumptions about the imaging process and generic properties of what appears. This paper explores some advantages of applying knowledge and hypotheses about what appears in the image. The knowledge and hypotheses used here come from domain knowledge and edge-matching. Specifically, we are able to find solutions to some problems that cannot be solved by other methods and gain advantages in terms of computation speed over similar approaches. Further, we can fully automate the derivation of the approximate shape of an object. This paper demonstrates the efficacy of using knowledge in the basic operation of an early vision operator, and so introduces a new paradigm for computer vision that may be applied to other early vision operators.

KNOWLEDGE-BASED SCENE EXPLORATION USING COMPUTER VISION AND LEARNED ANALYSIS STRATEGIES

In this contribution we demonstrate how the task of visual scene exploration can be solved by a knowledge-based vision system. During scene exploration, the system searches for a fixed number of a priori known objects in a static scene. If not all objects are visible using the initial camera set-up, the camera parameters have to be adjusted and the camera has to be moved by the system. This problem is reduced to the choice of optimal camera actions. The information about the objects and the camera actions is uniformly represented in a semantic network. In addition, a control algorithm is provided that finds the optimal assignment from objects to parts of a scene based on a suitable analysis strategy. This strategy is acquired by the system itself using reinforcement learning methods. The paper focuses on aspects of knowledge representation concerning the integration of camera actions and on the integration of reinforcement learning methods in a semantic network formalism and applies them in a realistic setup. Experiments are shown for images of two office rooms.

User-agent cooperation in multiagent IVUS image segmentation

Automated interpretation of complex images requires elaborate knowledge and model-based image analysis, but often needs interaction with an expert as well. This research describes expert interaction with a multiagent image interpretation system using only a restricted vocabulary of high-level user interactions. The aim is to minimize inter- and intra-observer variability by keeping the total number of interactions as low and simple as possible. The multiagent image interpretation system has elaborate high-level knowledge-based control over low-level image segmentation algorithms. Agents use contextual knowledge to keep the number of interactions low but, when in doubt, present the user with the most likely interpretation of the situation. The user, in turn, can correct, supplement, and/or confirm the results of image-processing agents. This is done at a very high level of abstraction such that no knowledge of the underlying segmentation methods, parameters or agent functioning is needed. High-level interaction thereby replaces more traditional contour correction methods like inserting points and/or (re)drawing contours. This makes it easier for the user to obtain good results, while inter- and intra-observer variability are kept minimal, since the image segmentation itself remains under control of image-processing agents. The system has been applied to intravascular ultrasound (IVUS) images. Experiments show that with an average of 2-3 high-level user interactions per correction, segmentation results substantially improve while the variation is greatly reduced. The achieved level of accuracy and repeatability is equivalent to that of manual drawing by an expert.

Comparing Bayesian neural network algorithms for classifying segmented outdoor images

In this paper we investigate the Bayesian training of neural networks for region labelling of segmented outdoor scenes; the data are drawn from the Sowerby Image Database of British Aerospace. Neural networks are trained with two Bayesian methods, (i) the evidence framework of MacKay (1992a,b) and (ii) a Markov Chain Monte Carlo method due to Neal (1996). The performance of the two methods is compared to evaluating the empirical learning curves of neural networks trained with the two methods. We also investigate the use of the Automatic Relevance Determination method for input feature selection.

Automatic segmentation and classification of outdoor images using neural networks

The paper describes how neural networks may be used to segment and label objects in images. A self-organising feature map is used for the segmentation phase, and we quantify the quality of the segmentations produced as well as the contribution made by colour and texture features. A multi-layer perception is trained to label the regions produced by the segmentation process. It is shown that 91.1% of the image area is correctly classified into one of eleven categories which include cars, houses, fences, roads, vegetation and sky.