Emulation as an integrating principle for cognition

Touching events predict human action segmentation in brain and behavior

Recognizing the actions of others depends on segmentation into meaningful events. After decades of research in this area, it remains still unclear how humans do this and which brain areas support underlying processes. Here we show that a computer vision-based model of touching and untouching events can predict human behavior in segmenting object manipulation actions with high accuracy. Using this computational model and functional Magnetic Resonance Imaging (fMRI), we pinpoint the neural networks underlying this segmentation behavior during an implicit action observation task. Segmentation was announced by a strong increase of visual activity at touching events followed by the engagement of frontal, hippocampal and insula regions, signaling updating expectation at subsequent untouching events. Brain activity and behavior show that touching-untouching motifs are critical features for identifying the key elements of actions including object manipulations.

Revealing the Computational Meaning of Neocortical Interarea Signals

To understand the function of the neocortex, which is a hierarchical distributed network, it is useful giving meaning to the signals transmitted between these areas from the computational viewpoint. The overall anatomical structure or organs related to this network, including the neocortex, thalamus, and basal ganglia, has been roughly revealed, and much physiological knowledge, though often fragmentary, is being accumulated. The computational theories involving the neocortex have also been developed considerably. By introducing the assumption “The signals transmitted by interarea axonal projections of pyramidal cells in the neocortex carry different meanings for each cell type, common to all areas,” derived from its nature as a distributed network in the neocortex, allows us to specify the computational meanings of interarea signals. In this paper, first, the types of signals exchanged between neocortical areas are investigated, taking into account biological constraints, and employing theories such as predictive coding, reinforcement learning, representation emulation theory, and BDI logic as theoretical starting points, two types of feedforward signals (observation and deviation) and three types of feedback signals (prediction, plan, and intention) are identified. Next, based on the anatomical knowledge of the neocortex and thalamus, the pathways connecting the areas are organized and summarized as three corticocortical pathways and two thalamocortical pathways. Using this summation as preparation, this paper proposes a hypothesis that gives meaning to each type of signals transmitted in the different pathways in the neocortex, from the viewpoint of their functions. This hypothesis reckons that the feedforward corticocortical pathway transmits observation signals, the feedback corticocortical pathway transmits prediction signals, and the corticothalamic pathway mediated by core relay cells transmits deviation signals. The thalamocortical pathway, which is mediated by matrix relay cells, would be responsible for transmitting the signals that activate a part of prediction signals as intentions, due to the reason that the nature of the other available feedback pathways are not sufficient for conveying plans and intentions as signals. The corticocortical pathway, which is projected from various IT cells to the first layer, would be responsible for transmitting signals that activate a part of prediction signals as plans.

Human creativity, evolutionary algorithms, and predictive representations: The mechanics of thought trials

Creative thinking is arguably the pinnacle of cerebral functionality. Like no other mental faculty, it has been omnipotent in transforming human civilizations. Probing the neural basis of this most extraordinary capacity, however, has been doggedly frustrated. Despite a flurry of activity in cognitive neuroscience, recent reviews have shown that there is no coherent picture emerging from the neuroimaging work. Based on this, we take a different route and apply two well established paradigms to the problem. First is the evolutionary framework that, despite being part and parcel of creativity research, has no informed experimental work in cognitive neuroscience. Second is the emerging prediction framework that recognizes predictive representations as an integrating principle of all cognition. We show here how the prediction imperative revealingly synthesizes a host of new insights into the way brains process variation-selection thought trials and present a new neural mechanism for the partial sightedness in human creativity. Our ability to run offline simulations of expected future environments and action outcomes can account for some of the characteristic properties of cultural evolutionary algorithms running in brains, such as degrees of sightedness, the formation of scaffolds to jump over unviable intermediate forms, or how fitness criteria are set for a selection process that is necessarily hypothetical. Prospective processing in the brain also sheds light on how human creating and designing - as opposed to biological creativity - can be accompanied by intentions and foresight. This paper raises questions about the nature of creative thought that, as far as we know, have never been asked before.

Visual Illusions Based on Processes: New Classification System Needed

Illusions have mainly been classified according to their phenomenological appearance. Here, I plead for a new classification approach based on processing areas or mechanisms. Classifying visual illusions according to processing areas or mechanisms may not only be valuable for a better understanding of the visual system but also for diagnostics of impairments, degenerative effects, and lesions (from retina to striate and extra-striate cortex).

The basal ganglia select the expected sensory input used for predictive coding

While considerable evidence supports the notion that lower-level interpretation of incoming sensory information is guided by top-down sensory expectations, less is known about the source of the sensory expectations or the mechanisms by which they are spread. Predictive coding theory proposes that sensory expectations flow down from higher-level association areas to lower-level sensory cortex. A separate theory of the role of prediction in cognition describes "emulations" as linked representations of potential actions and their associated expected sensation that are hypothesized to play an important role in many aspects of cognition. The expected sensations in active emulations are proposed to be the top-down expectation used in predictive coding. Representations of the potential action and expected sensation in emulations are claimed to be instantiated in distributed cortical networks. Combining predictive coding with emulations thus provides a theoretical link between the top-down expectations that guide sensory expectations and the cortical networks representing potential actions. Now moving to theories of action selection, the basal ganglia has long been proposed to select between potential actions by reducing inhibition to the cortical network instantiating the desired action plan. Integration of these isolated theories leads to the novel hypothesis that reduction in inhibition from the basal ganglia selects not just action plans, but entire emulations, including the sensory input expected to result from the action. Basal ganglia disinhibition is hypothesized to both initiate an action and also allow propagation of the action's associated sensory expectation down towards primary sensory cortex. This is a novel proposal for the role of the basal ganglia in biasing perception by selecting the expected sensation, and initiating the top-down transmission of those expectations in predictive coding.

Attention and prediction in human audition: a lesson from cognitive psychophysiology

Attention is a hypothetical mechanism in the service of perception that facilitates the processing of relevant information and inhibits the processing of irrelevant information. Prediction is a hypothetical mechanism in the service of perception that considers prior information when interpreting the sensorial input. Although both (attention and prediction) aid perception, they are rarely considered together. Auditory attention typically yields enhanced brain activity, whereas auditory prediction often results in attenuated brain responses. However, when strongly predicted sounds are omitted, brain responses to silence resemble those elicited by sounds. Studies jointly investigating attention and prediction revealed that these different mechanisms may interact, e.g. attention may magnify the processing differences between predicted and unpredicted sounds. Following the predictive coding theory, we suggest that prediction relates to predictions sent down from predictive models housed in higher levels of the processing hierarchy to lower levels and attention refers to gain modulation of the prediction error signal sent up to the higher level. As predictions encode contents and confidence in the sensory data, and as gain can be modulated by the intention of the listener and by the predictability of the input, various possibilities for interactions between attention and prediction can be unfolded. From this perspective, the traditional distinction between bottom-up/exogenous and top-down/endogenous driven attention can be revisited and the classic concepts of attentional gain and attentional trace can be integrated.

The many faces of precision (Replies to commentaries on "Whatever next? Neural prediction, situated agents, and the future of cognitive science")

An appreciation of the many roles of “precision-weighting” (upping the gain on select populations of prediction error units) opens the door to better accounts of planning and “offline simulation,” makes suggestive contact with large bodies of work on embodied and situated cognition, and offers new perspectives on the “active brain”. Combined with the complex affordances of language and culture, and operating against the essential backdrop of a variety of more biologically basic ploys and stratagems, the result is a maximally context-sensitive, restless, constantly self-reconfiguring architecture.

TEST: a tropic, embodied, and situated theory of cognition

TEST is a novel taxonomy of knowledge representations based on three distinct hierarchically organized representational features: Tropism, Embodiment, and Situatedness. Tropic representational features reflect constraints of the physical world on the agent's ability to form, reactivate, and enrich embodied (i.e., resulting from the agent's bodily constraints) conceptual representations embedded in situated contexts. The proposed hierarchy entails that representations can, in principle, have tropic features without necessarily having situated and/or embodied features. On the other hand, representations that are situated and/or embodied are likely to be simultaneously tropic. Hence, although we propose tropism as the most general term, the hierarchical relationship between embodiment and situatedness is more on a par, such that the dominance of one component over the other relies on the distinction between offline storage versus online generation as well as on representation-specific properties.

Surprised at all the entropy: hippocampal, caudate and midbrain contributions to learning from prediction errors

Influential concepts in neuroscientific research cast the brain a predictive machine that revises its predictions when they are violated by sensory input. This relates to the predictive coding account of perception, but also to learning. Learning from prediction errors has been suggested for take place in the hippocampal memory system as well as in the basal ganglia. The present fMRI study used an action-observation paradigm to investigate the contributions of the hippocampus, caudate nucleus and midbrain dopaminergic system to different types of learning: learning in the absence of prediction errors, learning from prediction errors, and responding to the accumulation of prediction errors in unpredictable stimulus configurations. We conducted analyses of the regions of interests' BOLD response towards these different types of learning, implementing a bootstrapping procedure to correct for false positives. We found both, caudate nucleus and the hippocampus to be activated by perceptual prediction errors. The hippocampal responses seemed to relate to the associative mismatch between a stored representation and current sensory input. Moreover, its response was significantly influenced by the average information, or Shannon entropy of the stimulus material. In accordance with earlier results, the habenula was activated by perceptual prediction errors. Lastly, we found that the substantia nigra was activated by the novelty of sensory input. In sum, we established that the midbrain dopaminergic system, the hippocampus, and the caudate nucleus were to different degrees significantly involved in the three different types of learning: acquisition of new information, learning from prediction errors and responding to unpredictable stimulus developments. We relate learning from perceptual prediction errors to the concept of predictive coding and related information theoretic accounts.